A roof image segmentation method based on hybrid framework
Through the roof image segmentation method based on hybrid frames, the deep convolutional neural network and multi-scale information fusion module are used, combined with the boundary loss function, the problems of poor roof topology processing capabilities and insufficient automation training in the existing technology are solved, and the roof is realized Fast, efficient and precise segmentation of images.
Patent Information
- Application Number
- CN202111525707.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-14
AI Technical Summary
The existing roof profile segmentation method has poor processing capabilities and lacks automated training when processing roof topology in aerial images. It relies on manual parameters to set the parameters and cannot effectively process a large amount of data.
Using a roof image segmentation method based on a hybrid framework, a hybrid model including learning module and inference module is constructed. The initial contour is extracted through a deep convolutional neural network and the parameters are updated using backpropagation. Combining the multi-scale information fusion module and boundary loss function, it realizes Fast, efficient and precise segmentation of roof images.
The accurate boundary topological structure segmentation of roof images is realized, segmentation accuracy and automated training capabilities are improved, dependence on manual processing is reduced, and fuzzy boundary situations can be effectively handled.
Smart Images

Figure CN114359558B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and in particular relates to a roof image segmentation method based on a hybrid framework. Background Art
[0002] Image segmentation technology has a significant position in various fields. Among them, extracting roof contours from remote sensing images and aerial images has important applications in urban planning, land cover analysis and natural disaster relief operations. Existing roof contour segmentation methods are limited to a large amount of manual processing, which requires a lot of manpower and financial resources, and the accuracy cannot meet the requirements. With the rapid development of computer technology, deep learning has achieved remarkable results in the field of images. Deep learning reduces the need for manual processing and improves efficiency. However, how to deal with the fuzzy boundaries of roofs and image topological structures is still a major problem facing deep convolutional neural networks. In traditional segmentation methods, the active contour model uses a continuous curve to represent the target contour, and updates the curve based on the curve evolution theory and gradient descent method, so it has a good ability to handle the topological changes of the segmentation target boundary. However, the training of the traditional active contour model still relies on manually set parameters and cannot carry out training on large amounts of data. Summary of the invention
[0003] The main purpose of the present invention is to propose a roof image segmentation method based on a hybrid framework, which can solve the problems of poor processing ability and automatic training of the topological structure of the roof outline in aerial images in the existing image segmentation methods. Based on the hybrid framework, the method establishes a fast, efficient and accurate building outline segmentation model, and based on the segmentation model, realizes the segmentation of the roof image.
[0004] To achieve the above object, the technical solution adopted by the present invention is:
[0005] The embodiment of the present invention provides a roof image segmentation method based on a hybrid framework, comprising:
[0006] S1. Build the roof sample dataset required for training;
[0007] S2, constructing a hybrid model including a learning module and a reasoning module; inputting the data of the roof sample data set into the learning module;
[0008] S3, using the deep convolutional neural network constructed in the learning module to learn the image sample and extract the initial outline of the roof in the image;
[0009] S4, using the reasoning module to adjust the initial contour extracted by the learning module, and back-propagating to update the parameters of the learning module;
[0010] S5. Complete the training of the hybrid model through multiple iterations, and use the hybrid model to complete the segmentation of the roof image to be processed.
[0011] Furthermore, in step S1, the roof sample data set includes a plurality of original images and label data corresponding to the original images; the label data uses binarization to describe the background and segment the target for the pixel values.
[0012] Further, in step S2, the learning module is based on DenseNet, and includes an image downsampling module, a multi-scale information fusion module and an image upsampling module connected in sequence;
[0013] The image downsampling module consists of a convolution layer, a maximum pooling layer, three average pooling layers and four dense convolution blocks, which are used to extract image features;
[0014] The multi-scale information fusion module is composed of four pooling layers of different sizes: 1×1, 2×2, 3×3, and 6×6, to obtain feature maps of multiple sizes and fuse them;
[0015] The image upsampling module consists of 4 convolutional layers and 5 deconvolutional layers to obtain the initial feature image.
[0016] Furthermore, the reasoning module is a CV model, wherein the loss function used in back propagation to update the parameters of the learning module is:
[0017] L loss =L Dice +L1
[0018] L1=||φ-φ GT ||
[0019] Among them, L Dice is the Dice loss function, L1 is the boundary loss function; φ GT It is expressed as the active contour form of Ground truth, φ is the result output by the reasoning module, and ||.|| is the absolute error.
[0020] The beneficial effects of the present invention are:
[0021] The present invention proposes a roof image segmentation method based on a hybrid framework, including: constructing a roof sample data set required for training; constructing a hybrid model including a learning module and an inference module; inputting data of the roof sample data set into the learning module; using the deep convolutional neural network constructed in the learning module to learn the image sample and extract the initial contour of the roof in the image; using the inference module to adjust the initial contour extracted by the learning module and back propagating to update the parameters of the learning module; completing the hybrid model training through multiple iterations, and using the hybrid model to complete the segmentation of the roof image to be processed. The present invention segments the roof image based on the hybrid framework, better retains the roof contour information by constructing the learning module and the inference module, and ensures that the accurate boundary topology structure can be obtained in the case of blurred boundaries.
[0022] Furthermore, this method gives full play to the advantages of the hybrid framework, adds a multi-scale information fusion module to the learning module to obtain more detailed information, and uses the reasoning module to improve the segmentation accuracy of the learning module; at the same time, the boundary loss function is introduced to make the segmentation result closer to the edge of the segmentation target. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A flow chart of a roof image segmentation method based on a hybrid framework provided by an embodiment of the present invention;
[0024] Figure 2 A schematic diagram of a framework for roof image segmentation based on a hybrid framework provided by an embodiment of the present invention;
[0025] Figure 3 A structural diagram of a learning module provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0026] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.
[0027] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0028] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0029] The present invention provides a roof image segmentation method based on a hybrid framework, referring to Figure 1 As shown, the method includes:
[0030] S1. Build the roof sample dataset required for training;
[0031] S2, constructing a hybrid model including a learning module and a reasoning module; inputting the data of the roof sample data set into the learning module;
[0032] S3, using the deep convolutional neural network constructed in the learning module to learn the image sample and extract the initial outline of the roof in the image;
[0033] S4, using the reasoning module to adjust the initial contour extracted by the learning module, and back-propagating to update the parameters of the learning module;
[0034] S5. Complete the training of the hybrid model through multiple iterations, and use the hybrid model to complete the segmentation of the roof image to be processed.
[0035] like Figure 2 As shown, the roof image to be segmented is input into the learning module of the hybrid model to extract features; then the extracted features are input into the reasoning module to obtain the segmentation result. In the embodiment of the present invention, the method segments the roof image based on the hybrid framework, and better retains the roof contour information by constructing the learning module and the reasoning module, and ensures that the accurate boundary topology structure can be obtained in the case of blurred boundaries.
[0036] The following is a detailed description of each of the above steps:
[0037] In the above step S1, the roof sample data set includes multiple original images and label data corresponding to the original images; the label data uses binary pixel values to describe the background and segmentation target. The original data and the label data constitute the image data set required for training. For example, in the label data, a pixel value of 0 represents the background, a pixel value of 1 represents the segmentation target, and the image size is 512 pixels × 512 pixels.
[0038] In the above step S2, the learning module is based on DenseNet and adds a multi-scale information fusion module to obtain feature maps of multiple sizes and fuse them to better retain detail information; finally, the extracted contour is converted into a level set form suitable for inference module training through a signed distance function.
[0039] This learning module refers to Figure 3 As shown, it includes an image downsampling module, a multi-scale information fusion module and an image upsampling module which are connected in sequence; wherein the image downsampling module is composed of a convolution layer, a maximum pooling layer, three average pooling layers and four dense convolution blocks for extracting image features; the multi-scale information fusion module is composed of four pooling layers of different sizes, 1×1, 2×2, 3×3, and 6×6, to obtain feature maps of multiple sizes and fuse them; the image upsampling module is composed of 4 convolution layers and 5 deconvolution layers to obtain the initial feature image.
[0040] The above step S3 comprises:
[0041] S31, input training image data into a learning module;
[0042] S32. The multi-scale information fusion module in the learning module consists of four pooling layers of different sizes, and the pooling layer sizes are 1×1, 2×2, 3×3, and 6×6. The global information is retained at different scales, and then the features of different sizes are fused to retain more detailed information;
[0043] S33, the learning module converts the extracted contour into a level set form suitable for reasoning module training through a signed distance function.
[0044] In the above step S4, the reasoning module is a CV model, and the reasoning module obtains a fine-tuning result through the evolution of the level set, and updates the learning module parameters by calculating the error between the fine-tuning result and the groundtruth.
[0045] The loss function used in back propagation to update the parameters of the learning module is:
[0046] L loss =L Dice +L1
[0047] L1=||φ-φ GT ||
[0048] Among them, L Dice is the Dice loss function, L1 is the boundary loss function; φ GT It is expressed as the active contour form of Ground truth (correct labeled data), φ is the result output by the reasoning module, and ||.|| is the absolute error.
[0049] The above step S5 completes the hybrid model training through multiple iterations, and uses the hybrid model to complete the segmentation of the roof image to be processed. The stop iteration condition is set manually, and usually stops iteration when the inference module reaches the maximum number of inferences or there is no change.
[0050] The roof image segmentation method based on a hybrid framework provided in an embodiment of the present invention can be used for the segmentation of roofs in images such as remote sensing and aerial images. The present invention establishes a fast, efficient and accurate building contour segmentation model based on the hybrid framework to complete the segmentation of the roof image to be processed. Compared with the existing methods, the present invention introduces a multi-scale information fusion module in the deep convolutional neural network used in the learning module to better retain contextual information to improve segmentation accuracy. The initial contour obtained by the deep neural network is input into the reasoning module, and the reasoning module is fine-tuned based on the initial contour. The defined loss function is then used to backpropagate to the designed learning module for learning.
[0051] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A roof image segmentation method based on a hybrid framework, characterized in that: include: S1. Construct a rooftop sample dataset of remote sensing images and aerial images required for training; S2, constructing a hybrid model including a learning module and a reasoning module; inputting the data of the roof sample data set into the learning module; S3, using the deep convolutional neural network constructed in the learning module to learn the image sample and extract the initial outline of the roof in the image; S4, using the reasoning module to adjust the initial contour extracted by the learning module, and back-propagating to update the parameters of the learning module; S5, completing the hybrid model training through multiple iterations, and using the hybrid model to complete the segmentation of the roof image to be processed; Wherein, in step S2, the learning module is based on DenseNet, and includes an image downsampling module, a multi-scale information fusion module and an image upsampling module connected in sequence; The image downsampling module consists of a convolution layer, a maximum pooling layer, three average pooling layers and four dense convolution blocks, which are used to extract image features; The multi-scale information fusion module is composed of four pooling layers of different sizes: 1×1, 2×2, 3×3, and 6×6, to obtain feature maps of multiple sizes and fuse them; The image upsampling module consists of 4 convolutional layers and 5 deconvolutional layers to obtain the initial feature image.
2. The roof image segmentation method based on hybrid framework according to claim 1, characterized in that: In step S1, the roof sample data set includes a plurality of original images and label data corresponding to the original images; the label data uses binary pixel values to describe the background and segment the target.
3. The roof image segmentation method based on hybrid framework according to claim 1, characterized in that: The inference module is a CV model, in which the loss function used in back-propagation updating the parameters of the learning module is: L loss =L Dice +L1 L1=||φ-φ GT || Among them, L Dice is the Dice loss function, L1 is the boundary loss function; φ GT It is expressed as the active contour form of Ground truth, φ is the result output by the reasoning module, and ||.|| is the absolute error.
