A neural network building segmentation method and system combined with wavelet transform

Abstract

The application belongs to the technical field of geographic information systems, and discloses a neural network building segmentation method and system combined with wavelet transformation, which comprises the following steps: obtaining original aerial images, marking the original aerial images by using the mean marking method, and classifying different building labels according to sizes; performing Haar wavelet transformation on the classified building labels, making labels of different sizes, and performing feature mapping on the labels by using a convolutional neural network to generate high-level semantic features; training a wavelet transformation network based on a double-task regression learning method and the high-level semantic features to obtain the trained wavelet transformation network; inputting the original aerial images into the trained wavelet transformation network for inference, performing post-processing on the model output, and outputting complete building semantic segmentation results; and the neural network combined with wavelet transformation is used for downsampling, the double-task learning high-level semantic prior and labels can be outputted, and the building segmentation precision is improved.

Description

Technical Field

[0001] This invention relates to the field of geographic information system technology, and in particular to a method and system for building segmentation in aerial imagery using a neural network incorporating wavelet transform. Background Technology

[0002] Building roof extraction from aerial imagery is a fundamental task for key applications such as geographic information updating and 3D city modeling, directly impacting the level of automation in ground feature mapping. Current technologies primarily employ deep learning methods for pixel-level segmentation, such as U-Net and its variants, visual Transformer-based networks, and instance-based segmentation methods (e.g., Mask R-CNN).

[0003] However, several challenges remain. First, in convolutional neural network-based methods, downsampling leads to the loss of detailed information, easily resulting in the loss of small targets, edge details, and discontinuities in large buildings. Second, existing methods that directly generate all building predictions at the highest level do not fully utilize the characteristics of features at different levels. For example, low-level features retain more details, which helps capture small targets, while high-level features establish broader global information, which is beneficial for capturing large targets. Third, neural network-based methods typically learn labels (masks) directly from the input (RGB), while the input and labels often have significant differences. This usually requires more resources to design more efficient neural networks to fit this process.

[0004] Therefore, existing technologies still suffer from problems such as loss of downsampling information, inaccurate segmentation, and large differences between input and label. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for building segmentation of aerial images using a neural network combined with wavelet transform, in order to solve the problems of loss of downsampling information, inaccurate segmentation, and large differences between input and label in the existing technology.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides a method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, comprising: The original aerial images were acquired, and the original aerial images were labeled using the mean labeling method, and different building labels were classified according to size. The classified building labels are processed by Haar wavelet transform to create labels of different sizes, and the labels are then mapped using a convolutional neural network to generate high-level semantic features. The wavelet transform network is trained based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network. The original aerial imagery is input into the trained wavelet transform network for inference, and the model output is post-processed to output a complete semantic segmentation result for the building.

[0007] In one implementation, the step of labeling the original aerial image using the mean labeling method and classifying different building labels according to size includes: Take the average value on the R channel, G channel and B channel, merge the three channels into a single channel, and replace the pixel value of the corresponding area with the average pixel value of the building area after rounding. Subtract the preset value from the preset position of the remaining pixel value to obtain the labeled label. The statistical data collection includes the area range of all individual building rooftops. Based on the marked labels, buildings are classified into three sizes: small, medium, and large, according to the individual area range of 0~R / 3, R / 3~2R / 3, and 2R / 3~R. The resulting building labels are then obtained.

[0008] In one implementation, the classified building labels are processed using Haar wavelet transform to create labels of different sizes, and the labels are then feature-mapped using a convolutional neural network to generate high-level semantic features, including: The classified building labels are processed by Haar wavelet transform to decompose them into their respective wavelet coefficients. The decomposed wavelet coefficients are then input into the convolutional neural network, and the high-level semantic features of the labels are obtained after feature mapping.

[0009] In one implementation, the trained wavelet transform network includes a pre-layer and a post-layer. The front layer is used to capture local texture features and edge features of the image; The post-layer is used to mine local contextual features of the image based on the self-attention of the moving window, and to model global dependencies under global self-attention, refining global information and local details in the final prediction head; In each downsampling process of the post-layer, Haar wavelet decomposition is used for processing.

[0010] In one implementation, training the wavelet transform network based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network includes: Based on the combined loss function, a pre-defined high-level semantic feature is learned in the front layer using a simple learning mechanism, and the output is obtained through the back layer; wherein, the output obtained by the back layer is used to learn the wavelet coefficients of the building label; Alternatively, based on the combined loss function, a fast learning mechanism can be used in the post-layer to learn to map the high-level semantic features back to the corresponding wavelet coefficients, and a fine-tuning method can be used to jointly regress the wavelet coefficients of the building labels with the pre-layer.

[0011] In one implementation, the combined loss function is: ; ; ; ; in, Indicates the total loss. , These represent the losses for the approximation coefficients and the other three coefficients in the wavelet coefficients, respectively. This represents the loss of the intermediate layer of the wavelet transform network described above. Indicates coefficient; This represents a loss function that combines mean squared error and mean absolute error, used to compare the differences between the predicted image and the target image at the pixel level. This represents a loss function based on image structure similarity, used to compare the differences between the predicted image and the target image in terms of brightness, contrast, and structure. This represents the perceptual loss function, used to compare the differences between the predicted image and the target image in the feature space.

[0012] In one implementation, the original aerial imagery is input into the trained wavelet transform network for inference, and the model output is post-processed to output a complete building semantic segmentation result, including: The original aerial imagery is input into the trained wavelet transform network for inference, and wavelet coefficients of three sizes (large, medium, and small) are output. Three wavelet coefficients were subjected to inverse wavelet transform to generate three class labeling maps. The region growing method is used to find similar regions for each class labeling image, and regions with too few pixels are excluded from the found similar regions. The remaining similar regions are set as the segmented target regions; wherein the pixel values ​​of the similar regions meet preset conditions. The target region selected from each class labeling map is set to 1, and the remaining regions are set to 0, resulting in three segmentation maps of different sizes. The segmentation maps are then added together, and the regions with values ​​greater than 0 are designated as building regions, thus obtaining the final semantic segmentation result for buildings.

[0013] Secondly, the present invention provides a building segmentation system for aerial imagery based on a neural network incorporating wavelet transform, comprising: The labeling and classification module is used to acquire the original aerial images, label the original aerial images using the mean labeling method, and classify different building labels according to size; The feature mapping module is used to perform Haar wavelet transform on the classified building labels to create labels of different sizes, and to perform feature mapping on the labels through a convolutional neural network to generate high-level semantic features. The dual-task regression training module is used to train the wavelet transform network based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network. The inference and post-processing module is used to input the original aerial image into the trained wavelet transform network for inference, and to post-process the model output to output a complete semantic segmentation result of the building.

[0014] Thirdly, the present invention provides a terminal comprising: a processor and a memory, the memory storing a neural network aerial image building segmentation program incorporating wavelet transform, wherein the neural network aerial image building segmentation program incorporating wavelet transform, when executed by the processor, is used to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described in the first aspect.

[0015] Fourthly, the present invention also provides a computer-readable storage medium storing a neural network aerial image building segmentation program incorporating wavelet transform, which, when executed by a processor, is used to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described in the first aspect.

[0016] The present invention, by employing the above technical solution, has the following effects: This invention incorporates wavelet decomposition into the downsampling process of the network, avoiding the loss of downsampling information. Leveraging the fault tolerance of wavelet transform, it describes the segmentation problem as a regression problem, classifying and extracting buildings of three different sizes (large, medium, and small), thus refining the division of labor at different levels. Furthermore, this invention uses a dual-task regression learning mechanism to output high-level semantic priors and labels learned through dual tasks, shortening the distance between input and output. In the intermediate layers of the network, it deepens the network's understanding of the labels by learning their high-level semantics, thereby improving the accuracy of building segmentation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the neural network aerial image building segmentation method combining wavelet transform in this invention.

[0019] Figure 2 This is a technical roadmap of the present invention.

[0020] Figure 3 This is a schematic diagram of the overall architecture of WaveletNet (wavelet transform network) in this invention.

[0021] Figure 4 This is a schematic diagram of the main module structure of WaveletNet (wavelet transform network) in this invention.

[0022] Figure 5 This is a schematic diagram of the size-based extraction method in this invention.

[0023] Figure 6 This is a schematic diagram of the Easy-Learning mechanism in this invention.

[0024] Figure 7 This is a schematic diagram of the Fast-Learning mechanism in this invention.

[0025] Figure 8 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0026] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0028] Exemplary methods Existing aerial imagery building segmentation techniques still face several challenges. First, in convolutional neural network-based methods, downsampling leads to the loss of detail, easily resulting in the loss of small targets, edge details, and discontinuities in large buildings. Second, existing methods that directly generate all building predictions at the highest level do not fully utilize the characteristics of features at different levels. For example, low-level features retain more details, which helps capture small targets, while high-level features establish broader global information, which is beneficial for capturing large targets. Third, neural network-based methods typically learn labels (masks) directly from the input (RGB), while the input and labels often have significant differences. This usually requires more resources to design more efficient neural networks to fit this process.

[0029] Therefore, existing technologies still suffer from problems such as loss of downsampling information, inaccurate segmentation, and large differences between input and label.

[0030] To address the above-mentioned technical problems, this invention provides a method for building segmentation based on a neural network using wavelet transform. The method includes: acquiring original aerial images; labeling the original aerial images using the mean labeling method and classifying them into different building labels according to size; performing Haar wavelet transform processing on the classified building labels to create annotations of different sizes, and using a convolutional neural network to perform feature mapping on the annotations to generate high-level semantic features; training the wavelet transform network based on a dual-task regression learning method and high-level semantic features to obtain a trained wavelet transform network; inputting the original aerial images into the trained wavelet transform network for inference, and post-processing the model output to output a complete building semantic segmentation result. This invention, by combining wavelet transform with a neural network for downsampling, can output dual-task learning of high-level semantic priors and labels, thus improving the accuracy of building segmentation.

[0031] like Figure 1 As shown, this embodiment of the invention provides a method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, comprising the following steps: Step S100: Acquire the original aerial image, label the original aerial image using the mean labeling method, and classify the buildings into different labels according to their size.

[0032] In this embodiment, a semantic segmentation network WaveletNet (wavelet transform network) combining wavelet technology and a learning mechanism that combines Easy-Learning and its variant Fast-Learning is provided: First, to alleviate the problem of information loss during downsampling, given the information conservation property of wavelet transform, all downsampling operations in WaveletNet adopt the Haar wavelet transform method. Second, in order to address the problem of segmentation loss and discontinuity caused by changes in building size, and to make full use of features at different levels, this embodiment divides buildings into three types: large, medium and small, based on their area. It adopts an attention mechanism and uses features at different levels to autonomously respond to buildings of different sizes. Third, WaveletNet transforms the pixel-level classification problem into a regression problem. To make the regression problem simpler, it proposes the Easy-Learning learning mechanism and its variant, the Fast-Learning learning mechanism. It relies on methods such as shortening the distance between the input and the label to make the network easier to learn.

[0033] The technical approach in this embodiment is as follows: Figure 2 As shown, the details are as follows: First, aerial imagery is input, labeled using the mean labeling method, and different buildings are classified by size. Then, Haar wavelet transform is performed to create labels of different sizes. Next, a neural network is used to map the features of these labels, generating high-level semantic priors. Following this, WaveletNet dual-task regression is used to learn the labels and high-level semantic priors, thereby training a robust parametric model. During inference, the aerial imagery is input into this model to obtain its output. Finally, the model output is post-processed to produce a complete semantic segmentation result for the buildings.

[0034] In the above technical approach, the input data in this embodiment are RGB aerial images and semantic label maps. After end-to-end supervised training using a WaveletNet neural network combined with an Easy-Learning or Fast-Learning learning mechanism, a well-trained parameter model is finally obtained. After obtaining the trained wavelet transform network, inputting RGB aerial images again will allow the model to output a segmentation map with 0 (background) and 1 (building) labels.

[0035] In the process of labeling, classification and feature mapping, the mean labeling method is first used to label the buildings in the aerial image. The buildings are divided into three sizes: small, medium and large. The labels of the three sizes are then subjected to Haar wavelet decomposition and feature mapping to obtain the high-level semantic prior of the labels.

[0036] Specifically, in one implementation of this embodiment, step S100 includes the following steps: Step S101: Take the average value of the R channel, G channel and B channel, merge the three channels into a single channel, and replace the pixel value of the corresponding area with the average pixel value of the building area after rounding. Subtract the preset value from the preset position (i.e. the position close to the integer) of the remaining pixel value to obtain the marked label. Step S102: Collect data on the area range of all individual building rooftops. Classify the buildings into small, medium, and large sizes based on the area range of 0~R / 3, R / 3~2R / 3, and 2R / 3~R from the marked labels, and obtain the classified building labels.

[0037] In this embodiment, the label classification and feature mapping process is as follows: 1) The mean labeling method is used to label buildings in aerial imagery. Specifically, the mean values ​​of the R, G, and B channels are taken, these three channels are merged into a single channel, and the average pixel value of the building area is used as the labeling factor. (i represents different individual building areas) Replace the pixel value of that area, and subtract 0.5 for the remaining pixel values ​​that are close to integers, thus achieving the labeling. The labeled label is denoted as Marked_Label (the labeled label).

[0038] 2) Collect data on the area range (0-R) of all individual building rooftops. The area is measured by the number of pixels of each individual building. In Marked_Label, buildings of three sizes are classified into small, medium and large based on the range of individual area: 0~R / 3, R / 3~2R / 3, and 2R / 3~R. These constitute three sizes of Marked_Label.

[0039] like Figure 1 As shown, this embodiment of the invention provides a method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, comprising the following steps: Step S200: Perform Haar wavelet transform on the classified building labels to create labels of different sizes, and use a convolutional neural network to perform feature mapping on the labels to generate high-level semantic features.

[0040] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201: Perform Haar wavelet transform on the classified building labels to decompose them into their respective wavelet coefficients. Input the decomposed wavelet coefficients into the convolutional neural network and obtain the high-level semantic features of the labels after feature mapping.

[0041] In this embodiment, the label classification and feature mapping process further includes: 3) Perform Haar wavelet decomposition on each Marked_Label to obtain its respective wavelet coefficients. Then, apply convolutional neural networks (convolutional modules) to the decomposed coefficients for feature mapping, obtaining the high-level semantic prior of the label, denoted as... .

[0042] like Figure 1 As shown, this embodiment of the invention provides a method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, comprising the following steps: Step S300: Train the wavelet transform network based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network.

[0043] In this embodiment, the trained wavelet transform network includes a pre-layer and a post-layer. The pre-layer is used to capture local texture features and edge features of the image. The post-layer is used to mine local context features of the image based on moving window self-attention, and to model global dependencies under global self-attention, refining global information and local details in the final prediction head. In each downsampling process of the post-layer, Haar wavelet decomposition is used for processing.

[0044] In this embodiment, the architecture of WaveletNet is summarized as follows: Figure 3 As shown, it is divided into pre-processing and post-processing layers, adopting a sequential structure of CNN-Transformer-CNN (Convolutional Network-Deep Learning Network-Convolutional Network). CNN acts as the pre-processing feature extractor, while convolution excels at capturing fine-grained features such as local textures and edges, while maintaining low complexity, making it suitable for processing high-resolution features. In the intermediate stage (the first processing stage of the post-processing layer), moving window self-attention is used to process medium-resolution features, further mining local contextual features. In the later stage (the second processing stage of the post-processing layer), a global self-attention module is used to further model global dependencies. In the final FCA (Adaptive Fine-Grained Channel Attention) prediction head, CNN is used to further refine global information and local details. Simultaneously, each downsampling step employs Haar wavelet decomposition. Compared to traditional downsampling methods (such as strided convolution and pooling), wavelet decomposition has an energy conservation relationship, meaning the energy (information) of the original image is conserved before and after the wavelet transform. Thanks to this, no information is lost throughout the entire process, which is highly advantageous for capturing small targets and detailed textures. In summary, this structure ensures that the model can capture both local details and global dependencies simultaneously, while maintaining a moderate computational cost. The specific structure of the modules in the wavelet transform network is as follows: Figure 4 As shown.

[0045] Furthermore, considering the characteristics of wavelet coefficients: among the four wavelet coefficients (LL, HL, LH, HH, where LL represents approximation coefficients, LH represents horizontal coefficients, HL represents vertical coefficients, and HH represents diagonal coefficients), LL represents the overall features of the image at a larger scale and differs significantly from the other three coefficients (LH, HL, HH), while LH, HL, and HH have high similarity. Therefore, the four wavelet coefficients are classified into two categories: LL is one category, and LH, HL, and HH are another. In the wavelet transform network, LL has its own branch, while the other three share another branch. To preserve the features of the original wavelet coefficients as much as possible, the features of the first wavelet decomposition are fused into the coefficients of each subsequent wavelet decomposition using a dynamic gating mechanism.

[0046] In semantic segmentation, small targets are often lost, while large targets are discontinuous. It's generally believed that shallow layers retain features of both small and large targets, while deeper layers, with their larger receptive fields, retain features of large targets but ignore many small ones. This embodiment, to more clearly define and refine the division of labor among the network layers, categorizes buildings into large, medium, and small targets based on size, and employs channel attention (ECA) to allow the network to autonomously assign different weights to features of different resolutions. The process is as follows: Figure 5 As shown. In Figure 5 In the process shown, the network uses multi-resolution feature maps from different layers, along with an attention mechanism, to extract small, medium, and large targets separately, and finally merges them to form the final segmentation result. Compared to the approach of directly predicting the entire segmentation map, this method aims to refine the extraction task, enabling the network to make reasonable use of features and improve its recognition of targets of different sizes.

[0047] In this embodiment, based on the WaveletNet architecture and processing principles described above, the WaveletNet dual-task learning mechanism is used to train WaveletNet.

[0048] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301: Based on the combined loss function, a pre-defined high-level semantic feature (i.e., a high-level semantic feature close to the building label) is learned in the front layer using a simple learning mechanism, and the output is obtained through the back layer; wherein, the output obtained by the back layer is used to learn the wavelet coefficients of the building label; Step S302: Based on the combined loss function, a fast learning mechanism is used to learn to map the high-level semantic features back to the corresponding wavelet coefficients in the post-layer, and a fine-tuning method is used to jointly regress the wavelet coefficients of the building labels in the pre-layer.

[0049] In this embodiment, WaveletNet transforms the pixel-level classification problem into a regression problem, leveraging the characteristics of inverse wavelet transform. Typical semantic segmentation tasks use a softmax function (normalized exponential function) to map the final result to a probability map of 0-1, generating the segmentation result. WaveletNet, however, regresses the wavelet coefficients of the Marked_Label decomposition, obtaining the segmentation result through the inverse transform of the learned wavelet coefficients. The advantage of this is that when the wavelet coefficients are subjected to a small random perturbation and then inversely transformed back, the result remains almost unchanged after rounding. This means that learning wavelet coefficients allows the neural network to have greater fault tolerance.

[0050] In this embodiment, WaveletNet employs a dual-task learning mechanism of Easy-Learning or Fast-Learning. The Easy-Learning method involves the WaveletNet front layer (intermediate layer features) learning high-level semantic features close to the Marked_Label. Then, the output is obtained through a post-layer. The output is responsible for learning the coefficients of the Marked_Label decomposition. Both are learned simultaneously, as is the Easy-Learning mechanism. Figure 6 As shown. The mechanism of Fast-Learning is: the front layer is responsible for learning to map the original RGB input to... The post-layer is responsible for learning to map the output of the pre-layer (intermediate layer features) back to wavelet coefficients. Specifically, the post-layer first learns to map the output of the pre-layer (intermediate layer features) back to wavelet coefficients. Mapping back to coefficients (Phase 1), followed by fine-tuning to learn the joint pre-layer (still regression). The wavelet coefficients of the regressed Marked_Label (stage 2) are used in the Fast-Learning mechanism. Figure 7 As shown. This mechanism has two main purposes: 1) Mean labeling can increase the similarity between the label and the original RGB image, which is helpful for regression tasks; 2) High-level semantic information generally has some common characteristics. It is easier for a neural network to learn a feature that has also been mapped by the neural network than to directly learn a label. The intermediate layers of the network learn such high-level semantic prior knowledge, which enables the network to deeply understand the features of the target.

[0051] In this embodiment, based on the aforementioned dual-task learning mechanism, considering that directly comparing the pixel-level differences between the generated and target images using general regression losses (such as MAE loss and MSE loss, where MAE loss is the mean absolute error loss and MSE loss is the mean squared error loss) can easily lead to overly smooth generated images, resulting in the loss of details and texture, this method employs a combined loss function to mitigate this situation. Furthermore, among the four wavelet coefficients, the pixel accuracy of LL (approximation coefficients) is almost identical to the accuracy of the final result, indicating that LL contributes the most to the final result. Therefore, strong supervision is applied to LL; specifically, the combined loss function is: ; ; ; ; in, Indicates the total loss. , These represent the losses for the approximation coefficients and the other three coefficients in the wavelet coefficients, respectively. This represents the loss of the intermediate layer of the wavelet transform network described above. Indicates coefficient; This represents a loss function that combines mean squared error and mean absolute error, used to compare the difference between the predicted image and the target image at the pixel level, aiming to bring the predicted image closer to the target image at the pixel level. The loss function based on image structure similarity is used to compare the differences between the predicted image and the target image in terms of brightness, contrast, and structure, with the aim of bringing the predicted image and the target image closer in terms of brightness, contrast, and structure (this loss function compares locally and then averages globally). The perceptual loss function is used to extract high-level features of an image through a pre-trained deep neural network, which are then used to compare the differences between the predicted image and the target image in the feature space.

[0052] The specific loss function is as follows: ; ; ; in, , These represent the target image and the predicted image, respectively. It is a hyperparameter; This represents the local mean (brightness) of the predicted image. Representative image , The covariance (structural correlation). Represents the local standard deviation (contrast) of an image. , To maintain stability, the denominator should not be zero; The first one representing the feature extraction network layer, , , It is the first The channel height and width of the layer feature map.

[0053] In this embodiment, based on the aforementioned combined loss function, the training strategy for the wavelet transform network is specifically as follows: During training, data augmentation strategies are employed, including: Random rotation and color jittering are employed; the Adam optimizer is used, with an initial learning rate of 0.01; to maximize model performance, if the accuracy on the validation set fails to improve for three consecutive rounds, the learning rate is reduced to 0.7 times the current learning rate, and training terminates when the current learning rate falls below 1e-5; to conserve computational resources, L_3 loss is added when the learning rate first decreases, and the feature mapping parameters are initialized using the parameters of any two convolutions in the network (remaining unchanged); L_Percetual loss is added after the learning rate decreases five times. During the second stage of Fast-Learning training, the learning rate of subsequent layers is reduced to 1 / 10 of the initial learning rate.

[0054] This embodiment uses the aforementioned dual-task learning mechanism, combined loss function, and training strategy to obtain the trained wavelet transform network, which is then used for inference.

[0055] like Figure 1 As shown, this embodiment of the invention provides a method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, comprising the following steps: Step S400: Input the original aerial image into the trained wavelet transform network for inference, and perform post-processing on the model output to output a complete semantic segmentation result of the building.

[0056] Specifically, in one implementation of this embodiment, step S400 includes the following steps: Step S401: Input the original aerial image into the trained wavelet transform network for inference and output wavelet coefficients of three sizes: large, medium and small. Step S402: Perform inverse wavelet transform on the three wavelet coefficients to generate three class labeling maps; Step S403: Find similar regions for each class labeling image using the region growing method, and exclude regions with too low a number of pixels from the found similar regions, and set the remaining similar regions as the segmented target regions; wherein, the pixel values ​​of the similar regions meet a preset condition (i.e., the pixel values ​​of the similar regions are close to the average pixel value). Step S404: Set the target region selected from each class labeling map to 1 and the remaining regions to 0 to obtain three segmentation maps of different sizes. Add the segmentation maps together and set the positions with values ​​greater than 0 as building regions to obtain the final building semantic segmentation result.

[0057] In this embodiment, the original aerial image is input again for inference. The network outputs wavelet coefficients (3×4) of three sizes: large, medium, and small. Post-processing is performed according to the following steps: 1) First, perform inverse wavelet transform on the three wavelet coefficients to generate three class labeling maps.

[0058] 2) Use the region growing method to find similar regions (with similar pixel values) in each class labeling image. The pixel values ​​of these similar regions should be close. (Integer value), and finally exclude the regions with too few similar pixels. The remaining similar regions are the segmented targets.

[0059] 3) Set the selected areas to 1 and the remaining areas to 0 to form three sizes of segmentation maps. Finally, add the three segmentation maps together and mark the positions with values ​​greater than 0 as building areas to form the final segmentation result.

[0060] In summary, this embodiment employs WaveletNet, a segmentation network incorporating wavelet decomposition. This integrates wavelet decomposition into downsampling, mitigating information loss. Leveraging the fault tolerance of wavelet transform, the segmentation problem is described as a regression problem. Furthermore, it refines the division of labor at different levels by classifying and extracting data for buildings of three different sizes: large, medium, and small. In the specific processing, WaveletNet uses the wavelet coefficients of the regression label as its task. It extracts the LL coefficients and other coefficients from the input decomposition using two branches, progressively mining local and global information in a CNN-Transformer-CNN sequence. Downsampling is performed through feature fusion and wavelet transform. Finally, multi-level features are combined using an ECA attention mechanism to predict buildings of three sizes: large, medium, and small.

[0061] Based on WaveletNet, this embodiment also incorporates Easy-Learning and Fast-Learning mechanisms to support the regression task. These mechanisms shorten the distance between input and output using mean labeling and deepen the network's understanding of labels by learning high-level semantics in the intermediate layers. In the specific learning process, labels are first created on the input using mean labeling, and then the high-level semantic prior of the labels is mapped through the neural network. Easy-Learning, in conjunction with the pre-layer and output layers, learns both the high-level semantic prior and the labels. Fast-Learning first learns to map the high-level semantic prior back to the labels through the network's post-layer layers, and then learns both the high-level semantic prior and the labels through transfer learning, in conjunction with the pre-layer and output layers.

[0062] This embodiment achieves the following technical effects through the above technical solution: This embodiment incorporates wavelet decomposition into the downsampling process of the network, avoiding the loss of downsampling information. By leveraging the fault tolerance of wavelet transform, the segmentation problem is described as a regression problem. It classifies and extracts buildings of three different sizes (large, medium, and small), refining the division of labor at different levels. This embodiment outputs high-level semantic priors and labels learned through a dual-task regression learning mechanism, which can shorten the distance between input and output. Furthermore, by learning the high-level semantics of the labels in the middle layer of the network, the network deepens its understanding of the labels, thereby improving the accuracy of building segmentation.

[0063] Exemplary device Based on the above embodiments, the present invention also provides a neural network aerial image building segmentation system combining wavelet transform, comprising: The labeling and classification module is used to acquire the original aerial images, label the original aerial images using the mean labeling method, and classify different building labels according to size; The feature mapping module is used to perform Haar wavelet transform on the classified building labels to create labels of different sizes, and to perform feature mapping on the labels through a convolutional neural network to generate high-level semantic features. The dual-task regression training module is used to train the wavelet transform network based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network. The inference and post-processing module is used to input the original aerial image into the trained wavelet transform network for inference, and to post-process the model output to output a complete semantic segmentation result of the building.

[0064] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 8 As shown.

[0065] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a computer-readable storage medium and internal memory; the computer-readable storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the computer-readable storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0066] When executed by a processor, this computer program is used to implement a neural network-based building segmentation method for aerial imagery that incorporates wavelet transform.

[0067] It will be understood by those skilled in the art that Figure 8 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0068] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a neural network aerial image building segmentation program incorporating wavelet transform, the neural network aerial image building segmentation program incorporating wavelet transform being executed by the processor to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described above.

[0069] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a neural network aerial image building segmentation program incorporating wavelet transform, which, when executed by a processor, is used to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described above.

[0070] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0071] In summary, this invention provides a method and system for building segmentation based on aerial imagery using a neural network incorporating wavelet transform. The method includes: acquiring original aerial imagery; labeling the original aerial imagery using the mean labeling method and classifying buildings according to size; performing Haar wavelet transform processing on the classified building labels to create annotations of different sizes, and using a convolutional neural network to perform feature mapping on the annotations to generate high-level semantic features; training the wavelet transform network based on a dual-task regression learning method and high-level semantic features to obtain a trained wavelet transform network; inputting the original aerial imagery into the trained wavelet transform network for inference, and post-processing the model output to output a complete semantic segmentation result for the buildings; this invention improves building segmentation accuracy by combining wavelet transform with a neural network for downsampling, enabling dual-task learning of high-level semantic priors and labels.

[0072] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for building segmentation in aerial imagery using a neural network incorporating wavelet transform, characterized in that, include: The original aerial imagery is acquired and labeled using the mean labeling method. Different building labels are then categorized according to size. This process includes: taking the mean values ​​from the R, G, and B channels; merging the three channels into a single channel; replacing the pixel values ​​of the corresponding area with the rounded average pixel value of the building area; and subtracting a preset value from the preset position of the remaining pixel values ​​to obtain the labeled labels. The area range of all individual building rooftops is statistically analyzed. From the labeled labels, buildings are categorized into small, medium, and large sizes based on their area ranges: 0~R / 3, R / 3~2R / 3, and 2R / 3~R. These are the categorized building labels. The classified building labels are processed by Haar wavelet transform to create labels of different sizes, and the labels are feature-mapped through a convolutional neural network to generate high-level semantic features. This includes: processing the classified building labels by Haar wavelet transform, decomposing them to obtain their respective wavelet coefficients, and inputting the decomposed wavelet coefficients into the convolutional neural network. After feature mapping, the high-level semantic features of the labels are obtained. The wavelet transform network is trained based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network. The trained wavelet transform network includes a pre-layer and a post-layer. The pre-layer is used to capture local texture features and edge features of the image. The post-layer is used to mine local context features of the image based on moving window self-attention, and to model global dependencies under global self-attention, refining global information and local details in the final prediction head. In each downsampling process of the post-layer, Haar wavelet decomposition is used for processing. The original aerial imagery is input into the trained wavelet transform network for inference, and the model output is post-processed to output a complete semantic segmentation result for the building.

2. The method for building segmentation in aerial imagery using a neural network incorporating wavelet transform according to claim 1, characterized in that, The wavelet transform network is trained using the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network, including: Based on the combined loss function, a pre-defined high-level semantic feature is learned in the front layer using a simple learning mechanism, and the output is obtained through the back layer; wherein, the output obtained by the back layer is used to learn the wavelet coefficients of the building label; Alternatively, based on the combined loss function, a fast learning mechanism can be used in the post-layer to learn to map the high-level semantic features back to the corresponding wavelet coefficients, and a fine-tuning method can be used to jointly regress the wavelet coefficients of the building labels with the pre-layer.

3. The method for building segmentation in aerial imagery using a neural network incorporating wavelet transform according to claim 2, characterized in that, The combined loss function is: ； ； ； ； in, Indicates the total loss. , These represent the losses for the approximation coefficients and the other three coefficients in the wavelet coefficients, respectively. This represents the loss of the intermediate layer of the wavelet transform network described above. Indicates coefficient; This represents a loss function that combines mean squared error and mean absolute error, used to compare the differences between the predicted image and the target image at the pixel level. This represents a loss function based on image structure similarity, used to compare the differences between the predicted image and the target image in terms of brightness, contrast, and structure. This represents the perceptual loss function, used to compare the differences between the predicted image and the target image in the feature space.

4. The method for building segmentation in aerial imagery using a neural network incorporating wavelet transform according to claim 1, characterized in that, The process involves inputting the original aerial imagery into the trained wavelet transform network for inference, and post-processing the model output to output a complete building semantic segmentation result, including: The original aerial imagery is input into the trained wavelet transform network for inference, and wavelet coefficients of three sizes (large, medium, and small) are output. Three wavelet coefficients were subjected to inverse wavelet transform to generate three class labeling maps. The region growing method is used to find similar regions for each class labeling image, and regions with too few pixels are excluded from the found similar regions. The remaining similar regions are set as the segmented target regions; wherein the pixel values ​​of the similar regions meet preset conditions. The target region selected from each class labeling map is set to 1, and the remaining regions are set to 0, resulting in three segmentation maps of different sizes. The segmentation maps are then added together, and the regions with values ​​greater than 0 are designated as building regions, thus obtaining the final semantic segmentation result for buildings.

5. A neural network aerial image building segmentation system combining wavelet transform, used to implement the neural network aerial image building segmentation method combining wavelet transform as described in any one of claims 1-4, characterized in that, include: The labeling and classification module is used to acquire the original aerial images, label the original aerial images using the mean labeling method, and classify different building labels according to size; The feature mapping module is used to perform Haar wavelet transform on the classified building labels to create labels of different sizes, and to perform feature mapping on the labels through a convolutional neural network to generate high-level semantic features. The dual-task regression training module is used to train the wavelet transform network based on the dual-task regression learning method and the high-level semantic features to obtain the trained wavelet transform network. The inference and post-processing module is used to input the original aerial image into the trained wavelet transform network for inference, and to post-process the model output to output a complete semantic segmentation result of the building.

6. A terminal, characterized in that, include: The processor and memory, the memory storing a neural network aerial image building segmentation program incorporating wavelet transform, the neural network aerial image building segmentation program incorporating wavelet transform being executed by the processor to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a neural network aerial image building segmentation program incorporating wavelet transform, which, when executed by a processor, is used to implement the operation of the neural network aerial image building segmentation method incorporating wavelet transform as described in any one of claims 1-4.

Images