Building extraction model construction method and device, equipment and storage medium

Abstract

The invention provides a building extraction model construction method and device, equipment and a storage medium, and the method comprises the steps: obtaining a building image, and carrying out the preprocessing of the building image, and obtaining a training set; inputting the training set into a double-current decoder, outputting position features of different resolutions at each layer through a position extraction network, and outputting basic feature maps of different finesses at each layer through an overall extraction network; fusing the position features of the corresponding layers and the basic feature maps into N comprehensive feature maps, and inputting the basic feature map of the last layer of the overall extraction network into a global capture model to obtain a decoding feature map of the first layer of a decoder; inputting the decoding feature map of the previous layer of the decoder and the corresponding comprehensive feature map into an attention gating structure to obtain a decoding feature map of the current layer, and taking the decoding feature map of the (N + 1) th layer of the decoder as a final feature map; and inputting the final feature map into a prediction layer to extract a building, and repeatedly iterating and updating model parameters through extraction errors to obtain a final building extraction model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for constructing a building extraction model. Background Art

[0002] Buildings are an important part of cities. Understanding their distribution and changes is of great significance to urban planning, land resource utilization and disaster assessment.

[0003] In the related technologies, building information extraction methods are divided into two categories: traditional image processing methods and deep learning methods. The essence of traditional image extraction methods is to classify buildings based on their underlying features such as edges, contours, colors, textures and shapes. However, the above methods are greatly affected by human subjective factors, and the complexity and automation of the building extraction process are low. The use of deep learning methods can demonstrate better learning and generalization capabilities, but traditional semantic segmentation usually only uses multi-layer convolution and pooling for feature extraction operations, which easily ignores the spatial details of buildings, resulting in blurred building edges, incomplete extraction of irregular buildings, omission or misdetection of small buildings, etc., making the building extraction accuracy low and the effect poor.

[0004] Based on the above analysis of the development status of this technical field, the existing technical solutions lack the use of complex building extraction models that capture local information, global dependency information and other spatial distributions. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device, equipment and storage medium for constructing a building extraction model, aiming to solve the above-mentioned problems in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a method for constructing a building extraction model is provided, comprising:

[0007] Obtain building images and preprocess them to obtain training sets;

[0008] The training set is input into a two-stream decoder, where the two-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network. The position extraction network outputs position features of different resolutions at each layer, and the overall extraction network outputs basic feature maps of different fineness at each layer.

[0009] The position features and basic feature maps of the corresponding layers are fused into N comprehensive feature maps, and the basic feature maps of the last layer of the overall extraction network are input into the global capture model to obtain the decoded feature maps of the first layer of the decoder;

[0010] The decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map are input into the attention gate structure to obtain the decoded feature map of the current layer, and the decoded feature map of the N+1th layer of the decoder is used as the final feature map;

[0011] The final feature map is input into the prediction layer to extract the building, and the model parameters are iteratively updated through the extraction error to obtain the final building extraction model.

[0012] According to a second aspect of an embodiment of the present invention, there is provided a device for constructing a building extraction model, comprising:

[0013] A preprocessing module is used to obtain building images and perform preprocessing to obtain a training set;

[0014] A preliminary extraction module is used to input the training set into a dual-stream decoder, wherein the dual-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network. The position extraction network outputs position features of different resolutions at each layer, and the overall extraction network outputs basic feature maps of different fineness at each layer.

[0015] The feature fusion module is used to fuse the position features and basic feature maps of the corresponding layers into N comprehensive feature maps, and input the basic feature map of the last layer of the overall extraction network into the global capture model to obtain the decoding feature map of the first layer of the decoder;

[0016] The attention gating module is used to input the decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map into the attention gating structure to obtain the decoded feature map of the current layer, and use the decoded feature map of the N+1th layer of the decoder as the final feature map;

[0017] The iterative training module is used to input the final feature map into the prediction layer to extract buildings, and iteratively update the model parameters through the extraction error to obtain the final building extraction model.

[0018] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for constructing a building extraction model as provided in the first aspect of the present disclosure are implemented.

[0019] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps of the method for constructing a building extraction model provided in the first aspect of the present disclosure are implemented.

[0020] The technical solution provided by the embodiment of the present invention includes the following beneficial effects: using a dual-stream decoder to extract position features and basic feature maps respectively, and performing corresponding layer feature splicing, each decoder uses a multi-level structure to effectively capture features of different resolutions or levels of precision; using a pyramid model to add position information to obtain the spatial distribution and relationship of different features, improve the model's ability to locate the target space, and serve as a hub between the encoder and decoder; the decoded feature map obtained in the previous layer and the corresponding comprehensive feature map are input into the attention gate structure to facilitate the integration of information and emphasize important parts, so as to improve the accuracy of the model in extracting buildings.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 is a flow chart of a method for constructing a building extraction model according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a single-layer Transformer according to an embodiment of the present invention;

[0025] Figure 3 Schematic diagram of a pyramid pooling model according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of an attention gating structure of an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of the overall framework of the building extraction model of an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of a device for constructing a building extraction model according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0031] Method Embodiment

[0032] According to an embodiment of the present invention, a method for constructing a building extraction model is provided. Figure 1 is a flow chart of a method for constructing a building extraction model according to an embodiment of the present invention. Figure 1 As shown, the method for constructing a building extraction model according to an embodiment of the present invention specifically includes:

[0033] In step S110, the building image is acquired and preprocessed to obtain a training set, which specifically includes:

[0034] In the embodiment of the present invention, the building image refers to a remote sensing image, and the query selects a public remote sensing image building data set. Since changes in lighting, weather and seasons may affect the appearance of the building, and there are a large number of shadows and trees around the building, which may cause incomplete edge extraction of the building, and the inter-class similarity and inter-class difference between the objects may cause the omission of buildings, etc., the data set includes high-resolution images in diversified scenes;

[0035] The high-resolution building images are subjected to relevant preprocessing such as cropping and grouping. Preferably, more training samples can be generated by adding noise, rotating, flipping, etc. to simulate different lighting conditions, vegetation coverage, etc., and the test set and validation set are generated accordingly;

[0036] In step S120, the training set is input into a dual-stream decoder, wherein the dual-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network, wherein the position extraction network outputs position features of different resolutions at each layer, and the overall extraction network outputs basic feature maps of different fineness at each layer, specifically including:

[0037] The position extraction network specifically includes: a multi-layer Transformer encoder, wherein each layer of the multi-layer Transformer encoder includes multiple processing layers, and the processing layer includes a self-attention layer and a feedforward network. The multi-layer Transformer encoder is a lightweight structure;

[0038] The position features of different resolutions output at each layer through the position extraction network include:

[0039] Through the Overlap Patch division operation, each image in the training set is divided into overlapping patches of fixed size, and the overlapping patches are input into the first layer of the Transformer encoder. The original Transformer image embedding method often divides the image into non-overlapping patches, which easily ignores the edge information of the building;

[0040] In each layer of the Transformer encoder, a self-attention layer with a scaling factor is added to process the relationship between overlapping patches in the upper layer. The self-attention layer with a scaling factor is the Efficient Self-Attn module, which can effectively reduce the computational complexity of each self-attention layer. The self-attention layer in the transformer can directly merge overlapping patches by designing relevant convolution parameters, ensuring the continuity of patch edges and avoiding edge discontinuity problems in semantic segmentation tasks.

[0041] The mutual relations are sorted out through the feed-forward network of Mix-FNN including zero-padded 3×3 depth convolution, and the position information is added to obtain the position features of the corresponding layer; Mix-FNN replaces the position embedding module in the traditional Transformer to obtain the position information;

[0042] The relevant features of the position information are transmitted to the next layer of Transformer to generate features at different resolutions to ensure local continuity; since each layer of Transformer needs to merge patches, complex merging methods will reduce processing efficiency;

[0043] Figure 2 is a schematic diagram of a single-layer Transformer according to an embodiment of the present invention, such as Figure 2 As shown, the structure of one layer in the multi-layer Transformer encoder is shown. Norm and Linear represent norms to prevent overfitting and linear output, respectively. Spatial Reduction represents the reduction of spatial dimensions in the self-attention layer. Multi-Head Attention represents multi-head attention. Multiple groups of processing layers can enhance the learning ability of the model. It should be noted that in the embodiment of the present invention, each layer of Transformer needs to merge adjacent patches into a larger patch.

[0044] In a hierarchical Transformer encoder, it is usually not necessary to merge patches at every layer. However, the probability of artifacts in remote sensing images or building images is higher than that in other types of images. Merging overlapping patches can significantly reduce artifacts and improve edge continuity.

[0045] The overall extraction network specifically includes: a multi-layer convolutional neural network. The convolution of the initial layer can learn relatively simple features, such as edges and textures, and the deep convolution can learn more abstract and complete features, effectively extracting deep semantic features.

[0046] The dual-stream decoder finally fully obtains the global dependency information of the image, so as to capture the global dependency information in the image, improve the global information interaction ability of the image, and more accurately understand the role of the content in the image.

[0047] In step S130, the position features and basic feature maps of the corresponding layers are fused into N comprehensive feature maps, and the basic feature maps of the last layer of the overall extraction network are input into the global capture model to obtain the decoded feature maps of the first layer of the decoder, specifically including:

[0048] The position features of the corresponding layer and the basic feature map are fused into N comprehensive feature maps through upsampling technology;

[0049] The global capture model specifically includes: a pyramid pooling model MSICM with position information, wherein the pyramid pooling model includes a coordinate attention layer and a pooling layer. In an embodiment of the present invention, the MSICM serves as a convolution module for communicating between an encoder and a decoder;

[0050] The basic feature map of the last layer of the overall extraction network is input into the global capture model to obtain the decoded feature map of the first layer of the decoder, which specifically includes:

[0051] Get the basic feature map of the last layer of the overall extraction network as the input of the pyramid pooling model;

[0052] The coordinate attention layer extracts the spatial and contextual features of the input and obtains the coordinate attention feature map.

[0053] The input is converted into a plurality of feature maps of different sizes by using an adaptive average pooling method through a pooling layer. In the embodiment of the present invention, the input is converted into four feature maps of different scales, namely 1×1, 2×2, 3×3, and 6×6, respectively, including image features of regions of different sizes;

[0054] For each scale feature map, 1×1 convolution is used to reduce the channel dimension of feature maps of different sizes, and bilinear difference processing is used to adjust the size of the feature map. Bilinear difference upsampling can adjust the image size, which is equivalent to deconvolution, restoring the size and the basic features of the last layer. Figure 1 To achieve this, the restored feature maps of different sizes are spliced ​​to obtain the pyramid pooling global feature map;

[0055] The basic feature map, coordinate attention feature map and pyramid pooling global feature map of the original input are spliced ​​to obtain a comprehensive spliced ​​feature map, which is input into a layer of convolutional neural network to obtain the decoding feature map of the first layer of the decoder.

[0056] Figure 3 Schematic diagram of a pyramid pooling model according to an embodiment of the present invention. Figure 3 As shown in the figure, the structure of the pyramid pooling model is shown, where C represents the number of channels and Conv represents convolution.

[0057] By adding a pyramid pooling module with position information at the end of the encoder, rich multi-scale information and deep semantic information are captured to address the problem of difficult segmentation of similar buildings in complex scenes.

[0058] In step S140, the decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map are input into the attention gate structure to obtain the decoded feature map of the current layer, and the decoded feature map of the N+1th layer of the decoder is used as the final feature map, which specifically includes:

[0059] The correspondence refers to the correspondence of scale. If the comprehensive feature maps are arranged from small to large in scale, the decoded feature map of the i-th layer of the decoder corresponds to the N-i+1-th comprehensive feature map, and the solution is obtained layer by layer;

[0060] It is necessary to adjust the sizes of the two feature maps to the same, convolve the decoding feature map and the corresponding comprehensive feature map obtained in the previous layer respectively, and then splice the convolved decoding feature map and the comprehensive feature map to obtain a signal enhancement map. The signal enhancement map is linearly activated, convolved, nonlinearly activated and resampled in sequence to obtain the attention weight coefficient α;

[0061] The attention weight coefficient α is multiplied point-to-point with the comprehensive feature map to obtain the decoded feature map of the current layer, thereby continuously increasing the weight of the local area of ​​interest and suppressing the weight of the non-interest area. As the training progresses, the auxiliary model focuses on the desired area, gradually strengthens the weight of the local ROI through the soft attention method, effectively suppresses the activation in irrelevant areas, reduces the redundant parts of the jump connection, and enables the model to learn the areas that need to focus attention, thereby improving the segmentation performance;

[0062] Figure 4 is a schematic diagram of the attention gating structure of an embodiment of the present invention, such as Figure 4 As shown, the structural framework for fusing the decoded feature map and the comprehensive feature map is shown;

[0063] In step S150, the final feature map is input into the prediction layer to extract the building, and the model parameters are updated repeatedly through the extraction error to obtain the final building extraction model CLUNet, which specifically includes:

[0064] A prediction layer is added to the final feature map. The prediction layer is a simple prediction head module. The low-resolution feature map is upsampled to restore the size of the original image and generates a prediction result corresponding to each pixel position to obtain the semantic segmentation result.

[0065] The method further comprises:

[0066] In step S160, the constructed building extraction model is used to extract buildings on the image, a quantitative evaluation is performed on the constructed building extraction model, and corresponding ablation tests are performed to verify the effectiveness of each module.

[0067] The above technical solutions of the embodiments of the present invention are illustrated with reference to the following drawings.

[0068] Figure 5 Schematic diagram of the overall framework of the building extraction model of the embodiment of the present invention. Figure 5 As shown in the figure, the network structure diagram including a dual-stream encoder, a fusion of decoding feature maps and comprehensive feature maps, a pyramid pooling model MSICM including position information, and a decoder is shown. AG represents the attention gating structure.

[0069] In summary, in response to the existing problems, the method for constructing a building extraction model of this invention uses a dual-stream decoder to extract position features and basic feature maps respectively, and performs corresponding layer feature splicing. Each decoder uses a multi-level structure to effectively capture features of different resolutions or fineness; a self-attention layer including a scaling factor is used in the multi-layer Transformer encoder to reduce the computational complexity, and Mix-FNN is used to replace the position embedding module in the traditional Transformer to obtain position information, and each layer needs to merge patches to greatly reduce building artifacts and improve edge continuity. The model complexity is reduced by merging patches at each layer; the pyramid model MSICM is used to add position information to obtain the spatial distribution and relationship of different features, improve the model's ability to locate the target space, and serve as a hub between the encoder and the decoder. In order to retain more multi-scale semantic information and spatial detail information, the clarity and integrity of the building edges in complex scenes are improved, and the generalization ability of the network is enhanced; the decoded feature map obtained in the previous layer and the corresponding comprehensive feature map are input into the attention gate structure to facilitate the integration of information and emphasize important parts to improve the accuracy of the model in extracting buildings.

[0070] Device Embodiment

[0071] According to an embodiment of the present invention, a device for constructing a building extraction model is provided. Figure 6 Schematic diagram of a device for constructing a building extraction model according to an embodiment of the present invention. Figure 6As shown, the device for constructing a building extraction model according to an embodiment of the present invention specifically includes:

[0072] A preprocessing module 60 is used to obtain building images and perform preprocessing to obtain a training set;

[0073] The preliminary extraction module 62 is used to input the training set into the dual-stream decoder, wherein the dual-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network, and outputs position features of different resolutions at each layer through the position extraction network, and outputs basic feature maps of different fineness at each layer through the overall extraction network, which is specifically used for:

[0074] The position extraction network specifically includes: a multi-layer Transformer encoder, wherein each layer of the multi-layer Transformer encoder includes multiple processing layers, and the processing layer includes a self-attention layer and a feedforward network;

[0075] The position features of different resolutions output at each layer through the position extraction network include:

[0076] Divide each image in the training set into overlapping patches of fixed size and feed the overlapping patches into the first layer of the Transformer encoder;

[0077] In each layer of the Transformer encoder, the mutual relationship between the overlapping patches in the upper layer is processed by adding a self-attention layer with a scaling factor, and the mutual relationship is sorted out through a feedforward network including Mix-FNN, and the position information is added to obtain the position features of the corresponding layer.

[0078] The overall extraction network specifically includes: a multi-layer convolutional neural network.

[0079] The feature fusion module 64 is used to fuse the position features and basic feature maps of the corresponding layers into N comprehensive feature maps, input the basic feature map of the last layer of the overall extraction network into the global capture model, and obtain the decoded feature map of the first layer of the decoder, which is specifically used for:

[0080] The global capture model specifically includes: a pyramid pooling model with position information, wherein the pyramid pooling model includes a coordinate attention layer and a pooling layer;

[0081] The basic feature map of the last layer of the overall extraction network is input into the global capture model to obtain the decoded feature map of the first layer of the decoder, which specifically includes:

[0082] Get the basic feature map of the last layer of the overall extraction network as the input of the pyramid pooling model;

[0083] The spatial features and context features of the input are extracted through the coordinate attention layer to obtain the coordinate attention feature map;

[0084] The input is converted into multiple feature maps of different sizes using the adaptive average pooling method through the pooling layer, the channel dimension of the feature maps of different sizes is reduced using 1×1 convolution, and the size of the feature map is adjusted using bilinear interpolation to restore the size and the basic features of the last layer. Figure 1 To achieve this, the restored feature maps of different sizes are spliced ​​to obtain the pyramid pooling global feature map;

[0085] The basic feature map of the original input, the coordinate attention feature map and the pyramid pooling global feature map are spliced ​​to obtain a comprehensive splicing feature map, and the comprehensive splicing feature map is input into a layer of convolutional neural network to obtain the decoding feature map of the first layer of the decoder;

[0086] By adding a pyramid pooling module with position information at the end of the encoder, rich multi-scale information and deep semantic information are captured to address the problem of difficult segmentation of similar buildings in complex scenes.

[0087] The attention gating module 66 is used to input the decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map into the attention gating structure to obtain the decoded feature map of the current layer, and use the decoded feature map of the N+1th layer of the decoder as the final feature map, specifically for:

[0088] The decoding feature map and the corresponding comprehensive feature map obtained in the previous layer are convolved respectively, and then the decoding feature map and the comprehensive feature map after convolution are spliced ​​to obtain the signal enhancement map. The signal enhancement map is linearly activated, convolved, nonlinearly activated and resampled in sequence to obtain the attention weight coefficient α;

[0089] The attention weight coefficient α is multiplied point-to-point with the comprehensive feature map to obtain the decoded feature map of the current layer, thereby continuously improving the weight of the local area of ​​interest and suppressing the weight of the non-interest area. As the training progresses, the auxiliary model focuses on the desired area, and gradually strengthens the weight of the local ROI through the soft attention method, effectively suppressing the activation in irrelevant areas, reducing the redundant parts of the jump connection, and enabling the model to learn the areas that need to focus attention, thereby improving the segmentation performance.

[0090] The iterative training module 68 is used to input the final feature map into the prediction layer to extract the building, and iteratively update the model parameters through the extraction error to obtain the final building extraction model.

[0091] The device further comprises:

[0092] The application module 610 is used to extract buildings on the image using the constructed building extraction model.

[0093] In summary, in response to the existing problems, the device for constructing a building extraction model of this invention uses a dual-stream decoder to extract position features and basic feature maps respectively, and performs corresponding layer feature splicing. Each decoder uses a multi-level structure to effectively capture features of different resolutions or fineness; a self-attention layer including a scaling factor is used in the multi-layer Transformer encoder to reduce the computational complexity, and Mix-FNN is used to replace the position embedding module in the traditional Transformer to obtain position information, and each layer needs to merge patches to greatly reduce building artifacts and improve edge continuity, and the model complexity is reduced by merging patches at each layer; the pyramid model MSICM is used to add position information to obtain the distribution and relationship of different features in space, improve the model's ability to locate the target space, and serve as a hub between the encoder and the decoder. In order to retain more multi-scale semantic information and spatial detail information, the clarity and integrity of the building edges in complex scenes are improved, and the generalization ability of the network is enhanced; the decoded feature map obtained in the previous layer and the corresponding comprehensive feature map are input into the attention gate structure to facilitate the integration of information and emphasize important parts to improve the accuracy of the model in extracting buildings.

[0094] Electronic device embodiment

[0095] Figure 7 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device 700 may include at least one processor 710 and a memory 720. The processor 710 may execute instructions stored in the memory 720. The processor 710 is connected to the memory 720 through a data bus. In addition to the memory 720, the processor 710 may also be connected to an input device 730, an output device 740, and a communication device 750 through a data bus.

[0096] The processor 710 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0097] The memory 720 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0098] In the embodiment of the present disclosure, executable instructions are stored in the memory 720, and the processor 710 can read the executable instructions from the memory 720 and execute the instructions to implement all or part of the steps of the method for constructing a building extraction model in any of the above exemplary embodiments.

[0099] Computer Readable Storage Medium Embodiments

[0100] In addition to the above-mentioned methods and devices, the exemplary embodiments of the present disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product, wherein the computer product includes computer program instructions that can be executed by a processor to implement all or part of the steps described in the method for constructing a building extraction model in any of the above-mentioned exemplary embodiments.

[0101] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages ​​and scripting languages ​​(e.g., Python). The program code may be executed entirely on the user computing device, partially on the user computing device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0102] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of readable storage media include: a static random access memory (SRAM) with one or more wires electrically connected, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk, or any suitable combination of the above.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a building extraction model, characterized in that: include: Obtain building images and preprocess them to obtain training sets; Inputting the training set into a dual-stream decoder, wherein the dual-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network, wherein the position extraction network outputs position features of different resolutions at each layer, and the overall extraction network outputs basic feature maps of different fineness at each layer; The position features and the basic feature maps of the corresponding layers are fused into N comprehensive feature maps, and the basic feature maps of the last layer of the overall extraction network are input into the global capture model to obtain the decoding feature maps of the first layer of the decoder; The decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map are input into the attention gate structure to obtain the decoded feature map of the current layer, and the decoded feature map of the N+1th layer of the decoder is used as the final feature map; The final feature map is input into the prediction layer to extract the building, and the model parameters are iteratively updated through the extraction error to obtain the final building extraction model.

2. The method according to claim 1, characterized in that The method further comprises: The buildings on the image are extracted using the constructed building extraction model.

3. The method according to claim 1, characterized in that The position extraction network specifically includes: a multi-layer Transformer encoder, wherein each layer of the multi-layer Transformer encoder includes multiple processing layers, and the processing layer includes a self-attention layer and a feedforward network; Outputting position features of different resolutions at each layer through the position extraction network specifically includes: Divide each image in the training set into overlapping patches of fixed size, and input the overlapping patches into the first layer of the Transformer encoder; In each layer of the Transformer encoder, the relationship between overlapping patches in the upper layer is processed by a self-attention layer with a scaling factor, and the relationship is sorted out through a feedforward network including Mix-FNN, and the position information is added to obtain the position features of the corresponding layer.

4. The method according to claim 1, characterized in that The overall extraction network specifically includes: a multi-layer convolutional neural network.

5. The method according to claim 1, characterized in that The global capture model specifically includes: a pyramid pooling model with position information, wherein the pyramid pooling model includes a coordinate attention layer and a pooling layer; The step of inputting the basic feature map of the last layer of the overall extraction network into the global capture model to obtain the decoded feature map of the first layer of the decoder specifically includes: Get the basic feature map of the last layer of the overall extraction network as the input of the pyramid pooling model; Extracting spatial features and contextual features of the input through the coordinate attention layer to obtain a coordinate attention feature map; The input is converted into a plurality of feature maps of different sizes by using an adaptive average pooling method through the pooling layer, the channel dimensions of the feature maps of different sizes are reduced by using a 1×1 convolution, and the size of the feature map is adjusted by using a bilinear difference process, the restored size is consistent with the basic feature map of the last layer, and the restored feature maps of different sizes are spliced ​​to obtain a pyramid pooling global feature map; The basic feature map of the original input, the coordinate attention feature map and the pyramid pooling global feature map are spliced ​​to obtain a comprehensive splicing feature map, and the comprehensive splicing feature map is input into a layer of convolutional neural network to obtain a decoding feature map of the first layer of the decoder.

6. The method according to claim 1, characterized in that The step of inputting the decoded feature map of the previous layer of the decoder and the corresponding comprehensive feature map into the attention gating structure to obtain the decoded feature map of the current layer specifically includes: The decoding feature map and the corresponding comprehensive feature map obtained in the previous layer are convolved respectively, and then the decoding feature map and the comprehensive feature map after convolution are spliced ​​to obtain a signal enhancement map, and the signal enhancement map is linearly activated, convolved, nonlinearly activated and resampled in sequence to obtain an attention weight coefficient; The attention weight coefficient is multiplied point-by-point by the comprehensive feature map to obtain a decoding feature map of the current layer.

7. A device for constructing a building extraction model, characterized in that: include: A preprocessing module is used to obtain building images and perform preprocessing to obtain a training set; A preliminary extraction module, used for inputting the training set into a dual-stream decoder, wherein the dual-stream decoder includes an N-layer position extraction network and an N+1-layer overall extraction network, wherein the position extraction network outputs position features of different resolutions at each layer, and the overall extraction network outputs basic feature maps of different fineness at each layer; A feature fusion module is used to fuse the position features and the basic feature maps of the corresponding layer into N comprehensive feature maps, and input the basic feature map of the last layer of the overall extraction network into the global capture model to obtain a decoded feature map of the first layer of the decoder; The attention gating module is used to input the decoding feature map of the previous layer of the decoder and the corresponding comprehensive feature map into the attention gating structure to obtain the decoding feature map of the current layer, and use the decoding feature map of the N+1th layer of the decoder as the final feature map; The iterative training module is used to input the final feature map into the prediction layer to extract buildings, and iteratively update the model parameters through the extraction error to obtain the final building extraction model.

8. The device according to claim 7, characterized in that The device further comprises: The application module is used to extract buildings on the image using the constructed building extraction model.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for constructing a building extraction model according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the method for constructing a building extraction model according to any one of claims 1 to 6 are implemented.

Images