Building object extraction method, system, device and storage medium of remote sensing image

Abstract

The application discloses a kind of building object extraction method, system, equipment and storage medium of remote sensing image, the method includes: building detection is carried out to remote sensing image into target detection network, obtains multiple initial contours;The contour feature vector of each initial contour is input into contour optimization network to carry out vertex prediction, obtains multiple vertex offset of each optimized contour;The feature map of remote sensing image is processed by contour attention module to obtain each vertex weight coefficient;According to each original vertex coordinate, each vertex offset and each vertex weight coefficient, multiple vertex target coordinates of each optimized contour are obtained by coordinate iteration formula;Building object is extracted from the remote sensing image based on multiple vertex target coordinates.The application is divided into multiple initial contours according to detection frame, introduces edge attention module in contour optimization stage, adjusts the weight of each vertex of each initial contour in regression process, realizes the accurate extraction of building object from remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image information extraction technology, and particularly to a method, system, device, and storage medium for extracting architectural objects from remote sensing images. Background Technology

[0002] Buildings, as the most important component of a city, shape its structure and form, and convey important cognitive information and structural knowledge. Accurately extracting information from buildings is of great significance for optimizing urban structure and formulating urban planning.

[0003] Most current research treats building extraction as a semantic segmentation task. While semantic segmentation can achieve ideal accuracy in specific scenarios, its results cannot be directly applied to subsequent tasks. Some studies post-process semantic segmentation results to extract instance objects, but this approach relies heavily on the accuracy of the semantic segmentation results and may not be able to completely segment every object in complex scenes. Methods that directly predict building vertices are faster than semantic segmentation, but the quality of the extracted buildings is poor. Although subsequent research has improved these methods, the following problems remain: 1. Manually designed initial contours lack sufficient generalization, becoming inaccurate for different scenarios and complex building situations; 2. Inaccurate initial contours lead to errors or self-intersections during contour vertex movement; 3. Limited feature extraction of building contours results in inaccurate contour vertex movement, leading to limited accuracy in the final extracted contour.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, device, and storage medium for extracting building objects from remote sensing images, aiming to solve the technical problem of how to accurately extract building objects from remote sensing images.

[0006] To achieve the above objectives, the present invention provides a method for extracting building objects from remote sensing images, the method comprising:

[0007] The remote sensing image is input into the Backbone module in the target detection network for feature extraction to obtain a remote sensing image feature map. The remote sensing image feature map is then used to detect buildings and obtain multiple initial outlines of building objects.

[0008] Determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment;

[0009] Multiple contour feature vectors are input into a contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by combining a Backbone module, a Fusion module, and a Prediction module.

[0010] The remote sensing image feature map is processed by the contour attention module to obtain the contour boundary attention map. The contour boundary attention map is then bilinearly interpolated based on the coordinates of multiple original vertices corresponding to each initial contour segment to obtain the weight coefficients of multiple vertices corresponding to each initial contour segment.

[0011] Based on the coordinates of multiple original vertices corresponding to each initial contour segment, the offsets of multiple vertices corresponding to each optimized contour segment, and the weight coefficients of multiple vertices corresponding to each initial contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula.

[0012] The building object is extracted from the remote sensing image based on the target coordinates of multiple vertices corresponding to each segment of the optimized contour.

[0013] Optionally, the step of performing building detection on the remote sensing image feature map to obtain multiple initial outlines of the building object includes:

[0014] The remote sensing image feature map is downsampled to obtain a predicted heatmap, a predicted bias map, and a predicted attribute map;

[0015] The geometric center coordinates of the building object are determined using the heatmap head module based on the predicted heatmap.

[0016] The center coordinates of the detection frame corresponding to the building object are determined based on the geometric center coordinates.

[0017] The center coordinate offset of the detection box is determined by the offset head module based on the predicted offset map.

[0018] The detection box attribute information is determined by the wh head module based on the predicted attribute map.

[0019] The detection box of the building object is generated based on the center coordinates of the detection box, the offset of the center coordinates, and the attribute information of the detection box.

[0020] Based on the detection box of the building object, multiple initial contours of the building object are obtained through vertex sampling.

[0021] Optionally, the step of inputting multiple contour feature vectors into a contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment includes:

[0022] Multiple contour feature vectors are input into the Backbone module in the contour optimization network to obtain different levels of contour feature information corresponding to each initial contour segment.

[0023] The Fusion module within the contour optimization network processes the contour feature information of different levels corresponding to each initial contour segment to obtain the contour feature information of each initial contour segment after pooling.

[0024] Based on the contour feature information after pooling of each initial contour segment, the Predication module in the contour optimization network obtains multiple vertex offsets corresponding to each optimized contour segment.

[0025] Optionally, the step of processing the remote sensing image feature map through the contour attention module to obtain the contour boundary attention map includes:

[0026] The remote sensing image feature map is superimposed and convolved by the contour attention module to obtain the contour boundary convolution map.

[0027] The contour boundary convolution map is processed by a Sigmoid activation layer to obtain a contour boundary prediction map;

[0028] The contour boundary prediction map is subjected to multi-layer convolution processing to obtain a contour prediction convolution map;

[0029] A contour boundary attention map is generated by passing the contour prediction convolutional map through the Sigmoid activation layer.

[0030] Optionally, the step of obtaining the target coordinates of multiple vertices corresponding to each optimized contour segment through a coordinate iteration formula based on the coordinates of multiple original vertices corresponding to each initial contour segment, the offsets of multiple vertices corresponding to each optimized contour segment, and the weight coefficients of multiple vertices corresponding to each initial contour segment includes:

[0031] Based on the multiple vertex offsets corresponding to each optimized contour segment and the multiple vertex weight coefficients corresponding to each initial contour segment, the target vertex offsets corresponding to each optimized contour segment are obtained through the vertex offset formula.

[0032] The vertex offset formula is:

[0033] (Δx′ k ,Δy′ k ) = Atten(x k ,y k )*(Δx k ,Δy k )

[0034] In the formula, k is the number of iterations, Atten(x) k x k) represents the weight coefficient of the k-th vertex, (Δx) k Δy k ) represents the offset of the vertex at the k-th time, (Δx′) k ,Δy′ k ) represents the vertex target offset for the kth time;

[0035] Based on the coordinates of multiple original vertices corresponding to each initial contour segment and the target offsets of multiple vertices corresponding to each optimized contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula.

[0036] The coordinate iteration formula is:

[0037]

[0038] In the formula, Let (Δx′) be the vertex target offset of point i in the (k-1)th iteration. k-1 ,Δy′ k-1 ) represents the vertex offset at the (k-1)th time. The original vertex coordinates of point i in the (k-1)th iteration are... Let i be the vertex target coordinates of point i in the kth iteration.

[0039] Furthermore, to achieve the above objectives, the present invention also proposes a building object extraction system for remote sensing images, the building object extraction system for remote sensing images comprising:

[0040] The initial module is used to input remote sensing images into the Backbone module in the target detection network for feature extraction, obtain remote sensing image feature maps, and perform building detection on the remote sensing image feature maps to obtain multiple initial outlines of building objects.

[0041] The determination module is used to determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment;

[0042] An optimization module is used to input multiple contour feature vectors into a contour optimization network for vertex prediction, and obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by a Backbone module, a Fusion module, and a Prediction module.

[0043] The optimization module is further configured to process the remote sensing image feature map through the contour attention module to obtain a contour boundary attention map, and to perform bilinear interpolation on the contour boundary attention map according to the coordinates of multiple original vertices corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment.

[0044] The optimization module is also used to obtain the target coordinates of multiple vertices corresponding to each segment of the optimized contour through a coordinate iteration formula based on the coordinates of multiple original vertices corresponding to each segment of the initial contour, the offsets of multiple vertices corresponding to each segment of the optimized contour, and the weight coefficients of multiple vertices corresponding to each segment of the initial contour.

[0045] The extraction module is used to extract the building object from the remote sensing image based on the coordinates of multiple vertex targets corresponding to each segment of the optimized contour.

[0046] Furthermore, to achieve the above objectives, the present invention also proposes a building object extraction device for remote sensing images, the device comprising: a memory, a processor, and a building object extraction program for remote sensing images stored in the memory and executable on the processor, the building object extraction program for remote sensing images being configured to implement the steps of the building object extraction method for remote sensing images as described above.

[0047] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a building object extraction program for remote sensing images, wherein when the building object extraction program for remote sensing images is executed by a processor, the steps of the building object extraction method for remote sensing images as described above are implemented.

[0048] This invention first inputs remote sensing imagery into the Backbone module of a target detection network for feature extraction, obtaining a remote sensing image feature map. Building detection is then performed on the remote sensing image feature map to obtain multiple initial contours of the building objects. The contour feature vectors and multiple original vertex coordinates corresponding to each initial contour segment are determined. Then, the multiple contour feature vectors are input into a contour optimization network for vertex prediction, obtaining multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed from the Backbone module, Fusion module, and Prediction module. Next, the remote sensing image feature map is processed by a contour attention module to obtain a contour boundary attention map. Bilinear interpolation is then performed on the contour boundary attention map based on the multiple original vertex coordinates corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment. Finally, based on the multiple original vertex coordinates, multiple vertex offsets, and multiple vertex weight coefficients corresponding to each initial contour segment, the target vertex coordinates corresponding to each optimized contour segment are obtained through a coordinate iteration formula. Building objects are then extracted from the remote sensing image based on these target vertex coordinates. This invention divides the initial contour into multiple segments based on the target detection bounding box. Each initial contour segment is input separately into the network for regression, thereby avoiding matching errors between predicted vertices and real vertices. In the contour optimization stage, a new Backbone network is proposed. Compared with previous networks, this Backbone can extract deeper features of the contour, and the predicted contour boundary is closer to the real boundary. Subsequently, an edge attention module is introduced in the contour optimization stage to adjust the weight of each vertex in the regression process, forcing vertices far from the real contour to move faster. Finally, the coordinates of multiple vertex targets corresponding to each optimized contour segment are obtained through coordinate iteration formula, thereby realizing the accurate extraction of building objects from remote sensing images. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the structure of the building object extraction device for remote sensing images of the hardware operating environment involved in the embodiments of the present invention;

[0050] Figure 2 This is a flowchart illustrating the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0051] Figure 3 This is a diagram of the overall structure of the CenterNet network in the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0052] Figure 4 This is a schematic diagram of the detection frame of the first embodiment of the method for extracting building objects from remote sensing images according to the present invention;

[0053] Figure 5This is a schematic diagram of contour segmentation of the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0054] Figure 6 This is a diagram showing the overall structure of the contour optimization network in the first embodiment of the remote sensing image building object extraction method of the present invention.

[0055] Figure 7 This is a structural diagram of the Backbone module within the contour optimization network of the first embodiment of the building object extraction method for remote sensing images of the present invention;

[0056] Figure 8 This is a flowchart of the contour attention module of the first embodiment of the remote sensing image building object extraction method of the present invention;

[0057] Figure 9 This is a structural block diagram of the first embodiment of the remote sensing image building object extraction system of the present invention.

[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0060] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a building object extraction device based on remote sensing images of the hardware operating environment involved in the embodiments of the present invention.

[0061] like Figure 1 As shown, the building object extraction device for remote sensing imagery may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage system independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the equipment for extracting architectural objects from remotely sensed images. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0063] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a program for extracting building objects from remote sensing images.

[0064] exist Figure 1 In the remote sensing image building object extraction device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the remote sensing image building object extraction device of the present invention can be set in the remote sensing image building object extraction device, and the remote sensing image building object extraction device calls the remote sensing image building object extraction program stored in the memory 1005 through the processor 1001 and executes the remote sensing image building object extraction method provided in the embodiment of the present invention.

[0065] This invention provides a method for extracting architectural objects from remote sensing images, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0066] In this embodiment, the method for extracting building objects from remote sensing images includes the following steps:

[0067] Step S10: Input the remote sensing image into the Backbone module in the target detection network for feature extraction to obtain a remote sensing image feature map, and perform building detection on the remote sensing image feature map to obtain multiple initial outlines of the building objects.

[0068] It is easy to understand that the execution subject of this embodiment can be a building object extraction system for remote sensing images with functions such as data processing, network communication and program operation, or other computer equipment with similar functions. This embodiment does not limit it.

[0069] It should be noted that remote sensing images contain a large number of objects. In order to obtain building objects of different locations and sizes, it is first necessary to locate the buildings in the remote sensing images and construct the initial outline of the building objects.

[0070] It should also be understood that reference Figure 3 , Figure 3This is a diagram of the overall structure of the CenterNet network in the first embodiment of the building object extraction method from remote sensing images of the present invention. Figure 3 The target detection network uses CenterNet as the network for building identification and localization, detecting potential building objects in remote sensing images.

[0071] Furthermore, the process of detecting buildings using remote sensing image feature maps and obtaining multiple initial outlines of building objects involves downsampling the remote sensing image feature maps to obtain a predicted heatmap, a predicted offset map, and a predicted attribute map. Based on the predicted heatmap, the geometric center coordinates of the building object are determined using the heatmap head module. Based on the geometric center coordinates, the center coordinates of the corresponding detection box are determined. Based on the predicted offset map, the center coordinate offset of the detection box is determined using the offset head module. Based on the predicted attribute map, the attribute information of the detection box is determined using the wh head module. The detection boxes of the building objects are generated based on the center coordinates, center coordinate offsets, and detection box attribute information. Based on the detection boxes of the building objects, multiple initial outlines of the building objects are obtained through vertex sampling.

[0072] In the specific implementation, the remote sensing image is processed by the Backbone module composed of CNNs in the CenterNet network for feature extraction. The extracted feature map (i.e., remote sensing image feature map) is sampled to obtain three maps for prediction, namely the offset map (i.e., prediction bias map), the wh map (i.e., prediction attribute map), and the he atmap map (i.e., prediction heatmap).

[0073] It should also be noted that a peak retrieval method is used, which determines the geometric center coordinates of several building objects from the heatmap using the heatmap head module. This method uses 3×3 max pooling to traverse each point of the heatmap p, while comparing the value (confidence score) of the point with all eight adjacent points. After obtaining a certain number of undetermined points, points with confidence scores greater than a threshold are selected as the final center points. The positions of these center points in the heatmap correspond to the center coordinates of the detection boxes of the building objects. Then, the offset head module determines the center coordinate offset of the detection boxes, and the wh head module determines the attribute information of the detection boxes, including the length and width information of the detection boxes.

[0074] In this embodiment, CenterNet obtains the center point and width and height information of the detected object through three predicted heads (i.e., the heatmap head module, the offset head module, and the wh head module), obtains the location of potential buildings in the remote sensing image, and generates a detection box for the building object at its location, referencing... Figure 4 , Figure 4This is a schematic diagram of the detection frame of the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0075] It should also be understood that a bounding box is assigned to each building. The bounding box is typically represented by the coordinates of its top-left and bottom-right corners, and its shape is usually a matrix. This rectangle is directly and uniformly sampled, reducing it from a rectangle represented by only 2 or 4 vertices to N vertices (N is usually set to 128). This new contour serves as the initial building contour. Then, using the four vertices of the target bounding box (i.e., the initial building contour) and the four intersection points between the building object and the bounding box, the initial building contour is divided into multiple initial contour segments (e.g., eight initial contour segments). These segments are then resampled, and contour optimization is performed locally within each segment. (Refer to...) Figure 5 , Figure 5 This is a schematic diagram of contour segmentation of the first embodiment of the method for extracting building objects from remote sensing images according to the present invention.

[0076] Step S20: Determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment.

[0077] It should also be noted that although the initial outline of the building can represent the building object to a certain extent, it is still not accurate enough compared to the actual outline of the building. Therefore, it is necessary to optimize the initial outline of the building by segmentation so that it can be closer to the actual outline.

[0078] It should be understood that determining that each initial contour segment has multiple corresponding original vertices requires pre-determining the contour feature vector and the coordinates of the multiple original vertices corresponding to each initial contour segment.

[0079] Step S30: Input multiple contour feature vectors into the contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by combining the Backbone module, Fusion module and Prediction module.

[0080] Furthermore, the processing method for inputting multiple contour feature vectors into the contour optimization network for vertex prediction and obtaining multiple vertex offsets corresponding to each optimized contour segment is as follows: multiple contour feature vectors are input into the Backbone module within the contour optimization network to obtain contour feature information at different levels corresponding to each initial contour segment; the Fusion module within the contour optimization network processes the contour feature information at different levels corresponding to each initial contour segment to obtain the contour feature information after pooling of each initial contour segment; and the Prediction module within the contour optimization network obtains multiple vertex offsets corresponding to each optimized contour segment based on the contour feature information after pooling of each initial contour segment.

[0081] In the specific implementation, refer to Figure 6 , Figure 6 This is a diagram of the overall structure of the contour optimization network in the first embodiment of the remote sensing image building object extraction method of the present invention. In the contour optimization network, the input of the network is the contour feature vector corresponding to the initial contour of each segment of the building object, and the output is the offset of multiple vertices corresponding to each optimized contour segment. The entire contour optimization network is divided into Backbone, Fusion and Prediction parts.

[0082] The multiple vertex offsets corresponding to each optimized contour segment can be understood as the multiple vertex offsets after the initial contour segment is optimized.

[0083] In the specific implementation, for the Backbone module:

[0084] A backbone simulating a Unet++ network was designed, with the network structure as follows: Figure 7 As shown, Figure 7 This is a structural diagram of the Backbone module within the contour optimization network of the first embodiment of the building object extraction method for remote sensing images of the present invention. Compared to the network structure of Unet++, the Backbone abandons the network depth and increases the network length. The Backbone involves three operations: downsampling, upsampling, and a basic convolutional block. The basic convolutional block consists of two layers of 1D convolution, ReLU, and BatchNorm. For each X in the first layer... 0,i , is X 0,i-1 After the basic convolutional block and X 1,i-1 It is obtained by adding the upsampling results; for each X in the second layer 1,i Except for X 1,0 It is X 0,0 The downsampling result is the result of the previous layer after processing the basic convolutional block and the results of the calculation (i.e., the contour feature information of different levels corresponding to each initial contour segment); each X in the first layer 0 ,i They all have the same dimensions, and each X 0,i All are stored in the State.

[0085] Fusion module:

[0086] The fusion part is to fuse features from different levels. The Stat output from the Bcakbone module is fused and then passed through a 1×1 convolutional layer, followed by max pooling, to obtain the contour feature information of each initial contour after pooling.

[0087] Prediction module:

[0088] The Prediction part consists of N layers of 1×1 convolutions and ReLU functions. Based on the contour feature information after pooling of the initial contour of each segment, the Prediction module in the contour optimization network obtains the offsets of multiple vertices corresponding to each optimized contour segment.

[0089] Step S40: Process the remote sensing image feature map through the contour attention module to obtain the contour boundary attention map, and perform bilinear interpolation on the contour boundary attention map according to the coordinates of multiple original vertices corresponding to each initial contour segment to obtain the multiple vertex weight coefficients corresponding to each initial contour segment.

[0090] Furthermore, the contour attention module processes the remote sensing image feature map to obtain the contour boundary attention map. This process involves performing a superimposed convolution on the remote sensing image feature map using the contour attention module to obtain a contour boundary convolution map; processing the contour boundary convolution map using a Sigmoid activation layer to obtain a contour boundary prediction map; performing multi-layer convolution on the contour boundary prediction map to obtain a contour prediction convolution map; and generating the contour boundary attention map based on the contour prediction convolution map using a Sigmoid activation layer.

[0091] In the specific implementation, refer to Figure 8 , Figure 8 The flowchart of the contour attention module in the first embodiment of the remote sensing image building object extraction method of the present invention is shown. An additional contour attention module is introduced. This module automatically learns the weight of each vertex in the contour and gives greater weight to vertices that are far away from the real contour, so that they move faster and increase the efficiency of network learning.

[0092] In this embodiment, the feature map obtained after contour initialization of the image, namely the remote sensing image feature map, is further stacked with N layers of convolution and processed using a Sigmoid activation layer to obtain a prediction map of the contour boundary, namely the contour boundary prediction map. This contour boundary prediction map is directly compared with the actual contour boundary of the image for loss calculation.

[0093] The contour boundary prediction map also goes through N layers of convolution, and finally uses a sigmoid activation layer to generate an attention map. Based on the position of the contour (i.e., the coordinates of multiple original vertices corresponding to each initial contour segment), bilinear interpolation is performed on the attention map to obtain the weight coefficient Atten of each vertex (i.e., the weight coefficients of multiple vertices corresponding to each initial contour segment).

[0094] Step S50: Based on the coordinates of multiple original vertices corresponding to each initial contour segment, the offsets of multiple vertices corresponding to each optimized contour segment, and the weight coefficients of multiple vertices corresponding to each initial contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula.

[0095] In this embodiment, the target offset of each vertex corresponding to each optimized contour is obtained by using the vertex offset formula based on the multiple vertex offsets corresponding to each optimized contour and the multiple vertex weight coefficients corresponding to each initial contour.

[0096] The vertex offset formula is:

[0097] (Δx ′ k ,Δy ′ k ) = Atten(x k ,y k )*(Δx k ,Δy k )

[0098] In the formula, k is the number of iterations, Atten(x) k x k ) represents the weight coefficient of the k-th vertex, (Δx) k Δy k ) represents the offset of the vertex at the k-th time, (Δx′) k ,Δy′ k ) represents the vertex target offset for the kth time;

[0099] Based on the coordinates of multiple original vertices corresponding to each initial contour segment and the target offsets of multiple vertices corresponding to each optimized contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula;

[0100] The coordinate iteration formula is:

[0101]

[0102] In the formula, Let (Δx′) be the vertex target offset of point i in the (k-1)th iteration. k-1 ,Δy′ k-1 ) represents the vertex offset at the (k-1)th time. The original vertex coordinates of point i in the (k-1)th iteration are... Let i be the vertex target coordinates of point i in the kth iteration.

[0103] It should also be noted that updating vertex coordinates iteratively involves updating the original vertex coordinates. offset from the vertex The sum gives the new coordinates of the vertex. Furthermore, this new coordinate will continue to be fed into the contour optimization network as before, outputting a new offset, where the number of iterations K can be set to 3.

[0104] Step S60: Extract the building object from the remote sensing image based on the multiple vertex target coordinates corresponding to each segment of the optimized contour.

[0105] It should be understood that by obtaining the target coordinates of each vertex after optimizing each segment of the outline in the above manner, the optimized outline of the building object can be determined based on the target coordinates of each vertex after optimizing each segment of the outline, and then the building object can be extracted from the remote sensing image based on the optimized outline of the building object.

[0106] In this embodiment, the remote sensing image is first input into the Backbone module of the target detection network for feature extraction to obtain a remote sensing image feature map. Building detection is then performed on the remote sensing image feature map to obtain multiple initial contours of the building objects. The contour feature vectors and multiple original vertex coordinates corresponding to each initial contour segment are determined. Then, the multiple contour feature vectors are input into the contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by the Backbone module, Fusion module, and Prediction module. Afterward, the remote sensing image feature map is processed by the contour attention module to obtain a contour boundary attention map. Bilinear interpolation is then performed on the contour boundary attention map based on the multiple original vertex coordinates corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment. Finally, the multiple vertex target coordinates corresponding to each optimized contour segment are obtained through a coordinate iteration formula based on the multiple original vertex coordinates corresponding to each initial contour segment, the multiple vertex offsets corresponding to each optimized contour segment, and the multiple vertex weight coefficients corresponding to each initial contour segment. Based on the multiple vertex target coordinates corresponding to each optimized contour segment, the building objects are extracted from the remote sensing image. In this embodiment, the initial contour is divided into multiple segments based on the target detection box. Each initial contour segment is input into the network for regression separately, thereby avoiding matching errors between predicted vertices and real vertices. In the contour optimization stage, a new Backbone network is proposed. Compared with the previous network, this Backbone can extract deeper features of the contour, and the predicted contour boundary is closer to the real boundary. Then, an edge attention module is introduced in the contour optimization stage to adjust the weight of each vertex in the regression process, forcing vertices far from the real contour to move faster. Finally, the coordinates of multiple vertex targets corresponding to each optimized contour segment are obtained through coordinate iteration formula, thereby realizing the accurate extraction of building objects from remote sensing images.

[0107] Reference Figure 9 , Figure 9 This is a structural block diagram of the first embodiment of the remote sensing image building object extraction system of the present invention.

[0108] like Figure 9 As shown, the building object extraction system for remote sensing images proposed in this embodiment of the invention includes:

[0109] The initial module 9001 is used to input remote sensing images into the Backbone module in the target detection network for feature extraction, obtain a remote sensing image feature map, and perform building detection on the remote sensing image feature map to obtain multiple initial outlines of building objects.

[0110] The module 9002 is used to determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment.

[0111] The optimization module 9003 is used to input multiple contour feature vectors into the contour optimization network for vertex prediction and obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by combining the Backbone module, the Fusion module and the Prediction module.

[0112] The optimization module 9003 is further configured to process the remote sensing image feature map through the contour attention module to obtain the contour boundary attention map, and to perform bilinear interpolation on the contour boundary attention map according to the coordinates of multiple original vertices corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment.

[0113] The optimization module 9003 is also used to obtain the target coordinates of multiple vertices corresponding to each segment of the optimized contour through a coordinate iteration formula based on the coordinates of multiple original vertices corresponding to each segment of the initial contour, the offsets of multiple vertices corresponding to each segment of the optimized contour, and the weight coefficients of multiple vertices corresponding to each segment of the initial contour.

[0114] Extraction module 9004 is used to extract the building object from the remote sensing image based on the coordinates of multiple vertex targets corresponding to each segment of the optimized contour.

[0115] In this embodiment, the remote sensing image is first input into the Backbone module of the target detection network for feature extraction to obtain a remote sensing image feature map. Building detection is then performed on the remote sensing image feature map to obtain multiple initial contours of the building objects. The contour feature vectors and multiple original vertex coordinates corresponding to each initial contour segment are determined. Then, the multiple contour feature vectors are input into the contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by the Backbone module, Fusion module, and Prediction module. Afterward, the remote sensing image feature map is processed by the contour attention module to obtain a contour boundary attention map. Bilinear interpolation is then performed on the contour boundary attention map based on the multiple original vertex coordinates corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment. Finally, the multiple vertex target coordinates corresponding to each optimized contour segment are obtained through a coordinate iteration formula based on the multiple original vertex coordinates corresponding to each initial contour segment, the multiple vertex offsets corresponding to each optimized contour segment, and the multiple vertex weight coefficients corresponding to each initial contour segment. Based on the multiple vertex target coordinates corresponding to each optimized contour segment, the building objects are extracted from the remote sensing image. In this embodiment, the initial contour is divided into multiple segments based on the target detection box. Each initial contour segment is input into the network for regression separately, thereby avoiding matching errors between predicted vertices and real vertices. In the contour optimization stage, a new Backbone network is proposed. Compared with the previous network, this Backbone can extract deeper features of the contour, and the predicted contour boundary is closer to the real boundary. Then, an edge attention module is introduced in the contour optimization stage to adjust the weight of each vertex in the regression process, forcing vertices far from the real contour to move faster. Finally, the coordinates of multiple vertex targets corresponding to each optimized contour segment are obtained through coordinate iteration formula, thereby realizing the accurate extraction of building objects from remote sensing images.

[0116] Other embodiments or specific implementations of the building object extraction system for remote sensing images of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0117] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0118] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0120] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for extracting architectural objects from remote sensing images, characterized in that, The method for extracting building objects from remote sensing images includes the following steps: The remote sensing image is input into the Backbone module in the target detection network for feature extraction to obtain a remote sensing image feature map. The remote sensing image feature map is then used to detect buildings to obtain multiple initial outlines of building objects. Determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment; Multiple contour feature vectors are input into a contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by combining a Backbone module, a Fusion module, and a Prediction module. The remote sensing image feature map is processed by the contour attention module to obtain the contour boundary attention map. The contour boundary attention map is then bilinearly interpolated based on the coordinates of multiple original vertices corresponding to each initial contour segment to obtain the weight coefficients of multiple vertices corresponding to each initial contour segment. Based on the coordinates of multiple original vertices corresponding to each initial contour segment, the offsets of multiple vertices corresponding to each optimized contour segment, and the weight coefficients of multiple vertices corresponding to each initial contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula. The building object is extracted from the remote sensing image based on the target coordinates of multiple vertices corresponding to each segment of the optimized contour.

2. The method as described in claim 1, characterized in that, The step of performing building detection on the remote sensing image feature map to obtain multiple initial outlines of building objects includes: The remote sensing image feature map is downsampled to obtain a predicted heatmap, a predicted bias map, and a predicted attribute map; The geometric center coordinates of the building object are determined using the heatmap head module based on the predicted heatmap. The center coordinates of the detection frame corresponding to the building object are determined based on the geometric center coordinates. The center coordinate offset of the detection box is determined by the offset head module based on the predicted offset map. The detection box attribute information is determined by the wh head module based on the predicted attribute map. The detection box of the building object is generated based on the center coordinates of the detection box, the offset of the center coordinates, and the attribute information of the detection box. Based on the detection box of the building object, multiple initial contours of the building object are obtained through vertex sampling.

3. The method as described in claim 2, characterized in that, The step of inputting multiple contour feature vectors into a contour optimization network for vertex prediction to obtain multiple vertex offsets corresponding to each optimized contour segment includes: Multiple contour feature vectors are input into the Backbone module in the contour optimization network to obtain different levels of contour feature information corresponding to each initial contour segment. The Fusion module within the contour optimization network processes the contour feature information of different levels corresponding to each initial contour segment to obtain the contour feature information of each initial contour segment after pooling. Based on the contour feature information after pooling of each initial contour segment, the Prediction module in the contour optimization network obtains multiple vertex offsets corresponding to each optimized contour segment.

4. The method according to any one of claims 1-3, characterized in that, The step of processing the remote sensing image feature map through the contour attention module to obtain the contour boundary attention map includes: The remote sensing image feature map is superimposed and convolved by the contour attention module to obtain the contour boundary convolution map. The contour boundary convolution map is processed by a Sigmoid activation layer to obtain a contour boundary prediction map; The contour boundary prediction map is subjected to multi-layer convolution processing to obtain a contour prediction convolution map; A contour boundary attention map is generated by passing the contour prediction convolutional map through the Sigmoid activation layer.

5. The method as described in claim 4, characterized in that, The step of obtaining the target coordinates of multiple vertices corresponding to each optimized contour segment through a coordinate iteration formula based on the coordinates of multiple original vertices corresponding to each initial contour segment, the offsets of multiple vertices corresponding to each optimized contour segment, and the weight coefficients of multiple vertices corresponding to each initial contour segment includes: Based on the multiple vertex offsets corresponding to each optimized contour segment and the multiple vertex weight coefficients corresponding to each initial contour segment, the target vertex offsets corresponding to each optimized contour segment are obtained through the vertex offset formula. The vertex offset formula is: (Δx ′ k ,Δy ′ k )=Atten(x k ,y k )*(Δx k ,Δy k ) In the formula, k is the number of iterations, Atten(x) k x k ) represents the weight coefficient of the k-th vertex, (Δx) k Δy k ) represents the offset of the vertex at the k-th time, (Δx′) k ,Δy′ k ) represents the vertex target offset for the kth time; Based on the coordinates of multiple original vertices corresponding to each initial contour segment and the target offsets of multiple vertices corresponding to each optimized contour segment, the target coordinates of multiple vertices corresponding to each optimized contour segment are obtained through coordinate iteration formula. The coordinate iteration formula is: In the formula, Let Δx be the vertex target offset of point i in the (k-1)th iteration. k-1 ,Δy k-1 ) represents the vertex offset at the (k-1)th time. The original vertex coordinates of point i in the (k-1)th iteration are... Let i be the vertex target coordinates of point i in the kth iteration.

6. A system for extracting architectural objects from remotely sensed images, characterized in that, The building object extraction system for the remote sensing image includes: The initial module is used to input remote sensing images into the Backbone module in the target detection network for feature extraction, obtain remote sensing image feature maps, and perform building detection on the remote sensing image feature maps to obtain multiple initial outlines of building objects. The determination module is used to determine the contour feature vector and multiple original vertex coordinates corresponding to each initial contour segment; An optimization module is used to input multiple contour feature vectors into a contour optimization network for vertex prediction, and obtain multiple vertex offsets corresponding to each optimized contour segment. The contour optimization network is constructed by combining a Backbone module, a Fusion module, and a Prediction module. The optimization module is further configured to process the remote sensing image feature map through the contour attention module to obtain a contour boundary attention map, and to perform bilinear interpolation on the contour boundary attention map according to the coordinates of multiple original vertices corresponding to each initial contour segment to obtain multiple vertex weight coefficients corresponding to each initial contour segment. The optimization module is also used to obtain the target coordinates of multiple vertices corresponding to each segment of the optimized contour through a coordinate iteration formula based on the coordinates of multiple original vertices corresponding to each segment of the initial contour, the offsets of multiple vertices corresponding to each segment of the optimized contour, and the weight coefficients of multiple vertices corresponding to each segment of the initial contour. The extraction module is used to extract the building object from the remote sensing image based on the coordinates of multiple vertex targets corresponding to each segment of the optimized contour.

7. A device for extracting architectural objects from remote sensing images, characterized in that, The device includes: a memory, a processor, and a building object extraction program for remote sensing images stored in the memory and executable on the processor, the building object extraction program for remote sensing images being configured to implement the steps of the building object extraction method for remote sensing images as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a building object extraction program for remote sensing images. When the building object extraction program for remote sensing images is executed by a processor, it implements the steps of the building object extraction method for remote sensing images as described in any one of claims 1 to 5.

Images