Method, device, equipment and medium for drawing vector outline of building
By combining a multi-task learning method with convolutional neural networks and recurrent neural networks, and using the edge information of buildings for model training, the problems of low accuracy and efficiency in drawing building vector contour maps are solved, and higher-precision building vector contour generation is achieved.
Patent Information
- Application Number
- CN202210725389.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-24
- Publication Date
- 2025-09-19
- Estimated Expiration
- 2042-06-24
AI Technical Summary
The existing technology has problems of insufficient accuracy and low efficiency in drawing building vector outlines, especially when building corners are missed, disordered and the edge conformity is low.
A multi-task learning method is adopted, combining convolutional neural networks and recurrent neural networks, and the positioning accuracy of building corners is improved through the synergy of edge detection and instance segmentation tasks, including edge detection on feature maps, edge detection on image feature maps using building feature maps, training using building edge information, and generating a building vector contour map drawing method through collaborative training of edge detection and instance segmentation tasks, using building edge information for model training, learning the corner position of the building roof, and outputting the building vector contour map.
The accuracy and efficiency of drawing building vector contour maps are improved, the post-processing process in the semantic segmentation method is reduced, and the accuracy of building corner positioning and vector contour generation is improved.
Smart Images
Figure CN114998379B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and deep neural networks, and in particular to a method, device, equipment and medium for drawing a vector outline of a building. Background Art
[0002] With the rapid development and application of artificial intelligence and deep neural network models, fundamental computer vision tasks such as image classification, semantic segmentation, instance segmentation, and panoptic segmentation have achieved unprecedented progress. The combination of Mask R-CNN object detection and Polygon-RNN polygon recognition has shown promising results in extracting building vector outlines. An instance segmentation algorithm, widely used in remote sensing image processing, balances efficiency and performance. This algorithm defines the object's location in the image using the coordinates, length, and width of an object detection box, and performs semantic segmentation using a mask branch. Furthermore, RCF has consistently demonstrated excellent performance in edge detection tasks. This network integrates image features from multiple stages into a unified framework. To improve the fit of polygons around building edges, the edge information in the image is fully utilized to guide the generation of building corner point sequences, effectively improving the accuracy of building vector outline extraction. Another semi-automatic image instance annotation method based on recurrent neural networks, called Polygon-RNN (Automatic Annotation of Image Segmentation Datasets) and its improved version, Polygon-RNN++, requires manually defining an initial rectangular box around each image instance target. A deep neural network then processes the image within the rectangular box and outputs the image instance annotation. Because the annotation is composed of specific contour vertices, using building corners as annotation points in the network output facilitates the generation of building vector outlines, thereby improving the rendering of building thematic maps. However, problems such as missing building corners, confusion, and poor edge conformance still exist.
[0003] As can be seen from the above, in the process of drawing the building vector outline map, how to improve the accuracy of the building vector outline map drawing and improve the efficiency of the building vector outline map drawing is a problem to be solved in this field. Summary of the Invention
[0004] In view of this, the present invention aims to provide a method, apparatus, device, and medium for drawing building vector outlines, which can effectively improve the accuracy and efficiency of drawing building vector outlines. The specific scheme is as follows:
[0005] In a first aspect, the present application discloses a method for drawing a vector outline of a building, comprising:
[0006] Acquire a building image, and perform image processing on the building image to obtain a feature map of the building image;
[0007] Performing an initialization operation on the feature map to obtain an initialized feature map, and performing edge detection processing on the feature map to obtain a processed feature map;
[0008] A first loss function is called to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and a second loss function is called and a pre-established recurrent neural network is used to perform a second training on the probability map to obtain a vector contour map of the building image.
[0009] Optionally, acquiring a building image and performing image processing on the building image to obtain a feature map of the building image includes:
[0010] Acquire the building image through the backbone network of the convolutional neural network;
[0011] The building image is acquired using a region proposal network and a region of interest feature alignment layer, and image processing is performed on the building image to obtain a feature map of the building image of a fixed size.
[0012] Optionally, performing an initialization operation on the feature map to obtain an initialized feature map includes:
[0013] Performing a semantic segmentation operation on the feature map using the instance mask branch to obtain a first initialized sub-feature map, and performing corresponding processing on the feature map using the target detection box regression branch and the target object classification branch to obtain a second initialized sub-feature map;
[0014] The initialization feature map is determined based on the first initialization sub-feature map and the second initialization sub-feature map.
[0015] Optionally, calling the first loss function to perform a first training on the initialized feature map and the processed feature map includes:
[0016] Determining a loss function for the instance mask branch, a loss function for the target detection box regression branch, a loss function for the target object classification branch, and a loss function for edge detection processing respectively to obtain a first loss function;
[0017] Calling a first loss function to perform a first training on the initialized feature map and the processed feature map.
[0018] Optionally, the performing a first training on the initialized feature map and the processed feature map to obtain a probability map includes:
[0019] The convolution feature is clipped using the second initialized sub-feature map according to a preset scaling scale to obtain a clipped feature map carrying a detection frame;
[0020] The feature map carrying the detection frame after the clipping and the processed feature map are spliced, and a probability map is determined in combination with the first initialized sub-feature map.
[0021] Optionally, obtaining the vector outline of the building image includes:
[0022] A vector contour map containing the coordinate vector of the building with the detection frame, the instance category of the building with the detection frame, and the coordinate sequence of the corner point of a single building is obtained.
[0023] Optionally, calling the second loss function and using a pre-established recurrent neural network to perform a second training on the probability map includes:
[0024] Using the pre-established corner point sequence branch, respectively determine the loss function of the entire building vector outline polygon, the loss function of the coordinate position of the building vector outline starting point, and the auxiliary loss function to obtain the second loss function;
[0025] The second loss function is called and a pre-established recurrent neural network is used to train the probability map for a second time according to preset training parameters.
[0026] In a second aspect, the present application discloses a device for drawing a vector outline of a building, comprising:
[0027] a feature map determining module, configured to acquire a building image and perform image processing on the building image to obtain a feature map of the building image;
[0028] An initialization module is used to perform an initialization operation on the feature map to obtain an initialized feature map, and perform edge detection processing on the feature map to obtain a processed feature map;
[0029] A training module is used to call a first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and call a second loss function and use a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image.
[0030] In a third aspect, the present application discloses an electronic device, comprising:
[0031] Memory, used to store computer programs;
[0032] The processor is used to execute the computer program to implement the aforementioned method for drawing a vector outline of a building.
[0033] In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned method for drawing a building vector outline are implemented.
[0034] It can be seen that the present application provides a method for drawing a vector contour map of a building, including acquiring a building image and performing image processing on the building image to obtain a feature map of the building image; performing an initialization operation on the feature map to obtain an initialized feature map, and performing edge detection processing on the feature map to obtain a processed feature map; calling a first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, calling a second loss function and using a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image. This application performs an initialization operation on the feature map of the building image to obtain an initialized feature map, and then performs edge detection on the feature map. The edge information of the building is used to learn the position coordinates of the corner points of the building roof during the model training process, thereby improving the positioning accuracy of the building corner points and improving the model performance of the building vector contour extraction on the high-resolution image. This application utilizes a multi-task learning method to add an edge detection task to the original instance segmentation task, which can effectively improve the synergy between multiple tasks of the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corner points and improve the generation accuracy of the building corner point sequence, thereby directly outputting the vector contour of the building, eliminating the need for post-processing processes such as edge regularization based on the semantic segmentation method, thereby improving the accuracy of drawing the building vector contour map and improving the efficiency of drawing the building vector contour map. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0036] Figure 1 This is a flow chart of a method for drawing a vector outline of a building disclosed in this application;
[0037] Figure 2 This is a specific flow chart of a method for drawing a vector outline of a building disclosed in this application;
[0038] Figure 3 This is a flow chart of a method for drawing a vector outline of a building disclosed in this application;
[0039] Figure 4 This is an output result diagram of a building vector outline drawing method disclosed in this application;
[0040] Figure 5 This is a schematic structural diagram of a device for drawing a vector outline of a building disclosed in this application;
[0041] Figure 6 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION
[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0043] With the rapid development and application of artificial intelligence and deep neural network models, fundamental computer vision tasks such as image classification, semantic segmentation, instance segmentation, and panoptic segmentation have achieved unprecedented progress. The combination of Mask R-CNN object detection and Polygon-RNN polygon recognition has shown promising results in extracting building vector outlines. An instance segmentation algorithm, widely used in remote sensing image processing, balances efficiency and performance. This algorithm defines the object's location in the image using the coordinates, length, and width of an object detection box, and performs semantic segmentation using a mask branch. Furthermore, RCF has consistently demonstrated excellent performance in edge detection tasks. This network integrates image features from multiple stages into a unified framework. To improve the fit of polygons around building edges, the edge information in the image is fully utilized to guide the generation of building corner point sequences, effectively improving the accuracy of building vector outline extraction. There is also a semi-automatic image instance annotation method based on recurrent neural networks, called Polygon-RNN (automatic annotation of image segmentation datasets) and its improved version Polygon-RNN++. This method requires manually giving an initial rectangular box around each image instance target, and then processing the image within the rectangular box through a deep neural network and outputting the annotation of the image instance. Since the annotation is composed of specific contour vertices, using the corner points of the building as the annotation points output by the network will be conducive to the generation of the building vector contour, thereby improving the drawing of the building thematic map. However, there are still problems such as omission and confusion of building corner points and low edge conformity. As can be seen from the above, in the process of drawing the building vector contour map, how to improve and increase the accuracy of the building vector contour map drawing and improve the efficiency of the building vector contour map drawing is a problem to be solved in this field.
[0044] See also Figure 1 As shown, the embodiment of the present invention discloses a method for drawing a vector outline of a building, which may specifically include:
[0045] Step S11: Acquire a building image, and perform image processing on the building image to obtain a feature map of the building image.
[0046] In this embodiment, the building image is acquired through the backbone network of the convolutional neural network, and then the building image is acquired using the region proposal network and the region of interest feature alignment layer, and the building image is processed to obtain a feature map of the building image of a fixed size.
[0047] For example, Figure 2As shown, this application uses the middle convolutional layer of the backbone network in the Mask R-CNN network structure as the input of the RPN (region proposal network), and then uses the region of interest feature alignment layer to make the image features in the target candidate area generate a fixed-size feature map to perform target detection box regression, target object classification and semantic segmentation of instance masks.
[0048] Step S12: performing an initialization operation on the feature map to obtain an initialized feature map, and performing edge detection processing on the feature map to obtain a processed feature map.
[0049] In this embodiment, a semantic segmentation operation is performed on the feature map using an instance mask branch to obtain a first initialized sub-feature map, and the feature map is processed accordingly using a target detection box regression branch and a target object classification branch to obtain a second initialized sub-feature map. Then, the initialized feature map is determined based on the first initialized sub-feature map and the second initialized sub-feature map, and edge detection processing is performed on the feature map to obtain a processed feature map.
[0050] Step S13: Calling the first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, calling the second loss function and using a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image.
[0051] In this embodiment, after obtaining the processed feature map, the loss function for the instance mask branch, the loss function for the target detection box regression branch, the loss function for the target object classification branch, and the loss function for edge detection processing are determined respectively to obtain a first loss function, and then the first loss function is called to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and then the second loss function is called and a pre-established recurrent neural network is used to perform a second training on the probability map to obtain a vector contour map of the building image.
[0052] It can be understood that after calling the first loss function to train the initialized feature map and the processed feature map for the first time, the convolution feature of the second initialized sub-feature map is cut off according to a preset scaling scale to obtain a feature map with a detection frame after cutting off, and then the feature map with a detection frame after cutting off and the processed feature map are spliced, and a probability map is determined in combination with the first initialized sub-feature map, and then the second loss function is called and the pre-established recurrent neural network is used to train the probability map for a second time to obtain a vector contour map of the building image.
[0053] This application utilizes a multi-task learning method to add an edge detection task to the instance segmentation task, which can effectively improve the synergy between multiple tasks in the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corners and improve the generation accuracy of the building corner sequence, thereby directly outputting the vector outline of the building, eliminating the need for post-processing such as edge regularization based on the semantic segmentation method, and improving the work efficiency of building mapping. Since the task of corner sequence recognition and the instance segmentation task are not related, based on the Mask R-CNN pre-training model, the target detection branch, the semantic segmentation branch and the RCF-based edge recognition task are approximated and jointly trained (that is, the second training mentioned above), and then the corner sequence branch (that is, the RNN decoder) is connected for fine-tuning separately, and finally the coordinate vector of the building carrying the detection frame, the instance category of the building carrying the detection frame and the vector outline of the single building corner coordinate sequence are output.
[0054] In this embodiment, a building image is acquired and image processing is performed on the building image to obtain a feature map of the building image; an initialization operation is performed on the feature map to obtain an initialized feature map, and edge detection processing is performed on the feature map to obtain a processed feature map; a first loss function is called to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and a second loss function is called and a pre-established recurrent neural network is used to perform a second training on the probability map to obtain a vector contour map of the building image. This application performs an initialization operation on the feature map of the building image to obtain an initialized feature map, and then performs edge detection on the feature map. The edge information of the building is used to learn the position coordinates of the corner points of the building roof during the model training process, thereby improving the positioning accuracy of the building corner points and improving the model performance of the building vector contour extraction on the high-resolution image. This application utilizes a multi-task learning method to add an edge detection task to the original instance segmentation task, which can effectively improve the synergy between multiple tasks of the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corner points and improve the generation accuracy of the building corner point sequence, thereby directly outputting the vector contour of the building, eliminating the need for post-processing processes such as edge regularization based on the semantic segmentation method, thereby improving the accuracy of drawing the building vector contour map and improving the efficiency of drawing the building vector contour map.
[0055] See also Figure 3 As shown, the embodiment of the present invention discloses a method for drawing a building vector outline. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, it may include:
[0056] Step S21: Acquire a building image, and perform image processing on the building image to obtain a feature map of the building image.
[0057] Step S22: performing an initialization operation on the feature map to obtain an initialized feature map, and performing edge detection processing on the feature map to obtain a processed feature map.
[0058] For more detailed processing procedures in steps S21, S22, and S23, please refer to the aforementioned disclosed embodiments, which will not be described again here.
[0059] Step S23: Call the first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, call the second loss function and use the pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map containing the coordinate vector of the building carrying the detection frame, the instance category of the building carrying the detection frame, and the coordinate sequence of the corner point of a single building.
[0060] In this embodiment, after the initialization feature map and the processed feature map are trained for the first time to obtain a probability map, the pre-established corner point sequence branch is used to determine the loss function of the building vector outline polygon as a whole, the loss function of the building vector outline starting point coordinate position, and the auxiliary loss function to obtain the second loss function, and then the second loss function is called and the pre-established recurrent neural network is used to train the probability map for a second time according to preset training parameters, and finally a vector contour map containing the coordinate vector of the building carrying the detection frame, the instance category of the building carrying the detection frame, and the coordinate sequence of the corner point of a single building is obtained.
[0061] For example, Figure 2As shown, this application uses a pre-trained model of Mask R-CNN based on the COCO (Microsoft Common Objects inContext) dataset, selects ResNet (deep residual network) 101 as the feature extraction layer of the convolutional neural network (CNN), and uses the intermediate convolution layer of the backbone network as the input of RPN. The image in the target candidate area is processed through the region of interest feature alignment layer to generate a fixed-size feature map, that is, a fixed-size featuremap is obtained according to the feature map (feature map) and region proposals (candidate area) in the image, and then the Mask Branch (mask branch) is used to perform semantic segmentation on the feature map to obtain a first initialized sub-feature map, and then the Box regression (target detection box regression branch) and Classification (target object classification branch) in FCN (full convolutional neural network) are used to process the feature map to obtain a second initialized sub-feature map, and then the RCF (Richer Convolutional Features, an edge detection network model) processes the feature map to obtain a processed feature map, and then performs the first training for edge pixel detection on the first initialized sub-feature map, the second initialized sub-feature map and the processed feature map to obtain a probability map, and uses the probability map as an enhanced feature. For example, the input image size is 224×224, and after the above steps, a probability map of 224×224×1 is finally output. The convolutional features are then clipped according to the scaling scale, and the clipped features are spliced with the feature map of edge detection and sent to the RNN (Recurrent Neural Network) decoder for a second training through the RNN decoder. The probability map can be trained for a second time according to preset training parameters. For example, the number of training iterations is set to 20,000, batch size = 16, the learning rate is 0.001, and a 0.1-fold attenuation is performed every 100 iterations. By setting the training parameters, its weight can be greatly reduced, even to 0. Before the first training process, it is necessary to determine the first loss function. The specific formula of the first loss function is as follows:
[0062] L1=L cls +L box +L mask +L RCF ;
[0063] Among them, L cls is the loss function generated during the target object classification branch processing, L boxis the loss function generated in the target detection frame regression branch process, L mask is the loss function generated in the semantic segmentation process, L RCF is the loss function generated in the edge detection process.
[0064] Calculate the first loss function, and then call the first loss function for the first training. The specific formula of the second loss function is as follows:
[0065] L2=L polygon +L vertex +L edge ;
[0066] Among them, L polygon is the loss function of the building vector outline polygon as a whole, L vertex is the loss function of the coordinate position of the contour starting point, where L edge To assist the loss, it prevents the gradient from disappearing in the integrated deep network.
[0067] The second loss function is calculated, and then the second loss function is called for the second training. After passing through the RNN decoder, the coordinate vectors of the detection frame positions of all buildings in the image, the categories of the instances in the detection frame, and the coordinate sequence of the corner points of a single building in the detection frame will be output.
[0068] This application utilizes a multi-task learning method to add an edge detection task to the instance segmentation task, which can effectively improve the synergy between multiple tasks in the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corners and improve the generation accuracy of the building corner sequence, thereby directly outputting the vector contour of the building, eliminating the need for post-processing such as edge regularization based on the semantic segmentation method, and improving the efficiency of building mapping. Since the corner sequence recognition task and the instance segmentation task are not related, this allows the target detection branch, the semantic segmentation branch, and the RCF-based edge recognition task to be approximated and jointly trained (that is, the second training mentioned above) based on the Mask R-CNN pre-training model, and then the corner sequence branch (that is, the RNN decoder) is connected for fine-tuning separately, and finally the coordinate vector of the building carrying the detection frame, the instance category of the building carrying the detection frame, and the vector contour map of the single building corner coordinate sequence are output, as shown in the following example. Figure 4 shown.
[0069] This application is connected to the edge recognition network and uses the edge information of the building to improve the Polygon-RNN polygon recognition method. And during the model training process, the coordinates of the corner points of the building roof are clearly learned, thereby improving the positioning accuracy of the building corner points and improving the model performance of the building vector contour extraction on high-resolution images. This method models the boundary of the building roof as a closed polygon, and improves the geometric shape of the building roof by encouraging the vertices of the polygon to be close to the corner points of the building. This method simulates the process of manually outlining the roof contours of buildings in remote sensing images or aerial images in the geographic information industry, thereby drawing vector contours of buildings with regular shapes.
[0070] In this embodiment, a building image is acquired and image processing is performed on the building image to obtain a feature map of the building image; an initialization operation is performed on the feature map to obtain an initialized feature map, and edge detection processing is performed on the feature map to obtain a processed feature map; a first loss function is called to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and a second loss function is called and a pre-established recurrent neural network is used to perform a second training on the probability map to obtain a vector contour map of the building image. This application performs an initialization operation on the feature map of the building image to obtain an initialized feature map, and then performs edge detection on the feature map. The edge information of the building is used to learn the position coordinates of the corner points of the building roof during the model training process, thereby improving the positioning accuracy of the building corner points and improving the model performance of the building vector contour extraction on the high-resolution image. This application utilizes a multi-task learning method to add an edge detection task to the original instance segmentation task, which can effectively improve the synergy between multiple tasks of the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corner points and improve the generation accuracy of the building corner point sequence, thereby directly outputting the vector contour of the building, eliminating the need for post-processing processes such as edge regularization based on the semantic segmentation method, thereby improving the accuracy of drawing the building vector contour map and improving the efficiency of drawing the building vector contour map.
[0071] See also Figure 5 As shown, the embodiment of the present invention discloses a device for drawing a vector outline of a building, which may specifically include:
[0072] A feature map determining module 11 is configured to acquire a building image and perform image processing on the building image to obtain a feature map of the building image;
[0073] An initialization module 12 is used to perform an initialization operation on the feature map to obtain an initialized feature map, and perform edge detection processing on the feature map to obtain a processed feature map;
[0074] The training module 13 is used to call the first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and call the second loss function and use a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image.
[0075] In this embodiment, a building image is acquired and image processing is performed on the building image to obtain a feature map of the building image; an initialization operation is performed on the feature map to obtain an initialized feature map, and edge detection processing is performed on the feature map to obtain a processed feature map; a first loss function is called to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and a second loss function is called and a pre-established recurrent neural network is used to train the probability map for a second time to obtain a vector contour map of the building image. This application performs an initialization operation on the feature map of the building image to obtain an initialized feature map, and then performs edge detection on the feature map. The edge information of the building is used to learn the position coordinates of the corner points of the building roof during the model training process, thereby improving the positioning accuracy of the building corner points and improving the model performance of the building vector contour extraction on the high-resolution image. This application utilizes a multi-task learning method to add an edge detection task to the original instance segmentation task, which can effectively improve the synergy between multiple tasks of the network. At the same time, the edge detection task is used to enhance the positioning accuracy of the building corner points and improve the generation accuracy of the building corner point sequence, thereby directly outputting the vector contour of the building, eliminating the need for post-processing processes such as edge regularization based on the semantic segmentation method, thereby improving the accuracy of drawing the building vector contour map and improving the efficiency of drawing the building vector contour map.
[0076] In some specific embodiments, the feature map determination module 11 may specifically include:
[0077] A building image acquisition module, configured to acquire the building image through a convolutional neural network backbone network;
[0078] The feature map acquisition module is used to acquire the building image using a region proposal network and a region of interest feature alignment layer, and perform image processing on the building image to obtain a feature map of the building image of a fixed size.
[0079] In some specific embodiments, the initialization module 12 may specifically include:
[0080] a sub-feature map determination module, configured to perform a semantic segmentation operation on the feature map using the instance mask branch to obtain a first initialized sub-feature map, and to perform corresponding processing on the feature map using the target detection box regression branch and the target object classification branch to obtain a second initialized sub-feature map;
[0081] An initialization feature map determining module is used to determine the initialization feature map based on the first initialization sub-feature map and the second initialization sub-feature map.
[0082] In some specific embodiments, the training module 13 may specifically include:
[0083] a first loss function determination module, configured to respectively determine a loss function for the instance mask branch, a loss function for the target detection box regression branch, a loss function for the target object classification branch, and a loss function for edge detection processing, to obtain a first loss function;
[0084] The first training module is used to call the first loss function to perform the first training on the initialized feature map and the processed feature map.
[0085] In some specific embodiments, the training module 13 may specifically include:
[0086] A clipping module, configured to clip the convolution feature of the second initialized sub-feature map according to a preset scaling scale to obtain a clipped feature map carrying a detection frame;
[0087] A probability map determination module is used to splice the feature map carrying the detection frame after the interception and the processed feature map, and determine a probability map in combination with the first initialized sub-feature map.
[0088] In some specific embodiments, the training module 13 may specifically include:
[0089] The vector contour drawing module is used to obtain a vector contour map including the coordinate vector of the building carrying the detection frame, the instance category of the building carrying the detection frame, and the coordinate sequence of the corner point of a single building.
[0090] In some specific embodiments, the training module 13 may specifically include:
[0091] A second loss function determination module is used to use a pre-established corner point sequence branch to respectively determine the loss function of the entire building vector outline polygon, the loss function of the coordinate position of the starting point of the building vector outline, and the auxiliary loss function to obtain the second loss function;
[0092] The second training module is used to call the second loss function and use the pre-established recurrent neural network to perform a second training on the probability map according to preset training parameters.
[0093] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the method for drawing a building vector outline image performed by the electronic device disclosed in any of the aforementioned embodiments.
[0094] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0095] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.
[0096] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, enabling the processor 21 to calculate and process data 223 in the memory 22. The operating system 221 may be Windows, Unix, Linux, or other operating systems. In addition to including computer programs capable of performing the building vector outline drawing method performed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks. The data 223 may include data transmitted from external devices and received by the building vector outline drawing device, as well as data collected by its own input / output interface 25.
[0097] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0098] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the method for drawing a vector outline of a building disclosed in any of the aforementioned embodiments are implemented.
[0099] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0100] The above is a detailed introduction to the method, device, equipment and storage medium for drawing a vector outline of a building provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for drawing a vector outline of a building, characterized in that: include: Acquire a building image, and perform image processing on the building image to obtain a feature map of the building image; Performing an initialization operation on the feature map to obtain an initialized feature map, and performing edge detection processing on the feature map to obtain a processed feature map; Calling a first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, calling a second loss function and using a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image; The probability map is a probability map obtained after the first training for edge pixel detection on the first initialized sub-feature map, the second initialized sub-feature map, and the processed feature map; the processed feature map is obtained by processing the feature map using RCF, the first initialized sub-feature map is obtained by performing a semantic segmentation operation on the feature map using the mask branch, and the second initialized sub-feature map is obtained by processing the feature map using the target detection box regression branch and the target object classification branch in the fully convolutional neural network; The method further includes: respectively determining a loss function for an instance mask branch, a loss function for a target detection box regression branch, a loss function for a target object classification branch, and a loss function for edge detection processing to obtain a first loss function; The specific formula of the first loss function is as follows: ； in, is the loss function generated during the target object classification branch processing, is the loss function generated in the target detection frame regression branch process, is the loss function generated in the semantic segmentation process, is the loss function generated in the edge detection process; Acquiring a building image and performing image processing on the building image to obtain a feature map of the building image, including: acquiring the building image through a convolutional neural network backbone network; acquiring the building image using a region proposal network and a region of interest feature alignment layer, and performing image processing on the building image to obtain a feature map of the building image of a fixed size; The first loss function is called to perform the first training on the initialized feature map and the processed feature map, and the convolution feature of the second initialized sub-feature map is intercepted according to the preset scaling scale to obtain the intercepted feature map with the detection frame. The feature map with the detection frame and the processed feature map are then spliced together, and the probability map is determined in combination with the first initialized sub-feature map. The second loss function is then called and the pre-established recurrent neural network is used to train the probability map for the second time to obtain a vector contour map of the building image.
2. The method for drawing a building vector outline according to claim 1, wherein: The initializing operation on the feature map to obtain an initialized feature map includes: Performing a semantic segmentation operation on the feature map using the instance mask branch to obtain a first initialized sub-feature map, and performing corresponding processing on the feature map using the target detection box regression branch and the target object classification branch to obtain a second initialized sub-feature map; The initialization feature map is determined based on the first initialization sub-feature map and the second initialization sub-feature map.
3. The method for drawing a building vector outline according to claim 2, wherein: The first training of the initialized feature map and the processed feature map to obtain a probability map includes: The convolution feature is clipped using the second initialized sub-feature map according to a preset scaling scale to obtain a clipped feature map carrying a detection frame; The feature map carrying the detection frame after the clipping and the processed feature map are spliced, and a probability map is determined in combination with the first initialized sub-feature map.
4. The method for drawing a building vector outline according to any one of claims 1 to 3, characterized in that: The step of obtaining a vector outline of the building image comprises: A vector contour map containing the coordinate vector of the building with the detection frame, the instance category of the building with the detection frame, and the coordinate sequence of the corner point of a single building is obtained.
5. The method for drawing a building vector outline according to claim 1, wherein: The calling of the second loss function and using a pre-established recurrent neural network to perform a second training on the probability map includes: Using the pre-established corner point sequence branch, respectively determine the loss function of the entire building vector outline polygon, the loss function of the coordinate position of the building vector outline starting point, and the auxiliary loss function to obtain the second loss function; The second loss function is called and a pre-established recurrent neural network is used to train the probability map for a second time according to preset training parameters.
6. A device for drawing a vector outline of a building, characterized in that: include: a feature map determining module, configured to acquire a building image and perform image processing on the building image to obtain a feature map of the building image; An initialization module is used to perform an initialization operation on the feature map to obtain an initialized feature map, and perform edge detection processing on the feature map to obtain a processed feature map; A training module is configured to call a first loss function to perform a first training on the initialized feature map and the processed feature map to obtain a probability map, and call a second loss function and use a pre-established recurrent neural network to perform a second training on the probability map to obtain a vector contour map of the building image; The probability map is a probability map obtained after the first training for edge pixel detection on the first initialized sub-feature map, the second initialized sub-feature map, and the processed feature map; the processed feature map is obtained by processing the feature map using RCF, the first initialized sub-feature map is obtained by performing a semantic segmentation operation on the feature map using the mask branch, and the second initialized sub-feature map is obtained by processing the feature map using the target detection box regression branch and the target object classification branch in the fully convolutional neural network; The device is further configured to respectively determine a loss function for an instance mask branch, a loss function for a target detection frame regression branch, a loss function for a target object classification branch, and a loss function for edge detection processing to obtain a first loss function; The specific formula of the first loss function is as follows: ； in, is the loss function generated during the target object classification branch processing, is the loss function generated in the target detection frame regression branch process, is the loss function generated in the semantic segmentation process, is the loss function generated in the edge detection process; Acquiring a building image and performing image processing on the building image to obtain a feature map of the building image, including: acquiring the building image through a convolutional neural network backbone network; acquiring the building image using a region proposal network and a region of interest feature alignment layer, and performing image processing on the building image to obtain a feature map of the building image of a fixed size; The first loss function is called to perform the first training on the initialized feature map and the processed feature map, and the convolution feature of the second initialized sub-feature map is intercepted according to the preset scaling scale to obtain the intercepted feature map with the detection frame. The feature map with the detection frame and the processed feature map are then spliced together, and the probability map is determined in combination with the first initialized sub-feature map. The second loss function is then called and the pre-established recurrent neural network is used to train the probability map for the second time to obtain a vector contour map of the building image.
7. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for drawing a building vector outline map according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the method for drawing a building vector outline map according to any one of claims 1 to 5 is implemented.