Building monomer image segmentation method, device and equipment and computer readable medium

Abstract

The embodiment of the invention discloses a building monomer image segmentation method, device and equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: acquiring remote sensing image data and a mapping information set of a target area; preprocessing the remote sensing image data to obtain preprocessed remote sensing image data; performing information extraction on the preprocessed remote sensing image data to obtain contour information of each building; generating an initial region three-dimensional model according to the contour information of each building; according to the mapping information set, performing mapping processing on the three-dimensional scene of the initial area to obtain a target three-dimensional model; performing point cloud extraction on the target three-dimensional model to obtain a point cloud data set; performing building segmentation processing on the point cloud data set to obtain a building point cloud data set; and carrying out single segmentation on the building point cloud data set to obtain single image information of each building. According to the embodiment, the precision of carrying out monomer segmentation on the building image is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, equipment, and computer-readable medium for segmenting a building image. Background Art

[0002] By performing 3D modeling on urban areas, we can obtain a 3D model of the urban area. By segmenting the building units from the 3D model, we can obtain independent building unit models that can be managed and operated separately, which can help the construction and refined management of smart cities and other fields. At present, the main methods used for 3D modeling of urban areas are geometric modeling, scanning modeling, fractal modeling, and drone oblique image modeling. And the building units are separated from the 3D model of the urban area through vector cutting unitization, physical unitization and other unitization technologies.

[0003] However, in practice, when the above method is used to segment building monomer images, the following technical problems often occur: For some complex models, geometric modeling may require a lot of time and manpower to manually create and adjust the shape; scanning modeling requires the use of high-quality scanners, resulting in high equipment costs for modeling; fractal modeling is based on mathematical rules, involves a lot of mathematical calculations, and cannot accurately control the shape and size of the model, resulting in a long modeling time; when the building is high, the bottom is blocked by other objects, resulting in low imaging accuracy of the drone's oblique image, which leads to low accuracy of the generated three-dimensional model and the need to configure drones, resulting in high equipment costs for modeling. The monomerized model produced by vector cutting monomerization may have obvious jagged edges and irregular shapes at the bottom of the edge, resulting in low accuracy of the cut building monomer model. Physical monomerization extracts the base of the building and performs monomerization processing, but for buildings with irregular shapes or complex structures, the accuracy of monomer segmentation is poor.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a method, device, electronic device and computer-readable medium for segmenting a building image to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for segmenting a single image of a building, the method comprising: acquiring remote sensing image data and a mapping information set of a target area; preprocessing the remote sensing image data to obtain preprocessed remote sensing image data; extracting information from the preprocessed remote sensing image data to obtain contour information of each building; generating an initial area three-dimensional model corresponding to the target area based on the contour information of each building; mapping the initial area three-dimensional model based on the mapping information set to obtain a target three-dimensional model; performing point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model; performing building segmentation processing on the point cloud data set to obtain a building point cloud data set; performing monomer segmentation on the building point cloud data set to obtain monomer image information of each building.

[0008] In a second aspect, some embodiments of the present disclosure provide a device for segmenting a single image of a building, the device comprising: an acquisition unit, configured to acquire remote sensing image data and a mapping information set of a target area; a preprocessing unit, configured to preprocess the remote sensing image data to obtain preprocessed remote sensing image data; a first extraction unit, configured to extract information from the preprocessed remote sensing image data to obtain contour information of each building; a generation unit, configured to generate an initial area three-dimensional model corresponding to the target area according to the contour information of each building; a mapping unit, configured to perform mapping processing on the initial area three-dimensional model according to the mapping information set to obtain a target three-dimensional model; a second extraction unit, configured to perform point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model; a first segmentation unit, configured to perform building segmentation processing on the point cloud data set to obtain a building point cloud data set; and a second segmentation unit, configured to perform monomer segmentation on the building point cloud data set to obtain monomer image information of each building.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the building monomer image segmentation method of some embodiments of the present disclosure, the equipment cost of urban modeling is reduced, and the accuracy of urban modeling and the accuracy of monomer segmentation of building images are improved. Specifically, the reasons for the high equipment cost of urban modeling, the low accuracy of urban modeling and the low accuracy of monomer segmentation of building images are: for some complex models, geometric modeling may require a lot of time and manpower to manually create and adjust the shape; scanning modeling requires the use of high-quality scanners, resulting in high equipment costs for modeling; fractal modeling is based on mathematical rules, involves a lot of mathematical calculations, and cannot accurately control the shape and size of the model, resulting in a long time for modeling; when the building is high, the bottom is blocked by other objects, resulting in low imaging accuracy of the drone oblique image, resulting in low accuracy of the generated three-dimensional model, and the need to configure a drone, resulting in high equipment costs for modeling. The monomerized model generated by vector cutting monomerization may have obvious jagged and irregular shapes at the bottom of the edge, resulting in low accuracy of the monomer model of the cut building. Physical monomerization extracts the base of the building and performs monomerization processing, but for buildings with irregular shapes or complex structures, the accuracy of monomer segmentation is poor. Based on this, the building monomer image segmentation method of some embodiments of the present disclosure first obtains the remote sensing image data and the mapping information set of the target area. Thus, the remote sensing image data and the mapping information set of the target area can be obtained. Then, the remote sensing image data is preprocessed to obtain the preprocessed remote sensing image data, thereby obtaining the preprocessed remote sensing image data, and the noise in the remote sensing image data can be removed. Afterwards, the preprocessed remote sensing image data is extracted to obtain the contour information of each building. Thus, the contour information of each building included in the remote sensing image data can be obtained. Afterwards, according to the contour information of each building, an initial regional three-dimensional model corresponding to the target area is generated. Thus, the initial regional three-dimensional model corresponding to the target area can be obtained. Then, according to the mapping information set, the initial regional three-dimensional model is mapped to obtain the target three-dimensional model. Thus, the target three-dimensional model can be obtained. Among them, the target three-dimensional model can be the initial regional three-dimensional model after mapping. Afterwards, it is configured to extract point cloud from the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model. Thus, the point cloud data corresponding to the above target three-dimensional model can be obtained. Next, the above point cloud data set is subjected to building segmentation processing to obtain a building point cloud data set. Thus, a building point cloud data set after building segmentation processing can be obtained. Among them, the above building point cloud data set can be a point cloud data set after removing ground point cloud data and background point cloud data. Finally, the above building point cloud data set is subjected to monomer segmentation to obtain the image information of each building monomer.Thus, the image information of each building monomer can be obtained. Among them, the above-mentioned image information of each building monomer can represent each building monomer included in the above-mentioned building image information. Also because the realized building monomer image segmentation method can generate a high-precision three-dimensional model of the target area building based on the remote sensing image data and mapping information set of the target area, and extract the point cloud data of the target area building three-dimensional model, perform building point cloud segmentation based on the point cloud data, and then perform fine-grained monomer segmentation on the segmented building cloud data to obtain the image information of each building monomer. Since the remote sensing image data has low acquisition cost, wide coverage, high resolution, and does not require a large amount of complex calculations, the modeling accuracy is high, the time consumption is short, and more manual participation is not required. The building monomer segmentation based on point cloud data can accurately capture the surface morphology and details of the building, thereby improving the accuracy of monomer segmentation of the building image. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the method for segmenting a building monomer image according to the present disclosure; Figure 2 is a schematic structural diagram of some embodiments of the building monomer image segmentation device according to the present disclosure; Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0020] Figure 1 The process 100 of some embodiments of the method for segmenting a building single image according to the present disclosure is shown. The method for segmenting a building single image comprises the following steps: Step 101, obtaining remote sensing image data and a map information set of a target area.

[0021] In some embodiments, the execution subject (e.g., a computing device) of the building monomer image segmentation method may obtain remote sensing image data and a map information set of the target area. The target area may be any urban area. The remote sensing image data may be images and data information of the urban area obtained by remote sensing technology. The data information may be reflection, radiation, or scattering information of the urban surface. The remote sensing image data may also include surface elevation data of the target area. The surface elevation data may be the heights of each point in the target area relative to a specific vertical reference. For example, the specific vertical reference may be the sea level. The map information set may be each on-site photo obtained by taking on-site photos of the target area or each picture obtained after retouching each on-site photo. The retouching of each on-site photo may include cropping, adjusting resolution, cutting out, etc. In practice, the execution subject may obtain the remote sensing image data and map information set of the target area from an associated storage server.

[0022] Step 102, preprocessing the remote sensing image data to obtain preprocessed remote sensing image data.

[0023] In some embodiments, the execution entity may preprocess the remote sensing image data to obtain preprocessed remote sensing image data. In practice, first, the execution entity may use the radiation correction tool radiometriccorrection to perform radiation calibration processing on the remote sensing image data to obtain radiation calibrated remote sensing image data. Then, a geometric correction algorithm may be used to perform geometric correction on the remote sensing image data after radiation calibration processing to obtain geometrically corrected remote sensing image data. Afterwards, the dark pixel method may be used to perform atmospheric correction on the geometrically corrected remote sensing image data to preprocess the remote sensing image data to obtain preprocessed remote sensing image data. Among them, the geometric correction algorithm may be but is not limited to one of the following: RPC model algorithm, polynomial model algorithm.

[0024] Step 103: extract information from the pre-processed remote sensing image data to obtain the outline information of each building.

[0025] In some embodiments, the execution subject may extract information from the pre-processed remote sensing image data to obtain the outline information of each building. The outline information of each building may characterize the outline and position of each building included in the remote sensing image data. The building outline information may include position information and elevation data. The position information may be the geographical location coordinates of the building. The elevation data may be the height of the building relative to the specific vertical reference. In practice, the execution subject may extract information from the pre-processed remote sensing image data in various ways to obtain the outline information of each building.

[0026] In some optional implementations of some embodiments, the execution subject may extract information from the pre-processed remote sensing image data to obtain the outline information of each building through the following steps: The first step is to perform image enhancement on the preprocessed remote sensing image data to obtain remote sensing image enhanced data. The remote sensing image enhanced data may be image enhanced remote sensing image data. In practice, the execution subject may use a filter enhancement method to perform image enhancement on the preprocessed remote sensing image data to obtain remote sensing image enhanced data. The filter enhancement method may be, but is not limited to, one of the following: median filtering, mean filtering, and sharpening filtering.

[0027] The second step is to perform building recognition on the remote sensing image enhancement data to obtain building recognition result information. The building recognition result information can be a binary image of the building contour. For example, the area with a pixel value of 1 in the binary image of the building contour can represent the building area. The area with a pixel value of 0 in the binary image of the building contour can represent the non-building area.

[0028] The third step is to determine the location information of each building according to the building recognition result information, wherein the location information of each building can be the three-dimensional coordinates of each building in the world coordinate system.

[0029] The fourth step is to determine the elevation data of each building according to the surface elevation data of the target area included in the remote sensing image data and the location information of each building. In practice, the execution entity may determine each height corresponding to the location information of each building in the surface elevation data of the target area as the elevation data of each building.

[0030] In the fifth step, for each of the above-mentioned building location information, the above-mentioned building location information and the building elevation data corresponding to the above-mentioned building location information among the above-mentioned building elevation data are determined as building outline information.

[0031] In some optional implementations of some embodiments, the execution subject may perform building recognition on the remote sensing image enhancement data to obtain building recognition result information through the following steps: The first step is to perform region extraction on the remote sensing image enhancement data to obtain each remote sensing image region data. Among them, the remote sensing image region data in each remote sensing image region data can represent the rectangular boundary including the building extracted from the remote sensing image enhancement data. The remote sensing image region data in each remote sensing image region data can include corner point coordinates, region height and region width. The above corner point coordinates can be the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates or lower right corner coordinates of the above rectangular boundary. In practice, first, the above execution subject can use a corner point detection algorithm to perform corner point detection on the remote sensing image enhancement data to obtain each corner point information. Then, a rectangle fitting algorithm is used to perform rectangle fitting on each corner point information to obtain remote sensing image region data. The above corner point detection algorithm can be but not limited to one of the following: SIFT algorithm, Harris algorithm or SURF algorithm. The above rectangle fitting algorithm can be but not limited to one of the following: Hough transform, RANSAC algorithm.

[0032] The second step is to extract features from the above-mentioned remote sensing image region data to obtain feature information of each region. Among them, the regional feature information in the above-mentioned regional feature information can be a feature vector of the remote sensing image region data. In practice, the above-mentioned execution subject can use a convolution operation to extract features from the above-mentioned remote sensing image region data to obtain feature information of each region. Here, the specific setting of the relevant parameters of the above-mentioned convolution operation is not limited.

[0033] The third step is to classify the above-mentioned regional feature information to obtain various classification result information. Among them, the classification result information in the above-mentioned classification result information can represent the probability that the remote sensing image area data includes a building. In practice, the above-mentioned execution subject can use a softmax function to classify the above-mentioned regional feature information to obtain various classification result information.

[0034] In the fourth step, each remote sensing image area data corresponding to each classification result information that meets the preset classification condition in the above classification result information is determined as each building image area data. The above preset classification condition may be that the classification result information is greater than the preset classification result threshold. Here, the specific setting of the above preset classification result threshold is not limited. For example, the preset classification result threshold may be 85%. The above building image area data may be each remote sensing image area data whose classification result information is greater than the above preset classification result threshold.

[0035] The fifth step is to adjust the position of the above-mentioned image area data of each building to obtain the image area data of each building after the position adjustment. In practice, first, the execution subject can use an edge detection algorithm to perform edge detection on the above-mentioned image area data of each building to obtain various edge information. The above-mentioned edge detection algorithm can be a Canny algorithm. Then, the findContours function in OpenCV can be used to extract the contours of the above-mentioned edge information to obtain various contour extraction information. Finally, the contour approximation algorithm can be used to simplify the contours of the above-mentioned contour extraction information to obtain the simplified contour extraction information as the image area data of each building after the position adjustment. The above-mentioned contour approximation algorithm can be a Douglas-Peucker algorithm.

[0036] In the sixth step, the image area data of each building after the position adjustment is subjected to regional redundancy elimination processing to obtain the target image area data. The target image area data may be the image area data of each building after the regional redundancy elimination processing. In practice, the execution subject may adopt a non-maximum suppression algorithm to perform regional redundancy elimination processing on the image area data of each building after the position adjustment to obtain the target image area data.

[0037] In the seventh step, the above-mentioned target image area data are determined as building recognition result information.

[0038] Next, when the position adjustment process of the above-mentioned building image area data is performed using the method disclosed in the present invention, the following technical problem 2 further exists: In the case of complex backgrounds, irregular building shapes or occlusions, edge detection and contour analysis are difficult to accurately extract the boundaries of buildings, resulting in poor accuracy in position adjustment of building image area data based on the building boundaries extracted by edge detection and contour analysis.

[0039] In some optional implementations of some embodiments, the execution subject may perform position adjustment processing on the above-mentioned respective building image region data through the following steps to obtain the position-adjusted respective building image region data: For each of the above building image area data, perform the following steps: The first step is to input the regional feature information corresponding to the above-mentioned building image area data into a preset position regression model to obtain the position offset information corresponding to the above-mentioned building image area data. Among them, the above-mentioned preset position regression model can be a regression model trained with the regional feature information corresponding to the building image area data as input and the position offset information corresponding to the above-mentioned building image area data as the expected output. The above-mentioned position offset information can be a position offset amount that characterizes the prediction of the above-mentioned preset position regression model. The above-mentioned position offset amount can include a corner point coordinate offset and a scaling factor. The above-mentioned corner point coordinate offset can characterize the coordinate difference between the corner point coordinates of the above-mentioned building image area data and the corner point coordinates of the above-mentioned real building image area data. The above-mentioned scaling factor can characterize the scaling ratio of the width and height of the above-mentioned building image area data and the above-mentioned real building image area data. The above-mentioned corner point coordinate offset can include a horizontal coordinate offset value and a vertical coordinate offset value. The above-mentioned scaling factor can include a height scaling ratio and a width scaling ratio. The above-mentioned real building image area data can be the real rectangular boundary of the building included in the above-mentioned building image area data in the above-mentioned remote sensing image enhancement data.

[0040] The second step is to update the building image area data according to the position offset information to obtain the building image area data after position adjustment. In practice, the execution subject can adopt various methods to update the building image area data according to the position offset information to obtain the building image area data after position adjustment.

[0041] In some optional implementations of some embodiments, the execution subject may perform regional update on the building image region data according to the position offset information to obtain the building image region data after position adjustment through the following steps: In the first step, the horizontal axis coordinates of the corner point coordinates included in the building image area data are determined as the horizontal coordinates to be updated.

[0042] In the second step, the vertical axis coordinates of the corner point coordinates included in the above-mentioned building image area data are determined as the vertical coordinates to be updated.

[0043] The third step is to determine the area height included in the above-mentioned building image area data as the height to be updated.

[0044] The fourth step is to determine the area width included in the above-mentioned building image area data as the width to be updated.

[0045] In the fifth step, the sum of the product of the horizontal coordinate offset value included in the position offset information and the width to be updated and the horizontal coordinate to be updated is determined as the updated corner point horizontal coordinate.

[0046] In the sixth step, the product of the ordinate offset value included in the position offset information and the height to be updated and the ordinate to be updated are determined as the updated corner point ordinate.

[0047] In the seventh step, the updated corner point horizontal coordinate and the updated corner point vertical coordinate are determined as the updated corner point coordinates.

[0048] In the eighth step, the product of the height scaling value raised to the power of the real number e and the height to be updated is determined as the height of the update area. The height scaling value may be the height scaling ratio included in the position offset information.

[0049] In the ninth step, the product of the width scaling value raised to the power of the real number e and the width to be updated is determined as the update region width. The width scaling value may be the width scaling ratio included in the position offset information.

[0050] In the tenth step, the updated corner point coordinates, the updated area height and the updated area width are determined as the building image area data after position adjustment.

[0051] In some optional implementations of some embodiments, the above-mentioned execution subject may train and obtain a preset position regression model through the following steps: The first step is to obtain a sample set. The samples in the sample set may include a sample feature vector of a sample image, sample region boundary information corresponding to the sample image, and initial region boundary information. The sample region boundary information may be a real rectangular boundary of a building included in the sample image information. The initial region boundary information may be a rectangular boundary of a building obtained by performing region extraction on the sample image.

[0052] In the second step, based on the sample set, perform the following training steps: In a first training step, a sample feature vector of at least one sample in the sample set is input into an initial position regression model to obtain predicted position offset information of the sample feature vector corresponding to the at least one sample. The initial position regression model may be a regression model. The predicted position offset information may be position offset information predicted by the regression model. The predicted position offset information may include predicted corner point coordinate offset and predicted scaling factor. The predicted corner point coordinate offset may include predicted horizontal coordinate offset value and predicted vertical coordinate offset value. The predicted scaling factor may include predicted height scaling ratio and predicted width scaling ratio.

[0053] The second training step is to determine the predicted region boundary information of the sample feature vector corresponding to the at least one sample according to the predicted position offset information of the sample feature vector corresponding to the at least one sample and the initial region boundary information of the at least one sample. The predicted boundary information may be the region boundary information after the initial region boundary information is offset and corrected according to the predicted position offset information. In practice, for each initial region boundary information in the initial region boundary information of the at least one sample, the execution subject may perform region update on the initial region boundary information according to the predicted position offset information, and obtain the initial region boundary information after region update as the predicted region boundary information. Here, the specific step of performing region update on the initial region boundary information according to the predicted position offset information may refer to the step of performing region update on the building image region data according to the position offset information in step 103, which will not be repeated here.

[0054] The third training step is to determine whether the initial position regression model reaches the preset optimization target based on the prediction region boundary information of the sample feature vector of the at least one sample and the sample region boundary information of the sample feature vector of the at least one sample. The preset optimization target can be that the boundary information error value between the prediction region boundary information of the sample feature vector of the at least one sample and the sample region boundary information of the sample feature vector of the at least one sample is less than the preset value. Here, the specific setting of the preset value is not limited. In practice, first, for each prediction region boundary information in the prediction region boundary information of the sample feature vector of the at least one sample, the execution subject can determine the distance between the corner point coordinates included in the prediction region boundary information and the corner point coordinates included in the sample region boundary information corresponding to the prediction region boundary information as the coordinate distance, and determine the difference between the area corresponding to the prediction region boundary information and the area corresponding to the sample region boundary information corresponding to the prediction region boundary information as the area difference. Then, the average value of each coordinate distance obtained is determined as the mean distance. After that, the average value of each area difference obtained is determined as the mean area difference. Finally, a weighted sum is performed on the mean distance and the mean area difference to obtain a boundary information error value. Here, the specific setting of the weights of the mean distance and the mean area difference is not limited.

[0055] In a fourth training step, in response to determining that the initial position regression model achieves the above-mentioned preset optimization target, the initial position regression model is determined as a preset position regression model.

[0056] In a fifth training step, in response to determining that the initial position regression model does not reach the above-mentioned preset optimization target, the parameters of the initial position regression model are adjusted, and the above-mentioned training step is performed again using a sample set consisting of unused samples.

[0057] The first step to the second step of the above technical solution and its related contents serve as an inventive point of the embodiment of the present disclosure, which solves the above technical problem 2 "When the background is complex, the building shape is irregular or there is occlusion, it is difficult for edge detection and contour analysis to accurately extract the boundary of the building. As a result, the accuracy of the position adjustment processing of the building image area data based on the building boundary extracted by edge detection and contour analysis is poor". The factors that lead to the poor accuracy of the position adjustment processing of the building image area data are often as follows: When the background is complex, the building shape is irregular or there is occlusion, it is difficult for edge detection and contour analysis to accurately extract the boundary of the building. If the above factors are solved, the effect of improving the accuracy of the position adjustment processing of the building image area data can be achieved. In order to achieve this effect, the present disclosure introduces a position regression model, which can handle various complex backgrounds, shapes and occlusions through a large amount of data training, and can learn the complex features of the building, so that a set of position offsets can be predicted, and the building image area data is adjusted according to the predicted position offsets, thereby achieving high-precision rectangular frame correction to improve the accuracy of the position adjustment processing of the building image area data.

[0058] Step 104: Generate an initial three-dimensional regional model corresponding to the target area according to the outline information of each building.

[0059] In some embodiments, the execution entity may generate an initial regional three-dimensional model corresponding to the target area according to the outline information of each building. The initial regional three-dimensional model may be a three-dimensional model including each building in the target area. In practice, the execution entity may generate an initial regional three-dimensional model corresponding to the target area according to the outline information of each building through a three-dimensional modeling tool. For example, the three-dimensional modeling tool may be, but is not limited to, one of the following: Maya, 3Dmax, Zbrush.

[0060] Step 105, mapping the initial region three-dimensional model according to the mapping information set to obtain a target three-dimensional model.

[0061] In some embodiments, the execution subject may perform mapping processing on the initial region 3D model according to the mapping information set to obtain a target 3D model. The target 3D model may be the initial region 3D model after mapping. In practice, the execution subject may use the Substance Painter tool to perform mapping processing on the initial region 3D model according to the mapping information set to obtain a target 3D model.

[0062] Step 106 , extracting a point cloud from the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model.

[0063] In some embodiments, the execution entity may perform point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model. The point cloud data set may be individual point cloud data extracted from the target three-dimensional model. The point cloud data in the point cloud data set may be point cloud three-dimensional coordinates in a world coordinate system. In practice, the execution entity may use a point cloud extraction tool to perform point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model. The point cloud extraction tool may be Meshlab.

[0064] Step 107, performing building segmentation processing on the point cloud data set to obtain a building point cloud data set.

[0065] In some embodiments, the execution subject may perform building segmentation processing on the point cloud data set to obtain a building point cloud data set. The building point cloud data set may be a point cloud data set after removing ground point cloud data and background point cloud data. In practice, the execution subject may perform building segmentation processing on the point cloud data set to obtain a building point cloud data set in various ways.

[0066] In some optional implementations of some embodiments, the execution subject may perform building segmentation processing on the point cloud data set to obtain a building point cloud data set through the following steps: The first step is to perform ground segmentation on the above point cloud data set to obtain a ground segmented point cloud data set. In practice, the above execution entity may delete each point cloud data whose vertical axis coordinates in the point cloud data set are less than or equal to a preset height threshold from the above point cloud data set to obtain a ground segmented point cloud data set. Here, the specific setting of the above preset height threshold is not limited.

[0067] The second step is to perform building segmentation on the point cloud data set after ground segmentation to obtain a building point cloud data set. In practice, the above-mentioned execution subject can use a segmentation algorithm based on region growing to perform building segmentation on the point cloud data set after ground segmentation to obtain a building point cloud data set.

[0068] Step 108, segmenting the building point cloud data set into individual units to obtain individual image information of each building.

[0069] In some embodiments, the execution subject may perform monomer segmentation on the building point cloud data set to obtain image information of each building unit. The image information of each building unit may represent each building unit included in the building image information. In practice, the execution subject may adopt various methods to perform monomer segmentation on the building point cloud data set to obtain image information of each building unit.

[0070] In some optional implementations of some embodiments, the execution subject may perform monomer segmentation on the building point cloud data set to obtain monomer image information of each building through the following steps: The first step is to perform noise reduction processing on the above-mentioned building point cloud data set to obtain the building point cloud data set after noise reduction processing. In practice, the above-mentioned execution subject can use a filtering algorithm to perform noise reduction processing on the above-mentioned building point cloud data set to obtain the building point cloud data set after noise reduction processing. Among them, the above-mentioned filtering algorithm can be but is not limited to one of the following: statistical filtering, Gaussian filtering, median filtering, bilateral filtering.

[0071] In the second step, point cloud clustering is performed on the building point cloud data set after noise reduction processing to obtain each point cloud data cluster. The above-mentioned each point cloud data cluster can be each cluster after clustering. In practice, the above-mentioned execution subject can use a clustering algorithm to perform point cloud clustering processing on the building point cloud data set after noise reduction processing to obtain each point cloud data cluster. The above-mentioned clustering algorithm can be but not limited to one of the following: K-means clustering, hierarchical clustering, DBSCAN clustering.

[0072] The third step is to generate confidence score information of each building corresponding to each point cloud data cluster according to the above-mentioned point cloud data clusters. Among them, the building confidence score information in the above-mentioned building confidence score information can be the probability that the point cloud data cluster belongs to the building point cloud data. In practice, first, the above-mentioned execution subject can extract the shape features of the above-mentioned each point cloud data cluster to obtain each shape feature information. Then, the above-mentioned each shape feature information can be input into the pre-trained building classification model to obtain each building probability as each building confidence score information. Among them, the shape feature information in the above-mentioned each shape feature information can characterize the shape of the point cloud data cluster. The above-mentioned building classification model can be a logistic regression model trained with the shape feature information corresponding to the above-mentioned point cloud data cluster as input and the building probability corresponding to the above-mentioned point cloud data cluster as the expected output.

[0073] In the fourth step, each point cloud data cluster corresponding to each building confidence score information that meets the preset confidence score condition in the above-mentioned building confidence score information is determined as each target point cloud data cluster. Among them, the above-mentioned preset confidence score condition can be that the building confidence score information is greater than a preset confidence score threshold. Here, the specific setting of the above-mentioned preset confidence score threshold is not limited.

[0074] In the fifth step, clustering and merging the above target point cloud data clusters are performed to obtain merged point cloud data clusters. The merged point cloud data clusters may be clusters obtained after clustering and merging the above target point cloud data clusters.

[0075] In the sixth step, each of the above-mentioned merged point cloud data clusters is subjected to boundary correction processing, and each of the merged point cloud data clusters after boundary correction processing is obtained as each standard point cloud data cluster. In practice, the above-mentioned execution subject can use a morphological corrosion method to perform boundary correction processing on each of the above-mentioned merged point cloud data clusters, and each of the merged point cloud data clusters after boundary correction processing is obtained as each standard point cloud data cluster.

[0076] In the seventh step, an identifier is assigned to each of the above-mentioned standard point cloud data clusters to obtain each identifier information, wherein the identifier information in the above-mentioned each identifier information can uniquely identify the standard point cloud data cluster.

[0077] In the eighth step, the above-mentioned standard point cloud data clusters are visualized to obtain the images of each building. Among them, the building images in the above-mentioned building images can correspond one-to-one to the identification information in the above-mentioned identification information. The above-mentioned building images can be monomer images of each building included in the above-mentioned target three-dimensional model. In practice, the above-mentioned execution entity can use the PCL (Point Cloud Library) tool to visualize the above-mentioned merged point cloud data clusters to obtain the images of each building.

[0078] In the ninth step, for each of the building images, the building image and the identification information corresponding to the building image in the identification information are determined as the building single body image information.

[0079] In some optional implementations of some embodiments, the execution subject may perform shape feature extraction on the above-mentioned point cloud data clusters to obtain various shape feature information through the following steps: In the first step, the execution subject may use a minimum bounding box algorithm to determine each minimum bounding box of each point cloud data cluster, wherein the minimum bounding box of each minimum bounding box may include a length, a width and a height.

[0080] In the second step, for each of the above minimum bounding boxes, perform the following steps: In the first sub-step, the ratio of the maximum value to the minimum value of the length in the minimum bounding box is determined as the length stretch information.

[0081] In the second sub-step, the ratio of the maximum value to the minimum value of the width in the minimum bounding box is determined as the width stretching information.

[0082] In the third sub-step, the ratio of the maximum value to the minimum value of the height in the above minimum bounding box is determined as the height stretch information.

[0083] In a fourth sub-step, the length stretch information, the width stretch information and the height stretch information are determined as stretch information.

[0084] The fifth sub-step is to determine the ratio of the height of the minimum bounding box to the bottom area of ​​the minimum bounding box as the flatness information. The bottom area of ​​the minimum bounding box may be the product of the length and width of the minimum bounding box.

[0085] The sixth sub-step is to determine the above-mentioned stretch information and the above-mentioned flatness information as shape feature information.

[0086] Next, when the present disclosure is used to generate the confidence score information of each building corresponding to each of the above-mentioned point cloud data clusters, there is further the following technical problem three: Only the geometric features of the point cloud data cluster are used to perform logistic regression on the point cloud data cluster to obtain the probability that the point cloud data cluster belongs to the building. Other features of the point cloud data cluster are not integrated, resulting in poor accuracy of the probability that the obtained cloud data cluster belongs to the building.

[0087] In some optional implementations of some embodiments, the execution subject may generate confidence score information of each building corresponding to each point cloud data cluster according to each point cloud data cluster through the following steps: In the first step, the number of point cloud data in each of the above-mentioned merged point cloud data clusters is determined as point cloud quantity information to obtain each point cloud quantity information. The point cloud quantity information in each of the above-mentioned point cloud quantity information can correspond one-to-one to the merged point cloud data clusters in the above-mentioned merged point cloud data clusters.

[0088] The second step is to extract shape features from each of the above point cloud data clusters to obtain each shape feature information. The shape feature information in each of the above shape feature information can correspond one-to-one to the point cloud data clusters in each of the above point cloud data clusters. Here, the specific steps of extracting shape features from each of the above point cloud data clusters to obtain each shape feature information can refer to the steps of extracting shape features from each of the above point cloud data clusters to obtain each shape feature information in step 108, which will not be repeated here.

[0089] The third step is to determine the point cloud density of each point cloud data cluster in each of the above point cloud data clusters to obtain each point cloud density information. The point cloud density information in each of the above point cloud density information can correspond to the point cloud data cluster in each of the above point cloud data clusters one by one. The above point cloud density information can characterize the distribution density of each point cloud of each of the above point cloud data clusters.

[0090] The fourth step is to determine the elevation distribution information of each point cloud data cluster in the above-mentioned point cloud data clusters to obtain the elevation distribution information of each point cloud. Among them, the point cloud elevation distribution information in the above-mentioned each point cloud elevation distribution information can correspond one-to-one with the point cloud data cluster in the above-mentioned each point cloud data cluster. The point cloud elevation distribution information in the above-mentioned each point cloud elevation distribution information can be the average value of the vertical axis coordinates of each point cloud data in the point cloud data cluster. In practice, for each point cloud data cluster in the above-mentioned each point cloud data cluster, the above-mentioned execution entity can determine the average value of the vertical axis coordinates of each point cloud data in the above-mentioned point cloud data cluster as the point cloud elevation distribution information to obtain the elevation distribution information of each point cloud.

[0091] In the fifth step, for each of the above point cloud data clusters, extract the flatness feature of the above point cloud data cluster to obtain each flatness feature information. The flatness feature information in the above each flatness feature information can represent the flatness of the point cloud data cluster.

[0092] The sixth step is to determine the symmetry features of each point cloud data cluster in the above-mentioned point cloud data clusters to obtain each symmetry feature information. Among them, the symmetry feature information in the above-mentioned each symmetry feature information can characterize the probability of the point cloud data cluster having rotational symmetry. In practice, the above-mentioned execution entity can input each point cloud data cluster in the above-mentioned each point cloud data cluster into a pre-trained point cloud cluster symmetry feature extraction model to obtain each symmetry feature information. The above-mentioned point cloud cluster symmetry feature extraction model can be a neural network model trained with point cloud data clusters as input and the symmetry feature information of the corresponding point cloud data clusters as the expected output. For example, the neural network model can be a convolutional neural network model.

[0093] In the seventh step, the surface roughness information of each point cloud data cluster in the above-mentioned point cloud data clusters is determined to obtain each surface roughness information. The surface roughness information in the above-mentioned surface roughness information may be the roughness of the surface represented by the point cloud data cluster.

[0094] Step 8: For each point cloud data cluster in the above point cloud data clusters, perform the following steps: In the first sub-step, the point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information corresponding to the above point cloud data clusters are normalized to obtain the normalized point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information. In practice, the above execution entity can use the Z-score normalization method to normalize the point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information corresponding to the above point cloud data clusters.

[0095] In the second sub-step, the normalized point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information are input into a pre-trained building confidence scoring model to obtain the building confidence score information corresponding to the above point cloud data clustering. The above building confidence scoring model can be a logistic regression model trained with the point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information of the point cloud data clustering as input and the building confidence score information corresponding to the above point cloud data clustering as the expected output.

[0096] The first to eighth steps of the above technical solution and their related contents serve as an inventive point of the embodiments of the present disclosure, and solve the above technical problem three "only the geometric features of the point cloud data cluster are used to perform logistic regression on the point cloud data cluster to obtain the probability that the point cloud data cluster belongs to the building, without integrating other features of the point cloud data cluster, resulting in poor accuracy of the obtained probability that the cloud data cluster belongs to the building". The factors that lead to the poor accuracy of the obtained probability that the cloud data cluster belongs to the building are often as follows: only the geometric features of the point cloud data cluster are used to perform logistic regression on the point cloud data cluster to obtain the probability that the point cloud data cluster belongs to the building, without integrating other features of the point cloud data cluster. If the above factors are solved, the effect of improving the accuracy of the obtained probability that the cloud data cluster belongs to the building can be achieved. In order to achieve this effect, the present disclosure introduces the point cloud quantity information, shape feature information, point cloud density information, point cloud elevation distribution information, flatness feature information, symmetry feature information and surface roughness information of the point cloud data cluster, and inputs these features together into the pre-trained logistic regression model to obtain the probability that the cloud data cluster belongs to the building. Not only the geometric characteristics of the point cloud data cluster are considered, but also the point cloud quantity information, point cloud density information, point cloud elevation distribution information, symmetry feature information and surface roughness information, which can improve the accuracy of building probability prediction for point cloud data clusters, thereby improving the accuracy of the probability that the obtained cloud data cluster belongs to a building.

[0097] In some optional implementations of some embodiments, the execution subject may determine the point cloud density of each point cloud data cluster in the above-mentioned point cloud data clusters through the following steps to obtain each point cloud density information: For each of the above point cloud data clusters, perform the following steps: In the first step, for each point cloud data in the above point cloud data clustering, perform the following steps: In a first sub-step, the distance between the point cloud data and each point cloud data in the point cloud data cluster that is different from the point cloud data is determined as each point cloud distance.

[0098] In the second sub-step, the smallest point cloud distance among the above point cloud distances is determined as the target distance.

[0099] In the second step, the average value of the obtained distances to each target is determined as the point cloud density information corresponding to the above point cloud data clustering.

[0100] In some optional implementations of some embodiments, the execution subject may perform cluster merging processing on the above-mentioned target point cloud data clusters through the following steps to obtain each merged point cloud data cluster: In the first step, for each target point cloud data cluster in the above target point cloud data clusters, perform the following steps: In the first sub-step, the average value of the vertical axis coordinates of each target point cloud data in the above target point cloud data cluster is determined as the average elevation information.

[0101] In the second sub-step, the average coordinate value of each target point cloud data in the above target point cloud data cluster is determined as the center point coordinate information.

[0102] The third sub-step is to extract the principal axis of the target point cloud data clusters by using a principal component analysis tool to obtain the cluster principal axis as cluster direction information, wherein the cluster direction information may be a cluster principal axis direction vector.

[0103] In the second step, the above target point cloud data are clustered into initial clusters.

[0104] In the third step, based on each initial clustering, perform the following iterative steps: In the first iteration step, any initial cluster is selected from each initial cluster as the target initial cluster.

[0105] In the second iterative step, each initial cluster different from the above initial cluster is determined as each initial cluster to be processed.

[0106] In the third iterative step, for each of the above-mentioned initial clusters to be processed, the similarity information between the above-mentioned initial clusters to be processed and the above-mentioned initial clusters is determined according to the average elevation information, center point coordinate information, cluster direction information corresponding to the above-mentioned initial clusters to be processed and the average elevation information, center point coordinate information, cluster direction information corresponding to the above-mentioned initial clusters, to obtain each piece of similarity information. Among them, the similarity information in the above-mentioned similarity information can represent the probability that the above-mentioned initial cluster to be processed and the above-mentioned target initial cluster belong to the same building.

[0107] In the fourth iteration step, each of the initial clusters to be processed corresponding to each of the similarity information satisfying the preset similarity condition among the obtained similarity information is determined as each of the initial clusters to be merged. The preset similarity condition may be that the similarity information is greater than a preset similarity threshold. Here, the specific setting of the preset similarity threshold is not limited.

[0108] In the fifth iterative step, the target initial cluster and each of the initial clusters to be merged are combined into a merged point cloud data cluster.

[0109] In the sixth iterative step, the above-mentioned initial clusters to be merged are deleted from the initial clusters to update the initial clusters to obtain updated initial clusters.

[0110] In the seventh iterative step, the merged point cloud data cluster is added as an initial cluster to each updated initial cluster, and each updated initial cluster is used as each initial cluster, and the iterative steps are performed again.

[0111] Optionally, in response to determining that the similarity information between any two initial clusters in each initial cluster does not satisfy the above-mentioned preset similarity condition, the above-mentioned execution subject may determine each obtained updated initial cluster as each merged point cloud data cluster.

[0112] In some optional implementations of some embodiments, for each point cloud data cluster in the above-mentioned point cloud data clusters, the above-mentioned execution subject may extract flatness features for the above-mentioned point cloud data clusters through the following steps to obtain each flatness feature information: In the first step, a plane fitting is performed on the point cloud data clusters according to a plane fitting algorithm to obtain the best fitting plane corresponding to the point cloud data clusters. The best fitting plane may be a plane equation.

[0113] In the second step, the distance from each point cloud data in each point cloud data in the point cloud data clustering to the best fitting plane information is determined as a distance residual value to obtain each distance residual value.

[0114] The third step is to determine the average of the above distance residual values ​​as the average residual value.

[0115] The fourth step is to determine the arithmetic square root of the average of the square values ​​of the above distance residual values ​​as the standard residual value.

[0116] In the fifth step, the average residual value and the standard residual value are determined as flatness feature information.

[0117] In some optional implementations of some embodiments, the execution subject may determine the surface roughness information of each point cloud data cluster in the above-mentioned point cloud data clusters through the following steps to obtain each surface roughness information: For each of the above point cloud data clusters, perform the following steps: In the first step, for each point cloud data in each point cloud data in the above point cloud data clustering, perform the following steps: In the first sub-step, each point cloud data within a preset neighborhood range of the point cloud data in the point cloud data clustering is determined as each neighborhood point cloud data. The preset neighborhood range may be a circular area with the coordinates of the point cloud data as the center coordinate and a preset radius value as the radius.

[0118] The second sub-step is to perform surface fitting on each of the above-mentioned neighborhood point cloud data according to a surface fitting algorithm to obtain surface information. The surface fitting algorithm may be a least squares method or an implicit surface method. The surface information may be a surface expression.

[0119] In a third sub-step, the normal vector of the surface information is divided by the modulus of the normal vector of the surface information to obtain a normalized normal vector corresponding to the point cloud data.

[0120] In the second step, for each point cloud data in the above point cloud data clustering, perform the following steps: In the first sub-step, a set of adjacent point cloud data corresponding to the point cloud data is determined according to the K-nearest neighbor method.

[0121] In the second sub-step, the angle between the normalized normal vector corresponding to the above point cloud data and the normalized normal vector of each adjacent point cloud data in the above adjacent point cloud data set is determined as the point cloud difference value to obtain each point cloud difference value.

[0122] In a third sub-step, an average value of the above-mentioned point cloud difference values ​​is determined as the roughness value corresponding to the above-mentioned point cloud data.

[0123] In the third step, the average value of each roughness value obtained is determined as the surface roughness information corresponding to the above point cloud data clustering.

[0124] Optionally, after step 108, the execution subject may also perform three-dimensional reconstruction on the obtained image information of each building to obtain a three-dimensional model of the building in the target area. The three-dimensional model of the building in the target area may be a high-precision three-dimensional model of urban buildings. In practice, the execution subject may use a three-dimensional reconstruction tool to perform three-dimensional reconstruction on the obtained image information of each building to obtain a three-dimensional model of the building in the target area. The three-dimensional reconstruction tool may be, but is not limited to, one of the following: Meshroom, MicMac, OpenMVG.

[0125] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the building monomer image segmentation method of some embodiments of the present disclosure, the equipment cost of urban modeling is reduced, and the accuracy of urban modeling and the accuracy of monomer segmentation of building images are improved. Specifically, the reasons for the high equipment cost of urban modeling, the low accuracy of urban modeling and the low accuracy of monomer segmentation of building images are: for some complex models, geometric modeling may require a lot of time and manpower to manually create and adjust the shape; scanning modeling requires the use of high-quality scanners, resulting in high equipment costs for modeling; fractal modeling is based on mathematical rules, involves a lot of mathematical calculations, and cannot accurately control the shape and size of the model, resulting in a long time for modeling; when the building is high, the bottom is blocked by other objects, resulting in low imaging accuracy of the drone oblique image, resulting in low accuracy of the generated three-dimensional model, and the need to configure a drone, resulting in high equipment costs for modeling. The monomerized model generated by vector cutting monomerization may have obvious jagged and irregular shapes at the bottom of the edge, resulting in low accuracy of the monomer model of the cut building. Physical monomerization extracts the base of the building and performs monomerization processing, but for buildings with irregular shapes or complex structures, the accuracy of monomer segmentation is poor. Based on this, the building monomer image segmentation method of some embodiments of the present disclosure first obtains the remote sensing image data and the mapping information set of the target area. Thus, the remote sensing image data and the mapping information set of the target area can be obtained. Then, the remote sensing image data is preprocessed to obtain the preprocessed remote sensing image data, thereby obtaining the preprocessed remote sensing image data, and the noise in the remote sensing image data can be removed. Afterwards, the preprocessed remote sensing image data is extracted to obtain the contour information of each building. Thus, the contour information of each building included in the remote sensing image data can be obtained. Afterwards, according to the contour information of each building, an initial regional three-dimensional model corresponding to the target area is generated. Thus, the initial regional three-dimensional model corresponding to the target area can be obtained. Then, according to the mapping information set, the initial regional three-dimensional model is mapped to obtain the target three-dimensional model. Thus, the target three-dimensional model can be obtained. Among them, the target three-dimensional model can be the initial regional three-dimensional model after mapping. Afterwards, it is configured to extract point cloud from the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model. Thus, the point cloud data corresponding to the above target three-dimensional model can be obtained. Next, the above point cloud data set is subjected to building segmentation processing to obtain a building point cloud data set. Thus, a building point cloud data set after building segmentation processing can be obtained. Among them, the above building point cloud data set can be a point cloud data set after removing ground point cloud data and background point cloud data. Finally, the above building point cloud data set is subjected to monomer segmentation to obtain the image information of each building monomer.Thus, the image information of each building monomer can be obtained. Among them, the above-mentioned image information of each building monomer can represent each building monomer included in the above-mentioned building image information. Also because the realized building monomer image segmentation method can generate a high-precision three-dimensional model of the target area building based on the remote sensing image data and mapping information set of the target area, and extract the point cloud data of the target area building three-dimensional model, perform building point cloud segmentation based on the point cloud data, and then perform fine-grained monomer segmentation on the segmented building cloud data to obtain the image information of each building monomer. Since the remote sensing image data has low acquisition cost, wide coverage, high resolution, and does not require a large amount of complex calculations, the modeling accuracy is high, the time consumption is short, and more manual participation is not required. The building monomer segmentation based on point cloud data can accurately capture the surface morphology and details of the building, thereby improving the accuracy of monomer segmentation of the building image.

[0126] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a building single image segmentation device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0127] like Figure 2 As shown, the building monomer image segmentation device 200 of some embodiments includes: an acquisition unit 201, a preprocessing unit 202, a first extraction unit 203, a generation unit 204, a mapping unit 205, a second extraction unit 206, a first segmentation unit 207 and a second segmentation unit 208. Among them, the acquisition unit 201 is configured to acquire the remote sensing image data and mapping information set of the target area; the preprocessing unit 202 is configured to preprocess the above remote sensing image data to obtain the preprocessed remote sensing image data; the first extraction unit 203 is configured to extract information from the preprocessed remote sensing image data to obtain the contour information of each building; the generation unit 204 is configured to generate an initial area three-dimensional model corresponding to the above target area according to the above contour information of each building; the mapping unit 205 is configured to perform mapping processing on the above initial area three-dimensional model according to the above mapping information set to obtain the target three-dimensional model; the second extraction unit 206 is configured to perform point cloud extraction on the above target three-dimensional model to obtain a point cloud data set corresponding to the above target three-dimensional model; the first segmentation unit 207 is configured to perform building segmentation processing on the above point cloud data set to obtain a building point cloud data set; the second segmentation unit 208 is configured to perform monomer segmentation on the above building point cloud data set to obtain the monomer image information of each building.

[0128] It can be understood that the units described in the building monomer image segmentation device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0129] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0130] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0131] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0132] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0133] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0134] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0135] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the remote sensing image data and mapping information set of the target area; preprocesses the remote sensing image data to obtain preprocessed remote sensing image data; extracts information from the preprocessed remote sensing image data to obtain the contour information of each building; generates an initial area three-dimensional model corresponding to the target area according to the contour information of each building; maps the initial area three-dimensional model according to the mapping information set to obtain the target three-dimensional model; extracts point cloud from the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model; performs building segmentation on the point cloud data set to obtain a building point cloud data set; performs monomer segmentation on the building point cloud data set to obtain monomer image information of each building.

[0136] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be set in a processor, for example, it may be described as: a processor includes an acquisition unit, a preprocessing unit, a first extraction unit, a generation unit, a mapping unit, a second extraction unit, a first segmentation unit, and a second segmentation unit. The names of these units do not constitute a limitation on the units themselves in some cases. For example, the acquisition unit may also be described as a "unit for acquiring a set of remote sensing image data and mapping information of a target area."

[0139] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0140] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for segmenting a building monomer image, comprising: Obtain remote sensing image data and map information collection of the target area; Preprocessing the remote sensing image data to obtain preprocessed remote sensing image data; Extract information from the pre-processed remote sensing image data to obtain the outline information of each building; Generating an initial regional three-dimensional model corresponding to the target area according to the outline information of each building; According to the mapping information set, mapping processing is performed on the initial area three-dimensional model to obtain a target three-dimensional model; Performing point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model; Performing building segmentation processing on the point cloud data set to obtain a building point cloud data set; The building point cloud data set is segmented into individual units to obtain individual building image information.

2. The method according to claim 1, wherein: The method further comprises: The obtained image information of each building is three-dimensionally reconstructed to obtain a three-dimensional model of the building in the target area.

3. The method according to claim 1, wherein: The remote sensing image data includes surface elevation data of the target area; And extracting information from the pre-processed remote sensing image data to obtain the outline information of each building includes: Perform image enhancement on the preprocessed remote sensing image data to obtain remote sensing image enhanced data; Performing building recognition on the remote sensing image enhancement data to obtain building recognition result information; Determine the location information of each building according to the building recognition result information; Determine the elevation data of each building according to the surface elevation data of the target area and the location information of each building included in the remote sensing image data; For each piece of building location information among the pieces of building location information, the building location information and the building elevation data corresponding to the building location information among the pieces of building elevation data are determined as building outline information.

4. The method according to claim 3, wherein: The step of performing building recognition on the remote sensing image enhancement data to obtain building recognition result information includes: Performing region extraction on the remote sensing image enhancement data to obtain region data of each remote sensing image; Extracting features from the regional data of each remote sensing image to obtain feature information of each region; Classify the regional feature information to obtain classification result information; Determine each remote sensing image region data corresponding to each classification result information satisfying a preset classification condition in each classification result information as each building image region data; Performing position adjustment processing on the image area data of each building to obtain the image area data of each building after position adjustment; Performing regional redundancy elimination processing on the image area data of each building after position adjustment to obtain the data of each target image area; The target image area data are determined as building recognition result information.

5. The method according to claim 4, wherein: The step of performing position adjustment processing on the image area data of each building to obtain the image area data of each building after position adjustment includes: For each building image region data in the respective building image region data, the following steps are performed: Inputting the regional feature information corresponding to the building image region data into a preset position regression model to obtain the position offset information corresponding to the building image region data; The building image region data is regionally updated according to the position offset information to obtain the building image region data after position adjustment.

6. The method according to claim 1, wherein: The step of performing building segmentation processing on the point cloud data set to obtain a building point cloud data set includes: Performing ground segmentation on the point cloud data set to obtain a ground segmented point cloud data set; The point cloud data set after ground segmentation is segmented into buildings to obtain a building point cloud data set.

7. The method according to claim 1, wherein: The step of segmenting the building point cloud data set into individual units to obtain individual image information of each building includes: Performing noise reduction processing on the building point cloud data set to obtain a noise-reduced building point cloud data set; Performing point cloud clustering processing on the building point cloud data set after noise reduction processing to obtain each point cloud data cluster; Generating confidence score information of each building corresponding to each point cloud data cluster according to each point cloud data cluster; Determining each point cloud data cluster corresponding to each building confidence score information satisfying a preset confidence score condition in the each building confidence score information as each target point cloud data cluster; Performing cluster merging processing on each of the target point cloud data clusters to obtain each merged point cloud data cluster; Performing boundary correction processing on each of the merged point cloud data clusters to obtain each of the merged point cloud data clusters after the boundary correction processing as each of the standard point cloud data clusters; Assigning an identifier to each of the standard point cloud data clusters to obtain respective identifier information; Performing visualization processing on the clustering of each standard point cloud data to obtain each building image, wherein the building images in each building image correspond one-to-one to the identification information in each identification information; For each of the building images, the building image and the identification information corresponding to the building image in the identification information are determined as the building single image information.

8. A building monomer image segmentation device, comprising: An acquisition unit is configured to acquire remote sensing image data and a map information set of a target area; A preprocessing unit is configured to preprocess the remote sensing image data to obtain preprocessed remote sensing image data; A first extraction unit is configured to extract information from the pre-processed remote sensing image data to obtain the outline information of each building; A generating unit, configured to generate an initial area three-dimensional model corresponding to the target area according to the outline information of each building; A mapping unit is configured to perform mapping processing on the initial area three-dimensional model according to the mapping information set to obtain a target three-dimensional model; A second extraction unit is configured to perform point cloud extraction on the target three-dimensional model to obtain a point cloud data set corresponding to the target three-dimensional model; A first segmentation unit is configured to perform building segmentation processing on the point cloud data set to obtain a building point cloud data set; The second segmentation unit is configured to perform monomer segmentation on the building point cloud data set to obtain monomer image information of each building.

9. An electronic device, comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images