Building boundary identification method based on live-action three-dimensional technology and multi-source fusion data

Abstract

The embodiment of the invention discloses a building boundary identification method based on a live-action three-dimensional technology and multi-source fusion data. One specific embodiment of the method comprises the following steps: carrying out shielding object identification on each initial building three-dimensional image in an initial building three-dimensional image set; building segmentation is carried out on the building area remote sensing image; extracting a building object feature information set corresponding to each building segmentation object included in the building segmentation image; according to the building object feature information set and the building three-dimensional image set, performing live-action three-dimensional modeling on the building area to be subjected to right registration; performing point cloud extraction on the live-action three-dimensional building model; and according to the building point cloud data set, extracting boundary information of each building in the building area to be subjected to right registration. According to the embodiment, the three-dimensional image of the sheltered building can be collected again, so that the accuracy of boundary right confirmation is ensured, the accuracy of building boundary extraction of right confirmation registration is improved, and the error of building right confirmation registration is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of building boundary recognition, and specifically to a building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data. Background Art

[0002] At present, the unified registration and confirmation of natural resources is being carried out on a large scale across the country. Natural resource registration is mainly carried out through field data collection, internal data analysis and field data verification. From the perspective of natural resource registration data, most of them are based on plane map patches, most of which only have simple plane remote sensing maps, and the analysis function of three-dimensional spatial levels is relatively limited. In addition, since some low buildings are easily blocked (for example, by trees), if only plane remote sensing maps are used to confirm the boundary level of buildings, it is easy to cause errors in building registration. Summary of the invention

[0003] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present application propose a building boundary recognition method, a computer device and a computer-readable storage medium based on real-scene three-dimensional technology and multi-source fusion data to solve the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present application provide a method for identifying building boundaries based on real-life 3D technology and multi-source fusion data, the method comprising: obtaining multi-source fusion data of a building area corresponding to a building area to be registered for property rights confirmation, wherein the multi-source fusion data of the building area comprises: a remote sensing image of a building area corresponding to the building area to be registered for property rights confirmation and an initial 3D image set of buildings; performing occlusion identification on each initial 3D image of a building in the initial 3D image set to generate an occlusion identification result to obtain an occlusion identification result set; in response to determining that there is an occlusion identification result that satisfies an occlusion condition in the occlusion identification result set, re-collecting the building 3D image corresponding to the occlusion identification result that satisfies the occlusion condition, and updating the initial 3D image set of buildings to obtain an updated initial 3D image of a building. An image set is used as a three-dimensional image set of buildings; using a pre-trained building matching model, the remote sensing image of the above-mentioned building area is segmented to obtain a building segmentation image; the building object feature information set corresponding to each building segmentation object included in the above-mentioned building segmentation image is extracted, wherein one building segmentation object corresponds to one building object feature information; based on the above-mentioned building object feature information set and the above-mentioned three-dimensional building image set, real-life three-dimensional modeling is performed on the above-mentioned building area to be confirmed and registered, and a real-life three-dimensional building model corresponding to the building area to be confirmed and registered is obtained; point cloud extraction is performed on the above-mentioned real-life three-dimensional building model to obtain a building point cloud data set corresponding to the above-mentioned real-life three-dimensional building model; based on the above-mentioned building point cloud data set, the boundary information of each building in the above-mentioned building area to be confirmed and registered is extracted.

[0006] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method described in any implementation of the first aspect is implemented.

[0007] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0008] The above-mentioned embodiments of the present application have the following beneficial effects: through the building boundary recognition method based on real scene three-dimensional technology and multi-source fusion data of some embodiments of the present application, the accuracy of building boundary extraction for real estate rights confirmation and registration is improved, and the error of building rights confirmation and registration is reduced. First, the multi-source fusion data of the building area corresponding to the building area to be confirmed and registered is obtained, wherein the multi-source fusion data of the building area includes: the remote sensing image of the building area corresponding to the building area to be confirmed and registered and the initial building three-dimensional image set. Thus, data support is provided for establishing a real scene three-dimensional building model of the building area to be confirmed and registered. Secondly, each initial building three-dimensional image in the above-mentioned initial building three-dimensional image set is subjected to occlusion recognition to generate an occlusion recognition result, and obtain an occlusion recognition result set; in response to determining that there is an occlusion recognition result that meets the occlusion condition in the above-mentioned occlusion recognition result set, the building three-dimensional image corresponding to the occlusion recognition result that meets the occlusion condition is re-collected, and the initial building three-dimensional image set is updated to obtain the updated initial building three-dimensional image set as the building three-dimensional image set. Thus, the three-dimensional image of the occluded building can be re-collected to ensure the accuracy of boundary confirmation. Next, the building area remote sensing image is segmented by using the pre-trained building matching model to obtain a building segmentation image; thus, each building in the building area remote sensing image can be segmented and marked. Thus, it is convenient to determine the boundary of each building. Afterwards, the building object feature information set corresponding to each building segmentation object included in the building segmentation image is extracted, wherein one building segmentation object corresponds to one building object feature information. Thus, the outline (building object feature information) of each building can be extracted. Then, according to the building object feature information set and the building three-dimensional image set, the real-scene three-dimensional modeling of the building area to be confirmed and registered is performed to obtain a real-scene three-dimensional building model corresponding to the building area to be confirmed and registered. Thus, a real-scene three-dimensional building model corresponding to the building area to be confirmed and registered can be established, which is convenient for extracting the boundary of each building. Finally, the real-scene three-dimensional building model is extracted from the real-scene three-dimensional building model to obtain a building point cloud data set corresponding to the real-scene three-dimensional building model; according to the building point cloud data set, the boundary information of each building in the building area to be confirmed and registered is extracted. Thus, the accuracy of building boundary extraction for real estate confirmation and registration is improved, and the error of building confirmation and registration is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0010] Figure 1 It is a flowchart of some embodiments of the building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data according to the present application; Figure 2 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0011] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0012] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0013] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0014] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0015] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0016] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0017] Figure 1 The process 100 of some embodiments of the method for identifying building boundaries based on real-scene 3D technology and multi-source fusion data according to the present application is shown. The method for identifying building boundaries based on real-scene 3D technology and multi-source fusion data includes the following steps: Step 101, obtaining multi-source fusion data of a building area corresponding to the building area to be registered for property rights confirmation.

[0018] In some embodiments, the execution subject (e.g., a computing device) of the building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data can obtain multi-source fusion data of the building area corresponding to the building area to be confirmed and registered. The multi-source fusion data of the building area includes: the remote sensing image of the building area corresponding to the building area to be confirmed and registered and the initial three-dimensional image set of the building. The building area to be confirmed and registered may refer to the area where real estate confirmation and registration is required, including each building that needs real estate confirmation and registration. The building may be a residential building, a commercial building, a gymnasium, or other buildings. The remote sensing image of the building area may be an image and data information of the building area to be confirmed and registered obtained by remote sensing technology. The data information may be the reflection, radiation, or scattering information of the urban surface. The initial three-dimensional image set of the building may be each three-dimensional on-site photo obtained by taking on-site photos of each building in the building area to be confirmed and registered, or each three-dimensional image obtained by retouching each three-dimensional on-site photo. The retouching of each three-dimensional on-site photo may include cropping, adjusting resolution, cutting out, etc. For example, the initial three-dimensional image set of the building may be a three-dimensional image of each building in the building area to be confirmed and registered taken by a drone. For another example, the initial building three-dimensional image set may be a three-dimensional image of each building in the building area to be registered and confirmed, taken by a 3D camera.

[0019] Step 102 : performing occlusion recognition on each initial building 3D image in the initial building 3D image set to generate an occlusion recognition result, thereby obtaining an occlusion recognition result set.

[0020] In some embodiments, the execution subject may perform occlusion recognition on each initial building 3D image in the initial building 3D image set to generate an occlusion recognition result and obtain an occlusion recognition result set. Occlusion recognition may refer to identifying whether a building in the initial building 3D image is occluded. For example, a 3D image of a building taken by a drone may be occluded by trees. For example, an occlusion recognition model may be used to perform occlusion recognition on each initial building 3D image in the initial building 3D image set to generate an occlusion recognition result and obtain an occlusion recognition result set. For example, the occlusion recognition model may be a pre-trained neural network model that takes the initial building 3D image as input and takes the occlusion recognition result as output. For example, the occlusion recognition model may be a convolutional neural network (CNN), a recurrent neural network (RNN), a support vector machine (SVM), or a YOLOv3 model that has been trained to recognize occlusions. The occlusion recognition result may indicate whether the building in the initial building 3D image is occluded.

[0021] Step 103, in response to determining that there are occlusion identification results that meet the occlusion conditions in the above-mentioned occlusion identification result set, re-collect the building three-dimensional images corresponding to the occlusion identification results that meet the occlusion conditions, and update the initial building three-dimensional image set to obtain an updated initial building three-dimensional image set as the building three-dimensional image set.

[0022] In some embodiments, the execution subject may, in response to determining that there is an occlusion recognition result that satisfies the occlusion condition in the occlusion recognition result set, re-collect the three-dimensional image of the building corresponding to the occlusion recognition result that satisfies the occlusion condition, and update the initial three-dimensional building image set to obtain an updated initial three-dimensional building image set as the building three-dimensional image set. The occlusion condition may be: the occlusion recognition result indicates that the building in the initial three-dimensional building image is occluded. That is, for the three-dimensional image of the building corresponding to the occlusion recognition result that satisfies the occlusion condition, the associated three-dimensional image acquisition device (3D camera or three-dimensional image camera device) may be controlled to re-collect the unobstructed three-dimensional image of the corresponding building. Afterwards, the initial three-dimensional building image of the corresponding building in the initial three-dimensional building image set is replaced with the re-collected three-dimensional image to obtain an updated initial three-dimensional building image set as the building three-dimensional image set.

[0023] Step 104 , using the pre-trained building matching model, perform building segmentation on the remote sensing image of the building area to obtain a building segmentation image.

[0024] In some embodiments, the above-mentioned execution entity can use a pre-trained building matching model to perform building segmentation on the above-mentioned building area remote sensing image to obtain a building segmentation image. The building matching model can be a pre-trained neural network model that takes the building area remote sensing image as input and the building segmentation image as output. The building segmentation image may refer to a remote sensing image after identifying and marking each building in the building area remote sensing image. For example, the building matching model can be a Segment Anything Model (SAM) model, a DINOv2 model, or a Mask2Former model. For another example, the building matching model can be a CNN convolutional neural network model or a target detection model such as YOLO, SSD, FastR-CNN, NanoDet, etc.

[0025] In an actual application scenario, the execution subject can perform building segmentation on the remote sensing image of the building area through the following steps to obtain a building segmentation image: The first step is to preprocess the remote sensing image of the building area to obtain a preprocessed remote sensing image of the building area. For example, first, the execution entity may use a radiation correction tool, radiometric correction, to perform radiation calibration on the remote sensing image of the building area to obtain a remote sensing image of the building area after radiation calibration. Then, a geometric correction algorithm may be used to perform geometric correction on the remote sensing image of the building area after radiation calibration to obtain a remote sensing image of the building area after geometry correction. Afterwards, a dark pixel method may be used to perform atmospheric correction on the remote sensing image of the building area after geometry correction to preprocess the remote sensing image of the building area to obtain a preprocessed remote sensing image of the building area. The geometric correction algorithm may be, but is not limited to, one of the following: an RPC model algorithm, a polynomial model algorithm.

[0026] In the second step, the pre-processed building area remote sensing image is input into a pre-trained building matching model to obtain a building segmentation image. The building matching model is a pre-trained network model for matching and identifying building images in the pre-processed building area remote sensing image and marking them.

[0027] The building matching model can be trained through the following steps: The first step is to obtain a sample set of remote sensing images of building areas. The sample remote sensing images of building areas include: sample remote sensing images of building areas and sample segmented image labels of buildings. The sample segmented image labels of buildings can represent images after segmentation and labeling of each building in the sample remote sensing images of building areas.

[0028] The second step is to determine the initial building matching model. The above-mentioned initial building matching model includes: an initial building feature extraction layer, an initial building outline feature extraction layer, an initial multi-level building image feature encoding layer, an initial building feature fusion layer and an initial building segmentation image recognition output layer. The initial building feature extraction layer can be used to extract the image features of the building area in the remote sensing image of the sample building area. The initial building feature extraction layer can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc. The initial building outline feature extraction layer can be an extraction network layer for extracting the outline of the building area in the remote sensing image of the sample building area. For example, the initial building outline feature extraction layer can be an edge detection layer. The initial residual drug image area feature extraction layer can be an extraction layer that takes the output of the initial residual drug image outline extraction layer as input and takes the residual drug image area feature as output. The initial multi-level building image feature encoding layer may include: an initial building image feature encoding layer (encoder) and an initial building image feature decoding layer (decoder), the initial building image feature encoding layer includes: an initial building image feature decoupling layer (including an autoencoder network, an LSTM network) and an initial building image feature fusion layer (VAE (VariationalAutoEncoder, variational encoder), the initial building image feature fusion layer includes: an initial fusion layer, an initial reparameterization layer. The initial fusion layer may be a Feature Pyramid Network (FPN) or a Self-Attention Mechanism network. The initial reparameterization layer may include: DyRep (DynamicRe-parameterization) and ACNet (Asymmetric Convolution Net). The initial building segmentation image recognition output layer may be an output layer (Output Layer) for outputting a building segmentation image. The initial building feature fusion layer may be a feature fusion layer (Feature Fusion).

[0029] The third step is to train the initial building matching model based on the remote sensing image sample set of the building area to obtain a trained building matching model.

[0030] The third step may include: The first step is to select a building area remote sensing image sample from the building area remote sensing image sample set as the target building area remote sensing image sample. A building area remote sensing image sample can be randomly selected from the building area remote sensing image sample set as the target building area remote sensing image sample.

[0031] In the second step, the sample building area remote sensing images included in the target building area remote sensing image samples are input into the initial building feature extraction layer to obtain the initial building features.

[0032] The third step is to input the above initial building features into the initial multi-level building image feature encoding layer to obtain the initial multi-level building image features.

[0033] The fourth step is to input the sample building area remote sensing image included in the target building area remote sensing image sample into the initial building outline feature extraction layer to obtain at least one initial building outline feature.

[0034] The fifth step is to input the above-mentioned initial multi-level building image features and the above-mentioned at least one initial building outline feature into the initial building feature fusion layer to obtain the building image fusion feature.

[0035] The sixth step is to input the above building image fusion features into the initial building segmentation image recognition output layer to obtain the initial building segmentation image.

[0036] The seventh step is to determine the loss value between the initial building segmentation image and the corresponding sample building segmentation image label. The loss value between the initial building segmentation image and the corresponding sample building segmentation image label can be determined by a damage function. The loss function can be: mean square error loss (MSE Loss) function, cross-entropy loss (Cross-Entropy Loss) function, and mean absolute error (MAE) function.

[0037] In the eighth step, in response to determining that the loss value is less than or equal to the preset loss value, the initial building matching model is determined as the trained building matching model.

[0038] Therefore, the building matching model constructed in this application can comprehensively capture the multi-level information of the building image area, enhance the efficiency and accuracy of the autoencoder in feature disentanglement, help the model capture the complex nonlinear relationship between each building in the building image area, and improve the accuracy of building matching marking.

[0039] Step 105: extracting a building object feature information set corresponding to each building segmentation object included in the building segmentation image.

[0040] In some embodiments, the execution subject may extract a set of building object feature information corresponding to each building segmentation object included in the building segmentation image. One building segmentation object corresponds to one building object feature information. For example, the building object feature information corresponding to each building segmentation object included in the building segmentation image may be extracted through a building feature extraction model. The building feature extraction model may be a pre-trained feature extraction model that takes a building segmentation image as input and takes building object feature information corresponding to each building segmentation object as output. For example, the building feature extraction model may be a U-Net model, a deep learning model combining edge supervision and feature fusion, or a fusion cascade CRFs and U-Net deep learning model.

[0041] In an actual application scenario, the execution subject may extract a building object feature information set corresponding to each building segmentation object included in the building segmentation image through the following steps: The first step is to perform image enhancement on the building segmentation image to obtain an enhanced building segmentation image. For example, the execution subject may use a filter enhancement method to perform image enhancement on the building segmentation image to obtain an enhanced building segmentation image. The filter enhancement method may be: median filtering, mean filtering, sharpening filtering.

[0042] The second step is to perform region extraction on the enhanced building segmentation image to obtain each image region data. The image region data in each image region data may represent the rectangular boundary including the building extracted from the enhanced building segmentation image. The image region data in each image region data may include corner point coordinates, region height and region width. The corner point coordinates may be the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates or lower right corner coordinates of the rectangular boundary. For example, first, the execution subject may use a corner point detection algorithm to perform corner point detection on the enhanced building segmentation image to obtain each corner point information. Then, a rectangle fitting algorithm is used to perform rectangle fitting on each corner point information to obtain image region data. The corner point detection algorithm may be: SIFT algorithm, Harris algorithm or SURF algorithm. The rectangle fitting algorithm may be: Hough transform, RANSAC algorithm.

[0043] The third step is to extract features from the above-mentioned image region data to obtain feature information of each region. The region feature information in the above-mentioned region feature information may be a feature vector of the image region data. For example, the execution subject may use a convolution operation to extract features from the above-mentioned image region data to obtain feature information of each region. Here, the specific setting of the relevant parameters of the above-mentioned convolution operation is not limited.

[0044] The fourth step is to classify the above-mentioned regional feature information to obtain various classification result information. Among them, the classification result information in the above-mentioned classification result information can represent the probability that the regional feature information includes a building. In practice, the above-mentioned execution body can use a softmax function to classify the above-mentioned regional feature information to obtain various classification result information.

[0045] In the fifth step, each image area data corresponding to each classification result information satisfying the classification condition in the above classification result information is determined as each building image area data. The above classification condition may be that the classification result information is greater than the classification result threshold. Here, there is no limitation on the specific setting of the above classification result threshold. For example, the classification result threshold may be 85%. The above building image area data may be each building image area data whose classification result information is greater than the classification result threshold.

[0046] The sixth step is to adjust the position of the above-mentioned building image area data to obtain the building image area data after the position adjustment. For example, first, the execution body can use an edge detection algorithm to perform edge detection on the above-mentioned building image area data to obtain various edge information. The above-mentioned edge detection algorithm can be a Canny algorithm. Then, the findContours function in OpenCV can be used to extract the contours of the above-mentioned edge information to obtain various contour extraction information. Finally, the contour approximation algorithm can be used to simplify the contours of the above-mentioned contour extraction information to obtain the simplified contour extraction information as the building image area data after the position adjustment. The above-mentioned contour approximation algorithm can be a Douglas-Peucker algorithm.

[0047] In the seventh step, the image region data of each building after the position adjustment is subjected to regional redundancy elimination processing to obtain the image data of each building as the building object feature information set. The feature information of each building object may be the image region data of each building after the regional redundancy elimination processing. For example, the execution subject may adopt a non-maximum suppression algorithm to perform regional redundancy elimination processing on the image region data of each building after the position adjustment to obtain the image data of each building as the building object feature information set.

[0048] Step 106, based on the building object feature information set and the building three-dimensional image set, real-scene three-dimensional modeling is performed on the building area to be registered for property rights confirmation to obtain a real-scene three-dimensional building model corresponding to the building area to be registered for property rights confirmation.

[0049] In some embodiments, the execution subject may perform real-life three-dimensional modeling on the building area to be confirmed and registered based on the building object feature information set and the building three-dimensional image set to obtain a real-life three-dimensional building model corresponding to the building area to be confirmed and registered. Among them, real-life three-dimensional modeling may refer to establishing three-dimensional model images (3D images) of each building corresponding to the building area to be confirmed and registered. For example, first, a three-dimensional modeling tool may be used to construct an initial real-life three-dimensional building model corresponding to the building area to be confirmed and registered based on the building object feature information set. For example, the three-dimensional modeling tool may be: Maya, 3Dmax, Zbrush. Then, the execution subject may use the Substance Painter tool to perform mapping processing on the initial real-life three-dimensional building model based on the building three-dimensional image set to obtain a real-life three-dimensional building model.

[0050] In an actual application scenario, the execution entity can perform real-life 3D modeling of the building area to be registered through the following steps: The first step is to generate an initial regional three-dimensional model corresponding to the aforementioned building area to be confirmed and registered based on the aforementioned building object feature information set. For example, a three-dimensional modeling tool can be used to construct an initial regional three-dimensional model corresponding to the aforementioned building area to be confirmed and registered based on the aforementioned building object feature information set. For example, the three-dimensional modeling tool can be: Maya, 3Dmax, Zbrush.

[0051] In the second step, the initial region 3D model is filled with building images according to the 3D building image set to obtain an initial real-scene 3D building model. The Substance Painter tool can be used to fill the 3D building image set into the initial region 3D model to obtain an initial real-scene 3D building model.

[0052] The third step is to input the above-mentioned initial real-life 3D building model into the pre-trained 3D model image reconstruction model to obtain a reconstructed real-life 3D building model as the real-life 3D building model corresponding to the building area to be registered for property rights confirmation. The 3D model image reconstruction model can be a pre-trained neural network model that takes the initial real-life 3D building model as input and reconstructs the real-life 3D building model as output. For example, the 3D model image reconstruction model can be a deep convolutional neural network (ConvNet) for multi-view stereoscopic vision (MVS) reconstruction. That is, the 3D model image reconstruction model can be a DeepMVS model. For another example, the 3D model image reconstruction model can be a V-Net model, a HighResNet model, or an EfficientNet3D model.

[0053] The three-dimensional model image reconstruction model can be trained by the following steps: The first step is to obtain a sample set of real-scene 3D building model images, wherein the real-scene 3D building model image samples have corresponding sample reconstruction labels, and the sample reconstruction labels can represent the reconstructed sample real-scene 3D building model images.

[0054] The second step is to determine the initial 3D model image reconstruction model. The initial 3D model image reconstruction model includes: a first convolution layer, a multi-scale aggregation network, a second convolution layer and an upsampling layer, and the multi-scale aggregation network includes: a graph aggregation layer and an aggregation convolution layer. The first convolution layer, the second convolution layer and the aggregation convolution layer can be used for convolution processing. The upsampling layer can be used for upsampling processing. The graph aggregation layer can be used for graph aggregation processing.

[0055] As an example, the first convolution layer, the second convolution layer, and the aggregation convolution layer may be convolution network models, respectively. The upsampling layer may be an upsampling network model. The graph aggregation layer may be a GAL (Graph Attentional Layer) network model.

[0056] The third step is to select a target real 3D building model image sample from the real 3D building model image sample set. A real 3D building model image sample can be randomly selected from the real 3D building model image sample set as the target real 3D building model image sample.

[0057] In the fourth step, the real-scene 3D building model image included in the target real-scene 3D building model image sample is input into the first convolution layer to obtain a first 3D building model convolution image.

[0058] In the fifth step, the first three-dimensional building model convolution image is locally aggregated to obtain a first aggregated three-dimensional building model image. The first three-dimensional building model convolution image can be locally aggregated using a preset aggregation algorithm to obtain the first aggregated three-dimensional building model image. The preset aggregation algorithm can be a GraphClustering algorithm.

[0059] In the sixth step, the first aggregated three-dimensional building model image and the first three-dimensional building model convolution image are input into the graph aggregation layer included in the multi-scale aggregation network to obtain a second aggregated three-dimensional building model image.

[0060] In the seventh step, the second three-dimensional building model image is input into the aggregation convolution layer included in the multi-scale aggregation network to obtain an aggregated three-dimensional building model convolution image.

[0061] In the eighth step, the aggregated three-dimensional building model convolution image is input into the second convolution layer to obtain a second three-dimensional building model convolution image.

[0062] In the ninth step, the first three-dimensional building model convolution image and the second three-dimensional building model convolution image are subjected to XOR processing to obtain an initial three-dimensional building model convolution image. The XOR processing of the first three-dimensional building model convolution image and the second three-dimensional building model convolution image may be: the pixel value of each pixel point included in the first three-dimensional building model convolution image is subjected to XOR processing with the pixel value of the corresponding pixel point in the second three-dimensional building model convolution image, and each obtained pixel value is used as the pixel value of the pixel point included in the initial three-dimensional building model convolution image to obtain the initial three-dimensional building model convolution image.

[0063] In the tenth step, the initial three-dimensional building model convolution image is input into the upsampling layer to obtain the initial reconstructed real-scene three-dimensional building model image.

[0064] In the eleventh step, based on a preset image reconstruction loss function, a reconstruction loss value between the initial reconstructed real-life 3D building model image and the corresponding sample reconstruction label is determined. As an example, the above image reconstruction loss function may be: L1Loss (L1 norm loss) function, SSIM Loss (Structural Similarity Loss) function, or MS-SSIM Loss (Multi Scale Structural Similarity Index Measure Loss) function.

[0065] In the twelfth step, in response to determining that the reconstruction loss value is less than or equal to a preset threshold, the initial three-dimensional model image reconstruction model is determined as the trained three-dimensional model image reconstruction model.

[0066] Thus, the model can be reconstructed by the trained 3D model image, and the 3D model image can be convolved to distinguish the feature information of each image pixel in the 3D model image. Then, the image obtained by convolution can be locally aggregated to enhance the relative information between each pixel. Then, the locally aggregated image can be further aggregated to enhance the position information of each pixel. Subsequently, the processed image and the image before processing can be fused to further improve the clarity of the 3D model image so as to facilitate the subsequent recognition of the building boundary.

[0067] Step 107 , performing point cloud extraction on the real-scene three-dimensional building model to obtain a building point cloud data set corresponding to the real-scene three-dimensional building model.

[0068] In some embodiments, the execution subject may perform point cloud extraction on the real-scene three-dimensional building model to obtain a building point cloud data set corresponding to the real-scene three-dimensional building model. The building point cloud data set may be the point cloud data of each building extracted from the real-scene three-dimensional building model. The building point cloud data in the building point cloud data set may be the point cloud three-dimensional coordinates of the building in the world coordinate system. For example, the execution subject may use a point cloud extraction tool to perform point cloud extraction on the real-scene three-dimensional building model to obtain a point cloud data set corresponding to the real-scene three-dimensional building model. The point cloud extraction tool may be Meshlab. Afterwards, the point cloud data set may be ground segmented to obtain a point cloud data set after ground segmentation. The execution subject may delete each point cloud data whose vertical axis coordinates of the point cloud data in the point cloud data set are less than or equal to a preset height threshold from the point cloud data set to obtain a point cloud data set after ground segmentation. Next, the point cloud data set after ground segmentation is segmented into buildings to obtain a building point cloud data set. For example, the execution subject may use a segmentation algorithm based on region growing to perform building segmentation on the point cloud data set after ground segmentation to obtain a building point cloud data set.

[0069] Step 108: extracting the boundary information of each building in the area of ​​buildings to be registered and confirmed based on the building point cloud dataset.

[0070] In some embodiments, the execution subject may extract the boundary information of each building in the area of ​​the building to be registered for property rights confirmation based on the building point cloud data set. For example, the boundary box in the point cloud data of each building may be extracted by an edge detection algorithm, and then the building identification of each building may be determined. Afterwards, the boundary box of the building point cloud data and the corresponding building identification may be combined into the boundary information.

[0071] In an actual application scenario, the execution entity can extract the boundary information of each building in the area of ​​the building to be registered for property rights confirmation through the following steps: The first step is to perform noise reduction processing on the building point cloud dataset to obtain a noise-reduced building point cloud dataset. The execution subject may use a filtering algorithm to perform noise reduction processing on the building point cloud dataset to obtain a noise-reduced building point cloud dataset. The filtering algorithm may be: Gaussian filtering, bilateral filtering.

[0072] The second step is to perform point cloud clustering processing on the noise-reduced building point cloud dataset to obtain various point cloud data clusters. The above-mentioned execution subject can use a clustering algorithm to perform point cloud clustering processing on the noise-reduced building point cloud dataset to obtain various point cloud data clusters. The above-mentioned clustering algorithm can be: K-means clustering, DBSCAN clustering.

[0073] The third step is to perform boundary correction processing on each of the above point cloud data clusters, and obtain each point cloud data cluster after boundary correction processing as each standard point cloud data cluster. The above execution subject can use a morphological corrosion method to perform boundary correction processing on each of the above point cloud data clusters, and obtain each point cloud data cluster after boundary correction processing as each standard point cloud data cluster.

[0074] The fourth step is to identify the building for each of the above standard point cloud data clusters to obtain the identification information of each building. That is, the identification of each building can be obtained first, and then the standard point cloud data cluster corresponding to each building is marked with the corresponding building identification to obtain the identification information of each building.

[0075] The fifth step is to visualize the above-mentioned standard point cloud data clusters to obtain the images of each building monomer. Among them, one standard point cloud data cluster corresponds to one building monomer image. The above-mentioned building images can be monomer images of each building included in the above-mentioned real-life three-dimensional building model. For example, the above-mentioned execution subject can use the PCL (Point Cloud Library) tool to visualize the above-mentioned standard point cloud data clusters to obtain the images of each building monomer.

[0076] In step 6, for each of the above-mentioned building monomer images, the image boundary information of the above-mentioned building monomer image and the corresponding building identification information are determined as the boundary information of the building. For example, the image boundary information of each building monomer image can be detected by an edge detection algorithm. The image boundary information can include the boundary of the building and the corresponding coordinate information.

[0077] Furthermore, for each building's boundary information, the following registration steps are performed: The first step is to obtain the user information and building identification corresponding to the above building. The user information may refer to the personal information of the owner of the building, and may include: name, identity information.

[0078] The second step is to map the user information, building identification and boundary information of the building into registration and confirmation information of the building. A mapping relationship between the user information, building identification and boundary information of the building can be established.

[0079] The third step is to store the registration and confirmation information of the above-mentioned building in a related building registration and confirmation database. The building registration and confirmation database may be a pre-established database for storing the registration and confirmation information of the building. Figure 2A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a terminal.

[0080] like Figure 2 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0081] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data.

[0082] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0083] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data.

[0084] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0086] In one embodiment, the processor is used to run a computer program stored in a memory to implement the following steps: obtaining multi-source fusion data of a building area corresponding to the building area to be registered for property rights confirmation, wherein the multi-source fusion data of the building area includes: a remote sensing image of the building area corresponding to the building area to be registered for property rights confirmation and an initial three-dimensional building image set; performing occlusion identification on each initial three-dimensional building image in the initial three-dimensional building image set to generate an occlusion identification result to obtain an occlusion identification result set; in response to determining that there is an occlusion identification result that meets the occlusion condition in the occlusion identification result set, re-collecting the three-dimensional building image corresponding to the occlusion identification result that meets the occlusion condition, and updating the initial three-dimensional building image set to obtain an updated initial three-dimensional building image set. as a three-dimensional image set of buildings; using a pre-trained building matching model to perform building segmentation on the remote sensing image of the above-mentioned building area to obtain a building segmentation image; extracting a building object feature information set corresponding to each building segmentation object included in the above-mentioned building segmentation image, wherein one building segmentation object corresponds to one building object feature information; based on the above-mentioned building object feature information set and the above-mentioned three-dimensional building image set, performing real-life three-dimensional modeling of the above-mentioned building area to be confirmed and registered, to obtain a real-life three-dimensional building model corresponding to the building area to be confirmed and registered; performing point cloud extraction on the above-mentioned real-life three-dimensional building model to obtain a building point cloud data set corresponding to the above-mentioned real-life three-dimensional building model; based on the above-mentioned building point cloud data set, extracting the boundary information of each building in the above-mentioned building area to be confirmed and registered.

[0087] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the building boundary recognition method based on real-scene three-dimensional technology and multi-source fusion data of the present application.

[0088] The computer-readable storage medium may be an internal storage unit of the computer device in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., provided on the computer device.

[0089] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0090] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A building boundary recognition method based on real-scene 3D technology and multi-source fusion data, characterized in that: include: Acquire multi-source fusion data of a building area corresponding to the building area to be registered for property rights confirmation, wherein the multi-source fusion data of the building area includes: a remote sensing image of the building area corresponding to the building area to be registered for property rights confirmation and an initial three-dimensional image set of the building; Performing occlusion recognition on each initial building three-dimensional image in the initial building three-dimensional image set to generate an occlusion recognition result, thereby obtaining an occlusion recognition result set; In response to determining that there is an occlusion recognition result that satisfies the occlusion condition in the occlusion recognition result set, re-collecting the three-dimensional building image corresponding to the occlusion recognition result that satisfies the occlusion condition, and updating the initial three-dimensional building image set to obtain an updated initial three-dimensional building image set as the three-dimensional building image set; Using a pre-trained building matching model, the remote sensing image of the building area is segmented to obtain a building segmentation image; Extracting a building object feature information set corresponding to each building segmentation object included in the building segmentation image, wherein one building segmentation object corresponds to one building object feature information; According to the building object feature information set and the building three-dimensional image set, real-scene three-dimensional modeling is performed on the building area to be registered for property rights confirmation to obtain a real-scene three-dimensional building model corresponding to the building area to be registered for property rights confirmation; Performing point cloud extraction on the real-scene three-dimensional building model to obtain a building point cloud data set corresponding to the real-scene three-dimensional building model; Based on the building point cloud data set, the boundary information of each building in the building area to be registered for property rights confirmation is extracted.

2. The building boundary recognition method based on real scene three-dimensional technology and multi-source fusion data according to claim 1 is characterized in that: The method further comprises: For each building's boundary information, perform the following registration steps: Obtaining user information and building identification corresponding to the building; Mapping the user information, the building identification and the boundary information of the building into the registration and confirmation information of the building; The registration and title confirmation information of the building is stored in the associated building registration and title confirmation database.

3. The building boundary recognition method based on real scene three-dimensional technology and multi-source fusion data according to claim 1 is characterized in that: The method of using the pre-trained building matching model to segment the building area remote sensing image to obtain a building segmentation image includes: Preprocessing the remote sensing image of the building area to obtain a preprocessed remote sensing image of the building area; The pre-processed building area remote sensing image is input into a pre-trained building matching model to obtain a building segmentation image, wherein the building matching model is a pre-trained network model for matching and identifying building images in the pre-processed building area remote sensing image and marking them.

4. The building boundary recognition method based on real scene three-dimensional technology and multi-source fusion data according to claim 1 is characterized in that: The method of performing real-scene three-dimensional modeling on the building area to be registered for property rights confirmation according to the building object feature information set and the building three-dimensional image set to obtain a real-scene three-dimensional building model corresponding to the building area to be registered for property rights confirmation includes: Generating an initial regional three-dimensional model corresponding to the building area to be registered and confirmed according to the building object feature information set; According to the building three-dimensional image set, the initial area three-dimensional model is filled with building images to obtain an initial real-scene three-dimensional building model; The initial real-scene 3D building model is input into a pre-trained 3D model image reconstruction model to obtain a reconstructed real-scene 3D building model as the real-scene 3D building model corresponding to the building area to be registered and confirmed.

5. The building boundary recognition method based on real scene 3D technology and multi-source fusion data according to claim 1 is characterized in that: The step of extracting a building object feature information set corresponding to each building segmentation object included in the building segmentation image comprises: Performing image enhancement on the building segmentation image to obtain an enhanced building segmentation image; Performing region extraction on the enhanced building segmentation image to obtain data of each image region; Extracting features from the image region data to obtain feature information of each region; Classify the regional feature information to obtain classification result information; Determine each image region data corresponding to each classification result information satisfying the classification condition in each classification result information as each building image region data; Performing position adjustment processing on the image region data of each building to obtain the image region data of each building after position adjustment; The image region data of each building after position adjustment is subjected to regional redundancy elimination processing to obtain the image data of each building as a building object feature information set.

6. A computer device, characterized in that: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images