Building detection method, device and equipment based on remote sensing image and laser point cloud

Abstract

The embodiment of the invention discloses a building detection method, device and equipment based on a remote sensing image and a laser point cloud. A specific embodiment of the method comprises the following steps: carrying out single building segmentation processing on a building remote sensing image; for each building segmentation remote sensing image in the building segmentation remote sensing image group, the following processing steps are executed: an image edge node coordinate sequence and surface elevation information of the building segmentation remote sensing image are determined, and each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image; extracting single building point cloud data corresponding to the building segmentation remote sensing image from the laser point cloud information; and generating building detection information according to the single building point cloud data and the corresponding standard building three-dimensional data. According to the embodiment, the building area edge detection precision and the building detection efficiency are improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the computer field, and specifically to a building detection method, device, and equipment based on remote sensing images and laser point clouds. Background Art

[0002] Building detection (e.g., settlement detection, edge detection) is usually carried out by performing settlement detection using a building deformation detection device, manually drawing the vector outline of the building edge, and then correcting the projection deviation based on experience and measurement to determine the actual edge position of the building.

[0003] However, the above-mentioned building detection method usually has the following technical problems: the building deformation detection device needs to be updated and detected through the building deformation detection device, and the deformation detection device needs to be installed on-site, which is a cumbersome process; the vector outline of the building edge is manually drawn, resulting in inaccurate actual edge position of the building.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a building detection method, device, electronic device and computer-readable medium based on remote sensing images and laser point clouds to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a building detection method based on remote sensing images and laser point clouds, the method comprising: collecting remote sensing images and laser point cloud information corresponding to a target building area; preprocessing the remote sensing images to generate preprocessed remote sensing images; inputting the preprocessed remote sensing images into a pre-trained remote sensing image object edge detection model to obtain remote sensing image object edge results, wherein the remote sensing image object edge results represent a building edge detection frame in the preprocessed remote sensing images; extracting a building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing images; performing single building segmentation processing on the building remote sensing images to obtain a building segmentation remote sensing image group; For each building segmentation remote sensing image in the above-mentioned building segmentation remote sensing image group, the following processing steps are performed: determine the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmentation remote sensing image, wherein each edge node coordinate in the above-mentioned edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the above-mentioned building segmentation remote sensing image; according to the above-mentioned image edge node coordinate sequence and the above-mentioned surface elevation information, extract the single building point cloud data corresponding to the above-mentioned building segmentation remote sensing image from the above-mentioned laser point cloud information; generate building detection information according to the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data.

[0008] In a second aspect, some embodiments of the present disclosure provide a building detection device based on remote sensing images and laser point clouds, the device comprising: an acquisition unit, configured to acquire remote sensing images and laser point cloud information corresponding to a target building area; a preprocessing unit, configured to preprocess the remote sensing images to generate preprocessed remote sensing images; an input unit, configured to input the preprocessed remote sensing images into a pre-trained remote sensing image object edge detection model to obtain remote sensing image object edge results, wherein the remote sensing image object edge results represent a building edge detection frame in the preprocessed remote sensing images; an extraction unit, configured to extract the building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image; a segmentation unit, configured to segment the building remote sensing image into individual buildings. Segmentation processing is performed to obtain a group of building segmented remote sensing images; the detection unit is configured to perform the following processing steps for each building segmented remote sensing image in the above-mentioned building segmented remote sensing image group: determine the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmented remote sensing image, wherein each edge node coordinate in the above-mentioned edge node coordinate sequence represents a key node of the building segmented remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the above-mentioned building segmented remote sensing image; according to the above-mentioned image edge node coordinate sequence and the above-mentioned surface elevation information, extract the single building point cloud data corresponding to the above-mentioned building segmented remote sensing image from the above-mentioned laser point cloud information; according to the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data, generate building detection information.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the building detection method based on remote sensing images and laser point clouds of some embodiments of the present disclosure, the accuracy of building area edge detection and the efficiency of building settlement detection are improved. First, collect remote sensing images and laser point cloud information corresponding to the target building area; preprocess the above remote sensing images to generate preprocessed remote sensing images. Thus, a preprocessed remote sensing image can be obtained, and the noise in the remote sensing image can be removed. Secondly, the above preprocessed remote sensing image is input into a pre-trained remote sensing image ground object edge detection model to obtain a remote sensing image ground object edge result, wherein the above remote sensing image ground object edge result represents the building edge detection frame in the above preprocessed remote sensing image. Thus, it is convenient to extract the building in the remote sensing image. Next, extract the building remote sensing image corresponding to the building edge detection frame in the above preprocessed remote sensing image. Then, the above building remote sensing image is segmented into a single building to obtain a building segmentation remote sensing image group. Thus, it is convenient to detect each single building. Finally, for each building segmentation remote sensing image in the above-mentioned building segmentation remote sensing image group, the following processing steps are performed: determine the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmentation remote sensing image, wherein each edge node coordinate in the above-mentioned edge node coordinate sequence represents the key node of the building segmentation remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the above-mentioned building segmentation remote sensing image; according to the above-mentioned image edge node coordinate sequence and the above-mentioned surface elevation information, extract the single building point cloud data corresponding to the above-mentioned building segmentation remote sensing image from the above-mentioned laser point cloud information; according to the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data, generate building detection information. Since the remote sensing image data has a low acquisition cost, a wide coverage range, a high resolution, and does not require a large amount of complex calculations, the modeling accuracy is high, the time consumption is short, and it does not require much manual participation. In addition, since the building detection is performed through the point cloud, the accuracy of the building area edge detection and the efficiency of the building settlement detection are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of a building detection method based on remote sensing images and laser point clouds according to the present disclosure;

[0014] Figure 2It is a schematic diagram of the structure of some embodiments of the building detection device based on remote sensing images and laser point clouds according to the present disclosure;

[0015] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Figure 1 1 is a flowchart of some embodiments of the building detection method based on remote sensing images and laser point clouds according to the present disclosure. 100 of some embodiments of the building detection method based on remote sensing images and laser point clouds according to the present disclosure is shown. The building detection method based on remote sensing images and laser point clouds includes the following steps:

[0023] Step 101, collect remote sensing images and laser point cloud information corresponding to the target building area.

[0024] In some embodiments, the execution subject (e.g., computing device) of the building detection method based on remote sensing images and laser point clouds can collect remote sensing images and laser point cloud information corresponding to the target building area. The remote sensing image of the target building area can be collected by remote sensing technology / satellite technology. The laser point cloud information of the target building area can be collected by a 3D laser point cloud camera. The target building area can be an area where changes in building settlement need to be detected, and can include a target building. For example, the target building can be a teaching building / laboratory building.

[0025] Step 102: preprocess the remote sensing image to generate a preprocessed remote sensing image.

[0026] In some embodiments, the execution entity may preprocess the remote sensing image to generate a preprocessed remote sensing image. In practice, first, the execution entity may use a radiation correction tool, radiometric correction, to perform radiation calibration on the remote sensing image to obtain a radiation calibrated remote sensing image. Then, a geometric correction algorithm may be used to perform geometric correction on the radiation calibrated remote sensing image to obtain a corrected remote sensing image. After that, a dark pixel method may be used to perform atmospheric correction on the corrected remote sensing image to obtain a preprocessed remote sensing image. For example, the geometric correction algorithm may be an RPC model algorithm.

[0027] Step 103: input the pre-processed remote sensing image into a pre-trained remote sensing image object edge detection model to obtain a remote sensing image object edge result.

[0028] In some embodiments, the execution entity may input the preprocessed remote sensing image into a pre-trained remote sensing image object edge detection model to obtain a remote sensing image object edge result, wherein the remote sensing image object edge result represents the building edge detection frame in the preprocessed remote sensing image. The remote sensing image object edge detection model may be a pre-trained neural network model that takes the preprocessed remote sensing image as input and the remote sensing image object edge result as output. For example, the remote sensing image object edge detection model may be: HED (Holistically-Nested Edge Detection) model, RCF (Richer Convolutional Features) model, FD-RCF (Fully Dilated-RCF) model.

[0029] The remote sensing image object edge detection model can be trained by the following steps:

[0030] The first step is to obtain a set of building remote sensing images, which can be remote sensing images of various buildings collected historically.

[0031] In the second step, image enhancement processing is performed on each building remote sensing image in the above building remote sensing image set to generate an enhanced building remote sensing image and obtain an enhanced building remote sensing image set.

[0032] The second step may include the following sub-steps:

[0033] The first sub-step is to determine the histogram information of the remote sensing image of the building. The histogram information may represent the histogram of the histogram information of the remote sensing image. In practice, the execution subject may determine the histogram information of the histogram information of the remote sensing image through the cv2.calcHist() function in the OpenCV library.

[0034] The second sub-step is to perform equalization processing on the above histogram information to obtain equalized building histogram information.

[0035] The third sub-step is to perform grayscale value mapping processing on the equalized building histogram information to obtain the grayscale level corresponding to the remote sensing image of the above building.

[0036] The fourth sub-step is to determine the threshold information corresponding to the remote sensing image of the building according to the grayscale level and the cropping factor information. The threshold information can represent the histogram cropping threshold. In practice, first, the execution entity can determine the ratio of the number of pixels of the remote sensing image of the building to the grayscale level as the target ratio. Then, the product of the target ratio and the cropping factor information is determined as the threshold information. The cropping factor information can be a pre-set cropping probability.

[0037] The fifth sub-step is to update the building remote sensing image according to the threshold information to obtain an updated building remote sensing image. The execution subject can perform cropping on the building remote sensing image according to the threshold information by using a threshold segmentation algorithm to obtain a cropped building remote sensing image. Then, an interpolation algorithm can be used to perform pixel allocation processing on each pixel in the cropped building remote sensing image to obtain a pixel allocated building remote sensing image as an updated building remote sensing image.

[0038] The sixth sub-step is to perform histogram equalization processing on the updated building remote sensing image to obtain an enhanced building remote sensing image.

[0039] The third step is to annotate each enhanced building remote sensing image in the enhanced building remote sensing image set to generate an annotated enhanced building remote sensing image set as a building remote sensing image sample set. For example, the edge frame of the building in each enhanced building remote sensing image can be annotated.

[0040] The fourth step is to determine the initial remote sensing image object edge detection model. The initial remote sensing image object edge detection model includes: an initial remote sensing image object edge feature extraction layer, an initial multi-level object edge feature encoding layer, and an initial remote sensing image object edge frame output layer. For example, the initial remote sensing image object edge feature extraction layer can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc.

[0041] The fourth step may include the following sub-steps:

[0042] The first sub-step is to determine the initial remote sensing image object edge feature extraction layer.

[0043] The second sub-step is to determine the initial multi-level feature edge encoding layer. The initial multi-level feature edge encoding layer includes: an initial feature edge encoding layer (encoder) and an initial feature edge decoding layer (decoder), the initial feature edge encoding layer includes: an initial feature edge feature decoupling layer (including autoencoder network, LSTM network) and an initial feature edge feature fusion layer (VAE (Variational AutoEncoder, variational encoder), the initial feature edge feature fusion layer includes: an initial fusion layer, an initial re-parameterization layer. The initial fusion layer can be a feature pyramid network (Feature PyramidNetwork, FPN) or a self-attention mechanism (Self-Attention Mechanism) network. The initial re-parameterization layer can include: DyRep (Dynamic Re-parameterization) and ACNet (Asymmetric ConvolutionNet).

[0044] The third sub-step is to determine the initial remote sensing image ground object edge frame output layer. The initial remote sensing image ground object edge frame output layer can be used to output the output layer (Output Layer) of the remote sensing image ground object edge frame.

[0045] In the fourth sub-step, the initial remote sensing image object edge feature extraction layer, the initial multi-level object edge feature encoding layer and the initial remote sensing image object edge frame output layer are fused to generate an initial remote sensing image object edge detection model.

[0046] Step 5: Select a target building remote sensing image sample from the above building remote sensing image sample set. A building remote sensing image sample can be randomly selected from the above building remote sensing image sample set as the target building remote sensing image sample.

[0047] The sixth step is to train the initial remote sensing image object edge detection model according to the remote sensing image samples of the target building to obtain a trained remote sensing image object edge detection model.

[0048] The sixth step may include the following sub-steps:

[0049] In the first sub-step, the sample building remote sensing images included in the target building remote sensing image samples are input into the initial remote sensing image ground object edge feature extraction layer to obtain the initial remote sensing image ground object edge features.

[0050] In the second sub-step, the initial remote sensing image object edge features are input into the initial multi-level object edge feature coding layer to obtain the initial multi-level remote sensing image object edge features.

[0051] In the third sub-step, the initial multi-layer remote sensing image object edge features are input into the initial remote sensing image object edge frame output layer to obtain the initial remote sensing image object edge frame detection result.

[0052] The fourth sub-step is to determine the loss value between the initial remote sensing image object edge frame detection result and the corresponding sample label based on a preset loss function. The loss function can be: mean square error loss (MSE Loss) function, cross-entropy loss (Cross-Entropy Loss) function, mean absolute error (MAE) function.

[0053] In a fifth sub-step, in response to determining that the loss value is less than or equal to a preset loss value, the initial remote sensing image object edge detection model is determined as a trained remote sensing image object edge detection model.

[0054] Therefore, the remote sensing image object edge detection model constructed by the present invention can fully capture the multi-level information of the remote sensing image object edge and enhance the efficiency and accuracy of the autoencoder in feature unwrapping. In addition, an initial remote sensing image object edge frame output layer is constructed to help the model output the complex nonlinear relationship in the object edge and improve the accuracy of object edge detection.

[0055] Step 104: extracting the building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image.

[0056] In some embodiments, the execution subject may extract the building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image. For example, the building remote sensing image within the building edge detection frame in the preprocessed remote sensing image may be extracted.

[0057] Step 105, performing a single-building segmentation process on the building remote sensing image to obtain a group of building segmentation remote sensing images.

[0058] In some embodiments, the execution entity may perform a single-building segmentation process on the building remote sensing image to obtain a building segmentation remote sensing image group.

[0059] In practice, the execution subject may perform single building segmentation processing on the building remote sensing image through the following steps:

[0060] The first step is to perform initial region segmentation on the remote sensing image of the building to obtain a remote sensing region image group. The remote sensing region image can represent the rectangular boundaries of each building extracted from the remote sensing image of the building. Each remote sensing region image can correspond to corner point coordinates, region height and region width. The corner point coordinates can be the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates or lower right corner coordinates of the rectangular boundary. In practice, first, the execution entity can use a corner point detection algorithm to perform corner point detection on the remote sensing region image to obtain information on each corner point. Then, a rectangle fitting algorithm is used to perform rectangle fitting on each corner point information to obtain a remote sensing region image. The corner point detection algorithm can be: SIFT algorithm, Harris algorithm or SURF algorithm. The rectangle fitting algorithm can be: Hough transform, RANSAC algorithm.

[0061] The second step is to extract regional image features from each remote sensing regional image in the remote sensing regional image group to generate remote sensing regional features and obtain a remote sensing regional feature group. The remote sensing regional features may be feature vectors of the remote sensing regional images. In practice, the execution subject may use a CNN convolutional network to extract regional image features from each remote sensing regional image in the remote sensing regional image group.

[0062] The third step is to perform building recognition processing on the above remote sensing area feature group to obtain a building recognition result group. Among them, one remote sensing area feature corresponds to one building recognition result. Each remote sensing area feature can be input into a pre-trained building recognition model to obtain a building recognition result group. For example, the building recognition model can be a pre-trained neural network model that takes remote sensing area features as input and takes building recognition results as output. For example, the building recognition model can be an FCN (Fully Convolutional Networks) model. The building recognition results can be:

[0063] In the fourth step, each remote sensing area image corresponding to each building recognition result that meets the preset recognition condition in the above building recognition result group is determined as a building remote sensing area image group.

[0064] The fifth step is to adjust the position of each building remote sensing area image to generate a position-adjusted building remote sensing area image and obtain a position-adjusted building remote sensing area image group. For example, the execution subject may use an edge detection algorithm to perform edge detection on each of the building remote sensing area images to obtain edge information. The edge detection algorithm may be a Canny algorithm. Then, the findContours function in OpenCV may be used to extract the contours of each of the edge information to obtain contour extraction information. Finally, a contour approximation algorithm may be used to simplify the contours of each of the contour extraction information to obtain each simplified contour extraction information as a position-adjusted building remote sensing area image group. The contour approximation algorithm may be a Douglas-Peucker algorithm.

[0065] The sixth step is to perform regional redundancy elimination processing on the remote sensing area images of the buildings adjusted at various positions, and obtain the remote sensing area images of the buildings adjusted at various positions after redundancy elimination as the building segmentation remote sensing image group. The non-maximum suppression algorithm can be used to perform regional redundancy elimination processing on the remote sensing area images of the buildings adjusted at various positions, and obtain the remote sensing area images of the buildings adjusted at various positions after redundancy elimination as the building segmentation remote sensing image group.

[0066] Step 106, for each building segmentation remote sensing image in the above building segmentation remote sensing image group, perform the following processing steps:

[0067] Step 1061, determining the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmentation remote sensing image.

[0068] In some embodiments, the execution subject may determine the image edge node coordinate sequence and surface elevation information of the building segmented remote sensing image. Each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmented remote sensing image, and the image formed by sequentially connecting each edge node coordinate represents the contour boundary of the building segmented remote sensing image. The surface elevation information may be the heights of each point in the target building area relative to a specific vertical reference. The image edge node coordinates may refer to the latitude and longitude coordinates of each key node of the building segmented remote sensing image in the map.

[0069] Step 1062: extracting the single building point cloud data corresponding to the segmented remote sensing image of the building from the laser point cloud information according to the image edge node coordinate sequence and the surface elevation information.

[0070] In some embodiments, the execution entity may extract the point cloud data of the individual buildings corresponding to the segmented remote sensing image of the building from the laser point cloud information according to the image edge node coordinate sequence and the surface elevation information.

[0071] In practice, the execution subject may extract the single building point cloud data corresponding to the segmented remote sensing image of the building from the laser point cloud information through the following steps:

[0072] The first step is to perform edge segmentation on each point cloud data in the above laser point cloud information according to the above image edge node coordinate sequence to obtain a ground point cloud data set corresponding to the above image edge node coordinate sequence. For example, point cloud data with the same plane coordinates (XY coordinates in the point cloud) as the image edge node coordinates can be found in each point cloud data in the laser point cloud information, and the point cloud data in each point cloud data can be extracted to obtain a ground point cloud data set corresponding to the above image edge node coordinate sequence. For example, the point cloud data in each point cloud data can be extracted through a point cloud extraction tool (Meshlab) to obtain a ground point cloud data set corresponding to the above image edge node coordinate sequence. For another example, a segmentation algorithm based on region growing can be used to extract the point cloud data in each point cloud data to obtain a ground point cloud data set corresponding to the above image edge node coordinate sequence.

[0073] The second step is to perform noise reduction processing on the ground point cloud dataset to obtain a noise-reduced ground point cloud dataset. For example, the ground point cloud dataset can be subjected to noise reduction processing by a filtering algorithm to obtain a noise-reduced ground point cloud dataset. The filtering algorithm can be: Gaussian filtering, median filtering.

[0074] The third step is to perform point cloud clustering processing on the denoised ground point cloud dataset to obtain point cloud cluster groups. A clustering algorithm can be used to perform point cloud clustering processing on the denoised ground point cloud dataset to obtain point cloud cluster groups. The above clustering algorithm can be: K-means clustering, DBSCAN clustering.

[0075] The third step is to perform boundary correction processing on the above point cloud cluster group to obtain a boundary-corrected point cloud cluster group. A morphological erosion method can be used to perform boundary correction processing on the above point cloud cluster group to obtain a boundary-corrected point cloud cluster group.

[0076] Step 4: For each boundary-corrected point cloud cluster in the above boundary-corrected point cloud cluster group, perform the following processing steps:

[0077] 1. From each surface elevation data in the surface elevation information, each surface elevation data corresponding to the above-mentioned boundary correction point cloud cluster is intercepted as a target surface elevation data group.

[0078] 2. According to the target surface elevation data set, the boundary correction point cloud cluster is de-redundanted to obtain a de-redundant boundary correction point cloud cluster. For example, point cloud data whose vertical axis coordinates are greater than the corresponding target surface elevation data in the boundary correction point cloud cluster can be removed.

[0079] The fifth step is to fuse each redundant boundary-corrected point cloud cluster into the single building point cloud data corresponding to the above-mentioned building segmentation remote sensing image.

[0080] Step 1063: Generate building detection information based on the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data.

[0081] In some embodiments, the above-mentioned execution entity can generate building detection information based on the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data. The standard building three-dimensional data can be a three-dimensional model of a building constructed in a 1:1 ratio. First, the point cloud data of the single building can be visualized to obtain a visualized three-dimensional image of the building. The PCL (Point Cloud Library) tool can be used to visualize the point cloud data of the single building to obtain a visualized three-dimensional image of the building. Determine the error between each edge of the visualized building three-dimensional image and the corresponding edge in the standard building three-dimensional data. Finally, merge the various errors into building detection information. Thereby, it can be determined whether the building has undergone settlement changes.

[0082] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a building detection device based on remote sensing images and laser point clouds. These embodiments of building detection devices based on remote sensing images and laser point clouds are similar to Figure 1 Corresponding to the method embodiments shown, the building detection device based on remote sensing images and laser point clouds can be specifically applied to various electronic devices.

[0083] like Figure 2As shown, some embodiments of the building detection device 200 based on remote sensing images and laser point clouds include: an acquisition unit 201, a preprocessing unit 202, an input unit 203, an extraction unit 204, a segmentation unit 205 and a detection unit 206. The acquisition unit 201 is configured to acquire remote sensing images and laser point cloud information corresponding to the target building area; the preprocessing unit 202 is configured to preprocess the remote sensing images to generate preprocessed remote sensing images; the input unit 203 is configured to input the preprocessed remote sensing images into a pre-trained remote sensing image object edge detection model to obtain a remote sensing image object edge result, wherein the remote sensing image object edge result represents a building edge detection frame in the preprocessed remote sensing image; the extraction unit 204 is configured to extract the building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image; the segmentation unit 205 is configured to perform single building segmentation processing on the building remote sensing image to obtain a building segmentation remote sensing image. Image group; the detection unit 206 is configured to perform the following processing steps for each building segmentation remote sensing image in the above-mentioned building segmentation remote sensing image group: determine the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmentation remote sensing image, wherein each edge node coordinate in the above-mentioned edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the above-mentioned building segmentation remote sensing image; according to the above-mentioned image edge node coordinate sequence and the above-mentioned surface elevation information, extract the single building point cloud data corresponding to the above-mentioned building segmentation remote sensing image from the above-mentioned laser point cloud information; generate building detection information according to the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data.

[0084] It can be understood that the units recorded in the building detection device 200 based on remote sensing images and laser point clouds are similar to the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the building detection device 200 based on remote sensing images and laser point clouds and the units contained therein, and will not be described in detail here.

[0085] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0086] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0087] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0088] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0089] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0090] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0091] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: collects remote sensing images and laser point cloud information corresponding to the target building area; preprocesses the above-mentioned remote sensing images to generate preprocessed remote sensing images; inputs the above-mentioned preprocessed remote sensing images into a pre-trained remote sensing image object edge detection model to obtain remote sensing image object edge results, wherein the above-mentioned remote sensing image object edge results represent the building edge detection frame in the above-mentioned preprocessed remote sensing image; extracts the building remote sensing image corresponding to the above-mentioned building edge detection frame in the above-mentioned preprocessed remote sensing image; performs single-building segmentation processing on the above-mentioned building remote sensing image to obtain a building segmentation remote sensing image. group; for each building segmentation remote sensing image in the above-mentioned building segmentation remote sensing image group, the following processing steps are performed: determining the image edge node coordinate sequence and surface elevation information of the above-mentioned building segmentation remote sensing image, wherein each edge node coordinate in the above-mentioned edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the above-mentioned building segmentation remote sensing image; according to the above-mentioned image edge node coordinate sequence and the above-mentioned surface elevation information, extracting the single building point cloud data corresponding to the above-mentioned building segmentation remote sensing image from the above-mentioned laser point cloud information; generating building detection information according to the above-mentioned single building point cloud data and the corresponding standard building three-dimensional data.

[0092] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0094] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The described units may also be set in a processor, for example, it may be described as: a processor includes: an acquisition unit, a preprocessing unit, an input unit, an extraction unit, a segmentation unit and a detection unit. The names of these units do not constitute a limitation on the units themselves in some cases. For example, the acquisition unit may also be described as a "unit for acquiring remote sensing images and laser point cloud information corresponding to the target building area."

[0095] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0096] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A building detection method based on remote sensing images and laser point clouds, comprising: Collect remote sensing images and laser point cloud information corresponding to the target building area; Preprocessing the remote sensing image to generate a preprocessed remote sensing image; Inputting the pre-processed remote sensing image into a pre-trained remote sensing image object edge detection model to obtain a remote sensing image object edge result, wherein the remote sensing image object edge result represents a building edge detection frame in the pre-processed remote sensing image; Extracting a building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image; Performing a single-building segmentation process on the building remote sensing image to obtain a building segmentation remote sensing image group; For each building segmentation remote sensing image in the building segmentation remote sensing image group, the following processing steps are performed: Determine the image edge node coordinate sequence and surface elevation information of the building segmented remote sensing image, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmented remote sensing image, and an image surrounded by each edge node coordinate connected in sequence represents a contour boundary of the building segmented remote sensing image; Extracting the single building point cloud data corresponding to the segmented remote sensing image of the building from the laser point cloud information according to the image edge node coordinate sequence and the surface elevation information; Building detection information is generated based on the point cloud data of the single building and the corresponding three-dimensional data of the standard building.

2. The method according to claim 1, wherein: Before inputting the pre-processed remote sensing image into a pre-trained remote sensing image object edge detection model to obtain a remote sensing image object edge result, the method further includes: Obtain a set of remote sensing images of buildings; Performing image enhancement processing on each building remote sensing image in the building remote sensing image set to generate an enhanced building remote sensing image, thereby obtaining an enhanced building remote sensing image set; Annotating each enhanced building remote sensing image in the enhanced building remote sensing image set to generate an annotated enhanced building remote sensing image set as a building remote sensing image sample set; Determine an initial remote sensing image ground object edge detection model, wherein the initial remote sensing image ground object edge detection model includes: an initial remote sensing image ground object edge feature extraction layer, an initial multi-level ground object edge feature encoding layer and an initial remote sensing image ground object edge frame output layer; Selecting a target person building remote sensing image sample from the building remote sensing image sample set; According to the remote sensing image samples of the target building, the initial remote sensing image ground object edge detection model is trained to obtain a trained remote sensing image ground object edge detection model.

3. The method according to claim 2, wherein: The performing image enhancement processing on each building remote sensing image in the building remote sensing image set to generate an enhanced building remote sensing image comprises: Determining histogram information of the remote sensing image of the building; Performing equalization processing on the histogram information to obtain equalized building histogram information; Performing grayscale value mapping processing on the equalized building histogram information to obtain a grayscale level corresponding to the remote sensing image of the building; Determining threshold information corresponding to the remote sensing image of the building according to the grayscale level and the cropping factor information; According to the threshold information, the remote sensing image of the building is updated to obtain an updated remote sensing image of the building; The updated building remote sensing image is processed with histogram equalization to obtain an enhanced building remote sensing image.

4. The method according to claim 1, wherein: The step of extracting the single building point cloud data corresponding to the segmented remote sensing image of the building from the laser point cloud information according to the image edge node coordinate sequence and the surface elevation information comprises: According to the image edge node coordinate sequence, edge segmentation is performed on each point cloud data in the laser point cloud information to obtain a ground point cloud data set corresponding to the image edge node coordinate sequence; Performing denoising on the ground point cloud dataset to obtain a denoised ground point cloud dataset; Perform point cloud clustering processing on the denoised ground point cloud dataset to obtain point cloud cluster groups; Performing boundary correction processing on the point cloud cluster group to obtain a boundary-corrected point cloud cluster group; For each boundary-corrected point cloud cluster in the boundary-corrected point cloud cluster group, the following processing steps are performed: Extracting, from each surface elevation data in the surface elevation information, each surface elevation data corresponding to the boundary-corrected point cloud cluster as a target surface elevation data group; According to the target surface elevation data group, the boundary correction point cloud cluster is subjected to redundancy elimination processing to obtain a redundancy eliminated boundary correction point cloud cluster; Each redundant boundary-corrected point cloud cluster is fused into single-building point cloud data corresponding to the building segmentation remote sensing image.

5. A building detection device based on remote sensing images and laser point clouds, comprising: A collection unit is configured to collect remote sensing images and laser point cloud information corresponding to a target building area; A preprocessing unit, configured to preprocess the remote sensing image to generate a preprocessed remote sensing image; An input unit is configured to input the pre-processed remote sensing image into a pre-trained remote sensing image ground object edge detection model to obtain a remote sensing image ground object edge result, wherein the remote sensing image ground object edge result represents a building edge detection frame in the pre-processed remote sensing image; An extraction unit is configured to extract a building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image; A segmentation unit is configured to perform a single building segmentation process on the building remote sensing image to obtain a building segmentation remote sensing image group; The detection unit is configured to perform the following processing steps for each building segmentation remote sensing image in the building segmentation remote sensing image group: determine the image edge node coordinate sequence and surface elevation information of the building segmentation remote sensing image, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image surrounded by each edge node coordinate is sequentially connected to represent the contour boundary of the building segmentation remote sensing image; extract the single building point cloud data corresponding to the building segmentation remote sensing image from the laser point cloud information according to the image edge node coordinate sequence and the surface elevation information; generate building detection information according to the single building point cloud data and the corresponding standard building three-dimensional data.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images