Building detection method, device and equipment based on remote sensing image and laser point cloud

Abstract

Embodiments of the present disclosure disclose a building detection method and device based on remote sensing images and laser point clouds. A specific embodiment of the method includes: performing single building segmentation processing on a building remote sensing image; for each building segmentation remote sensing image in a building segmentation remote sensing image group, performing the following processing steps: determining image edge node coordinate sequences and ground elevation information of the building segmentation remote sensing image, wherein each edge node coordinate in the edge node coordinate sequences represents a key node of the building segmentation remote sensing image; extracting single building point cloud data corresponding to the building segmentation remote sensing image from laser point cloud information; and generating building detection information according to the single building point cloud data and corresponding standard building three-dimensional data. The embodiment improves the accuracy of building area edge detection and the efficiency of building detection.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer, in particular to a building detection method and device based on remote sensing image and laser point cloud. BACKGROUND

[0002] The building is detected (for example, settlement detection, edge detection), and the commonly used way is: the settlement detection is detected through the building deformation detection device, and the vector profile of the manually outlined building edge is corrected according to the experience and the measurement, and the actual edge position of the building is determined.

[0003] However, the building detection method has the following technical problems: the building deformation detection device needs to be updated and detected, and the deformation detection device needs to be installed on site, which is complicated; the vector profile of the manually outlined building edge is not accurate.

[0004] The above information disclosed in the background section is only intended to enhance the understanding of the background of the present inventive concept, and therefore, it can include information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0005] The summary section is provided to introduce the concepts briefly in a simplified form, which will be described in detail in the specific embodiments section. The summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure provide a building detection method and device based on remote sensing image and laser point cloud, an electronic device and a computer readable medium, to solve one or more of the technical problems mentioned in the background section.

[0007] In a first aspect, some embodiments of the present disclosure provide a building detection method based on remote sensing images and laser point clouds, the method comprising: collecting remote sensing images and laser point cloud information corresponding to a target building area; preprocessing the remote sensing images to generate preprocessed remote sensing images; inputting the preprocessed remote sensing images into a pre-trained remote sensing image feature edge detection model to obtain remote sensing image feature edge results, wherein the remote sensing image feature edge results represent building edge detection boxes in the preprocessed remote sensing images; extracting building remote sensing images corresponding to the building edge detection boxes in the preprocessed remote sensing images; performing single building segmentation processing on the building remote sensing images to obtain a building segmented remote sensing image group; for each building segmented remote sensing image in the building segmented remote sensing image group, performing the following processing steps: determining image edge node coordinate sequences and ground elevation information of the building segmented remote sensing image, wherein each edge node coordinate in the edge node coordinate sequences represents a key node of the building segmented remote sensing image, and an image formed by sequentially connecting each edge node coordinate represents the contour boundary of the building segmented remote sensing image; extracting single building point cloud data corresponding to the building segmented remote sensing image from the laser point cloud information according to the image edge node coordinate sequences and the ground elevation information; and generating building detection information according to the single building point cloud data and corresponding standard building three-dimensional data.

[0008] In a second aspect, some embodiments of the present disclosure provide a building detection device based on remote sensing images and laser point clouds, the device comprising: a collection unit configured to collect remote sensing images and laser point cloud information corresponding to a target building area; a preprocessing unit configured to preprocess the remote sensing images to generate preprocessed remote sensing images; an input unit configured to input the preprocessed remote sensing images into a pre-trained remote sensing image feature edge detection model to obtain remote sensing image feature edge results, wherein the remote sensing image feature edge results represent building edge detection boxes in the preprocessed remote sensing images; an extraction unit configured to extract building remote sensing images corresponding to the building edge detection boxes in the preprocessed remote sensing images; a segmentation unit configured to perform single building segmentation processing on the building remote sensing images to obtain a building segmented remote sensing image group; and a detection unit configured to, for each building segmented remote sensing image in the building segmented remote sensing image group, perform the following processing steps: determining image edge node coordinate sequences and ground elevation information of the building segmented remote sensing image, wherein each edge node coordinate in the edge node coordinate sequences represents a key node of the building segmented remote sensing image, and an image formed by sequentially connecting each edge node coordinate represents a contour boundary of the building segmented remote sensing image; extracting single building point cloud data corresponding to the building segmented remote sensing image from the laser point cloud information according to the image edge node coordinate sequences and the ground elevation information; and generating building detection information according to the single building point cloud data and corresponding standard building three-dimensional data.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method described in any of the implementations of the first aspect.

[0011] The above various embodiments of the present disclosure have the following beneficial effects: through the building detection method based on remote sensing images and laser point clouds of some embodiments of the present disclosure, the accuracy of building area edge detection and the efficiency of building subsidence detection are improved. First, remote sensing images and laser point cloud information corresponding to the target building area are collected; the remote sensing images are preprocessed to generate preprocessed remote sensing images. In this way, the preprocessed remote sensing images can be obtained, and the noise in the remote sensing images can be removed. Second, the preprocessed remote sensing images are input into a pre-trained remote sensing image ground feature edge detection model to obtain a remote sensing image ground feature edge result, wherein the remote sensing image ground feature edge result represents a building edge detection box in the preprocessed remote sensing images. In this way, the buildings in the remote sensing images can be extracted. Then, the building remote sensing images corresponding to the building edge detection box in the preprocessed remote sensing images are extracted. Then, the building remote sensing images are subjected to single building segmentation processing to obtain a building segmentation remote sensing image group. In this way, each single building can be detected. Finally, for each building segmentation remote sensing image in the building segmentation remote sensing image group, the following processing steps are performed: determining the image edge node coordinate sequence and the ground elevation information of the building segmentation remote sensing image, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image formed by sequentially connecting each edge node coordinate represents the contour boundary of the building segmentation remote sensing image; according to the image edge node coordinate sequence and the ground elevation information, single building point cloud data corresponding to the building segmentation remote sensing image is extracted from the laser point cloud information; and according to the single building point cloud data and the corresponding standard building three-dimensional data, building detection information is generated. Since the remote sensing image data has low acquisition cost, wide coverage, high resolution, and does not require a large amount of complex calculation, the modeling accuracy is high, the time consumption is short, and human intervention is not required. In addition, since the building is detected by point cloud, the accuracy of building area edge detection and the efficiency of building subsidence detection are improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by referring to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and elements and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of the building detection method based on remote sensing images and laser point clouds according to the present disclosure;

[0014] Figure 2is a structural schematic diagram of some embodiments of a building detection device based on remote sensing images and laser point clouds according to the present disclosure;

[0015] Figure 3 is a structural schematic diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings of the present disclosure and the embodiments are only for exemplary purposes and should not be used to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0018] It should be noted that the terms "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0022] Figure 1 is a flowchart of some embodiments of a building detection method based on remote sensing images and laser point clouds according to the present disclosure. The flowchart 100 of some embodiments of a building detection method based on remote sensing images and laser point clouds according to the present disclosure is shown. The building detection method based on remote sensing images and laser point clouds includes the following steps:

[0023] Step 101, collecting remote sensing images and laser point cloud information corresponding to a target building area.

[0024] In some embodiments, an execution subject (e.g., a computing device) of the building detection method based on remote sensing images and laser point clouds can collect remote sensing images and laser point cloud information corresponding to a target building area. The remote sensing images of the target building area can be collected by remote sensing technology / satellite technology. The laser point cloud information of the target building area can be collected by a 3D laser point cloud camera. The target building area can be an area that needs to detect building settlement changes and can include a target building. For example, the target building can be a teaching building / laboratory building.

[0025] Step 102, pre-processing the remote sensing images to generate pre-processed remote sensing images.

[0026] In some embodiments, the execution subject can pre-process the remote sensing images to generate pre-processed remote sensing images. In practice, first, the execution subject can perform radiometric calibration on the remote sensing images using a radiometric correction tool to obtain radiometrically calibrated remote sensing images. Then, the radiometrically calibrated remote sensing images can be geometrically corrected using a geometric correction algorithm to obtain corrected remote sensing images. After that, the corrected remote sensing images can be atmospherically corrected using a dark pixel method to obtain pre-processed remote sensing images. For example, the geometric correction algorithm can be an RPC model algorithm.

[0027] Step 103, inputting the pre-processed remote sensing images into a pre-trained remote sensing image building edge detection model to obtain remote sensing image building edge results.

[0028] In some embodiments, the execution subject can input the pre-processed remote sensing images into a pre-trained remote sensing image building edge detection model to obtain remote sensing image building edge results, wherein the remote sensing image building edge results represent building edge detection boxes in the pre-processed remote sensing images. The remote sensing image building edge detection model can be a pre-trained neural network model that takes pre-processed remote sensing images as input and outputs remote sensing image building edge results. For example, the remote sensing image building edge detection model can be a HED (Holistically-Nested Edge Detection) model, a RCF (Richer Convolutional Features) model, or a FD-RCF (Full Dilated-RCF) model.

[0029] The remote sensing image building edge detection model can be trained by the following steps:

[0030] First, a set of building remote sensing images is obtained. The set of building remote sensing images can be historical remote sensing images of various buildings.

[0031] Secondly, image enhancement processing is performed on each building remote sensing image in the building remote sensing image set to generate an enhanced building remote sensing image, thereby obtaining an enhanced building remote sensing image set.

[0032] The second step can include the following sub-steps:

[0033] In the first sub-step, histogram information of the building remote sensing image is determined. The histogram information can represent a histogram of the histogram information of the remote sensing image. In practice, the execution subject can determine the histogram information of the histogram information of the remote sensing image through the cv2.calcHist() function in the OpenCV library.

[0034] In the second sub-step, the histogram information is equalized to obtain equalized building histogram information.

[0035] In the third sub-step, the equalized building histogram information is subjected to gray value mapping processing to obtain a gray level corresponding to the building remote sensing image.

[0036] In the fourth sub-step, threshold information corresponding to the building remote sensing image is determined according to the gray level and the clipping factor information. The threshold information can represent a histogram clipping threshold. In practice, first, the execution subject can determine the ratio of the number of pixels of the building remote sensing image to the gray level as a target ratio. Then, the product of the target ratio and the clipping factor information is determined as the threshold information. The clipping factor information can be a pre-set clipping probability.

[0037] In the fifth sub-step, the building remote sensing image is updated according to the threshold information to obtain an updated building remote sensing image. The execution subject can perform clipping processing on the building remote sensing image according to the threshold information through a threshold segmentation algorithm to obtain a clipped building remote sensing image. Then, an interpolation algorithm can be used to perform pixel assignment processing on each pixel in the clipped building remote sensing image to obtain a building remote sensing image after pixel assignment processing as the updated building remote sensing image.

[0038] In the sixth sub-step, histogram equalization processing is performed on the updated building remote sensing image to obtain an enhanced building remote sensing image.

[0039] Thirdly, each enhanced building remote sensing image in the enhanced building remote sensing image set is labeled to generate a set of labeled enhanced building remote sensing images as a building remote sensing image sample set. For example, the edge frame of the building in each enhanced building remote sensing image can be labeled.

[0040] In the fourth step, an initial remote sensing image ground object edge detection model is determined. The initial remote sensing image ground object edge detection model includes an initial remote sensing image ground object edge feature extraction layer, an initial multi-level ground object edge feature coding layer, and an initial remote sensing image ground object edge frame output layer. For example, the initial remote sensing image ground object edge feature extraction layer can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc.

[0041] The fourth step can include the following sub-steps:

[0042] In the first sub-step, an initial remote sensing image ground object edge feature extraction layer is determined.

[0043] In the second sub-step, an initial multi-level ground object edge feature coding layer is determined. The initial multi-level ground object edge feature coding layer includes an initial ground object edge coding layer (encoder) and an initial ground object edge decoding layer (decoder). The initial ground object edge coding layer includes an initial ground object edge feature decoupling layer (including an auto-encoding network and an LSTM network) and an initial ground object edge feature fusion layer (VAE (Variational AutoEncoder)). The initial ground object edge feature fusion layer includes an initial fusion layer and an initial re-parameterization layer. The initial fusion layer can be a feature pyramid network (FPN) or a self-attention mechanism network. The initial re-parameterization layer can include DyRep (Dynamic Re-parameterization) and ACNet (Asymmetric ConvolutionNet).

[0044] In the third sub-step, an initial remote sensing image ground object edge frame output layer is determined. The initial remote sensing image ground object edge frame output layer can be an output layer for outputting a remote sensing image ground object edge frame.

[0045] In the fourth sub-step, the initial remote sensing image ground object edge feature extraction layer, the initial multi-level ground object edge feature coding layer, and the initial remote sensing image ground object edge frame output layer are fused to generate an initial remote sensing image ground object edge detection model.

[0046] In the fifth step, a target building remote sensing image sample is selected from the building remote sensing image sample set. A building remote sensing image sample can be randomly selected from the building remote sensing image sample set as a target building remote sensing image sample.

[0047] In the sixth step, the initial remote sensing image ground object edge detection model is trained according to the target building remote sensing image sample, and a trained remote sensing image ground object edge detection model is obtained.

[0048] The sixth step can include the following sub-steps.

[0049] In the first sub-step, the sample building remote sensing image included in the target building remote sensing image sample is input into the initial remote sensing image ground object edge feature extraction layer, and initial remote sensing image ground object edge features are obtained.

[0050] In the second sub-step, the initial remote sensing image ground object edge features are input into the initial multi-level ground object edge feature encoding layer, and initial multi-level remote sensing image ground object edge features are obtained.

[0051] In the third sub-step, the initial multi-level remote sensing image ground object edge features are input into the initial remote sensing image ground object edge box output layer, and initial remote sensing image ground object edge box detection results are obtained.

[0052] In the fourth sub-step, a loss value between the initial remote sensing image ground object edge box detection result and the corresponding sample label is determined based on a preset loss function. The loss function can be a mean square error loss (MSE Loss) function, a cross-entropy loss (Cross-Entropy Loss) function, or a mean absolute error (MAE) function.

[0053] In the fifth sub-step, in response to determining that the loss value is less than or equal to a preset loss value, the initial remote sensing image ground object edge detection model is determined as the trained remote sensing image ground object edge detection model.

[0054] Therefore, the remote sensing image ground object edge detection model constructed by the present disclosure can comprehensively capture multi-level information of remote sensing image ground object edges, enhance the efficiency and accuracy of the autoencoder in feature disentangling, and further construct an initial remote sensing image ground object edge box output layer to help the model output complex nonlinear relationships in the ground object edge, thereby improving the accuracy of ground object edge detection.

[0055] In step 104, a building remote sensing image corresponding to the building edge detection box in the preprocessed remote sensing image is extracted.

[0056] In some embodiments, the execution subject can extract a building remote sensing image corresponding to the building edge detection box in the preprocessed remote sensing image. For example, the building remote sensing image in the building edge detection box in the preprocessed remote sensing image can be extracted.

[0057] In step 105, a single building segmentation processing is performed on the building remote sensing image, and a building segmentation remote sensing image group is obtained.

[0058] In some embodiments, the execution subject can perform single building segmentation processing on the building remote sensing image to obtain a building segmented remote sensing image group.

[0059] In practice, the execution subject can perform single building segmentation processing on the building remote sensing image by the following steps:

[0060] Firstly, initial region segmentation is performed on the building remote sensing image to obtain a remote sensing region image group. Each remote sensing region image can represent the rectangular boundary of each building extracted from the building remote sensing image. Each remote sensing region image can correspond to corner point coordinates, region height and region width. The corner point coordinates can be the top-left corner coordinates or the top-right corner coordinates or the bottom-left corner coordinates or the bottom-right corner coordinates of the rectangular boundary. In practice, firstly, the execution subject can use a corner point detection algorithm to detect the corner points of the remote sensing region image to obtain each corner point information. Then, a rectangular fitting algorithm is used to fit the rectangular boundary of each corner point information to obtain the remote sensing region image. The corner point detection algorithm can be SIFT algorithm, Harris algorithm or SURF algorithm. The rectangular fitting algorithm can be Hough transform or RANSAC algorithm.

[0061] Secondly, region image feature extraction is performed on each remote sensing region image in the remote sensing region image group to generate remote sensing region features to obtain a remote sensing region feature group. The remote sensing region feature can be a feature vector of the remote sensing region image. In practice, the execution subject can use a CNN convolutional network to extract the region image features of each remote sensing region image in the remote sensing region image group.

[0062] Thirdly, building recognition processing is performed on the remote sensing region feature group to obtain a building recognition result group. Each remote sensing region feature corresponds to a building recognition result. Each remote sensing region feature can be input into a pre-trained building recognition model to obtain the building recognition result group. For example, the building recognition model can be a pre-trained neural network model that takes the remote sensing region feature as input and outputs the building recognition result. For example, the building recognition model can be a FCN (Fully Convolutional Networks) model. The building recognition result can be:

[0063] Fourthly, each remote sensing region image corresponding to each building recognition result in the building recognition result group that meets the preset recognition condition is determined as a building remote sensing region image group.

[0064] In the fifth step, the position adjustment processing is performed on each building remote sensing area image to generate a position adjustment building remote sensing area image, thereby obtaining a position adjustment building remote sensing area image group. For example, the execution subject can perform edge detection on each building remote sensing area image by using an edge detection algorithm to obtain each edge information. The edge detection algorithm can be a Canny algorithm. Then, the findContours function in OpenCV can be used to perform contour extraction on each edge information to obtain each contour extraction information. Finally, a contour approximation algorithm can be used to simplify each contour extraction information to obtain each simplified contour extraction information as the position adjustment building remote sensing area image group. The contour approximation algorithm can be a Douglas-Peucker algorithm.

[0065] In the sixth step, the region redundancy elimination processing is performed on each position adjustment building remote sensing area image to obtain each position adjustment building remote sensing area image after the redundancy elimination as a building segmentation remote sensing image group. A non-maximum suppression algorithm can be used to perform the region redundancy elimination processing on each position adjustment building remote sensing area image to obtain each position adjustment building remote sensing area image after the redundancy elimination as the building segmentation remote sensing image group.

[0066] In step 106, for each building segmentation remote sensing image in the building segmentation remote sensing image group, the following processing steps are performed:

[0067] In step 1061, the image edge node coordinate sequence and the ground elevation information of the building segmentation remote sensing image are determined.

[0068] In some embodiments, the execution subject can determine the image edge node coordinate sequence and the ground elevation information of the building segmentation remote sensing image. Each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image represented by sequentially connecting each edge node coordinate represents the contour boundary of the building segmentation remote sensing image. The ground elevation information can be the height of each point in the target building area relative to a specific vertical reference. The image edge node coordinate can refer to the latitude and longitude coordinates of each key node of the building segmentation remote sensing image in the map.

[0069] In step 1062, the single building point cloud data corresponding to the building segmentation remote sensing image is extracted from the laser point cloud information according to the image edge node coordinate sequence and the ground elevation information.

[0070] In some embodiments, the execution subject can extract the single building point cloud data corresponding to the building segmentation remote sensing image from the laser point cloud information according to the image edge node coordinate sequence and the ground elevation information.

[0071] In practice, the above execution subject can extract the single building point cloud data corresponding to the building segmented remote sensing image from the above laser point cloud information through the following steps:

[0072] First, according to the image edge node coordinate sequence, the edge of each point cloud data in the laser point cloud information is segmented to obtain the ground point cloud data set corresponding to the image edge node coordinate sequence. For example, the same point cloud data as the plane coordinates (XY coordinates in the point cloud) corresponding to the image edge node coordinates can be found in each point cloud data in the laser point cloud information, and the point cloud data in each point cloud data is extracted to obtain the ground point cloud data set corresponding to the image edge node coordinate sequence. For example, the point cloud extraction tool (Meshlab) can be used to extract the point cloud data in each point cloud data to obtain the ground point cloud data set corresponding to the image edge node coordinate sequence. For another example, a region growing-based segmentation algorithm can be used to extract the point cloud data in each point cloud data to obtain the ground point cloud data set corresponding to the image edge node coordinate sequence.

[0073] Second, the ground point cloud data set is denoised to obtain a denoised ground point cloud data set. For example, a filtering algorithm can be used to denoise the ground point cloud data set to obtain a denoised ground point cloud data set. The filtering algorithm can be Gaussian filtering, median filtering.

[0074] Third, the denoised ground point cloud data set is subjected to point cloud clustering processing to obtain a point cloud cluster group. A clustering algorithm can be used to perform point cloud clustering processing on the denoised ground point cloud data set to obtain a point cloud cluster group. The clustering algorithm can be K-means clustering, DBSCAN clustering.

[0075] Third, the point cloud cluster group is subjected to boundary correction processing to obtain a boundary-corrected point cloud cluster group. A morphological erosion method can be used to perform boundary correction processing on the point cloud cluster group to obtain a boundary-corrected point cloud cluster group.

[0076] Fourth, for each boundary-corrected point cloud cluster in the boundary-corrected point cloud cluster group, the following processing steps are performed:

[0077] 1. From each ground elevation data in the ground elevation information, the ground elevation data corresponding to the boundary-corrected point cloud cluster is extracted as a target ground elevation data group.

[0078] 2. According to the target ground elevation data group, the boundary-corrected point cloud cluster is subjected to redundancy elimination processing to obtain a redundancy-eliminated boundary-corrected point cloud cluster. For example, the point cloud data with a vertical axis coordinate greater than the corresponding target ground elevation data can be removed from the boundary-corrected point cloud cluster.

[0079] Fifthly, fuse each redundant boundary rectification point cloud cluster into single building point cloud data corresponding to the above building segmented remote sensing image.

[0080] Step 1063, generate building detection information according to the above single building point cloud data and corresponding standard building three-dimensional data.

[0081] In some embodiments, the above execution subject can generate building detection information according to the above single building point cloud data and corresponding standard building three-dimensional data. The standard building three-dimensional data can be a three-dimensional model of a 1:1 constructed building. First, the single building point cloud data can be visualized to obtain a visualized building three-dimensional image. The PCL (Point Cloud Library) tool can be used to visualize the single building point cloud data to obtain a visualized building three-dimensional image. The error of each edge of the visualized building three-dimensional image and the corresponding edge in the standard building three-dimensional data is determined. Finally, each error is combined into building detection information. Thus, it can be determined whether the building has undergone settlement changes.

[0082] Further reference Figure 2 , as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a building detection device based on remote sensing image and laser point cloud. These building detection device embodiments based on remote sensing image and laser point cloud correspond to those method embodiment shown in Figure 1 , the building detection device based on remote sensing image and laser point cloud can be specifically applied to various electronic devices.

[0083] As Figure 2As shown, the building detection device 200 based on remote sensing image and laser point cloud of some embodiments includes a collection unit 201, a preprocessing unit 202, an input unit 203, an extraction unit 204, a segmentation unit 205, and a detection unit 206. Among them, the collection unit 201 is configured to collect remote sensing image and laser point cloud information corresponding to the target building area; the preprocessing unit 202 is configured to preprocess the remote sensing image to generate a preprocessed remote sensing image; the input unit 203 is configured to input the preprocessed remote sensing image into a pre-trained remote sensing image ground feature edge detection model to obtain a remote sensing image ground feature edge result, wherein the remote sensing image ground feature edge result represents a building edge detection frame in the preprocessed remote sensing image; the extraction unit 204 is configured to extract a building remote sensing image corresponding to the building edge detection frame in the preprocessed remote sensing image; the segmentation unit 205 is configured to perform single building segmentation processing on the building remote sensing image to obtain a building segmented remote sensing image group; the detection unit 206 is configured to perform the following processing steps for each building segmented remote sensing image in the building segmented remote sensing image group: determine the image edge node coordinate sequence and the ground elevation information of the building segmented remote sensing image, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmented remote sensing image, and the image formed by sequentially connecting each edge node coordinate represents the contour boundary of the building segmented remote sensing image; according to the image edge node coordinate sequence and the ground elevation information, extract single building point cloud data corresponding to the building segmented remote sensing image from the laser point cloud information; and generate building detection information according to the single building point cloud data and the corresponding standard building three-dimensional data.

[0084] It can be understood that the units described in the building detection device 200 based on remote sensing image and laser point cloud are corresponding to the steps in the method described above. Figure 1 The operations, features and advantages described above for the method also apply to the building detection device 200 based on remote sensing image and laser point cloud and the units contained therein, and will not be repeated here.

[0085] Reference is made below to Figure 3 which shows a structural schematic diagram of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic device in some embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. Figure 3The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0086] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0087] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0088] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. When the computer program is executed by the processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0089] Note that the computer readable medium in some embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus or device, or that can be used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present disclosure, the computer readable signal medium can include a computer readable program code propagated in or on a carrier medium, in which the computer readable program code is embodied. Such propagated computer readable program code can take many forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any suitable medium, including but not limited to, wire, cable, wireless, RF, infrared or any suitable combination of the foregoing.

[0090] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0091] The computer readable medium can be included in the electronic device, or can exist separately from the electronic device. The computer readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: collect remote sensing image and laser point cloud information corresponding to a target building area; pre-process the remote sensing image to generate a pre-processed remote sensing image; input the pre-processed remote sensing image into a pre-trained remote sensing image feature edge detection model to obtain a remote sensing image feature edge result, wherein the remote sensing image feature edge result represents a building edge detection frame in the pre-processed remote sensing image; extract a building remote sensing image corresponding to the building edge detection frame in the pre-processed remote sensing image; perform single building segmentation processing on the building remote sensing image to obtain a building segmented remote sensing image group; for each building segmented remote sensing image in the building segmented remote sensing image group, perform the following processing steps: determine image edge node coordinate sequences and ground elevation information of the building segmented remote sensing image, wherein each edge node coordinate in the edge node coordinate sequences represents a key node of the building segmented remote sensing image, and an image formed by sequentially connecting each edge node coordinate represents a contour boundary of the building segmented remote sensing image; extract single building point cloud data corresponding to the building segmented remote sensing image from the laser point cloud information according to the image edge node coordinate sequences and the ground elevation information; and generate building detection information according to the single building point cloud data and corresponding standard building three-dimensional data.

[0092] Computer program code for carrying out operations of some embodiments of the disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0093] The flow and block diagrams in the drawings represent possible architectural, functional, and operational scenarios of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0094] The units described in some embodiments of the present disclosure can be implemented by software or by hardware. The described units can also be arranged in a processor, for example, a processor can be described as including a collection unit, a preprocessing unit, an input unit, an extraction unit, a segmentation unit, and a detection unit. In some cases, the names of these units do not constitute a limitation on the units themselves, for example, the collection unit can also be described as a unit that collects remote sensing images and laser point cloud information corresponding to a target building area.

[0095] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.

[0096] The above description is merely some of the preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above inventive concept. For example, the above features can be replaced with technical features disclosed in the embodiments of the present disclosure (but not limited to) having similar functions to form technical solutions.

Claims

1. A building detection method based on remote sensing imagery and laser point clouds, comprising: Collect remote sensing images and laser point cloud information of the corresponding target building area; The remote sensing images are preprocessed to generate preprocessed remote sensing images; The preprocessed remote sensing image is input into a pre-trained remote sensing image feature edge detection model to obtain remote sensing image feature edge results, wherein the remote sensing image feature edge results represent building edge detection boxes in the preprocessed remote sensing image. Extract the building remote sensing image corresponding to the building edge detection box from the preprocessed remote sensing image; The remote sensing images of the buildings are processed to segment individual buildings, resulting in a group of segmented remote sensing images of buildings. For each building segmentation remote sensing image in the building segmentation remote sensing image group, the following processing steps are performed: The image edge node coordinate sequence and ground elevation information of the building segmentation remote sensing image are determined, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image enclosed by sequentially connecting each edge node coordinate represents the contour boundary of the building segmentation remote sensing image. Based on the image edge node coordinate sequence and the ground elevation information, extract the point cloud data of individual buildings corresponding to the building segmentation remote sensing image from the laser point cloud information; Building detection information is generated based on the point cloud data of the individual building and the corresponding 3D data of the standard building.

2. The method according to claim 1, wherein, Before inputting the preprocessed remote sensing image into a pre-trained remote sensing image ground feature edge detection model to obtain the remote sensing image ground feature edge results, the method further includes: Acquire a set of remote sensing images of buildings; Image enhancement processing is performed on each building remote sensing image in the building remote sensing image set to generate enhanced building remote sensing images, thus obtaining the enhanced building remote sensing image set. Each augmented building remote sensing image in the augmented building remote sensing image set is labeled to generate a labeled augmented building remote sensing image set, which serves as a building remote sensing image sample set. An initial remote sensing image ground object edge detection model is determined, wherein the initial remote sensing image ground object edge detection model includes: an initial remote sensing image ground object edge feature extraction layer, an initial multi-level ground object edge feature encoding layer, and an initial remote sensing image ground object edge box output layer; Select remote image samples of the target building from the set of remote sensing images of the building; Based on remote sensing image samples of the target building, the initial remote sensing image feature edge detection model is trained to obtain the trained remote sensing image feature edge detection model.

3. The method according to claim 2, wherein, The step of performing image enhancement processing on each building remote sensing image in the building remote sensing image set to generate enhanced building remote sensing images includes: Determine the histogram information of the remote sensing image of the building; The histogram information is subjected to equalization processing to obtain equalized building histogram information; The grayscale value mapping process is performed on the equalized building histogram information to obtain the grayscale level of the corresponding building remote sensing image; Based on the grayscale level and cropping factor information, the threshold information corresponding to the remote sensing image of the building is determined; Based on the threshold information, the remote sensing image of the building is updated to obtain an updated remote sensing image of the building. Histogram equalization is performed on the updated building remote sensing images to obtain enhanced building remote sensing images.

4. The method according to claim 1, wherein, The step of extracting individual building point cloud data corresponding to the building segmentation remote sensing image from the laser point cloud information based on the image edge node coordinate sequence and the surface elevation information includes: Based on the image edge node coordinate sequence, edge segmentation is performed on each point cloud data in the laser point cloud information to obtain a ground point cloud dataset corresponding to the image edge node coordinate sequence; The ground point cloud dataset is subjected to noise reduction processing to obtain a noise-reduced ground point cloud dataset; Point cloud clustering is performed on the denoised ground point cloud dataset to obtain point cloud clusters; The point cloud clusters are subjected to boundary correction processing to obtain boundary-corrected point cloud clusters; For each boundary-corrected point cloud cluster in the boundary-corrected point cloud cluster group, perform the following processing steps: Extract each surface elevation data corresponding to the boundary correction point cloud cluster from each surface elevation data in the surface elevation information, and use it as the target surface elevation data group. Based on the target surface elevation data set, the boundary correction point cloud cluster is deredundanted to obtain the deredundant boundary correction point cloud cluster. The various deredundancy-eliminating boundary-corrected point cloud clusters are merged into individual building point cloud data corresponding to the building segmentation remote sensing image.

5. A building detection device based on remote sensing imagery and laser point clouds, comprising: The acquisition unit is configured to acquire remote sensing images and laser point cloud information of the corresponding target building area; A preprocessing unit is configured to preprocess the remote sensing image to generate a preprocessed remote sensing image. The input unit is configured to input the preprocessed remote sensing image into a pre-trained remote sensing image feature edge detection model to obtain remote sensing image feature edge results, wherein the remote sensing image feature edge results represent building edge detection boxes in the preprocessed remote sensing image. The extraction unit is configured to extract the building remote sensing image corresponding to the building edge detection box in the preprocessed remote sensing image; The segmentation unit is configured to perform individual building segmentation processing on the remote sensing image of the building to obtain a group of segmented remote sensing images of the building. The detection unit is configured to perform the following processing steps for each building segmentation remote sensing image in the building segmentation remote sensing image group: determining the image edge node coordinate sequence and surface elevation information of the building segmentation remote sensing image, wherein each edge node coordinate in the edge node coordinate sequence represents a key node of the building segmentation remote sensing image, and the image enclosed by sequentially connecting the coordinates of each edge node represents the contour boundary of the building segmentation remote sensing image; extracting the individual building point cloud data corresponding to the building segmentation remote sensing image from the laser point cloud information based on the image edge node coordinate sequence and the surface elevation information; and generating building detection information based on the individual building point cloud data and the corresponding standard building 3D data.

6. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images