Single tree data extraction method, device, electronic equipment and storage medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and storage medium for extracting single tree data, the method comprising: a first CHM established based on laser radar data and a first DOM established based on visible light data; high-precision matching of the first CHM and the first DOM to obtain the matched second CHM and the second DOM; spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree range of the second CHM; extracting the single tree results of the third CHM from the missing area, and merging the single tree results of the second CHM and the single tree results of the third CHM to form a target single tree result; extracting single tree data based on the target single tree result. The spectral and texture information of the optical DOM data and the high-precision structural information of the laser radar CHM are fully combined, so that the accuracy of single tree segmentation is improved, and it has good practicality and popularization.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, electronic device and storage medium for extracting single tree data. Background Art

[0002] Stand parameters are an important concept in forestry and ecology, which refers to a set of data describing the various characteristics and attributes of individual trees in a forest area. These parameters are of great significance for forest management and ecosystem research, because they can help researchers and foresters understand the growth and health of forests and their response to environmental changes.

[0003] LiDAR (Light Detection And Ranging) technology is an efficient active remote sensing technology that measures the distance and position of objects by emitting laser pulses and receiving reflected light signals. Due to its unique working principle, it is less affected by terrain and light, and can obtain a large range of three-dimensional surface structure data at one time, and can effectively estimate spatial structure information such as the height and crown width of individual trees. LiDAR technology has significant advantages in obtaining three-dimensional surface structure data, especially in the fields of forest resource management and ecological environment monitoring.

[0004] Extracting individual tree structure information from LiDAR data is an important task in forestry management and ecological research. There are many algorithms for extracting individual trees from LiDAR data, such as extracting individual tree information from the Digital Orthophoto Model (DOM) or the Canopy Height Model (CHM).

[0005] DOM images have limitations in extracting vegetation information. They can only provide spectral texture information on the surface of vegetation, and it is difficult to distinguish vertical vegetation structures under the same terrain conditions. There are cases where individual tree crowns are stuck to each other and are difficult to segment due to occlusion. Single tree segmentation based on CHM is greatly affected by the resolution of CHM. High-resolution CHM images require high-density LiDAR point cloud support, which requires a large amount of data processing. In addition, there are more concave points inside the crown, and the edge of the crown is more irregular. At present, the main method for extracting individual trees from DOM or CHM is the marked watershed method, but it has high requirements for data quality. The existing extraction methods cannot meet its requirements for data quality, and the accuracy of single tree extraction cannot be guaranteed. Summary of the invention

[0006] One purpose of the present application is to propose a single tree data extraction method, device, electronic device and storage medium, which can extract comprehensive and accurate single tree data and improve the accuracy of single tree results.

[0007] According to one aspect of an embodiment of the present application, a method for extracting single tree data is disclosed, the method comprising: performing high-precision matching on a first canopy height model CHM established based on lidar data and a first digital orthophoto model DOM established based on visible light data; performing high-precision matching on the first CHM and the first DOM to obtain a matched second CHM and a second DOM; spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM to obtain a missing area of ​​the single tree results of the second CHM; extracting the single tree results of a third CHM from the missing area, and merging the single tree results of the second CHM and the single tree results of the third CHM to form a target single tree result; and extracting single tree data based on the target single tree result.

[0008] As in an embodiment of the present invention, high-precision matching is performed on the first CHM and the first DOM to obtain the matched second CHM and the second DOM, including: high-precision matching is performed on the first CHM and the first DOM to obtain the displacement difference between the first CHM and the first DOM; the first DOM is moved toward the first CHM by the displacement difference, the first CHM is used as the matched second CHM, and the first DOM after moving the displacement difference is used as the matched second DOM; or the first CHM is moved toward the first DOM by the displacement difference, the first DOM is used as the matched second DOM, and the first CHM after moving the displacement difference is used as the matched second CHM.

[0009] As in an embodiment of the present invention, before spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM, the method further includes: segmenting the second DOM based on a deep learning neural network model to obtain the single tree results of the second DOM; after performing global median filtering on the second CHM, using a local extreme value method to obtain the data after the global median filtering to form the single tree results of the second CHM.

[0010] As in an embodiment of the present invention, the method also includes: cutting the training data and labeling the training data using a deep learning image annotation method to obtain sample data; training the initial model based on the sample data to obtain the deep learning neural network model.

[0011] As in an embodiment of the present invention, the second DOM is segmented based on a deep learning neural network model to obtain a single tree result of the second DOM, including: segmenting the second DOM based on the deep learning neural network model to obtain a binary distribution map of the second DOM; performing opening and closing operations on the binary distribution map and converting it into vector surface data to form a single tree result of the second DOM.

[0012] As in an embodiment of the present invention, spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree results of the second CHM includes: spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM, extracting the area in the second DOM that does not overlap with the single tree results of the second CHM, and forming the missing area of ​​the single tree results of the second CHM.

[0013] As in an embodiment of the present invention, the single tree result of the third CHM is extracted from the missing area, and the single tree result of the second CHM and the single tree result of the third CHM are merged to form a target single tree result, including: after performing global median filtering on the missing area, using the local extreme value method to obtain the data after the global median filtering to form the single tree result of the third CHM; merging the single tree result of the second CHM and the single tree result of the third CHM to form the target single tree result.

[0014] According to one aspect of an embodiment of the present application, a single tree data extraction device is disclosed, the device comprising: a first model building module, configured to match a first canopy height model CHM established based on lidar data and a first digital orthophoto model DOM established based on visible light data; a second data acquisition module, configured to perform high-precision matching on the first CHM and the first DOM to obtain the matched second CHM and second DOM; a missing data acquisition module, configured to spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree range of the second CHM; a single tree result forming module, configured to extract the single tree results of the third CHM from the missing area, and merge the single tree results of the second CHM and the single tree results of the third CHM to form a target single tree result; a target data extraction module, configured to extract single tree data based on the target single tree result.

[0015] According to one aspect of an embodiment of the present application, an electronic device is disclosed, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the methods provided in the above-mentioned various optional implementation modes.

[0016] According to one aspect of an embodiment of the present application, a computer program medium is disclosed, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the methods provided in the various optional implementations described above.

[0017] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementations.

[0018] In the embodiment of the present invention, the first canopy height model CHM established based on laser radar data and the first digital orthophoto model DOM established based on visible light data are matched with the first CHM and the first DOM with high precision to obtain the matched second CHM and the second DOM; the single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed to obtain the missing area of ​​the single tree range of the second CHM; the single tree results of the third CHM are extracted from the missing area, and the single tree results of the second CHM and the single tree results of the third CHM are merged to form the target single tree results; and the single tree data is extracted based on the target single tree results. The spectrum and texture information of the optical DOM data and the high-precision structural information of the laser radar CHM are fully combined, so that the accuracy of single tree segmentation is improved, which has good practicality and popularization.

[0019] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0020] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0022] Figure 1 A flow chart of the single tree data extraction method provided in the present application is shown.

[0023] Figure 2 A flow chart of another single tree data extraction method provided in the present application is shown.

[0024] Figure 3 A block diagram of a single tree data extraction device according to an embodiment of the present application is shown.

[0025] Figure 4 A hardware diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present application will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The accompanying drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0027] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present application. However, those skilled in the art will appreciate that the technical solution of the present application may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring the present application and making the various aspects of the present application obscure.

[0028] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0029] First, the terms in the embodiments of the present invention are explained below:

[0030] Canopy Height Model (CHM) is an important forest structure parameter dataset. It represents the vertical distance between branches and leaves in the forest canopy and the ground, and is a key indicator of the forest canopy structure. CHM is widely used in forest ecology, forestry management, and environmental protection, providing researchers with an effective tool to analyze and understand the structure and function of forest ecosystems.

[0031] Digital Orthophoto Model (DOM) is a geographic information system (GIS) product based on aerial or satellite remote sensing data. DOM is a high-resolution digital image that eliminates the effects of perspective distortion, camera tilt and terrain undulation through orthorectification technology, so that each pixel in the image has an accurate correspondence with the actual position on the ground.

[0032] The following is a detailed description of the single tree data extraction method proposed in the embodiment of the present invention in conjunction with a specific embodiment. The execution subject of the method can be any processor with computing and processing functions, such as a terminal.

[0033] Figure 1 The flowchart of the single tree data extraction method provided in the present application is shown. The execution subject of the method is explained by taking a terminal as an example. Figure 1 As shown, the method may include but is not limited to the following steps:

[0034] Step S110 , establishing a first canopy height model CHM based on the laser radar data and a first digital orthophoto model DOM based on the visible light data.

[0035] In the embodiment of the present invention, the visible light data does not refer to single visible light data (for example, lidar data is also single visible light data) but data that fully integrates the spectra and textures of multiple lights.

[0036] In an embodiment of the present invention, the terminal can obtain visible light data and laser radar data through the drone. For example, the drone can simultaneously obtain visible light data and laser radar data and transmit them to the terminal, and the terminal can extract single tree data through these two types of data. For another example, the laser radar data can also be data pre-stored in the terminal, and the drone obtains the visible light data corresponding to the laser radar data and transmits it to the terminal.

[0037] After acquiring the laser radar data and the visible light data, the terminal can respectively establish a first CHM based on the laser radar data and a first digital orthophoto model DOM based on the visible light data.

[0038] In an embodiment of the present invention, a specific method is provided to establish a first CHM based on lidar data: first, a digital elevation model (DEM) and a digital surface model (DSM) are generated based on the lidar data, and the CHM is obtained by subtracting the DEM from the DSM, that is, CHM=DSM-DEM.

[0039] In an embodiment of the present invention, a specific method is provided to establish a first digital orthophoto model DOM using visible light data: after preprocessing, geometric correction, orthophoto correction, image mosaicking and other steps are performed on the visible light data, the first DOM can be obtained.

[0040] Step S120: Perform high-precision matching on the first CHM and the first DOM to obtain the matched second CHM and the second DOM.

[0041] In the embodiment of the present invention, the first CHM and the first DOM can be matched with high precision to obtain the displacement difference between the first CHM and the first DOM, and then the first DOM is moved toward the first CHM by the displacement difference, and the first CHM is used as the matched second CHM, and the first DOM after the displacement difference is moved is used as the matched second DOM; or the first CHM is moved toward the first DOM by the displacement difference, and the first DOM is used as the matched second DOM, and the first CHM after the displacement difference is moved is used as the matched second CHM.

[0042] It should be noted that when the displacement difference between the first CHM and the first DOM is obtained, it can be achieved through image registration technology in the field of image processing and pattern recognition. The purpose of image registration is to align two or more images so that they have the same spatial reference in the same coordinate system. For example, after feature extraction of two images, feature matching is performed to calculate the displacement difference between the two images.

[0043] In the embodiment of the present invention, after the displacement difference is obtained, the second CHM and the second DOM can be obtained in one of the following two ways: when the first CHM does not move, the first DOM is moved toward the first CHM by the displacement difference, the first CHM is used as the second CHM after matching, and the first DOM after the displacement difference is moved is used as the second DOM after matching. Alternatively, when the first DOM does not move, the first CHM is moved toward the first DOM by the displacement difference, the first DOM is used as the second DOM after matching, and the first CHM after the displacement difference is moved is used as the second CHM after matching.

[0044] In an embodiment of the present invention, by performing high-precision matching on the first CHM and the first DOM, the matched second CHM and the second DOM are obtained, thereby achieving matching and fusion of the lidar data and the visible light data, thereby improving the accuracy of the fused second CHM and the second DOM.

[0045] Step S130: spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree results of the second CHM.

[0046] In the embodiment of the present invention, after obtaining the second DOM and the second CHM, the single tree result of the second DOM and the single tree result of the second CHM can be obtained. It should be noted that the single tree result of the second DOM is a polygon vector (PolygonVector), which is usually used to represent a continuous area, and the single tree result of the second CHM is a point vector (Point Vector), which is used to represent a discrete point.

[0047] In an embodiment of the present invention, after obtaining the single tree results of the second DOM and the second CHM, the single tree results of the second DOM and the single tree results of the second CHM can be spatially superimposed to extract the area in the second DOM that does not overlap with the single tree results of the second CHM, and the area is the omitted area of ​​the single tree results of the second CHM. It should be noted that in the embodiment of the present invention, what is extracted is the area in the second DOM that does not overlap with the single tree results of the second CHM. Since the single tree results of the second DOM are surface vectors, in the embodiment of the present invention, the non-overlapping surface vectors are actually used as the omitted area of ​​the single tree results of the second CHM, and then the single tree results of the third CHM are subsequently obtained from the omitted area, that is, the omitted results of the single tree results of the second CHM are obtained from the omitted area.

[0048] It should be pointed out that when the single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed to extract the area in the second DOM that does not overlap with the single tree results of the second CHM, the non-overlapping area can be obtained through spatial analysis and geometric operations. For example, after the data is prepared, the data is spatially queried and spatially superimposed, and then differential or symmetric difference is performed to obtain the non-overlapping area.

[0049] Step S140: extracting the single tree result of the third CHM from the missing area, and merging the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result.

[0050] In an embodiment of the present invention, a method for extracting a single tree result of the third CHM from a missing area and a method for obtaining a single tree result of the second CHM are the same, both of which are performed by performing global median filtering on a target area (for extracting a single tree result of the third CHM, the target area is the missing area, and for extracting a single tree result of the second CHM, the target area is the second CHM), and then using a local extreme value method to obtain the data after the global median filtering to form the single tree result of the third CHM, which will be described in detail later.

[0051] In an embodiment of the present invention, the non-overlapping surface vector is used as the missing area of ​​the single tree result of the second CHM, and then the single tree result of the third CHM is subsequently obtained from the missing area, that is, the missing result of the single tree result of the second CHM is obtained from the missing area, and the single tree result of the second CHM is merged with the single tree result of the third CHM to form the target single tree result.

[0052] Step S150: extracting single tree data based on the target single tree result.

[0053] In the embodiment of the present invention, after obtaining the target single tree result, the target single tree result is used as a seed point, and the second CHM is operated according to the marked watershed algorithm to obtain a watershed operation single tree segmentation result.

[0054] The marked watershed algorithm is an algorithm that can effectively reduce over-segmentation. The algorithm is based on knowing in advance how to place marks inside the target to be segmented, and can be clearly described using the overflow model. The input grayscale image is considered to be a landform surface. A hole is punched at each local minimum on the image. Water will slowly seep into the surface from these holes, starting from the minimum at the lowest point, and the water gradually floods the water pan of the image at the same speed. When the rising water levels from different minima meet, a dam is built to block it. Finally, the overflow model will reach a state where only the top of the dam is exposed. The top of these dams is the watershed, and the water accumulation area rising from different local minima is called a trough or water pan. The marked watershed algorithm is to place the starting hole position only in the marked area, and each mark is associated with a color. Overflow starts from the bottom of the terrain. Water of different colors starts to rise at the same rate from holes of the same color. If the water reaches an unmarked water pan, it will directly flood the water pan. Similarly, if two water flows of different colors meet, a dam needs to be built. The different colored areas in the result are the water pans associated with different marks.

[0055] After obtaining the single tree segmentation result of the watershed operation, the predetermined structural elements are used to perform opening and closing operations on the single tree segmentation result of the watershed operation to obtain the single tree segmentation result. Then, combined with the point cloud and CHM, the geographical location, tree height and crown width information of the single tree can be obtained.

[0056] In the embodiment of the present invention, since most of the forest areas have serious overlap and occlusion, the canopy shapes on the CHM are different. In addition, there are many extremely irregular shapes in the single tree segmentation results obtained by the watershed operation, so optimization is required. Because the crown is approximately circular when observed from a vertical angle, the opening operation can be used to optimize the crown to restore the true shape of the crown to the greatest extent.

[0057] In the embodiment of the present invention, a first canopy height model CHM established based on laser radar data and a first digital orthophoto model DOM established based on visible light data are matched with the first CHM and the first DOM with high precision to obtain a second DOM matching the first CHM and a second CHM matching the first DOM; the single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed to obtain the missing area of ​​the single tree range of the second CHM; the single tree results of the third CHM are extracted from the missing area, and the single tree results of the second CHM and the single tree results of the third CHM are merged to form the target single tree results; and the single tree data is extracted based on the target single tree results. The spectrum and texture information of the optical DOM data and the high-precision structural information of the laser radar CHM are fully combined, so that the accuracy of single tree segmentation is improved, which has good practicality and popularization.

[0058] The single tree data extraction method provided in this application is described in more detail below in conjunction with specific embodiments. Figure 2 FIG. 4 shows a flow chart of another single tree data extraction method provided in the present application, such as Figure 2 As shown, the method may include but is not limited to the following steps:

[0059] Step S201: Establish a first CHM based on the laser radar data.

[0060] Step S202: establishing a first DOM based on the simultaneously acquired visible light data.

[0061] Step S203: perform high-precision matching on the first CHM and the first DOM to obtain the matched second CHM and the second DOM.

[0062] In the embodiment of the present invention, the first CHM and the first DOM are matched with high precision to obtain the displacement difference between the first CHM and the first DOM, and then the first DOM is moved toward the first CHM by the displacement difference, and the first CHM is used as the matched second CHM, and the first DOM after the displacement difference is moved is used as the matched second DOM; or the first CHM is moved toward the first DOM by the displacement difference, and the first DOM is used as the matched second DOM, and the first CHM after the displacement difference is moved is used as the matched second CHM.

[0063] Step S204: segment the second DOM based on the deep learning neural network model to obtain individual tree results of the second DOM.

[0064] In an embodiment of the present invention, a deep learning neural network model can be obtained through training. Specifically, the training data is cut and labeled using a deep learning image labeling (Labelme) method to obtain sample data. The sample data is data that has been labeled with single tree crown segmentation. The initial model is trained based on the sample data to obtain the deep learning neural network model. The training data refers to the cut second DOM. Of course, it can also be the data after the second DOM obtained in step S203 is cut. When marking, manual marking can also be performed.

[0065] After obtaining the deep learning neural network model, the second DOM is segmented using the deep learning neural network model to obtain a binary distribution map of the second DOM, and the binary distribution map is converted into vector surface data after opening and closing operations to form a single tree result (surface vector) of the second DOM.

[0066] Step S205: After performing global median filtering on the second CHM, the data after the global median filtering is obtained using a local extremum method to form a single tree result of the second CHM.

[0067] In the embodiment of the present invention, the function of performing global median filtering on the second CHM is to eliminate the influence of image noise. The specific mathematical expression of the processing is as follows:

[0068] CHM Median =Median(CHM)(1)

[0069] Among them, CHM Median It is the second CHM after global median filtering, and Median() is the median filter function.

[0070] After global median filtering, the local extreme value method is used to obtain the data processed by the global median filter to form the single tree results of the second CHM.

[0071] It should be pointed out that the local extremum method uses the height information of the tree crown. There is at least one highest point in a tree crown, that is, the tree top position, but there may also be a situation where a tree crown has multiple local maxima. In order to determine the uniqueness of the tree top, the method provided by the present invention first performs median filtering on the CHM to smooth out the situation of multiple tree tops on the same tree crown; by processing the acquired position data of each tree top, a single tree result (point vector) based on the second CHM is formed.

[0072] Step S206: spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree results of the second CHM.

[0073] In the embodiment of the present invention, a spatial overlay analysis is performed on the surface vector of the single tree result based on the second DOM and the point vector of the single tree result based on the second CHM, and non-overlapping areas are extracted in the second DOM as the missing areas of the single tree result of the second CHM.

[0074] Step S207: extract the single tree result of the third CHM from the missing area, and merge the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result.

[0075] In an embodiment of the present invention, after the missing area is processed by global median filtering, the data after the global median filtering is obtained by using the local extreme value method to form a single tree result of the third CHM, and the single tree result of the second CHM and the single tree result of the third CHM are merged to form a target single tree result.

[0076] Step S208: using the target single tree result as a seed point, and obtaining a watershed operation single tree segmentation result according to the marked watershed algorithm.

[0077] Step S209: Use a predetermined structural element to perform an opening and closing operation on the single tree segmentation result of the watershed operation to obtain a single tree segmentation result.

[0078] Step S210: According to the single tree segmentation result, combined with the point cloud and CHM, the geographical location, tree height and crown width information of the single tree can be obtained.

[0079] In the embodiment of the present invention, a first canopy height model CHM established based on laser radar data and a first digital orthophoto model DOM established based on visible light data are matched with the first CHM and the first DOM with high precision to obtain a second DOM matching the first CHM and a second CHM matching the first DOM; the single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed to obtain the missing area of ​​the single tree range of the second CHM; the single tree results of the third CHM are extracted from the missing area, and the single tree results of the second CHM and the single tree results of the third CHM are merged to form the target single tree results; and the single tree data is extracted based on the target single tree results. The spectrum and texture information of the optical DOM data and the high-precision structural information of the laser radar CHM are fully combined, so that the accuracy of single tree segmentation is improved, which has good practicality and popularization.

[0080] Figure 3 A block diagram of a single tree data extraction device according to an embodiment of the present application is shown, the device comprising:

[0081] A first model building module 310 is configured to build a first canopy height model CHM based on the laser radar data and a first digital orthophoto model DOM based on the visible light data;

[0082] The second data acquisition module 320 is configured to perform high-precision matching on the first CHM and the first DOM to obtain a matched second CHM and a second DOM;

[0083] A missing data acquisition module 330 is configured to spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM to obtain a missing area of ​​the single tree range of the second CHM;

[0084] A single tree result forming module 340 is configured to extract the single tree result of the third CHM from the missing area, and merge the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result;

[0085] The target data extraction module 350 is configured to extract single tree data based on the target single tree result.

[0086] In an exemplary embodiment of the present application, the second data acquisition module 320 is configured as follows:

[0087] Perform high-precision matching on the first CHM and the first DOM to obtain a displacement difference between the first CHM and the first DOM;

[0088] Move the first DOM toward the first CHM by the displacement difference, use the first CHM as the matched second CHM, and use the first DOM after moving by the displacement difference as the matched second DOM; or

[0089] The first CHM is moved toward the first DOM by the displacement difference, the first DOM is used as the matched second DOM, and the first CHM after being moved by the displacement difference is used as the matched second CHM.

[0090] In an exemplary embodiment of the present application, the device further includes: a DOM single tree result acquisition module 360 ​​and a CHM single tree result acquisition module 370;

[0091] The DOM single tree result acquisition module 360 ​​is configured to segment the second DOM based on a deep learning neural network model to obtain single tree results of the second DOM.

[0092] The CHM single tree result acquisition module 370 is configured to perform global median filtering on the second CHM, and then use a local extremum method to acquire the data after the global median filtering to form the single tree result of the second CHM.

[0093] In an exemplary embodiment of the present application, the device also includes: a deep learning model training module 380, configured to crop the training data and label the training data using a deep learning image annotation method to obtain sample data; and train the initial model based on the sample data to obtain the deep learning neural network model.

[0094] In an exemplary embodiment of the present application, the DOM single tree result acquisition module 360 ​​is configured to segment the second DOM based on the deep learning neural network model to obtain a binary distribution map of the second DOM; perform opening and closing operations on the binary distribution map and convert it into vector surface data to form a single tree result of the second DOM.

[0095] In an exemplary embodiment of the present application, the missing data acquisition module 330 is configured to spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM, extract the area in the second DOM that does not overlap with the single tree results of the second CHM, and form the missing area of ​​the single tree results of the second CHM.

[0096] In an exemplary embodiment of the present application, the single tree result forming module 340 is configured to perform global median filtering on the missing area, and then use the local extreme value method to obtain the data after the global median filtering to form the single tree result of the third CHM; merge the single tree result of the second CHM and the single tree result of the third CHM to form the target single tree result.

[0097] In the embodiment of the present invention, the first canopy height model CHM established based on laser radar data and the first digital orthophoto model DOM established based on visible light data are matched with the first CHM and the first DOM with high precision to obtain the matched second CHM and the second DOM; the single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed to obtain the missing area of ​​the single tree range of the second CHM; the single tree results of the third CHM are extracted from the missing area, and the single tree results of the second CHM and the single tree results of the third CHM are merged to form the target single tree results; and the single tree data is extracted based on the target single tree results. The spectrum and texture information of the optical DOM data and the high-precision structural information of the laser radar CHM are fully combined, so that the accuracy of single tree segmentation is improved, which has good practicality and popularization.

[0098] Reference below Figure 4 The electronic device 40 according to an embodiment of the present application is described. Figure 4The electronic device 40 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0099] like Figure 4 As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: at least one processing unit 410, at least one storage unit 420, and a bus 430 connecting different system components (including the storage unit 420 and the processing unit 410).

[0100] The storage unit stores program codes, which can be executed by the processing unit 410, so that the processing unit 410 performs the steps according to various exemplary embodiments of the present invention described in the description of the exemplary method described above in this specification. For example, the processing unit 410 can perform the following steps: Figure 1 The steps shown in .

[0101] The storage unit 420 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 4201 and / or a cache storage unit 4202 , and may further include a read-only storage unit (ROM) 4203 .

[0102] The storage unit 420 may also include a program / utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0103] Bus 430 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0104] The electronic device 40 may also communicate with one or more external devices 500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 40, and / or communicate with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 450. The input / output (I / O) interface 450 is connected to a display unit 440. In addition, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 460. As shown, the network adapter 460 communicates with other modules of the electronic device 40 via a bus 430. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0105] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0106] In an exemplary embodiment of the present application, a computer-readable storage medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the method described in the above method embodiment.

[0107] According to one embodiment of the present application, a program product for implementing the method in the above method embodiment is also provided, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0108] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0109] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0110] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0111] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as JAVA, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0112] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0113] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0114] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0115] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

Claims

1. A method for extracting single tree data, characterized in that: The method comprises: Establishing the first canopy height model CHM based on the LiDAR data and establishing the first digital orthophoto model DOM based on the visible light data; Performing high-precision matching on the first CHM and the first DOM to obtain a matched second CHM and a second DOM, including: Perform high-precision matching on the first CHM and the first DOM to obtain a displacement difference between the first CHM and the first DOM; Move the first DOM to the first CHM by the displacement difference, use the first CHM as the matched second CHM, and use the first DOM after moving by the displacement difference as the matched second DOM; or Move the first CHM to the first DOM by the displacement difference, use the first DOM as the matched second DOM, and use the first CHM after moving by the displacement difference as the matched second CHM; Spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree results of the second CHM; Extracting the single tree result of the third CHM from the missing area, and merging the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result; Extract single tree data based on the target single tree result.

2. The method according to claim 1, characterized in that Before spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM, the method further includes: Segmenting the second DOM based on a deep learning neural network model to obtain a single tree result of the second DOM; After performing global median filtering on the second CHM, the data after the global median filtering is obtained using a local extreme value method to form a single tree result of the second CHM.

3. The method according to claim 1, characterized in that The method further comprises: The training data is cut and labeled using a deep learning image annotation method to obtain sample data; The initial model is trained based on the sample data to obtain the deep learning neural network model.

4. The method according to claim 2, characterized in that: Segmenting the second DOM based on a deep learning neural network model to obtain a single tree result of the second DOM includes: Segmenting the second DOM based on the deep learning neural network model to obtain a binary distribution map of the second DOM; The binary distribution map is converted into vector surface data after opening and closing operations are performed to form a single tree result of the second DOM.

5. The method according to claim 1, characterized in that Spatially superimposing the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree results of the second CHM, including: The single tree results of the second DOM and the single tree results of the second CHM are spatially superimposed, and the area in the second DOM that does not overlap with the single tree results of the second CHM is extracted to form the missing area of ​​the single tree results of the second CHM.

6. The method according to claim 1, characterized in that Extracting a single tree result of the third CHM from the missing region, and merging the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result, including: After performing global median filtering on the missing area, the data after the global median filtering is obtained using a local extreme value method to form a single tree result of the third CHM; The single tree result of the second CHM and the single tree result of the third CHM are combined to form a target single tree result.

7. A single tree data extraction device, characterized in that: The device comprises: A first model building module is configured to build a first canopy height model CHM based on the laser radar data and a first digital orthophoto model DOM based on the visible light data; A second data acquisition module is configured to perform high-precision matching on the first CHM and the first DOM to obtain a matched second CHM and a second DOM; A missing data acquisition module is configured to spatially superimpose the single tree results of the second DOM and the single tree results of the second CHM to obtain the missing area of ​​the single tree range of the second CHM; a single tree result forming module, configured to extract the single tree result of the third CHM from the missing area, and merge the single tree result of the second CHM and the single tree result of the third CHM to form a target single tree result; A target data extraction module, configured to extract single tree data based on the target single tree result; The second data acquisition module is configured to perform high-precision matching on the first CHM and the first DOM to obtain a displacement difference between the first CHM and the first DOM; Move the first DOM to the first CHM by the displacement difference, use the first CHM as the matched second CHM, and use the first DOM after moving by the displacement difference as the matched second DOM; or The first CHM is moved toward the first DOM by the displacement difference, the first DOM is used as the matched second DOM, and the first CHM after being moved by the displacement difference is used as the matched second CHM.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images