Method and device for complementing hillside data

Abstract

The invention provides a hillside data completion method and device, and relates to the technical field of three-dimensional point cloud data completion, and the method comprises the steps: obtaining a plurality of pieces of three-dimensional point cloud data of a hillside in an area in front of an unmanned plane; determining three-dimensional information of each piece of three-dimensional point cloud data in the plurality of pieces of three-dimensional point cloud data; according to the height value in the three-dimensional information of each piece of three-dimensional point cloud data, determining a two-dimensional image without the first region; according to the edge point of the first area, determining a repair mask which is used for specifying a second area needing to be complemented; determining a pixel value of each pixel point in the second region according to the repair mask and a completion algorithm; and according to the pixel value of each pixel point in the second area and the position of each pixel point in the second area, complementing the three-dimensional point cloud data missing from the hillside. According to the method, the height value in the three-dimensional information of the three-dimensional point cloud data is projected to the two-dimensional image, and the hillside data is complemented by adopting the complementation algorithm, so that the flight safety of the unmanned aerial vehicle is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional point cloud data completion, and more specifically, to a method and device for hillside data completion. Background Art

[0002] During the flight of drones, obstacles are often encountered. Currently, obstacles can be detected by laser radar, which can accurately perceive the shape of obstacles. However, since the laser radar is installed on the top, bottom or side of the drone, the field of view of the laser radar is limited, and it is difficult to scan the entire obstacle or terrain in all directions. For example, if the obstacle is a hillside, when the hillside is high, the laser radar cannot scan the entire terrain during the scanning process, which will cause the height information of the hillside to be lost, and the safety of the drone flight cannot be guaranteed.

[0003] In view of this, how to obtain the overall picture of the hillside when the LiDAR field of view is limited is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a method and device for completing hillside data, which aims to complete the three-dimensional point cloud data of the hillside when the laser radar field of view is limited, so as to ensure the safe flight of the drone.

[0005] In a first aspect, a method for completing hillside data is provided, comprising: obtaining multiple three-dimensional point cloud data of a hillside in an area in front of an unmanned aerial vehicle, the multiple three-dimensional point cloud data being obtained by scanning a laser radar carried by the unmanned aerial vehicle; determining three-dimensional information of each three-dimensional point cloud data in the multiple three-dimensional point cloud data according to the multiple three-dimensional point cloud data, the three-dimensional information including a height value; determining a two-dimensional image of a missing first area according to the height value in the three-dimensional information of each three-dimensional point cloud data; determining a repair mask according to edge points of the first area, the repair mask being used to specify a second area to be completed; determining a pixel value of each pixel in the second area according to the repair mask and a completion algorithm; and completing the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel in the second area and the position of each pixel in the second area.

[0006] Based on the above scheme, the acquired three-dimensional point cloud data is projected onto a two-dimensional plane to obtain a two-dimensional image and the first area missing in the two-dimensional image, and the pixel values ​​of the edge points of the first area are used as a repair mask, and then the second area is completed through a completion algorithm, and the pixel points in the completed second area are used as the height value of the missing obstacle, thereby completing the missing three-dimensional point cloud data of the obstacle. In this way, when the field of view is limited, the laser radar can convert the three-dimensional point cloud data into a two-dimensional image, and obtain all the three-dimensional point cloud data of the hillside through the completion algorithm, so that the drone can obtain the full picture of the hillside in time, thereby ensuring the safety of the drone flight.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the three-dimensional information of each three-dimensional point cloud data in the multiple three-dimensional point cloud data is determined based on the multiple three-dimensional point cloud data, including: mapping the multiple three-dimensional point cloud data into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, the three-dimensional information also including a horizontal coordinate value and a vertical coordinate value.

[0008] In combination with the first aspect, in certain implementations of the first aspect, a two-dimensional image in which the first area is missing is determined based on the height value in the three-dimensional information of each three-dimensional point cloud data, including: determining the resolution of the two-dimensional image and the position of the pixel points corresponding to each three-dimensional point cloud data in the two-dimensional image based on the horizontal coordinate value and the vertical coordinate value in the three-dimensional information of each three-dimensional point cloud data; determining the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image based on the height value in the three-dimensional information of each three-dimensional point cloud data; determining the two-dimensional image based on the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

[0009] Based on the above scheme, the height value in the three-dimensional information of each three-dimensional point cloud data is used as the pixel value of each pixel point in the two-dimensional image. In this way, the height value of the obstacle actually measured is used as the pixel value of the two-dimensional image, so that the acquired first area is more accurate, thereby improving the accuracy of the completed three-dimensional point cloud data.

[0010] In combination with the first aspect, in some implementations of the first aspect, determining the repair mask according to the edge points of the first area includes: using pixel values ​​of the edge points of the first area as the repair mask.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the pixel value of each pixel in the second area is determined according to the repair mask and the completion algorithm, including: determining a repair mask image according to the repair mask, the repair mask image being the same size as the two-dimensional image; setting a repair radius corresponding to each pixel in the second area; and determining the pixel value of each pixel in the second area according to the repair mask image, the repair radius and the completion algorithm.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the missing three-dimensional point cloud data of the hillside is completed based on the pixel value of each pixel point in the second area and the position of each pixel point in the second area, including: using the pixel value of each pixel point in the second area as the missing height value of the hillside; using the horizontal coordinate value and the vertical coordinate value of each pixel point in the second area as the missing horizontal coordinate value and the vertical coordinate value of the hillside; completing the missing three-dimensional point cloud data of the hillside according to the missing height value of the hillside and the missing horizontal coordinate value and vertical coordinate value of the hillside.

[0013] In a second aspect, a device for completing hillside data is provided, which includes an acquisition module, a determination module and a completion module. The acquisition module is used to acquire multiple three-dimensional point cloud data of the hillside in the area in front of the drone, and the multiple three-dimensional point cloud data are obtained by scanning the laser radar carried by the drone; the determination module is used to determine the three-dimensional information of each three-dimensional point cloud data in the multiple three-dimensional point cloud data according to the multiple three-dimensional point cloud data, and the three-dimensional information includes a height value; the determination module is also used to determine the two-dimensional image of the missing first area according to the height value in the three-dimensional information of each three-dimensional point cloud data; the determination module is also used to determine the repair mask according to the edge points of the first area, and the repair mask is used to specify the second area to be completed; the determination module is also used to determine the pixel value of each pixel in the second area according to the repair mask and the completion algorithm; the completion module is used to complete the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel in the second area.

[0014] In combination with the second aspect, in certain implementations of the second aspect, the determination module is specifically used to: map the multiple three-dimensional point cloud data into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, and the three-dimensional information also includes a horizontal coordinate value and a vertical coordinate value.

[0015] In combination with the second aspect, in certain implementations of the second aspect, the determination module is further specifically used to: determine the resolution of the two-dimensional image and the position of the pixel points corresponding to each three-dimensional point cloud data in the two-dimensional image according to the horizontal coordinate value and the vertical coordinate value in the three-dimensional information of each three-dimensional point cloud data; determine the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image according to the height value in the three-dimensional information of each three-dimensional point cloud data; determine the two-dimensional image according to the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

[0016] In a third aspect, a device for completing hillside data is provided, comprising a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute a method in any possible implementation of the first aspect described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic block diagram of a method for hillside data completion provided in this application.

[0018] Figure 2 It is a schematic diagram of the Cartesian coordinate system provided in this application.

[0019] Figure 3 It is a schematic diagram of the first area provided in this application.

[0020] Figure 4 It is a schematic block diagram of a device for completing hillside data provided by the present application. DETAILED DESCRIPTION

[0021] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0022] The present application relates to the field of obstacle detection technology, that is, scanning the area in front of a laser radar carrier by a laser radar to obtain point cloud data of the obstacle in front, and then processing the obtained point cloud data, for example, rasterizing the point cloud data to obtain the shape and position of the obstacle.

[0023] In some obstacle detection methods, when the laser radar scans the area ahead and obtains the point cloud data of the obstacle, due to the limited field of view of the laser radar, it is impossible to obtain the full picture of the obstacle during the scanning process, for example, it is impossible to obtain the full picture of the hillside. Usually, a laser radar is installed on the drone. When the field of view of the laser radar is limited, the hillside cannot be completely scanned during the laser radar scanning. In this way, part of the hillside data, such as height data, will be missing in the three-dimensional point cloud data obtained by the laser radar scanning. Therefore, when the scanned point cloud data is used to construct a three-dimensional map of the hillside, a part of the shape of the hillside is missing, resulting in the drone being unable to accurately obtain the full picture of the hillside, and thus unable to ensure the safety of the drone during flight.

[0024] In view of this, an embodiment of the present application provides a method for completing hillside data, by determining the two-dimensional image of the missing first area through the height value in the three-dimensional information of the acquired three-dimensional point cloud data, and then completing the two-dimensional image through an image completion algorithm, and then completing the data to the three-dimensional point cloud data, and using the completed three-dimensional point cloud data to obtain the overall picture of the hillside to ensure the safe flight of the drone.

[0025] Figure 1 A schematic block diagram of a method 100 for completing hillside data provided in an embodiment of the present application is shown.

[0026] like Figure 1 As shown, the method 100 for completing hillside data includes the following steps.

[0027] S110, obtaining a plurality of three-dimensional point cloud data of a hillside in the area in front of the drone, wherein the plurality of three-dimensional point cloud data is obtained by scanning a laser radar carried by the drone.

[0028] S120. Determine three-dimensional information of each three-dimensional point cloud data in the plurality of three-dimensional point cloud data according to the plurality of three-dimensional point cloud data, wherein the three-dimensional information includes a height value.

[0029] S130: Determine a two-dimensional image in which the first area is missing according to the height value in the three-dimensional information of each three-dimensional point cloud data.

[0030] S140: Determine a repair mask according to the edge points of the first region, where the repair mask is used to specify a second region to be completed.

[0031] S150: Determine the pixel value of each pixel in the second area according to the repair mask and the completion algorithm.

[0032] S160: Complete the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel point in the second area and the position of each pixel point in the second area.

[0033] In the embodiment of the present application, the carrier of the laser radar can be a drone, and the laser radar can be located on the top, bottom or side of the drone. By mapping the three-dimensional point cloud data detected by the laser radar, the shape and position of the hillside ahead can be identified to ensure the safe flight of the drone.

[0034] It should be noted that in the present application, a hillside can be understood as an obstacle existing in the area in front of the drone.

[0035] In the above step S110, the laser radar scans the scene to obtain the three-dimensional point cloud data in the field of view. For each scan result, the laser radar transmits the three-dimensional point cloud data to the controller via the transmission control protocol (TCP) or the user datagram protocol (UDP). The execution subject of the embodiment of the present application is the controller, which is used to control the drone to navigate in the air and perform various tasks. The controller may include a handheld controller, an autopilot controller, a ground control station, etc.

[0036] Specifically, the laser radar scans the area in front of the drone to obtain three-dimensional point cloud data related to the hillside in the front area, and the three-dimensional point cloud data is stored in a data packet, for example, a PCAP data packet. Further, the PCAP data packet is transmitted to the controller via TCP or UDP, and the controller parses the PCAP data packet to obtain the three-dimensional coordinates corresponding to each point cloud data in the three-dimensional point cloud data.

[0037] In the above step S120, the acquired three-dimensional point cloud data are mapped into a Cartesian coordinate system to determine the three-dimensional information of the plurality of three-dimensional point cloud data, wherein the three-dimensional information includes the height value of the hillside.

[0038] Specifically, the acquired three-dimensional point cloud data is mapped to the established Cartesian coordinate system to obtain the three-dimensional information of each three-dimensional point cloud data in the Cartesian coordinate system. The three-dimensional information of each three-dimensional point cloud data includes a horizontal coordinate value, a vertical coordinate value and a height value. Further, based on the height value, the two-dimensional image of the first area is determined.

[0039] Among them, according to multiple three-dimensional point cloud data, the three-dimensional information of each three-dimensional point cloud data in the multiple three-dimensional point cloud data is determined, including: mapping the multiple three-dimensional point cloud data into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, and the three-dimensional information also includes a horizontal coordinate value and a vertical coordinate value.

[0040] Figure 2 A schematic diagram of a Cartesian coordinate system provided in an embodiment of the present application is shown.

[0041] like Figure 2 As shown, the Cartesian coordinate system includes the origin O, the X-axis, the Y-axis, and the Z-axis. Among them, the Z-axis is perpendicular to the XOY plane formed by the X-axis and the Y-axis, the X-axis is to the right, and the Y-axis is counterclockwise at a 90° angle to the X-axis.

[0042] Furthermore, each of the three-dimensional point cloud data in the plurality of three-dimensional point cloud data is mapped to the Cartesian coordinate system to determine the three-dimensional information of each of the three-dimensional point cloud data in the plurality of three-dimensional point cloud data.

[0043] Specifically, suppose that each 3D point cloud data is mapped to the Cartesian coordinate system, and the position of each 3D point cloud data is obtained, and P i (x i ,y i ,z i ), where i represents the 3D point cloud data scanned by the i-th laser radar, x i Indicates the coordinate value of the i-th point cloud data on the X-axis, that is, the horizontal coordinate value; y i Indicates the coordinate value of the i-th point cloud data on the Y axis, that is, the ordinate value; z i Represents the coordinate value of the i-th point cloud data on the Z axis, that is, the vertical coordinate value.

[0044] In the above step S130, after the height value in the three-dimensional information of each three-dimensional point cloud data is obtained, a two-dimensional image with the first region missing can be determined by projecting the height value onto a two-dimensional plane.

[0045] Specifically, the height value in the three-dimensional information of each three-dimensional point cloud data is projected onto a two-dimensional plane to obtain a two-dimensional image, and when some three-dimensional point cloud data of the hillside is missing in the three-dimensional point cloud data, there will be a hole in the two-dimensional image, and the hole is the first area. That is to say, the first area is the area of ​​the hillside that the laser radar has not scanned due to its limited field of view.

[0046] Wherein, determining the two-dimensional image of the missing first area according to the height value in the three-dimensional information of each three-dimensional point cloud data includes the following steps.

[0047] Step 1: Determine the resolution of the two-dimensional image and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image according to the horizontal coordinate value and the vertical coordinate value in the three-dimensional information of each three-dimensional point cloud data.

[0048] Specifically, the resolution of the two-dimensional image and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image are determined according to the horizontal coordinate value and the vertical coordinate value in the three-dimensional coordinate corresponding to each three-dimensional point cloud data. The resolution of the two-dimensional image is the horizontal resolution × the vertical resolution, the horizontal resolution is the maximum horizontal coordinate value in the three-dimensional point cloud data, and the vertical resolution is the maximum vertical coordinate value in the three-dimensional point cloud data; for each three-dimensional point cloud data in the three-dimensional point cloud data, the horizontal coordinate value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image is the horizontal coordinate value of the three-dimensional point cloud data, and the vertical coordinate value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image is the vertical coordinate value of the three-dimensional point cloud data.

[0049] Step 2: Determine the pixel value of the pixel point corresponding to each 3D point cloud data in the 2D image according to the height value in the 3D information of each 3D point cloud data.

[0050] Specifically, for each three-dimensional point cloud data in the three-dimensional point cloud data, the height value in the three-dimensional coordinates corresponding to each three-dimensional point cloud data is used as the pixel value of each three-dimensional point cloud data in the two-dimensional image. That is to say, the pixel value of each three-dimensional point cloud data in the two-dimensional image is the height value corresponding to each three-dimensional point cloud data. In this way, the height value of the scanned obstacle is directly used as the pixel value in the two-dimensional image, which can ensure the accuracy when the first area is completed using the first completion algorithm after projection.

[0051] Step 3: Determine the two-dimensional image according to the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

[0052] Among them, the laser radar will cause the hillside height value to be missing due to the limited field of view during the scanning process. Therefore, when the three-dimensional point cloud data is projected onto the two-dimensional plane through the above steps, a hole will be formed on the obtained two-dimensional image, and the hole is the first area missing in the two-dimensional image. In other words, each pixel point in the first area is missing the corresponding pixel value, and the missing pixel values ​​in the first area need to be further supplemented by a completion algorithm.

[0053] Figure 3 A schematic diagram of the first area provided in an embodiment of the present application is shown.

[0054] like Figure 3 As shown in Figure 2, the three-dimensional point cloud data is projected onto the XOY plane, and the resulting two-dimensional image is as follows: Figure 3 As shown, the black area corresponds to the three-dimensional point cloud data of the hillside scanned by the laser radar, and the white area is the first area.

[0055] In the above step S140, according to the edge points of the first region, the pixel values ​​of the edge points of the first region are used as a repair mask, and the repair mask is used to specify the second region to be completed.

[0056] Specifically, the repair mask is determined according to the edge points of the first area, including: the pixel values ​​of the edge points of the first area are used as the repair mask. Among them, the pixel values ​​corresponding to the pixel points in the second area are the pixel values ​​that need to be supplemented, and the pixel values ​​that need to be supplemented are the height values ​​of the missing hillside. After obtaining the two-dimensional image, the edge points of the first area can be obtained, and the pixel values ​​of the edge points of the first area are used as the repair mask. The area that needs to be repaired can be determined through the repair mask, that is, the repair mask is used to identify the second area, so that the area that needs to be supplemented can be determined through the supplement algorithm later, thereby completing the missing pixel values ​​of the second area.

[0057] In the above step S150, the second region to be completed indicated by the repair mask is completed according to the repair mask and the completion algorithm, and then the pixel value of each pixel in the second region is determined.

[0058] Specifically, the completion algorithm may be any algorithm that can be used to repair an image, for example, an image repair algorithm based on a fast marching method (FMM) or a fluid dynamics-based image repair algorithm, wherein the FMM is also called the NPAINT_TELEA algorithm, and the fluid dynamics-based image repair algorithm is also called the NPAINT_NS algorithm.

[0059] It should be noted that, in the embodiment of the present application, the first area and the second area may be the same area, or the second area may be larger than the first area, or the second area may be smaller than the first area. When the second area is larger than the first area, it can be considered that when obtaining edge points of the first area, the number of edge points obtained is large. When the second area is smaller than the first area, it can be considered that when obtaining edge points of the first area, the number of edge points obtained is small.

[0060] In the embodiment of the present application, taking the completion algorithm as the FMM algorithm as an example, the obtained two-dimensional image is repaired, that is, the second area is completed, which specifically includes the following steps.

[0061] Step 1: Determine a repair mask image according to the repair mask, where the repair mask image has the same size as the two-dimensional image.

[0062] Specifically, a repair mask image with the same size as the above two-dimensional image is determined, and the repair mask in the repair mask image includes a non-zero value mask and a zero value mask, wherein the non-zero value mask represents an area that needs to be repaired, for example, the second area needs to be repaired, and the zero value mask represents an area that does not need to be repaired.

[0063] Step 2: Set the repair radius corresponding to each pixel point in the second area.

[0064] Specifically, in the process of completing the second area, the FMM algorithm first starts from the boundary of the second area, and searches for similar pixel values ​​within the repair radius for each pixel point according to the set repair radius to determine the pixel value corresponding to the pixel point.

[0065] Step 3: Determine the pixel value of each pixel in the second area according to the repair mask image, the repair radius and the completion algorithm.

[0066] Specifically, in the process of completing the second area, the FMM algorithm starts from the boundary of the second area, and advances from the outside to the inside according to the set repair radius until the pixel values ​​corresponding to all pixel points in the second area are completed. The FMM algorithm will calculate new pixel values ​​to update the non-zero pixel values ​​corresponding to the repair mask in the second area, obtain the missing pixel values ​​in the second area, and realize the completion of the two-dimensional image.

[0067] In the above step S160, after the second area is completed, the missing height value and the missing position of the hillside can be determined according to the pixel value of each pixel point in the completed second area and the position of each pixel point in the second area. Specifically, the following steps are included:

[0068] Step 1: Use the pixel value of each pixel point in the second area as the missing height value of the hillside.

[0069] Step 2: The horizontal coordinate value and the vertical coordinate value of each pixel point in the second area are used as the horizontal coordinate value and the vertical coordinate value of the missing hillside.

[0070] Step 3: Complete the missing three-dimensional point cloud data of the hillside according to the missing height value of the hillside and the missing horizontal and vertical coordinate values ​​of the hillside.

[0071] More specifically, the pixel value of each pixel point in the second area is used as the missing height value of the hillside, and the position information of each pixel point in the second area is used as the missing position information of the hillside to complete the completion of the three-dimensional point cloud data. Among them, the position information of each pixel point in the second area includes a horizontal coordinate value and a vertical coordinate value, then the horizontal coordinate value corresponding to the missing height value of the hillside is the horizontal coordinate value of each pixel point in the second area, and the vertical coordinate value corresponding to the missing height value of the hillside is the vertical coordinate value of each pixel point in the second area.

[0072] Furthermore, the missing height value and position information of the hillside are converted into three-dimensional coordinates and mapped in the Cartesian coordinate system to complete the three-dimensional point cloud data.

[0073] In the embodiment of the present application, the height value in the three-dimensional information of the acquired three-dimensional point cloud data is used to determine the two-dimensional image and the first area missing in the two-dimensional image, and the pixel value of the edge point of the first area is used as the repair mask, and then the completion algorithm is used to complete the second area that needs to be completed specified by the repair mask, and the pixel points in the completed second area are used as the missing height value of the hillside, thereby completing the missing three-dimensional point cloud data of the hillside. In this way, when the field of view is limited, the laser radar can convert the three-dimensional point cloud data into a two-dimensional image, and obtain the missing three-dimensional point cloud data of the hillside through the completion algorithm, so that the drone can obtain the full picture of the hillside in time, thereby ensuring the safety of the drone flight.

[0074] Figure 4 A schematic block diagram of a device 200 for completing hillside data provided in an embodiment of the present application is shown.

[0075] like Figure 4 As shown, the device 200 includes an acquisition module 210 , a determination module 220 and a completion module 230 .

[0076] The acquisition module 210 is used to acquire a plurality of three-dimensional point cloud data of the hillside in the area in front of the drone, and the plurality of three-dimensional point cloud data are obtained by scanning by a laser radar carried by the drone.

[0077] Specifically, the laser radar scans the area in front of the drone to obtain three-dimensional point cloud data related to the hillside in the front area, and the three-dimensional point cloud data is stored in a data packet, for example, a PCAP data packet. Further, the PCAP data packet is transmitted to the controller via TCP or UDP, and the controller parses the PCAP data packet to obtain the three-dimensional coordinates corresponding to each point cloud data in the three-dimensional point cloud data.

[0078] The determination module 220 is used to determine the three-dimensional information of each three-dimensional point cloud data in the plurality of three-dimensional point cloud data according to the plurality of three-dimensional point cloud data, wherein the three-dimensional information includes a height value.

[0079] Specifically, the acquired three-dimensional point cloud data is mapped to the established Cartesian coordinate system to obtain the three-dimensional information of each three-dimensional point cloud data in the Cartesian coordinate system. The three-dimensional information of each three-dimensional point cloud data includes a horizontal coordinate value, a vertical coordinate value and a height value. Further, based on the height value, the two-dimensional image of the first area is determined.

[0080] The determination module 220 is specifically used to map the multiple three-dimensional point cloud data into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, wherein the three-dimensional information also includes a horizontal coordinate value and a vertical coordinate value.

[0081] The determination module 220 is further configured to determine the two-dimensional image in which the first area is missing according to the height value in the three-dimensional information of each three-dimensional point cloud data.

[0082] Specifically, the height value in the three-dimensional information of each three-dimensional point cloud data is projected onto a two-dimensional plane to obtain a two-dimensional image, and when some three-dimensional point cloud data of the hillside is missing in the three-dimensional point cloud data, there will be a hole in the two-dimensional image, and the hole is the first area. That is to say, the first area is the area of ​​the hillside that the laser radar has not scanned due to its limited field of view.

[0083] The determining module 220 is further specifically configured to:

[0084] Determine the resolution of the two-dimensional image and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image according to the abscissa value and the ordinate value in the three-dimensional information of each three-dimensional point cloud data;

[0085] Determine, according to the height value in the three-dimensional information of each three-dimensional point cloud data, a pixel value of a pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image;

[0086] The two-dimensional image is determined according to the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

[0087] The determination module 220 is further configured to determine a repair mask according to the edge points of the first region, where the repair mask is used to specify the second region to be completed.

[0088] The determination module 220 is further configured to determine the pixel value of each pixel in the second area according to the repair mask and the completion algorithm.

[0089] The determination module 220 is further specifically configured to:

[0090] Determine a repair mask image according to the repair mask, where the repair mask image has the same size as the two-dimensional image;

[0091] Setting a repair radius corresponding to each pixel point in the second area;

[0092] The pixel value of each pixel point in the second area is determined according to the repair mask image, the repair radius and the completion algorithm.

[0093] Specifically, in the process of completing the second area, the FMM algorithm starts from the boundary of the second area, and advances from the outside to the inside according to the pixel values ​​within the set neighborhood radius until the pixel values ​​corresponding to all pixel points in the second area are completed. The FMM algorithm calculates new pixel values ​​to update the non-zero pixel values ​​corresponding to the repair mask in the second area, and obtains the missing pixel values ​​in the second area to complete the two-dimensional image.

[0094] The completion module 230 is used to complete the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel point in the second area.

[0095] The completion module 230 is specifically used for:

[0096] The pixel value of each pixel point in the second area is used as the missing height value of the hillside;

[0097] The horizontal coordinate value and the vertical coordinate value of each pixel point in the second area are used as the horizontal coordinate value and the vertical coordinate value of the missing hillside;

[0098] The missing three-dimensional point cloud data of the hillside are completed according to the missing height value of the hillside and the missing horizontal coordinate value and vertical coordinate value of the hillside.

[0099] More specifically, the pixel value of each pixel point in the second area is used as the missing height value of the hillside, and the position information of each pixel point in the second area is used as the missing position information of the hillside to complete the completion of the three-dimensional point cloud data. Among them, the position information of each pixel point in the second area includes a horizontal coordinate value and a vertical coordinate value, then the horizontal coordinate value corresponding to the missing height value of the hillside is the horizontal coordinate value of each pixel point in the second area, and the vertical coordinate value corresponding to the missing height value of the hillside is the vertical coordinate value of each pixel point in the second area.

[0100] Furthermore, the missing height value and position information of the hillside are converted into three-dimensional coordinates and mapped in the Cartesian coordinate system to complete the three-dimensional point cloud data.

[0101] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0102] Optionally, the computer-readable storage medium can be applied to the method of hillside data completion in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes in each method of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0103] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0104] Optionally, the computer program product can be applied to the method for completing hillside data in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes in each method in the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0105] The embodiment of the present application also provides a computer program.

[0106] Optionally, the computer program can be applied to the method for completing hillside data in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes in the various methods in the embodiments of the present application. For the sake of brevity, they are not described here.

[0107] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0108] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application.

[0109] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of the present application are not limited to this.

[0110] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application are the same as the meanings generally understood by those skilled in the art of the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application. The singular forms "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include majority forms, unless the context clearly indicates other meanings. In addition, the terms "first", "second" etc. are only used for description purposes and cannot be interpreted as indicating or suggesting relative importance.

[0111] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0114] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for completing hillside data, characterized in that: include: Acquire a plurality of three-dimensional point cloud data of a hillside in the area in front of the drone, wherein the plurality of three-dimensional point cloud data is obtained by scanning a laser radar carried by the drone; Determine three-dimensional information of each three-dimensional point cloud data in the plurality of three-dimensional point cloud data according to the plurality of three-dimensional point cloud data, wherein the three-dimensional information includes a height value; Determine a two-dimensional image where the first area is missing according to a height value in the three-dimensional information of each three-dimensional point cloud data; Determine a repair mask according to the edge points of the first area, where the repair mask is used to specify a second area to be completed; Determine a pixel value of each pixel in the second area according to the repair mask and the completion algorithm; The missing three-dimensional point cloud data of the hillside is completed according to the pixel value of each pixel point in the second area and the position of each pixel point in the second area.

2. The method according to claim 1, characterized in that Determining three-dimensional information of each three-dimensional point cloud data in the plurality of three-dimensional point cloud data according to the plurality of three-dimensional point cloud data includes: The multiple three-dimensional point cloud data are mapped into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, wherein the three-dimensional information further includes a horizontal coordinate value and a vertical coordinate value.

3. The method according to claim 2, characterized in that Determining the two-dimensional image of the missing first area according to the height value in the three-dimensional information of each three-dimensional point cloud data includes: Determine the resolution of the two-dimensional image and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image according to the abscissa value and the ordinate value in the three-dimensional information of each three-dimensional point cloud data; Determine, according to the height value in the three-dimensional information of each three-dimensional point cloud data, a pixel value of a pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image; The two-dimensional image is determined according to the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining a repair mask according to edge points of the first area includes: The pixel values ​​of the edge points of the first area are used as the repair mask.

5. The method according to any one of claims 1 to 3, characterized in that: Determining the pixel value of each pixel in the second area according to the repair mask and the completion algorithm includes: Determine a repair mask image according to the repair mask, wherein the repair mask image has the same size as the two-dimensional image; Setting a repair radius corresponding to each pixel point in the second area; A pixel value of each pixel point in the second area is determined according to the repair mask image, the repair radius and the completion algorithm.

6. The method according to any one of claims 1 to 3, characterized in that The method of completing the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel point in the second area and the position of each pixel point in the second area includes: Using the pixel value of each pixel point in the second area as the missing height value of the hillside; Using the horizontal coordinate value and the vertical coordinate value of each pixel point in the second area as the horizontal coordinate value and the vertical coordinate value of the missing hillside; The missing three-dimensional point cloud data of the hillside are completed according to the missing height value of the hillside and the missing horizontal coordinate value and vertical coordinate value of the hillside.

7. A device for completing hillside data, characterized in that: The device comprises an acquisition module, a determination module and a completion module. The acquisition module is used to acquire a plurality of three-dimensional point cloud data of a hillside in the area in front of the drone, wherein the plurality of three-dimensional point cloud data are obtained by scanning a laser radar carried by the drone; The determination module is used to determine the three-dimensional information of each three-dimensional point cloud data in the plurality of three-dimensional point cloud data according to the plurality of three-dimensional point cloud data, wherein the three-dimensional information includes a height value; The determination module is further used to determine the two-dimensional image of the missing first area according to the height value in the three-dimensional information of each three-dimensional point cloud data; The determining module is further used to determine a repair mask according to the edge points of the first area, wherein the repair mask is used to specify the second area to be completed; The determination module is further used to determine the pixel value of each pixel in the second area according to the repair mask and the completion algorithm; The completion module is used to complete the missing three-dimensional point cloud data of the hillside according to the pixel value of each pixel point in the second area.

8. The device according to claim 7, characterized in that The determination module is specifically used to: map the multiple three-dimensional point cloud data into a Cartesian coordinate system to determine the three-dimensional information of each three-dimensional point cloud data, wherein the three-dimensional information also includes a horizontal coordinate value and a vertical coordinate value.

9. The device according to claim 8, characterized in that The determining module is further specifically used for: Determine the resolution of the two-dimensional image and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image according to the abscissa value and the ordinate value in the three-dimensional information of each three-dimensional point cloud data; Determine, according to the height value in the three-dimensional information of each three-dimensional point cloud data, a pixel value of a pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image; The two-dimensional image is determined according to the resolution, the pixel value of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image, and the position of the pixel point corresponding to each three-dimensional point cloud data in the two-dimensional image.

10. A device for completing hillside data, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute: the method according to any one of claims 1 to 6.

Images