Three-dimensional surface reconstruction method and system based on single-frame real-time point cloud

Abstract

The invention discloses a three-dimensional surface reconstruction method and system based on single-frame real-time point cloud. The method comprises the following steps: collecting material surface point cloud data in real time; transferring the collected point cloud data from a laser radar coordinate system to a getting-off coordinate system, and filtering interference point cloud to obtain first point cloud data; converting the first point cloud data into a cylindrical coordinate system from a getting-off coordinate system, dividing the selected material area into a plurality of fan-shaped grids in the cylindrical coordinate system, and projecting the first point cloud data into each fan-shaped grid; calculating centroids of all points in each grid as grid sampling points; if the point set in the grid is empty, carrying out sampling point supplement on the empty grid; and constructing a quadrilateral grid based on the complemented grid sampling points and the grid indexes to which the grid sampling points belong, and rendering the quadrilateral grid to obtain a three-dimensional reconstruction result of the material surface. According to the method, the overall estimation value of the material area is completed on the basis of the data of the single-frame point cloud, and the timeliness and the construction quality of excavator construction are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of engineering machinery construction, and in particular to a three-dimensional surface reconstruction method and system based on single-frame real-time point cloud. Background Art

[0002] Excavators are engineering machinery that mainly digs soil and stone. As an efficient and fast construction machine, they are widely used in soil and stone construction in various construction projects. Unmanned excavators can operate autonomously in dangerous scenarios such as landslides and toxic gases, avoiding the driver from facing such dangerous scenarios. At the same time, they can also alleviate the excessive load on the driver's body caused by construction work. When operating autonomously, unmanned excavators need to accurately obtain the measurement results of the surface of the operating material in order to make autonomous decisions during the operation, plan the movement trajectory of the boom, dipper arm, and bucket, and complete the excavation action. The accuracy of the measurement results of the material surface is directly related to the efficiency of each excavation action, that is, it is directly related to the economic benefits generated by the unmanned excavator.

[0003] The existing solution for 3D reconstruction of material surfaces used by unmanned excavators is to place the Mid-100 LiDAR on a fixed bracket at a certain distance from the excavator, but not assembled with the excavator. Since the LiDAR does not move, the interference of motion factors on the point cloud can be minimized, but it is only applicable to fixed material areas selected in advance and cannot change with the movement of the excavator.

[0004] The Chinese patent application with application number 202310492389.5 discloses a method and device for three-dimensional reconstruction of the surface of an object based on point cloud data. The method collects point cloud data of the surface of an object based on the Livox Mid-70 laser radar, constructs a dense point cloud by superimposing the point cloud, and screens outliers by filtering. The XOY coordinate plane is divided into hseg regions, and the Z-axis direction is divided into vseg regions, and the processed point cloud is mapped to each subspace. The centroid of the point cloud set of each subspace is calculated as the sampling point. The subspaces are connected into a quadrilateral grid according to the geometric relationship, and the three-dimensional reconstruction result of the material surface is obtained. However, it relies on post-processing of the point cloud superposition to obtain the surface result, and needs to consider its own motion compensation. It is difficult to implement, the algorithm takes a long time, and does not consider the situation where the surface of the material pile is blocked during the movement of the excavator, and cannot effectively compensate for the blocked part of the point cloud.

[0005] The Chinese patent application number 202010941261.9 discloses a 2D laser radar-based excavator working surface reconstruction system and method. Based on the 2D laser radar installed on the excavator boom, the target plate and working surface installed on the rotating platform of the upper body of the excavator are scanned, and the original point cloud data is collected. In combination with the rotation angle sensor installed on the upper body of the excavator for collecting the rotation angle, the boom attitude angle and the three-dimensional contour surface coordinate data of the working surface are calculated and determined. However, the 2D laser radar only has plane information and is not sensitive to height information, making it difficult to accurately obtain a material surface with clear texture.

[0006] The Chinese patent application number 201910075865.7 discloses a material pile 3D reconstruction method, device, electronic device and computer readable medium. After receiving an instruction to control the rotation of the excavator body to transport materials, the method obtains the depth image of the excavated material pile collected by the binocular camera set on the side of the excavator's rotation direction, while keeping the binocular camera set on the other side of the excavator in a closed state, and its depth image sequence is not processed; after one cycle, the method performs 3D reconstruction based on the depth image sequence of the material pile to generate a 3D model of the material pile. However, the depth camera has low stability outdoors and is greatly affected by light.

[0007] In addition, the existing technology is limited to the solid waste scenario. The excavator itself does not perform any translational movement, but only digs and loads back and forth at a fixed point at the work point. It does not take into account the other working states of the excavator, and the working scene is relatively simple. Summary of the invention

[0008] The purpose of the present invention is to provide a three-dimensional surface reconstruction method and system based on a single-frame real-time point cloud, so as to solve the problem that the prior art relies on point cloud superposition post-processing to obtain three-dimensional surface reconstruction results, which is difficult to implement, the algorithm takes a long time, and does not consider the problem of occlusion on the surface of material piles during the movement of the excavator, thereby quickly, accurately and efficiently generating the three-dimensional surface of construction objects such as material piles.

[0009] To achieve the above object, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a three-dimensional surface reconstruction method based on a single-frame real-time point cloud, comprising:

[0011] The point cloud data of the material surface is collected in real time by using a laser radar; the laser radar is installed on the top of the excavator cab;

[0012] The collected point cloud data is transferred from the laser radar coordinate system to the excavator's off-vehicle coordinate system, and the interfering point cloud is filtered out to obtain the first point cloud data;

[0013] The first point cloud data is converted from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, the selected material area is divided into a plurality of sector grids in the cylindrical coordinate system, and the first point cloud data is projected into each sector grid;

[0014] Calculate the centroid of all points in each grid as the grid sampling point; if the point cloud in the grid is empty, fill in the sampling points of the empty grid;

[0015] A quadrilateral mesh is constructed based on the completed grid sampling points and the grid index to which the grid sampling points belong, and the quadrilateral mesh is rendered to obtain the three-dimensional reconstruction result of the material surface.

[0016] Furthermore, the real-time collection of material surface point cloud data includes:

[0017] At any time between the end of the excavator's rotation phase and the beginning of the excavation phase, the material surface point cloud data is collected.

[0018] Furthermore, according to the transfer relationship matrix from the laser radar coordinate system to the excavator off-vehicle coordinate system, the collected point cloud data is transferred from the laser radar coordinate system to the excavator off-vehicle coordinate system. The transfer relationship matrix from the laser radar coordinate system to the excavator off-vehicle coordinate system is:

[0019]

[0020] in, is the transfer relationship matrix from the laser radar coordinate system to the excavator's off-vehicle coordinate system, is the transfer relationship matrix from the laser radar coordinate system to the excavator coordinate system, is the transfer relationship matrix between the coordinate system on the excavator and the coordinate system off the excavator,

[0021]

[0022] Among them, R is the rotation matrix between the coordinate system on the excavator and the coordinate system off the excavator,

[0023]

[0024] is the rotation angle between the coordinate system of the excavator on board and the coordinate system of the excavator off board;

[0025] T is the translation vector between the coordinate system on the excavator and the coordinate system off the excavator:

[0026]

[0027] Wherein, Z is the height difference between the coordinate system on the excavator and the coordinate system off the excavator, and the superscript T is the transposition.

[0028] Further, filtering out interference point clouds includes:

[0029] The point cloud of the vehicle body and the point cloud of the moving parts are filtered out according to the size of the excavator itself, and some discrete points are filtered out through neighborhood radius filtering.

[0030] Furthermore, the material area is selected within a range of 5m <r < 11m,-45°< < 45°, (r, ) is the polar coordinate of a point.

[0031] Furthermore, the method for filling sampling points in the hole grid includes:

[0032] If the point cloud hole grid is at the edge of the material area, the width is one angle unit wide, and the grid hole state is retained;

[0033] If the point cloud hole grid is not at the edge of the material area, traverse all non-edge grids. If it is judged to be a hole grid, calculate the coordinates of the center point of the hole grid, use the calculated x and y coordinates of the center point coordinates as the x and y coordinates of the hole grid sampling point, search for the next non-empty grid sampling point of the hole grid along the x-axis direction, and perform linear interpolation based on the previous non-empty grid sampling point and the next non-empty grid sampling point of the hole grid to obtain the z coordinate of the hole grid sampling point.

[0034] Furthermore, constructing a quadrilateral mesh based on the completed grid sampling points and the grid indexes to which the grid sampling points belong includes:

[0035] The completed grid sampling points are used as feature points. Based on the grid index to which the grid sampling points belong, the corresponding three points of each grid sampling point are taken to construct the vertices, edges and diagonal edges of the quadrilateral mesh, and the splicing method of the quadrilateral mesh is determined according to whether the point set in the grid is empty.

[0036] Furthermore, constructing a quadrilateral mesh based on the completed grid sampling points and the grid indexes to which the grid sampling points belong includes:

[0037] For the grid sampling points at the edge of the material area or , do not continue to construct quadrilaterals, where width is the number of grids divided into the material area along the x direction, and height is the number of grids divided into the material area along the y direction. is the grid sampling point set;

[0038] For the remaining grid sampling points , according to the following formula, we can get the corresponding points P1, P2, and P3:

[0039]

[0040]

[0041]

[0042] Calculate the grid index of points P0, P1, P2, and P3, and query whether the grid point set corresponding to points P0, P1, P2, and P3 is a hole point based on the calculated grid index.

[0043] If none of the four points are empty points, a quadrilateral is constructed by splicing two triangles with a shared hypotenuse into a quadrilateral unit;

[0044] If one of the four points is a hole point, the remaining three points will be used to construct a triangle;

[0045] If two or more of the four points are empty points, no construction processing will be performed.

[0046] In a second aspect, the present invention provides a three-dimensional surface reconstruction system based on a single-frame real-time point cloud, comprising:

[0047] Laser radar, used for collecting material surface point cloud data in real time; the laser radar is installed on the top of the excavator cab;

[0048] The first coordinate conversion and preprocessing module is used to transfer the collected point cloud data from the laser radar coordinate system to the excavator off-vehicle coordinate system, and filter out the interfering point cloud to obtain the first point cloud data;

[0049] The second coordinate conversion and grid division module is used to convert the first point cloud data from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, divide the selected material area into a plurality of sector grids in the cylindrical coordinate system, and project the first point cloud data into each sector grid;

[0050] The grid sampling point calculation and point filling module is used to calculate the centroid of all points in each grid as the grid sampling point; if the point cloud in the grid is empty, the sampling points of the empty grid are filled;

[0051] The quadrilateral mesh generation module is used to construct a quadrilateral mesh based on the completed grid sampling points and the grid index to which the grid sampling points belong, and render the quadrilateral mesh to obtain a three-dimensional reconstruction result of the material surface.

[0052] Furthermore, the laser radar is installed on the top of the excavator cab in a downward tilted manner.

[0053] In a third aspect, the present invention provides a computing device comprising a processor and a memory, wherein the processor is used to implement the steps of the three-dimensional surface reconstruction method based on a single-frame real-time point cloud as described in any one of the first aspects when executing a computer program stored in the memory.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the three-dimensional surface reconstruction method based on a single-frame real-time point cloud as described in any one of the first aspects are implemented.

[0055] Compared with the prior art, the beneficial technical effects of the present invention are:

[0056] The present invention aims at the existing excavator sensor system, completes the overall estimation value of the material area based on the data of a single-frame point cloud, can quickly make a sampling estimation of the material area in the excavator's working space, and constructs the sampled grid points in the cylindrical coordinate system according to the working mode of the excavator, eliminating the steps of self-motion compensation and dense point downsampling required for multi-frame superposition of point clouds, saving algorithm time without losing the accuracy of three-dimensional surface reconstruction, and effectively improving the timeliness and construction quality of excavator construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of the method of the present invention;

[0058] Figure 2 This is a schematic diagram of the excavator coordinate system;

[0059] Figure 3 Schematic diagram of two ways of dividing quadrilateral grid;

[0060] Figure 4 Schematic diagram of the construction of the triangular unit at the edge;

[0061] Figure 5 Schematic diagram of the grid sampling points and quadrilateral grid generation effect;

[0062] Figure 6 This is a three-dimensional model diagram of an excavator;

[0063] Figure 7 This is a three-dimensional model diagram of a mining card;

[0064] Figure 8 This is the position diagram of the mining truck relative to the excavator;

[0065] Fig. 9 This is a visualization diagram of the overall model of materials, excavators, and mining trucks. DETAILED DESCRIPTION

[0066] The present invention will be further described below in conjunction with specific examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention.

[0067] The excavator usually rotates back and forth on the loading platform to complete the excavation of materials from the designated excavation area, and then transports them to the designated location (usually the mining truck location) for unloading. The loading platform is trapezoidal in shape, with a height of about 1.5-2.5 meters, a slope of about 75 degrees, and a width of 1 meter extended by the width of the excavation crawler. The material pile is located below the loading platform, and the surface features are concave material pits or raised material piles.

[0068] like Figure 1 As shown, the present invention provides a three-dimensional surface reconstruction method based on a single-frame real-time point cloud, comprising:

[0069] Step 1: collect material surface point cloud data in real time;

[0070] The present invention uses LiDAR (Light Detection and Ranging) to collect point cloud data on the surface of materials. LiDAR is the abbreviation of laser ranging system, which is a complex type of sensor that integrates three technologies: laser, global positioning system (GPS) and inertial measurement equipment. A major feature of LiDAR is that it is not affected by external ambient light, perceives environmental information in real time, and can obtain accurate and reliable three-dimensional data models. Therefore, LiDAR is suitable for three-dimensional reconstruction of construction environments in open-air scenes.

[0071] The frequency of the ranging information returned by the laser radar in the present invention is 10 Hz, the LiDAR model is ouster os 0, the horizontal field of view angle range is 360°, the angular resolution is about 0.35°, the vertical field of view angle range is 90°, and the angular resolution is about 0.70°.

[0072] In order to better observe the material pile area below the platform, the laser radar is installed on the top of the excavator cab in a downward tilt with an angle of about 36°.

[0073] During the excavator operation cycle, an excavation status parameter / Dig_Phase_Plan_Flag is used to record the current excavation status. It is agreed that when the excavator is performing a rotation action and a unloading action, / Dig_Phase_Plan_Flag is set to 0; and when the excavator is performing an excavation action, / Dig_Phase_Plan_Flag is set to 1. In the three stages, the excavator's field of view is small most of the time during the rotation stage. During the unloading stage, the material is not within the excavator's field of view and the position of the mining truck needs to be captured at this time. During the excavation stage, the working device overlaps with the material area, and the occlusion phenomenon is more obvious. After several tests and verifications, at the end of the rotation stage and the beginning of the excavation stage, that is, when / Dig_Phase_Plan_Flag changes from 0 to 1, the occlusion area is the smallest or almost non-existent. Selecting the point cloud at this moment can greatly alleviate the occlusion problem.

[0074] Therefore, in the present invention, the point cloud data of the material surface is collected at any time between the end of the excavator's rotation phase and the beginning of the excavation phase.

[0075] Step 2: Transfer the collected point cloud data from the laser radar coordinate system to the excavator's off-vehicle coordinate system, and filter out the interfering point cloud to obtain the first point cloud data;

[0076] like Figure 2 As shown, the present invention introduces the following coordinate systems:

[0077] under_carriage: the coordinate system of the excavator getting off the vehicle, which is consistent with the map coordinate system;

[0078] upper_carriage: The coordinate system of the excavator upper carriage, which is fixed relative to the lidar.

[0079] The laser radar is installed on the top of the excavator cab and is relatively fixed to the vehicle coordinate system. The transfer relationship matrix from the laser radar coordinate system to the excavator vehicle coordinate system can be obtained by prior sensor calibration. .

[0080] During the operation of the excavator, there is only one rotation angle between the coordinate system on the excavator and the coordinate system off the excavator in the heading angle. , thus we get the rotation matrix R:

[0081] .

[0082] Translation vector T between the coordinate system of getting on the vehicle and the coordinate system of getting off the vehicle:

[0083] ;

[0084] Where Z is the height difference between the coordinate system of getting on the vehicle and the coordinate system of getting off the vehicle, and the superscript T is the transposition.

[0085] The transfer relationship matrix between the real-time boarding coordinate system and the getting-off coordinate system is obtained as follows:

[0086]

[0087] Then the transfer relationship matrix from the laser radar coordinate system to the vehicle coordinate system is:

[0088]

[0089] According to the transfer relationship matrix from the laser radar coordinate system to the vehicle coordinate system , transfer the single-frame point cloud data collected in step 1 from the laser radar coordinate system to the excavator's off-vehicle coordinate system.

[0090] Then, the point cloud of the vehicle body and the point cloud of the moving parts are filtered out according to the size of the excavator itself, and some discrete points are filtered out by neighborhood radius filtering. The filtering method adopts the existing technology.

[0091] Step 3, converting the first point cloud data from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, dividing the selected material area into a plurality of sector grids in the cylindrical coordinate system, and projecting the first point cloud data into each sector grid;

[0092] The cylindrical coordinate system refers to a coordinate system that uses plane polar coordinates and z-direction height to define the spatial coordinates of an object. For an excavator, the working device is mainly composed of three major rods: the boom, the dipper stick, and the bucket, and other auxiliary connecting rods. During operation, the materials in front are loaded into the bucket through the rotation of each component around the hinge point. The overall operation is completed in a fan-shaped interval within a certain angle in the off-vehicle coordinate system. Therefore, it is more appropriate to use the cylindrical coordinate system to complete point cloud processing.

[0093] Project the point cloud to be processed in step 2 into the cylindrical coordinate system with the origin of the vehicle coordinate system as the origin.

[0094] For a point in a three-dimensional coordinate system , transformed into the cylindrical coordinate system, the coordinates after conversion are , the calculation formula is:

[0095]

[0096]

[0097] In this embodiment, the material area range is selected to be 5m < r < 11m, -45° < < 45°, divide the selected material area into several sector grids .

[0098] Among them, according to the coordinates Calculate the grid index to which point P belongs:

[0099]

[0100]

[0101] in , are the maximum and minimum values ​​of r, respectively. , They are The maximum and minimum values ​​of , width is the number of grids divided along the x direction, and height is the number of grids divided along the y direction.

[0102] Step 4: Calculate the centroid of all points in each grid as the grid sampling point; if the point cloud in the grid is empty, fill in the sampling points of the empty grid;

[0103] After projecting all point clouds to each sector grid, count the number of point clouds in each grid Traverse all grids, calculate the centroid of all points in each grid, and use it as the grid sampling point. The grid point sampling algorithm is as follows:

[0104]

[0105] Although the excavator is working on a high material platform, as the excavator continues to dig, there are still a few cases where the radar cannot illuminate the back of the material pile. The number of grid points in is 0.

[0106] The present invention adopts linear interpolation to fill in the sampling points of the hole grid, including two cases:

[0107] (1) The point cloud hole grid is located at the left and right edges of the material area, with a width of one angle unit, and the grid hole state is retained;

[0108] The point cloud data effect may be affected because the material area is at the edge of the radar's visible range. In this case, this part of the hole is retained to truly show the edge of the material pile;

[0109] (2) The point cloud hole grid is in the middle of the material, with a large width and unstable size:

[0110] It may be that the material pile presents convex material piles of varying sizes during the operation of the excavator. At this time, the back of the material pile is blocked, and the point cloud cannot be effectively collected, so linear interpolation is required.

[0111] Specifically, the method for completing the sampling points of the empty grid includes: traversing all non-edge grids to determine whether it is an empty grid. If so, record the index of the last non-empty grid, calculate the coordinates of the center point of the empty grid, use the x and y coordinates of the calculated center point coordinates as the x and y coordinates of the empty grid sampling point, and search for the next non-empty grid index of the empty grid along the x-axis direction. If found, linear interpolation is performed based on the last non-empty grid sampling point and the next non-empty grid sampling point of the empty grid to solve the z coordinate of the empty grid sampling point; if not found, jump out of the loop. The algorithm for completing the empty points is as follows:

[0112]

[0113] Step 5, constructing a quadrilateral mesh based on the completed grid sampling points and the grid indexes to which the grid sampling points belong, and rendering the quadrilateral mesh to obtain a three-dimensional reconstruction result of the material surface;

[0114] Quadrilateral mesh is a plane mesh division method. As the name implies, the smallest unit of its division is usually a quadrilateral, and occasionally a few special triangles are mixed in, such as at the edge of the surface. Compared with triangular mesh division, the vertices and edges of quadrilateral mesh are intuitively neater, and it also conforms to the calculation method of some characteristic edges and characteristic points of the material area in the case. Quadrilateral mesh is also often used in mining scenes to describe the surface morphology of material piles.

[0115] The present invention selects the centroid coordinates of the point set in the grid as the feature point. Therefore, it is impossible to ensure that the four points on the quadrilateral are located on the same plane. In fact, most quadrilateral grids are not on the same plane. In order to describe such irregular quadrilaterals, consider using two triangles with a common hypotenuse to splice into a quadrilateral unit. There are two splicing methods, such as Figure 3 shown.

[0116] The constructing of a quadrilateral grid based on the completed grid sampling points and the grid indexes to which the grid sampling points belong includes:

[0117] a) For the grid sampling points at the edge of the material area or , do not continue to construct the quadrilateral;

[0118] b) For the remaining grid sampling points , according to the following formula, we can get the corresponding points P1, P2, and P3:

[0119]

[0120]

[0121]

[0122] Make a quadrilateral , conforming to the right-hand spiral law, the overall normal vector points upward, which is intuitively consistent with the upper surface of the material pile;

[0123] Calculate the grid index of points P0, P1, P2, and P3, and query the grid point set corresponding to points P0, P1, P2, and P3 based on the calculated grid index Is there a hollow point in

[0124] If all four points are not empty points, then follow Figure 3 Construct the quadrilateral in the manner shown;

[0125] If one of the four points is a hole point, the remaining three points are Figure 4 Construct a triangle;

[0126] If two or more of the four points are empty points, no construction processing will be performed.

[0127] Then, the sampling point height of the current grid is calculated, and different colors are assigned to the quadrilateral grid according to the different sampling point heights of the quadrilateral grid.

[0128] The grid sampling points and quadrilateral mesh generation effects are as follows Figure 5 shown.

[0129] On the basis of the above, the material area is rendered in Rviz, and the excavator model and the mining truck model are added to realize three-dimensional surface visualization.

[0130] Under the Ubuntu 18.04 system, it is implemented based on the ROS system (Robot Operating System) and is tested on a 69-ton excavator, which can assist the excavator in achieving semi-autonomous excavation operations. RViz is an open source tool for visualizing robot systems. It is part of ROS and one of the most commonly used visualization tools in ROS. Rviz provides rich functions and customizable interfaces, allowing users to view robot models, real-time sensor data, and current construction environment maps in three dimensions, with a high degree of freedom. It supports various types of visualization objects, including point clouds, grid models, visualization robot models, and camera images. The present invention mainly uses the visualization robot model urdf of Rviz to load and display the three-dimensional model of the robot, and dynamically update it according to the actual joint state, so that users can intuitively understand the appearance and posture of the robot.

[0131] The 3D model of the excavator is roughly divided into five parts: the lower part, the upper part, the boom, the dipper, and the bucket. According to the Denavit-Hartenberg rule, a coordinate system is fixed at each connection of the five parts, and then a 4×4 homogeneous transformation matrix is ​​used to describe the spatial relationship between two adjacent connected parts. Through sequential transformations, the position and posture of the end effector relative to the base coordinate system can be finally derived, thereby establishing the kinematic equations of multiple moving parts. The effect of importing the excavator model is as follows Figure 6 shown.

[0132] The mining truck model is relatively simple. It does not need to consider the relative positions of the internal moving parts. The mining truck can be regarded as a whole. Its three-dimensional model is as follows: Figure 7 According to the relative position of the mining truck in the excavator coordinate system, the mining truck model is imported and obtained as follows Figure 8 The position of the mining truck relative to the excavator is shown.

[0133] The visualization plug-in is mainly based on the visualization_msgs / Marker type messages in rviz published to the ros space.

[0134] The overall visualization effect is as follows Fig. 9 shown.

[0135] The present invention also provides a three-dimensional surface reconstruction system based on a single-frame real-time point cloud, comprising:

[0136] Laser radar, used for collecting material surface point cloud data in real time; the laser radar is installed on the top of the excavator cab;

[0137] The first coordinate conversion and preprocessing module is used to transfer the collected point cloud data from the laser radar coordinate system to the excavator off-vehicle coordinate system, and filter out the interfering point cloud to obtain the first point cloud data;

[0138] The second coordinate conversion and grid division module is used to convert the first point cloud data from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, divide the selected material area into a plurality of sector grids in the cylindrical coordinate system, and project the first point cloud data into each sector grid;

[0139] The grid sampling point calculation and point filling module is used to calculate the centroid of all points in each grid as the grid sampling point; if the point cloud in the grid is empty, the sampling points of the empty grid are filled;

[0140] The quadrilateral mesh generation module is used to construct a quadrilateral mesh based on the completed grid sampling points and the grid index to which the grid sampling points belong, and render the quadrilateral mesh to obtain a three-dimensional reconstruction result of the material surface.

[0141] The present invention also provides a computing device, including a processor and a memory, wherein the processor is used to implement the steps of the aforementioned three-dimensional surface reconstruction method based on single-frame real-time point cloud when executing the computer program stored in the memory.

[0142] The present invention also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the aforementioned three-dimensional surface reconstruction method based on a single-frame real-time point cloud are implemented.

[0143] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0147] The present invention has been disclosed above with preferred embodiments, but they are not intended to limit the present invention. Any technical solutions obtained by adopting equivalent replacement or equivalent transformation solutions fall within the protection scope of the present invention.

Claims

1. A three-dimensional surface reconstruction method based on single-frame real-time point cloud, characterized in that: include: The point cloud data of the material surface is collected in real time by using a laser radar; the laser radar is installed on the top of the excavator cab; The collected point cloud data is transferred from the laser radar coordinate system to the excavator's off-vehicle coordinate system, and the interfering point cloud is filtered out to obtain the first point cloud data; The first point cloud data is converted from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, the selected material area is divided into a plurality of sector grids in the cylindrical coordinate system, and the first point cloud data is projected into each sector grid; Calculate the centroid of all points in each grid as the grid sampling point; If the point cloud in the grid is empty, fill in the empty grid with sampling points; A quadrilateral mesh is constructed based on the completed grid sampling points and the grid index to which the grid sampling points belong, and the quadrilateral mesh is rendered to obtain the three-dimensional reconstruction result of the material surface.

2. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The real-time collection of material surface point cloud data includes: At any time between the end of the excavator's rotation phase and the beginning of the excavation phase, the material surface point cloud data is collected.

3. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: According to the transfer relationship matrix from the laser radar coordinate system to the excavator off-vehicle coordinate system, the collected point cloud data is transferred from the laser radar coordinate system to the excavator off-vehicle coordinate system. The transfer relationship matrix from the laser radar coordinate system to the excavator off-vehicle coordinate system is: ； in, is the transfer relationship matrix from the laser radar coordinate system to the excavator's off-vehicle coordinate system, is the transfer relationship matrix from the laser radar coordinate system to the excavator coordinate system, is the transfer relationship matrix between the coordinate system on the excavator and the coordinate system off the excavator, ； in, R is the rotation matrix between the coordinate system on the excavator and the coordinate system off the excavator, ； is the rotation angle between the coordinate system of the excavator on board and the coordinate system of the excavator off board; T is the translation vector between the coordinate system on the excavator and the coordinate system off the excavator: ； in, Z is the height difference between the coordinate system on the excavator and the coordinate system off the excavator, and the superscript T represents the transposition.

4. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The filtering out of the interference point cloud comprises: The point cloud of the vehicle body and the point cloud of the moving parts are filtered out according to the size of the excavator itself, and some discrete points are filtered out through neighborhood radius filtering.

5. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The material area is selected within a range of 5m< r < 11m, -45°< <45°, is the polar coordinate of a point.

6. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The method of filling sampling points in the hole grid includes: If the point cloud hole grid is at the edge of the material area, the width is one angle unit wide, and the grid hole state is retained; If the point cloud hole grid is not at the edge of the material area, traverse all non-edge grids. If it is judged to be a hole grid, calculate the coordinates of the center point of the hole grid and convert the calculated center point coordinates into x , y The coordinates are used as the sampling points of the hole grid x , y Coordinates, search for the next non-empty grid sampling point of the hole grid along the x-axis direction, and perform linear interpolation based on the previous non-empty grid sampling point and the next non-empty grid sampling point of the hole grid to obtain the sampling point of the hole grid z coordinate.

7. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The constructing of a quadrilateral grid based on the completed grid sampling points and the grid indexes to which the grid sampling points belong includes: The completed grid sampling points are used as feature points. Based on the grid index to which the grid sampling points belong, the corresponding three points of each grid sampling point are taken to construct the vertices, edges and diagonal edges of the quadrilateral mesh, and the splicing method of the quadrilateral mesh is determined according to whether the point set in the grid is empty.

8. The three-dimensional surface reconstruction method based on single-frame real-time point cloud according to claim 1, characterized in that: The constructing of a quadrilateral grid based on the completed grid sampling points and the grid indexes to which the grid sampling points belong includes: For the grid sampling points at the edge of the material area or , do not continue to construct quadrilaterals, where width For material area along x The number of grid divisions in the direction, height For material area along y The number of grid divisions in the direction, is the grid sampling point set; For the remaining grid sampling points , according to the following formula to get the corresponding point P 1 、P 2 、P 3: ； ； ； Calculation Point P 0 、P 1 、P 2 、P 3 The grid index to which it belongs, query the point based on the calculated grid index P 0 、P 1 、P 2 、P 3 Whether the corresponding grid point set is a hole point, If none of the four points are empty points, a quadrilateral is constructed by splicing two triangles with a shared hypotenuse into a quadrilateral unit; If one of the four points is a hole point, the remaining three points will be used to construct a triangle; If two or more of the four points are empty points, no construction processing will be performed.

9. A three-dimensional surface reconstruction system based on single-frame real-time point cloud, characterized in that: include: Laser radar, used for collecting material surface point cloud data in real time; the laser radar is installed on the top of the excavator cab; The first coordinate conversion and preprocessing module is used to transfer the collected point cloud data from the laser radar coordinate system to the excavator off-vehicle coordinate system, and filter out the interfering point cloud to obtain the first point cloud data; The second coordinate conversion and grid division module is used to convert the first point cloud data from the excavator undercarriage coordinate system to a cylindrical coordinate system with the origin of the undercarriage coordinate system as the origin, divide the selected material area into a plurality of sector grids in the cylindrical coordinate system, and project the first point cloud data into each sector grid; The grid sampling point calculation and point filling module is used to calculate the centroid of all points in each grid as the grid sampling point; If the point cloud in the grid is empty, fill in the empty grid with sampling points; The quadrilateral mesh generation module is used to construct a quadrilateral mesh based on the completed grid sampling points and the grid index to which the grid sampling points belong, and render the quadrilateral mesh to obtain a three-dimensional reconstruction result of the material surface.

10. The three-dimensional surface reconstruction system based on single-frame real-time point cloud according to claim 9, characterized in that: The laser radar is installed on the top of the excavator cab in a downward tilted manner.

11. A computing device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the three-dimensional surface reconstruction method based on a single-frame real-time point cloud as described in any one of claims 1 to 8 when executing the computer program stored in the memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the three-dimensional surface reconstruction method based on a single-frame real-time point cloud as described in any one of claims 1 to 8 are implemented.

Images