A multi-sensor map construction method and device for active ballast machine

Abstract

The present invention belongs to the technical field of salt field mapping, and discloses a multi-sensor map construction method for a movable ballast machine, including joint calibration of an RGB-D camera and a laser radar to obtain the position and posture relationship between sensors; calling the RGB-D camera and the laser radar to obtain salt field environmental data information, and constructing a two-dimensional grid map based on the data collected by the laser radar; downsampling and filtering the point cloud data collected by the RGB-D camera, and performing a secondary elimination of ground point cloud data on the downsampling and filtered point cloud data; converting the second non-ground point to the laser radar coordinate system through the position and posture relationship between sensors, and superimposing the second non-ground point converted to the laser radar coordinate system on the two-dimensional grid map to obtain a final fusion map. The present invention solves the original method of manually operating the movable ballast machine to stir the salt nucleus of the salt field crystallization, and provides a prerequisite for unmanned operation of the movable ballast machine, so that the working efficiency of the movable ballast machine can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of salt field mapping, and in particular relates to a multi-sensor map construction method and device for a movable ballast machine. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the gradual development and improvement of sensor technology, industrial machines are gradually changing from doing heavy work at the beginning to being able to achieve some target requirements autonomously. For example, in the process of making salt in salt fields, it is necessary to repeatedly rake the salt nuclei in the salt fields so that the required salt crystals can be smoothly produced. At present, the operation of the active slag machine still needs to rely on manual operation, which requires a lot of manpower and financial investment, and increases production costs. The manual operation of the active slag machine is replaced by unmanned operation, and the active slag machine generates a map of the salt field environment through sensor technology, and autonomously rakes the salt field, which reduces labor costs and improves production efficiency.

[0004] For the problem of salt field map construction, 3D lidar can be used to obtain data information to construct a three-dimensional point cloud map. However, the price of this type of sensor is too high at present. In this application scenario, the cost is too high to be popularized. In addition, a single lidar cannot recognize color and texture, resulting in limited accuracy of the constructed map. Summary of the invention

[0005] One of the purposes of the present invention is to provide a multi-sensor map construction method for a movable ballast machine, which integrates a 2D lidar and an RGB-D camera to construct a two-dimensional fused grid map, provide a map model for the autonomous operation of the movable ballast machine, reduce the cost of map construction, and improve the integrity of the constructed map.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A multi-sensor map construction method for a live ballast machine, the multi-sensor map construction method for a live ballast machine comprising:

[0008] Step S1: Jointly calibrate the RGB-D camera and the lidar to obtain the pose relationship between the sensors;

[0009] Step S2: Calling the RGB-D camera and the laser radar to obtain salt pan environmental data information, and constructing a two-dimensional grid map based on the data collected by the laser radar;

[0010] Step S3: downsampling and filtering the point cloud data collected by the RGB-D camera, and performing a secondary ground point cloud data elimination operation on the downsampling and filtered point cloud data, including:

[0011] Step S31: dividing the space where the point cloud is located into voxel grids of a preset size by a voxel grid filtering method, using the voxel grid as the point cloud after downsampling filtering, and calculating the point cloud grid coordinates of the voxel grid;

[0012] Step S32: pre-selecting a ground point cloud from the downsampled and filtered point cloud, calculating a normal vector of the pre-selected ground point cloud by vector cross multiplication, and obtaining an initial ground point cloud equation by using the normal vector estimation;

[0013] Step S33: extracting the initial ground point in the downsampled filtered point cloud based on the initial ground point cloud equation, minimizing the distance function using the initial ground point to determine the optimal ground point cloud equation, taking the point in the downsampled filtered point cloud that meets the optimal ground point cloud equation as the first ground point, and the remaining points as the first non-ground point;

[0014] Step S34: extracting ground points from the first non-ground points using the DoN algorithm to obtain second ground points and second non-ground points;

[0015] Step S4: convert all second non-ground points into the laser radar coordinate system through the position and posture relationship between sensors, and superimpose the second non-ground points converted into the laser radar coordinate system onto the two-dimensional grid map to obtain a final fusion map.

[0016] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0017] Preferably, in step S1, the posture relationship between the sensors includes:

[0018] There is a point p in the RGB-D camera coordinate system i =(X i ,Y i ,Z i ) T , in the laser radar coordinate system, this point is p l =(X l ,Y l ,Z l ) T , then the pose conversion formula from RGB-D camera to LiDAR is as follows:

[0019]

[0020] In the formula, (X l ,Y l ,Zl ) is the midpoint p of the laser radar coordinate system l The coordinates of (X i ,Y i ,Z i ) is the midpoint p of the RGB-D camera coordinate system i The coordinates of , R and t are the rotation matrix and offset matrix from the RGB-D camera coordinate system to the lidar coordinate system obtained by joint calibration.

[0021] Preferably, in step S31, the point cloud grid coordinates of the voxel grid are calculated as follows:

[0022]

[0023]

[0024]

[0025] In the formula, (X t ,Y t ,Z t ) is the point cloud grid coordinate of the voxel grid, n is the total number of points in the current voxel grid, (X i ,Y i ,Z i ) are the coordinates of the i-th point in the current voxel grid.

[0026] Preferably, in step S33, extracting the initial ground points in the downsampled and filtered point cloud based on the initial ground point cloud equation includes:

[0027] Initialize the current number of iterations to T = 1 and the minimum distance threshold to d min , the minimum number of internal points threshold is m min , the maximum number of internal points is m max ;

[0028] The RANSAC algorithm is used to randomly select a set of data points P from the downsampled and filtered point cloud. n , where m min <|P n | <m max ,|P n | is the number of points in the selected set of data points, traversing the set of data points P n For all the points in the , calculate the distance d from the point to the ground plane where the initial ground point cloud equation is located;

[0029] Let d≤d min The point is defined as an interior point, d>d min A point is defined as an external point if the set of data points P n The number of inliers is less than the preset minimum inlier threshold mmin , then re-estimate the initial ground point cloud equation and re-use the RANSAC algorithm to randomly select points;

[0030] Otherwise, the group of data points P n Add the inner points in the ground point set, remove the duplicate points in the ground point set, and take the number of points in the ground point set as m. If m min <m<m max , then the RANSAC algorithm is used again to randomly select points until m ≥ m max Or the current number of iterations is T and reaches the set maximum value T max ,Finally, all the internal points in the ground point set are taken as the initial ground points.

[0031] Preferably, the ground point cloud equation is as follows:

[0032] Ax+By+Cz+D=0

[0033] Then the initial ground point minimizes the distance function as follows:

[0034]

[0035] In the formula, A, B, C, D are the coefficients of the ground point cloud equation, m is the number of initial ground points, and d j is the distance from the jth initial ground point to the ground plane where the ground point cloud equation is located.

[0036] Preferably, in step S34, the first non-ground point is subjected to secondary extraction of ground points by using a DoN algorithm, including:

[0037] For each point p in the first non-ground point k , perform normal vector estimation in different neighborhoods c1 and c2, c1>c2, and obtain two unit normal vectors n(p k ,c1) and n(p k ,c2), then the normal vector difference operator is Δn(p k ,c1,c2)=(n(p k ,c1)-n(p k ,c2)) / 2, if Δn is less than the preset threshold, then point p k is the second ground point, otherwise point p k is the second non-ground point.

[0038] The multi-sensor map construction method for a movable ballast machine provided by the present invention has the following beneficial effects compared with the prior art:

[0039] (1) According to the specific autonomous working environment of the ballast machine, the RGB-D camera point cloud data processing strategy is improved to perform secondary elimination of the ground point cloud. In combination with the specific circumstances during the actual operation of the ballast machine, the initial estimated ground point cloud equation is added to the method of the present invention, and the constraint of the minimum number of inliers is added. This enhances the method of the present invention's ability to cope with the initial selection of too few inliers and the insufficiency of the initial fitting of the ground point cloud equation, thereby enhancing the more complete performance of the method of the present invention in eliminating ground point clouds.

[0040] (2) In combination with the actual working conditions of the active ballast machine, a multi-sensor mapping method of two-dimensional laser radar and RGB-D camera is adopted to solve the shortcoming of insufficient mapping information of a single laser radar sensor. The object information of the camera is added on the basis of laser mapping, thereby enriching the ability of the method of the present invention to obtain environmental information in the field of autonomous mapping of the active ballast machine to a certain extent.

[0041] A second object of the present invention is to provide a multi-sensor map building device for a movable ballast machine, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the multi-sensor map building method for a movable ballast machine as described in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a multi-sensor map construction method for a movable ballast machine according to the present invention;

[0043] Figure 2 This is a flow chart of the first removal of ground points in the present invention;

[0044] Figure 3 This is a flow chart of the second elimination of ground points in the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0047] In order to overcome the defects of the existing technology, this embodiment selects a multi-sensor data fusion method, that is, a low-cost 2D lidar and RGB-D camera fusion mapping method is used to construct a two-dimensional grid map of the salt field, providing an environmental map for the autonomous operation of the active ballast machine.

[0048] like Figure 1 As shown, the present embodiment provides a multi-sensor map construction method for a ballast machine, comprising the following steps:

[0049] Step S1: Use the chessboard calibration method to jointly calibrate the RGB-D camera and lidar to obtain the pose relationship between the sensors.

[0050] Step S11: Start the lidar and RGB-D camera to record data packets, and change the posture of the chessboard during the recording process.

[0051] Step S12: Run the recorded data packet, then pause the run, open autoware, select the topic of the data packet, and select camera to radar as the calibration type; the calibration method used in step S12 is the camera_lidar_calibration function package in autoware.

[0052] Step S13: Run the data package again, select the corresponding area of ​​the calibration board in the autoware calibration interface for joint calibration, until the laser radar hits the calibration board, and then the posture relationship transformation parameters from the RGB-D camera to the laser radar are obtained. There is a point p in the RGB-D camera coordinate system. i =(X i ,Y i ,Z i ) T , in the laser radar coordinate system, this point is p l =(X l ,Y l ,Z l ) T , then the pose conversion formula from RGB-D camera to LiDAR is as follows:

[0053]

[0054] In the formula, (X l ,Y l ,Z l ) is the midpoint p of the laser radar coordinate system l The coordinates of (X i ,Y i ,Z i ) is the midpoint p of the RGB-D camera coordinate system iThe coordinates of the RGB-D camera coordinate system, R and t are the rotation matrix and offset matrix from the RGB-D camera coordinate system to the lidar coordinate system obtained by the joint calibration. R and t in the formula are obtained through the above calibration.

[0055] It should be noted that in this embodiment, the chessboard calibration method is used to jointly calibrate the RGB-D camera and the lidar. In other embodiments, other calibration methods may also be used, such as a geometric calibration method, a feature point calibration method, and the like.

[0056] Step S2: Call the RGB-D camera and lidar to obtain salt pan environmental data information, and construct a two-dimensional grid map through the Gmapping algorithm based on the data collected by the lidar.

[0057] Step S3: downsample and filter the point cloud data collected by the RGB-D camera, and perform a secondary extraction of ground point cloud data on the downsampled and filtered point cloud data. In this step, the present embodiment uses an improved strategy of RANSAC and DoN algorithms on the filtered point cloud data to secondary remove the ground point cloud data.

[0058] Step S31: Divide the space where the point cloud is located into voxel grids of a preset size by a voxel grid filtering method, use the voxel grid as the point cloud after downsampling filtering, and calculate the point cloud grid coordinates of the voxel grid.

[0059] This embodiment reduces the number of point clouds and compresses point clouds by downsampling, wherein the size of the voxel grid can be set according to actual conditions. The larger the voxel grid is set, the fewer grids are obtained, and the more point clouds are contained in each grid, which is beneficial to improve the matching speed, but it will also increase mismatches. On the contrary, a voxel grid that is too small cannot describe local details in detail. Therefore, in this embodiment, the preset size of the voxel grid is taken as 5cm*5cm*5cm. And the point cloud grid coordinates of the voxel grid are calculated as follows:

[0060]

[0061]

[0062]

[0063] In the formula, (X t ,Y t ,Z t ) is the point cloud grid coordinate of the voxel grid, n is the total number of points in the current voxel grid (the point is the point in the point cloud data collected by the RGB-D camera), (X i ,Y i ,Z i ) is the coordinate of the i-th point in the current voxel grid. In the subsequent steps, the point cloud after downsampling filtering is processed.

[0064] Step S32: pre-selecting a ground point cloud from the down-sampled filtered point cloud, calculating the normal vector of the pre-selected ground point cloud by vector cross multiplication, and obtaining an initial ground point cloud equation using the normal vector estimation.

[0065] In order to speed up the fitting speed of the ground point cloud equation, this embodiment randomly selects from the point cloud data below the camera horizontal line height when pre-selecting the ground point cloud, and the number of selected ground points is determined according to needs, at least the amount of data that supports the calculation of the ground point cloud equation. The obtained ground point cloud equation is as follows: Ax+By+Cz+D=0, where A, B, C, and D are the coefficients of the ground point cloud equation.

[0066] Step S33: Extract the initial ground points in the downsampled and filtered point cloud based on the initial ground point cloud equation, use the initial ground points to minimize the distance function, determine the optimal ground point cloud equation, and take the points in the downsampled and filtered point cloud that meet the optimal ground point cloud equation as the first ground points, and the remaining points as the first non-ground points.

[0067] In this embodiment, when dividing the initial ground points and the initial non-ground points, the RANSAC algorithm is introduced to randomly select points, such as Figure 2 As shown, the specific process is as follows:

[0068] Initialize the current number of iterations to T = 1 and the minimum distance threshold to d min , the minimum number of internal points threshold is m min , the maximum number of internal points is m max .

[0069] The RANSAC algorithm is used to randomly select a set of data points P from the downsampled and filtered point cloud. n ={p1,p2,p3,..}, where m min <|P n | <m max ,|P n | is the number of points in the selected set of data points, traversing the set of data points P n For all points in the , calculate the distance d from the point to the ground plane where the initial ground point cloud equation is located.

[0070] Let d≤d min The point is defined as an interior point, d>d min A point is defined as an external point if the set of data points P n The number of inliers is less than the preset minimum inlier threshold m min , then re-estimate the initial ground point cloud equation (using the method in step S32 for estimation), and re-use the RANSAC algorithm to randomly select points. nRecalculate and judge.

[0071] Otherwise, the group of data points P n Add the inner points in the ground point set, remove the duplicate points in the ground point set, and take the number of points in the ground point set as m. If m min <m<m max , then the RANSAC algorithm is used again to randomly select points, and according to the newly selected data point P n Recalculate and judge until m≥m max Or the current number of iterations is T and reaches the set maximum value T max ,Finally, all the inliers in the ground point set are used as the ,initial ground points, and the remaining points in the downsampled and filtered point ,cloud are used as the initial non-ground points.

[0072] The initial ground point minimization distance function in this embodiment is as follows:

[0073]

[0074] Where m is the number of initial ground points, d j is the distance from the jth initial ground point to the ground plane where the ground point cloud equation is located. After minimizing the distance constraint from the inner point to the optimal plane through the argmin() function, the optimal ground point cloud equation is obtained, that is, solving a ground point cloud equation Ax+By+Cz+D=0 with coefficients A, B, C, D, so that the sum of the distances from all inner points to the ground plane where the equation is located is minimized, then the equation is considered to be the optimal ground point cloud equation.

[0075] Step S34: extracting ground points from the first non-ground points using the DoN algorithm to obtain second ground points and second non-ground points.

[0076] In order to improve the accuracy of ground point extraction, this embodiment performs a secondary ground point extraction on the first non-ground point screened for the first time, such as Figure 3 As shown, the extraction process is as follows: for each point p in the first non-ground point k , perform normal vector estimation in different neighborhoods c1 and c2, c1>c2, and obtain two unit normal vectors n(p k ,c1) and n(p k ,c2), then the normal vector difference operator is Δn(p k ,c1,c2)=(n(p k ,c1)-n(p k ,c2)) / 2, if Δn is less than the preset threshold θ, then point p k is the second ground point, otherwise point p k is the second non-ground point.

[0077] The second non-ground points are the valid point clouds retained after elimination in this embodiment. The first ground points obtained by division in step S33 and the second ground points obtained by secondary extraction in step S34 are merged to obtain all the ground points that need to be eliminated.

[0078] Step S4: Fuse the two-dimensional grid map, align the coordinates through the posture relationship between the lidar and the RGB-D camera, and project the second non-ground point in the RGB-D point cloud data onto the two-dimensional grid map constructed by the lidar.

[0079] In order to improve the accuracy and completeness of mapping, this embodiment converts all second non-ground points into the lidar coordinate system through the posture relationship between sensors, and superimposes the second non-ground points converted into the lidar coordinate system onto the two-dimensional grid map to obtain the final fused map.

[0080] The ground point cloud disturbance (misidentified obstacle point cloud) existing in the mapping process of the active ballast machine is filtered out, and after being eliminated by two types of filtering methods, the remaining point cloud data (considered to be obstacle point cloud) is transformed and projected and fused with the grid map built by the laser radar to generate a two-dimensional grid map that can be used for navigation, and the map has high integrity and reliability. The present invention solves the original method of manually operating the active ballast machine to stir the salt nucleus of the salt field crystallization, provides the prerequisite for unmanned operation of the active ballast machine, and improves the working efficiency of the active ballast machine.

[0081] In another embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a multi-sensor map building method for a movable ballast machine.

[0082] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0083] In another embodiment, the present invention also provides a multi-sensor map building device for a movable ballast machine, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of a multi-sensor map building method for a movable ballast machine.

[0084] For the specific definition of a multi-sensor map building device for a movable ballast machine, please refer to the definition of a multi-sensor map building method for a movable ballast machine mentioned above, which will not be repeated here.

[0085] The memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program that can be run on the processor, and the processor implements the method in the embodiment of the present invention by running the computer program stored in the memory.

[0086] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0087] The processor may be an integrated circuit chip with data processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0088] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A multi-sensor map construction method for a ballast machine, characterized in that: The multi-sensor map construction method for a live ballast machine comprises: Step S1: Jointly calibrate the RGB-D camera and the lidar to obtain the pose relationship between the sensors; Step S2: Calling the RGB-D camera and the laser radar to obtain salt pan environmental data information, and constructing a two-dimensional grid map based on the data collected by the laser radar; Step S3: downsampling and filtering the point cloud data collected by the RGB-D camera, and performing a secondary ground point cloud data elimination operation on the downsampling and filtered point cloud data, including: Step S31: dividing the space where the point cloud is located into voxel grids of a preset size by a voxel grid filtering method, using the voxel grid as the point cloud after downsampling filtering, and calculating the point cloud grid coordinates of the voxel grid; Step S32: pre-selecting a ground point cloud from the downsampled and filtered point cloud, calculating a normal vector of the pre-selected ground point cloud by vector cross multiplication, and obtaining an initial ground point cloud equation by using the normal vector estimation; Step S33: extracting the initial ground point in the downsampled filtered point cloud based on the initial ground point cloud equation, using the initial ground point to minimize the distance function, determine the optimal ground point cloud equation, and use the point in the downsampled filtered point cloud that meets the optimal ground point cloud equation as the first ground point, and the remaining points as the first non-ground point; wherein, extracting the initial ground point in the downsampled filtered point cloud based on the initial ground point cloud equation includes: Initialize the current number of iterations to T = 1 and the minimum distance threshold to d min , the minimum number of internal points threshold is m min , the maximum number of internal points is m max ; The RANSAC algorithm is used to randomly select a set of data points P from the downsampled and filtered point cloud. n , where m min <|P n | <m max ,|P n | is the number of points in the selected set of data points, traversing the set of data points P n For all the points in the , calculate the distance d from the point to the ground plane where the initial ground point cloud equation is located; Let d≤d min The point is defined as an interior point, d>d min A point is defined as an external point if the set of data points P n The number of inliers is less than the preset minimum inlier threshold m min , then re-estimate the initial ground point cloud equation and re-use the RANSAC algorithm to randomly select points; Otherwise, the group of data points P n Add the inner points in the ground point set, remove the duplicate points in the ground point set, and take the number of points in the ground point set as m. If m min <m<m max , then the RANSAC algorithm is used again to randomly select points until m ≥ m max Or the current number of iterations is T and reaches the set maximum value T max ,Finally, all the internal points in the ground point set are taken as the initial ground points; Step S34: extracting ground points from the first non-ground points using the DoN algorithm to obtain second ground points and second non-ground points; Step S4: convert all second non-ground points into the laser radar coordinate system through the position and posture relationship between sensors, and superimpose the second non-ground points converted into the laser radar coordinate system onto the two-dimensional grid map to obtain a final fusion map.

2. The multi-sensor map construction method for a live ballast machine according to claim 1, characterized in that: In step S1, the posture relationship between the sensors includes: There is a point p in the RGB-D camera coordinate system i =(X i ,Y i ,Z i ) T , in the laser radar coordinate system, this point is p l =(X l ,Y l ,Z l ) T , then the pose conversion formula from RGB-D camera to LiDAR is as follows: In the formula, (X l ,Y l ,Z l ) is the midpoint p of the laser radar coordinate system l The coordinates of (X i ,Y i ,Z i ) is the midpoint p of the RGB-D camera coordinate system i The coordinates of , R and t are the rotation matrix and offset matrix from the RGB-D camera coordinate system to the lidar coordinate system obtained by joint calibration.

3. The multi-sensor map construction method for a ballast machine according to claim 1, characterized in that: In step S31, the point cloud grid coordinates of the voxel grid are calculated as follows: In the formula, (X t ,Y t ,Z t ) is the point cloud grid coordinate of the voxel grid, n is the total number of points in the current voxel grid, (X i ,Y i ,Z i ) are the coordinates of the i-th point in the current voxel grid.

4. The multi-sensor map construction method for a ballast machine according to claim 1, characterized in that: The ground point cloud equation is as follows: Ax+By+Cz+D=0 Then the initial ground point minimizes the distance function as follows: In the formula, A, B, C, D are the coefficients of the ground point cloud equation, m is the number of initial ground points, and d j is the distance from the jth initial ground point to the ground plane where the ground point cloud equation is located.

5. The multi-sensor map construction method for a ballast machine according to claim 1, characterized in that: In step S34, the first non-ground point is subjected to secondary extraction of ground points using the DoN algorithm, including: For each point p in the first non-ground point k , perform normal vector estimation in different neighborhoods c1 and c2, c1>c2, and obtain two unit normal vectors n(p k ,c1) and n(p k ,c2), then the normal vector difference operator is Δn(p k ,c1,c2)=(n(p k ,c1)-n(p k ,c2)) / 2, if Δn is less than the preset threshold, then point p k is the second ground point, otherwise point p k is the second non-ground point.

6. A multi-sensor map building device for a ballast machine, comprising a processor and a memory storing a plurality of computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

Images