Carry-scraper sensor external parameter calibration method and device, terminal and storage medium

Abstract

The invention discloses an external parameter calibration method for a carry-scraper sensor in the technical field of engineering machinery, and aims to solve the problem that in the prior art, when a front vehicle axle and a rear vehicle axle deflect, sensor signals at different angles are difficult to convert into the same coordinate system, and the sensing requirement of the sensor in a dynamic environment is affected. The method comprises the following steps of: calibrating a front vehicle sensor and a rear vehicle sensor to a rear axle coordinate system according to a transformation matrix of the rear vehicle sensor and the rear axle coordinate system and a transformation matrix of the front vehicle sensor and the rear axle coordinate system so as to realize external parameter calibration work; according to the method, the external parameters of the sensors located on the front vehicle and the rear vehicle relative to the axle coordinate system of the rear vehicle can be determined under the condition that the included angle deflects, so that signals sensed by the sensors on the front vehicle and the rear vehicle are effectively fused, and comprehensive and real-time sensing signals are provided for smooth operation of the unmanned system of the carry-scraper.

Description

Technical Field

[0001] The present invention relates to a method, a device, a terminal and a storage medium for calibrating external parameters of a scraper sensor, and belongs to the technical field of engineering machinery. Background Art

[0002] As core transportation equipment in confined spaces such as underground mines and tunnel projects, underground scrapers (Scrapers) face the challenge of unmanned operation, requiring multi-sensor collaborative perception under complex working conditions. Scrapers utilize an articulated dual-body structure, with the front and rear vehicles connected by a slew axis driven by a hydraulic cylinder, enabling dynamic deflection angles of up to ±45°. Frequent vehicle deflection during operations such as ore loading and roadway steering results in nonlinear transformations in the sensor coordinate systems of the front and rear vehicles. Traditional static calibration methods are unable to meet the perception requirements in dynamic environments.

[0003] During the construction of an unmanned scraper loader system, the angle between the front and rear axles often deflects. This often requires determining the external parameters of sensors such as lidar and cameras located at different locations, such as the front and rear vehicles, relative to the coordinate system of the rear vehicle's axle, as the deflection angle changes. This allows for the effective fusion of the signals sensed by the sensors attached to the front and rear vehicles, providing comprehensive, real-time perception signals for the smooth operation of the scraper loader system. Existing multi-sensor calibration techniques for ground vehicles typically assume a rigid connection between the vehicle bodies, and the sensor external parameters are determined once during static calibration. However, the articulated structure of underground scrapers causes relative motion between the front and rear vehicles, and the sensor external parameters change dynamically with the deflection angle. When the deflection angle is large, uncalibrated sensor fusion errors can lead to significant environmental perception deviations, seriously impacting the safety of unmanned operations.

[0004] In summary, existing scraper loader sensors mainly focus on optimizing navigation control algorithms, and there is no effective solution to the problem of sensor external parameter calibration under dynamic deflection. When the front and rear axles of the scraper loader deflect, it is difficult to convert sensor signals at different angles into the same coordinate system, affecting the sensor's perception requirements in a dynamic environment. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, terminal and storage medium for calibrating the external parameters of scraper loader sensors, which can determine the external parameters of sensors located at different positions such as the front vehicle and the rear vehicle relative to the coordinate system of the rear vehicle axle when the angle between the front vehicle axle and the rear vehicle axle is deflected, thereby effectively fusing the signals sensed by the sensors fixed on the front vehicle and the signals sensed by the sensors fixed on the rear vehicle, providing comprehensive and real-time perception signals for the smooth operation of the unmanned system of the scraper loader; providing a fast and convenient perception method for the perception of information at various angles on the scraper loader, significantly improving the scraper loader's perception of external information, with good effect and low cost.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for calibrating external parameters of a scraper sensor, comprising the following steps:

[0008] Determine the directions of the front axle coordinate system and the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system;

[0009] Obtain vehicle parameters, and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters;

[0010] Obtaining a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to a transformation matrix between the front vehicle sensor and the front axle coordinate system and a transformation matrix between the front axle coordinate system and the rear axle coordinate system;

[0011] According to the transformation matrix between the rear vehicle sensor and the rear axle coordinate system and the transformation matrix between the front vehicle sensor and the rear axle coordinate system, both the front vehicle sensor and the rear vehicle sensor are calibrated to the rear axle coordinate system to realize the external parameter calibration.

[0012] Furthermore, the front vehicle sensor includes a front vehicle laser radar and a front vehicle camera, and the rear vehicle sensor includes a rear vehicle laser radar and a rear vehicle camera.

[0013] Furthermore, the vehicle parameters include the length from the center of the front axle to the center of the rotary axis, the length from the center of the rear axle to the center of the rotary axis, and the angle between the front axle and the rear axle;

[0014] The transformation matrix of the front axle coordinate system and the rear axle coordinate system is obtained according to the vehicle parameters. The specific expression is as follows:

[0015] ;

[0016] Where: is the transformation matrix between the front axle coordinate system and the rear axle coordinate system; is the angle between the front axle and the rear axle; It is the length from the center of the front axle to the center of the rotating axis; It is the length from the center of the rear axle to the center of the rotary axis.

[0017] Furthermore, obtaining the transformation matrix of the front vehicle sensor and the front vehicle axle coordinate system specifically includes:

[0018] Get the transformation matrix between the front vehicle LiDAR and the front vehicle axle coordinate system, and get the transformation matrix between the front vehicle LiDAR and the front vehicle camera;

[0019] According to the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system and the transformation matrix of the front vehicle laser radar and the front vehicle camera, the transformation matrix of the front vehicle camera and the front vehicle axle coordinate system is obtained.

[0020] Furthermore, obtaining the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system specifically includes:

[0021] Step a: Determine the positions of the first auxiliary laser radars on both sides of the leading vehicle, and obtain the transformation matrix between the two first auxiliary laser radars using a calibration object; wherein the leading vehicle is in a horizontal state;

[0022] Step b: Scanning the preceding vehicle using the first auxiliary laser radar to obtain point cloud data of the preceding vehicle from two first auxiliary laser radars, and splicing the point cloud data of the preceding vehicle from the two first auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the preceding vehicle;

[0023] Step c: Extracting point clouds of the surfaces of the two front tires based on the 3D reconstructed point cloud data of the front vehicle, fitting the 3D coordinates of the center points of the front tires based on the point clouds of the front tires, and calculating the position coordinates and length of the front axle based on the 3D coordinates of the center points of the two front tires;

[0024] Step d: Compare the length of the front axle with a preset range. If the length of the front axle is within the preset range, execute step e. If the length of the front axle is not within the preset range, re-determine the position of the first auxiliary laser radar on both sides of the front vehicle and repeat steps a, b, c, and d.

[0025] Step e: Determine the positions of the second auxiliary laser radars on both sides of the rear vehicle, and obtain the transformation matrix between the two second auxiliary laser radars through the calibration object; wherein the rear vehicle is in a horizontal state;

[0026] Step f: Scanning the rear vehicle using the second auxiliary laser radar to obtain rear vehicle point cloud data from the two second auxiliary laser radars, and splicing the rear vehicle point cloud data from the two second auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the rear vehicle;

[0027] Step g: extracting point clouds of the surfaces of the two rear tires based on the 3D reconstructed point cloud data of the rear vehicle, fitting the 3D coordinates of the center points of the rear tires based on the point clouds of the rear tire surfaces, and calculating the position coordinates and length of the rear axle based on the 3D coordinates of the center points of the two rear tires;

[0028] Step h: Compare the length of the rear axle with a preset range. If the length of the rear axle is within the preset range, execute step i. If the length of the rear axle is not within the preset range, re-determine the position of the second auxiliary laser radar on both sides of the rear vehicle and repeat steps e, f, g, and h.

[0029] Step i: Determine the XY plane of the front axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the front axle coordinate system; and determine the transformation matrix between the first auxiliary laser radar and the front axle coordinate system based on the XY plane of the front axle coordinate system and the position coordinates of the front axle;

[0030] Step j: Obtain the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar, and obtain the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system based on the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar and the transformation matrix of the first auxiliary laser radar and the front vehicle axle coordinate system.

[0031] Furthermore, obtaining the transformation matrix of the rear vehicle sensor and the rear vehicle axle coordinate system specifically includes:

[0032] Get the transformation matrix between the rear vehicle LiDAR and the rear vehicle axle coordinate system, and get the transformation matrix between the rear vehicle LiDAR and the rear vehicle camera;

[0033] Obtain the transformation matrix of the rear vehicle camera and the rear axle coordinate system based on the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system and the transformation matrix of the rear vehicle laser radar and the rear vehicle camera;

[0034] The step of obtaining the transformation matrix of the rear vehicle laser radar and the rear vehicle axle coordinate system specifically includes:

[0035] Determine the XY plane of the rear axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the rear axle coordinate system; and determine the transformation matrix between the second auxiliary laser radar and the rear axle coordinate system based on the XY plane of the rear axle coordinate system and the position coordinates of the rear axle.

[0036] Obtain the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar, and obtain the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system based on the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar and the transformation matrix of the second auxiliary laser radar and the rear axle coordinate system.

[0037] Furthermore, the calibration of the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system specifically includes:

[0038] The calibration method of continuous transformation matrix conversion is adopted to calibrate the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system.

[0039] In a second aspect, the present invention provides a device for calibrating external parameters of a scraper sensor, the device comprising:

[0040] The first transformation module is used to determine the direction of the front axle coordinate system and the direction of the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system;

[0041] The second transformation module is used to obtain vehicle parameters and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters;

[0042] A third transformation module is used to obtain a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to the transformation matrix between the front vehicle sensor and the front axle coordinate system and the transformation matrix between the front axle coordinate system and the rear axle coordinate system;

[0043] Calibration module: used to calibrate both the front vehicle sensor and the rear vehicle axle coordinate system according to the transformation matrix between the rear vehicle sensor and the rear vehicle axle coordinate system and the transformation matrix between the front vehicle sensor and the rear vehicle axle coordinate system, thereby realizing external parameter calibration.

[0044] In a third aspect, the present invention provides a terminal including a processor and a storage medium;

[0045] The storage medium is used to store instructions;

[0046] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The scraper loader sensor external parameter calibration method of the present invention can perform real-time calibration on the external parameters of sensors located at different positions such as the front vehicle and the rear vehicle in the unmanned scraper loader system. When the angle between the front vehicle axle and the rear vehicle axle is deflected, the external parameters of the sensors located at different positions such as the front vehicle and the rear vehicle relative to the coordinate system of the rear vehicle axle can be determined, thereby effectively fusing the signals sensed by the sensors fixed on the front vehicle and the signals sensed by the sensors fixed on the rear vehicle, providing comprehensive and real-time perception signals for the smooth operation of the unmanned scraper loader system; providing a fast and convenient perception method for the perception of information at various angles on the scraper loader, significantly improving the scraper loader's perception of external information, with good effect and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 1 is a flow chart of a method for calibrating external parameters of a scraper sensor according to an embodiment of the present invention;

[0051] Figure 2 1 is a schematic diagram of a process for obtaining a transformation matrix of a rear vehicle laser radar and a rear axle coordinate system according to an embodiment of the present invention;

[0052] Figure 3 2 is a schematic structural diagram of a scraper sensor external parameter calibration device provided by an embodiment of the present invention;

[0053] Figure 4 is a structural schematic diagram of a scraper provided according to an embodiment of the present invention;

[0054] Figure 5 is a schematic diagram of the positions of the second auxiliary laser radar and the following vehicle provided in an embodiment of the present invention;

[0055] Figure 6 is a schematic diagram of the operation of a second auxiliary laser radar provided according to an embodiment of the present invention;

[0056] Figure 7 3. It is a schematic diagram of the positions of the second auxiliary laser radar and the scraper provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0058] The term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.

[0059] Example 1:

[0060] like Figure 1 As shown, the present invention provides a method for calibrating external parameters of a scraper sensor, comprising the following steps:

[0061] Determine the directions of the front axle coordinate system and the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system;

[0062] Obtain vehicle parameters, and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters;

[0063] Obtaining a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to a transformation matrix between the front vehicle sensor and the front axle coordinate system and a transformation matrix between the front axle coordinate system and the rear axle coordinate system;

[0064] According to the transformation matrix between the rear vehicle sensor and the rear axle coordinate system and the transformation matrix between the front vehicle sensor and the rear axle coordinate system, both the front vehicle sensor and the rear vehicle sensor are calibrated to the rear axle coordinate system to realize the external parameter calibration.

[0065] Specifically, the underground scraper selects two basic coordinate systems, namely the front axle coordinate system and the rear axle coordinate system, one is located at the center of the rear axle, and the other is located at the center of the front axle; the two coordinate systems are x-axis forward, y-axis to the left, and z-axis upward. The only difference between the front axle coordinate system and the rear axle coordinate system is the yaw angle, the pitch angle and roll angle are the same, and the center of the rear axle is connected to the center of the front axle through a rotary axis.

[0066] In one embodiment, the front vehicle sensor includes a front vehicle laser radar and a front vehicle camera, and the rear vehicle sensor includes a rear vehicle laser radar and a rear vehicle camera.

[0067] Specifically, the transformation matrix between the front vehicle LiDAR and the front vehicle camera is obtained by placing a rectangular box or other marker in the same field of view of the two. The transformation matrix between the rear vehicle LiDAR and the rear vehicle camera is obtained in the same way.

[0068] In one embodiment, the vehicle parameters include the length from the center of the front axle to the center of the rotating axis, the length from the center of the rear axle to the center of the rotating axis, and the angle between the front axle and the rear axle;

[0069] The transformation matrix of the front axle coordinate system and the rear axle coordinate system is obtained according to the vehicle parameters. The specific expression is as follows:

[0070] ;

[0071] Where: is the transformation matrix between the front axle coordinate system and the rear axle coordinate system; is the angle between the front axle and the rear axle; It is the length from the center of the front axle to the center of the rotating axis; It is the length from the center of the rear axle to the center of the rotary axis.

[0072] In one embodiment, obtaining the transformation matrix of the front vehicle sensor and the front vehicle axle coordinate system specifically includes:

[0073] Get the transformation matrix between the front vehicle LiDAR and the front vehicle axle coordinate system, and get the transformation matrix between the front vehicle LiDAR and the front vehicle camera;

[0074] According to the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system and the transformation matrix of the front vehicle laser radar and the front vehicle camera, the transformation matrix of the front vehicle camera and the front vehicle axle coordinate system is obtained.

[0075] like Figure 4 and Figure 5 As shown, in one embodiment, obtaining the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system specifically includes:

[0076] Step a: Determine the positions of the first auxiliary laser radars on both sides of the leading vehicle, and obtain the transformation matrix between the two first auxiliary laser radars using a calibration object; wherein the leading vehicle is in a horizontal state;

[0077] Step b: Scanning the preceding vehicle using the first auxiliary laser radar to obtain point cloud data of the preceding vehicle from two first auxiliary laser radars, and splicing the point cloud data of the preceding vehicle from the two first auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the preceding vehicle;

[0078] Step c: Extracting point clouds of the surfaces of the two front tires based on the 3D reconstructed point cloud data of the front vehicle, fitting the 3D coordinates of the center points of the front tires based on the point clouds of the front tires, and calculating the position coordinates and length of the front axle based on the 3D coordinates of the center points of the two front tires;

[0079] Step d: Compare the length of the front axle with a preset range. If the length of the front axle is within the preset range, execute step e. If the length of the front axle is not within the preset range, re-determine the position of the first auxiliary laser radar on both sides of the front vehicle and repeat steps a, b, c, and d.

[0080] Step e: Determine the positions of the second auxiliary laser radars on both sides of the rear vehicle, and obtain the transformation matrix between the two second auxiliary laser radars through the calibration object; wherein the rear vehicle is in a horizontal state;

[0081] Step f: Scanning the rear vehicle using the second auxiliary laser radar to obtain rear vehicle point cloud data from the two second auxiliary laser radars, and splicing the rear vehicle point cloud data from the two second auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the rear vehicle;

[0082] Step g: extracting point clouds of the surfaces of the two rear tires based on the 3D reconstructed point cloud data of the rear vehicle, fitting the 3D coordinates of the center points of the rear tires based on the point clouds of the rear tire surfaces, and calculating the position coordinates and length of the rear axle based on the 3D coordinates of the center points of the two rear tires;

[0083] Step h: Compare the length of the rear axle with a preset range. If the length of the rear axle is within the preset range, execute step i. If the length of the rear axle is not within the preset range, re-determine the position of the second auxiliary laser radar on both sides of the rear vehicle and repeat steps e, f, g, and h.

[0084] Step i: Determine the XY plane of the front axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the front axle coordinate system; and determine the transformation matrix between the first auxiliary laser radar and the front axle coordinate system based on the XY plane of the front axle coordinate system and the position coordinates of the front axle;

[0085] Step j: Obtain the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar, and obtain the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system based on the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar and the transformation matrix of the first auxiliary laser radar and the front vehicle axle coordinate system.

[0086] In one embodiment, obtaining the transformation matrix of the rear vehicle sensor and the rear vehicle axle coordinate system specifically includes:

[0087] Get the transformation matrix between the rear vehicle LiDAR and the rear vehicle axle coordinate system, and get the transformation matrix between the rear vehicle LiDAR and the rear vehicle camera;

[0088] Obtain the transformation matrix of the rear vehicle camera and the rear axle coordinate system based on the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system and the transformation matrix of the rear vehicle laser radar and the rear vehicle camera;

[0089] The step of obtaining the transformation matrix of the rear vehicle laser radar and the rear vehicle axle coordinate system specifically includes:

[0090] Determine the XY plane of the rear axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the rear axle coordinate system; and determine the transformation matrix between the second auxiliary laser radar and the rear axle coordinate system based on the XY plane of the rear axle coordinate system and the position coordinates of the rear axle.

[0091] Obtain the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar, and obtain the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system based on the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar and the transformation matrix of the second auxiliary laser radar and the rear axle coordinate system.

[0092] Specifically, take the car as an example, Figure 2 As shown, the acquisition of the transformation matrix of the rear vehicle laser radar and the rear vehicle axle coordinate system specifically includes the following steps:

[0093] Place the underground scraper on a flat surface with the front and rear axles parallel. Place two secondary auxiliary LiDARs on both sides of the scraper, ensuring that each secondary auxiliary LiDAR's scanning range covers a complete rear wheel and that both secondary LiDARs have a common viewing area with the rear vehicle's LiDAR. Optionally, the first and second auxiliary LiDARs are Avia LiDARs.

[0094] First, the transformation matrix between the two second auxiliary lidars is calibrated with the help of calibration objects such as rectangular boxes;

[0095] Two second-aid LiDARs are used to scan the vehicle behind the underground scraper. The vehicle body point cloud data scanned by the two second-aid LiDARs are then spliced ​​together to obtain the 3D reconstructed point cloud data of the vehicle behind.

[0096] Based on the 3D reconstructed point cloud data of the rear vehicle, the point clouds of the two rear tire surfaces are extracted. Based on the point clouds of the two rear tire surfaces, the 3D coordinates of the rear tire center points are fitted, and the position coordinates and length of the rear axle of the scraper are calculated.

[0097] If the calculated rear axle length meets the requirements, that is, is within the preset range, it means that the calculated rear axle length matches the actual length of the rear axle, and the next calibration step is carried out; if the error is large, the position of the second auxiliary lidar is readjusted, and the above steps are repeated to re-model the two rear tires and calculate the three-dimensional coordinates of the rear axle position;

[0098] Place the first auxiliary LiDAR on both sides of the scraper, ensuring that the scanning range of each first auxiliary LiDAR covers a complete front wheel. Then repeat the above steps to obtain the position coordinates and length of the front axle.

[0099] Determine the XY plane of the rear axle coordinate system according to the two axes of the front axle and the rear axle and the direction of the rear axle coordinate system;

[0100] Determine the transformation matrix Tba2al of the rear axle coordinate system relative to the second auxiliary laser radar based on the XY plane of the rear axle coordinate system and the three-dimensional coordinates of the rear axle;

[0101] Place a rectangular box or other calibration object in the common viewing area of ​​the second auxiliary laser radar and the rear vehicle laser radar to calibrate the transformation matrix Tbl2al between the rear vehicle laser radar and the second auxiliary laser radar;

[0102] According to the transformation matrix Tba2al of the rear axle coordinate system relative to the second auxiliary laser radar, and the transformation matrix Tbl2al between the rear laser radar and the second auxiliary laser radar, the transformation matrix from the rear laser radar to the rear axle coordinate system is calculated. .

[0103] Similarly, the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system can be obtained.

[0104] In one embodiment, calibrating both the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system specifically includes: using a calibration method of continuous conversion of a transformation matrix to calibrate both the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system.

[0105] Specifically, during the calibration process of the transformation matrix, the calibration method of continuous transformation of the transformation matrix or the method of comprehensive optimization of multiple transformation matrices can be used to process and optimize the calibration external parameters; the calibration method of continuous transformation of the transformation matrix is ​​simple and fast, and the external parameters of different sensors can be calculated through the transformation matrix external parameters of the intermediate components, and comprehensive calibration parameters can be obtained by calibrating fewer sensor parameters; the calibration method of comprehensive optimization of multiple transformation matrices is characterized by requiring transformation matrix external parameters between multiple types of sensors, and through nonlinear optimization, obtaining a transformation matrix external parameter calibration scheme with the minimum overall error of all sensor transformation matrix external parameters.

[0106] In one embodiment, if the size, relative installation position and angle of each component of the vehicle body are accurately known, the calibration scheme defined in this section can be used to calibrate the transformation matrix of the laser radar and the vehicle body coordinate system. When calibrating the laser radar and the vehicle body coordinate system, it is necessary to use auxiliary sensors such as laser radar and cameras located outside the vehicle body, as well as auxiliary calibration objects such as calibration plates or rectangular boxes. In this design, the rectangular box is used to perform external parameter calibration between the vehicle-mounted laser radar and the auxiliary laser radar. Specifically:

[0107] Taking the rear vehicle as an example, the steps for calibrating the second auxiliary laser radar and the vehicle coordinate system include:

[0108] Based on the parameters of the underground scraper provided by the manufacturer, such as the size and installation angle, ten sets of three-dimensional coordinates coordinates_be (rear vehicle coordinates) of the scraper surface feature points in the rear vehicle coordinate system are calculated;

[0109] Calibrate the transformation matrix Tbal2el between the second auxiliary laser radar and the rear vehicle laser radar through the calibration object;

[0110] Calibrate ten sets of 3D coordinates (coordinates_be) of the scraper surface feature points in the rear vehicle body coordinate system and 3D coordinates (coordinates_al) (auxiliary coordinates) of the second auxiliary laser radar coordinate system;

[0111] Based on ten sets of three-dimensional coordinates coordinates_be and ten sets of three-dimensional coordinates coordinates_al, calculate the transformation matrix Tal2be between the second auxiliary laser radar and the rear vehicle coordinate system through nonlinear optimization and other methods;

[0112] Obtain the transformation matrix Tel2be between the rear vehicle LiDAR and the rear vehicle coordinate system through Tal2be*Tbal2el;

[0113] Implement the transformation matrix calibration between the auxiliary radar and the vehicle-mounted lidar, specifically including:

[0114] This calibration scheme selects a rectangular box or a square box as the calibration object. The relative position deployment scheme of the second auxiliary laser radar, the rear vehicle laser radar and the calibration object is as follows: Figure 6 As shown, that is, the calibration object is placed in the common viewing area between the second auxiliary laser radar and the rear vehicle laser radar;

[0115] During the calibration process, it is necessary to avoid placing the calibration object parallel to the axis of the laser radar so that the laser radar can scan the three planes of the calibration object. The point cloud scanned by the laser radar can then be fitted to the three planes of the calibration object. The three planes are then used to fit the three-dimensional coordinates of a vertex of the rectangular box in the laser radar, as well as the equations of the three sides connected to the vertex. The second auxiliary laser radar and the rear vehicle laser radar each collect the coordinates of a vertex in the rectangular box and the equations of the three sides connected to the vertex. Based on the parameters of the rectangular box, the three-dimensional coordinates of the other vertices between the second auxiliary laser radar and the rear vehicle laser radar can be calculated. Based on the three-dimensional coordinates of the ten sets of rectangular vertices between the two laser radars, the transformation matrix Tbal2el between the second auxiliary laser radar and the rear vehicle laser radar is calculated through nonlinear optimization and other methods. Alternatively, a point cloud of the rectangular box surface can be manually generated and then registered with the point cloud of the rectangular box scanned by the laser radar through point cloud registration to obtain the accurate three-dimensional coordinates of the rectangular box vertices in the two laser radars.

[0116] Obtain the transformation matrix between the second auxiliary laser radar and the following vehicle's body coordinate system, specifically including:

[0117] The second auxiliary laser radar is connected to the underground loader Figure 7 , so that the second auxiliary laser radar can scan the area where the key feature points on the scraper body are located; similarly, ten sets of three-dimensional coordinates coordinates_al of the feature points on the scraper body in the auxiliary radar are obtained by plane fitting; based on the ten sets of three-dimensional coordinates coordinates_be of the key feature points on the scraper body in the body coordinate system and the ten sets of three-dimensional coordinates coordinates_al, the transformation matrix Tal2be between the second auxiliary laser radar and the rear body coordinate system is calculated by nonlinear optimization and other methods.

[0118] The present invention can calibrate the external parameters of sensors located at different positions such as the front vehicle and the rear vehicle in the unmanned system of the scraper loader in real time, and can determine the external parameters of the sensors located at different positions such as the front vehicle and the rear vehicle relative to the coordinate system of the rear vehicle axle when the angle between the front vehicle axle and the rear vehicle axle is deflected, thereby effectively fusing the signals sensed by the sensors fixed on the front vehicle and the signals sensed by the sensors fixed on the rear vehicle, providing comprehensive and real-time perception signals for the smooth operation of the unmanned system of the scraper loader; providing a fast and convenient perception method for the perception of information at various angles on the scraper loader, significantly improving the scraper loader's perception ability of external information, with good effect and low cost.

[0119] Furthermore, by using a square or rectangular box calibration object, the sparse point cloud scanned on different planes of the box is fitted to each plane of the box, and the 3D coordinates of the box's vertices relative to the laser coordinate system are accurately determined. This enables accurate detection of the 3D coordinates of key points in sparse point clouds, providing a reliable and effective solution for extrinsic parameter calibration of sparse point cloud LiDARs. This significantly improves the excavator's ability to perceive external information, with excellent results and low cost.

[0120] Example 2:

[0121] The present invention provides a device for calibrating external parameters of a scraper sensor, the device comprising:

[0122] The first transformation module is used to determine the direction of the front axle coordinate system and the direction of the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system;

[0123] The second transformation module is used to obtain vehicle parameters and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters;

[0124] A third transformation module is used to obtain a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to the transformation matrix between the front vehicle sensor and the front axle coordinate system and the transformation matrix between the front axle coordinate system and the rear axle coordinate system;

[0125] Calibration module: used to calibrate both the front vehicle sensor and the rear vehicle axle coordinate system according to the transformation matrix between the rear vehicle sensor and the rear vehicle axle coordinate system and the transformation matrix between the front vehicle sensor and the rear vehicle axle coordinate system, thereby realizing external parameter calibration.

[0126] Specifically, such as Figure 3 As shown, the first transformation module specifically includes a front vehicle coordinate system selection module, a rear vehicle coordinate system selection module, a front vehicle sensor relative to the front vehicle coordinate system calibration module, and a rear vehicle sensor relative to the rear vehicle coordinate system transformation matrix calibration module; the front vehicle coordinate system selection module is used to determine the direction of the front vehicle axle coordinate system; the rear vehicle coordinate system selection module is used to determine the direction of the rear axle coordinate system; the front vehicle sensor relative to the front vehicle coordinate system calibration module is used to obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system; the rear vehicle sensor relative to the rear vehicle coordinate system transformation matrix calibration module is used to obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system;

[0127] The second transformation module includes a front vehicle coordinate system to rear vehicle coordinate system conversion module, which is used to obtain vehicle parameters and obtain a transformation matrix between the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters;

[0128] The third transformation module includes a front vehicle sensor relative to the rear vehicle coordinate system transformation matrix calibration module, which is used to obtain the transformation matrix of the front vehicle sensor and the rear axle coordinate system based on the transformation matrix of the front vehicle sensor and the front axle coordinate system and the transformation matrix of the front axle coordinate system and the rear axle coordinate system;

[0129] The calibration module includes an external parameter calibration module for different sensors of the scraper, which is used to calibrate both the front vehicle sensor and the rear vehicle axle coordinate system according to the transformation matrix between the rear vehicle sensor and the rear axle coordinate system and the transformation matrix between the front vehicle sensor and the rear axle coordinate system, thereby realizing external parameter calibration.

[0130] The underground scraper uses the leading and trailing vehicle coordinate system selection modules to select two base coordinate systems: one centered on the trailing vehicle's axle, and the other centered on the leading vehicle's axle. These two coordinate systems have the x-axis pointing forward, the y-axis pointing left, and the z-axis pointing upward. The trailing and leading vehicle axle centers are linked by a rotary axis. Sensors mounted on the leading vehicle (each consisting of a lidar and a camera) fuse sensor equipment and are calibrated based on the coordinate system centered on the leading vehicle's axle. Sensors mounted on the trailing vehicle (each consisting of a lidar and a camera) are calibrated based on the coordinate system centered on the leading vehicle's axle.

[0131] Specifically, the calibration module of the front vehicle sensor relative to the front vehicle coordinate system: the calibration of the lidar in the front vehicle sensor relative to the front vehicle coordinate system can be calibrated through the calibration scheme of the lidar and the vehicle body coordinate system; the calibration of the camera in the front vehicle sensor relative to the front vehicle coordinate system can first calibrate the transformation matrix from the camera coordinate system to the lidar coordinate system, and then based on the external parameter calibration result from the lidar coordinate system to the front vehicle coordinate system, realize the calibration of the transformation matrix from the front vehicle camera to the front vehicle coordinate system.

[0132] The rear vehicle sensor relative to the rear vehicle coordinate system transformation matrix calibration module: the calibration of the laser radar in the rear vehicle sensor relative to the rear vehicle coordinate system can be calibrated through the calibration scheme of the laser radar and the vehicle body coordinate system. The calibration of the camera in the rear vehicle sensor relative to the rear vehicle coordinate system can first calibrate the transformation matrix from the camera coordinate system to the laser radar coordinate system, and then based on the external parameter calibration result from the laser radar coordinate system to the rear vehicle coordinate system, realize the calibration of the transformation matrix from the rear vehicle camera to the rear vehicle coordinate system.

[0133] Example 3:

[0134] An embodiment of the present invention further provides a terminal, including a processor and a storage medium;

[0135] The storage medium is used to store instructions;

[0136] The processor is configured to operate according to the instructions to execute the steps of the method described in embodiment 1.

[0137] Example 4:

[0138] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0139] Since the storage medium provided in the embodiment of the present invention can execute the method provided in the first embodiment of the present invention, it has the corresponding functional modules and beneficial effects of the execution method.

[0140] The method described in Example 1 can be implemented in the form of a software program and run on an electronic device including a processor to form a scraper sensor external parameter calibration device; it should be noted that the processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements; for example, it can be one or more central processing units (CPUs), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more microprocessors, or one or more field programmable gate arrays (FPGAs).

[0141] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0142] In a specific implementation, as an embodiment, the processor may include one or more CPUs; each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU); the processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions); electronic devices may include: servers, industrial control computers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., where the embedded computers include vehicles and robots, etc.

[0143] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0147] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for calibrating external parameters of scraper sensors, characterized in that: The following steps are involved: Determine the directions of the front axle coordinate system and the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system; Obtain vehicle parameters, and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters; Obtaining a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to a transformation matrix between the front vehicle sensor and the front axle coordinate system and a transformation matrix between the front axle coordinate system and the rear axle coordinate system; According to the transformation matrix between the rear vehicle sensor and the rear axle coordinate system and the transformation matrix between the front vehicle sensor and the rear axle coordinate system, both the front vehicle sensor and the rear vehicle sensor are calibrated to the rear axle coordinate system to realize the external parameter calibration.

2. The scraper sensor external parameter calibration method according to claim 1, characterized in that: The front vehicle sensor includes a front vehicle laser radar and a front vehicle camera, and the rear vehicle sensor includes a rear vehicle laser radar and a rear vehicle camera.

3. The scraper sensor external parameter calibration method according to claim 1, characterized in that: The vehicle parameters include the length from the center of the front axle to the center of the rotating axis, the length from the center of the rear axle to the center of the rotating axis, and the angle between the front axle and the rear axle; The transformation matrix of the front axle coordinate system and the rear axle coordinate system is obtained according to the vehicle parameters. The specific expression is as follows: ； Where: is the transformation matrix between the front axle coordinate system and the rear axle coordinate system; is the angle between the front axle and the rear axle; It is the length from the center of the front axle to the center of the rotating axis; It is the length from the center of the rear axle to the center of the rotary axis.

4. The scraper sensor external parameter calibration method according to claim 2, characterized in that: The obtaining of the transformation matrix of the front vehicle sensor and the front vehicle axle coordinate system specifically includes: Get the transformation matrix between the front vehicle LiDAR and the front vehicle axle coordinate system, and get the transformation matrix between the front vehicle LiDAR and the front vehicle camera; According to the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system and the transformation matrix of the front vehicle laser radar and the front vehicle camera, the transformation matrix of the front vehicle camera and the front vehicle axle coordinate system is obtained.

5. The scraper sensor external parameter calibration method according to claim 4, characterized in that: The step of obtaining the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system specifically includes: Step a: Determine the positions of the first auxiliary laser radars on both sides of the leading vehicle, and obtain the transformation matrix between the two first auxiliary laser radars using a calibration object; wherein the leading vehicle is in a horizontal state; Step b: Scanning the preceding vehicle using the first auxiliary laser radar to obtain point cloud data of the preceding vehicle from two first auxiliary laser radars, and splicing the point cloud data of the preceding vehicle from the two first auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the preceding vehicle; Step c: Extracting point clouds of the surfaces of the two front tires based on the 3D reconstructed point cloud data of the front vehicle, fitting the 3D coordinates of the center points of the front tires based on the point clouds of the front tires, and calculating the position coordinates and length of the front axle based on the 3D coordinates of the center points of the two front tires; Step d: Compare the length of the front axle with a preset range. If the length of the front axle is within the preset range, execute step e. If the length of the front axle is not within the preset range, re-determine the position of the first auxiliary laser radar on both sides of the front vehicle and repeat steps a, b, c, and d. Step e: Determine the positions of the second auxiliary laser radars on both sides of the rear vehicle, and obtain the transformation matrix between the two second auxiliary laser radars through the calibration object; wherein the rear vehicle is in a horizontal state; Step f: Scanning the rear vehicle using the second auxiliary laser radar to obtain rear vehicle point cloud data from the two second auxiliary laser radars, and splicing the rear vehicle point cloud data from the two second auxiliary laser radars to obtain three-dimensional reconstructed point cloud data of the rear vehicle; Step g: extracting point clouds of the surfaces of the two rear tires based on the 3D reconstructed point cloud data of the rear vehicle, fitting the 3D coordinates of the center points of the rear tires based on the point clouds of the rear tire surfaces, and calculating the position coordinates and length of the rear axle based on the 3D coordinates of the center points of the two rear tires; Step h: Compare the length of the rear axle with a preset range. If the length of the rear axle is within the preset range, execute step i. If the length of the rear axle is not within the preset range, re-determine the position of the second auxiliary laser radar on both sides of the rear vehicle and repeat steps e, f, g, and h. Step i: Determine the XY plane of the front axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the front axle coordinate system; and determine the transformation matrix between the first auxiliary laser radar and the front axle coordinate system based on the XY plane of the front axle coordinate system and the position coordinates of the front axle; Step j: Obtain the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar, and obtain the transformation matrix of the front vehicle laser radar and the front vehicle axle coordinate system based on the transformation matrix of the first auxiliary laser radar and the front vehicle laser radar and the transformation matrix of the first auxiliary laser radar and the front vehicle axle coordinate system.

6. The method for calibrating external parameters of scraper sensors according to claim 5, characterized in that: The step of obtaining the transformation matrix of the rear vehicle sensor and the rear vehicle axle coordinate system specifically includes: Get the transformation matrix between the rear vehicle LiDAR and the rear vehicle axle coordinate system, and get the transformation matrix between the rear vehicle LiDAR and the rear vehicle camera; Obtain the transformation matrix of the rear vehicle camera and the rear axle coordinate system based on the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system and the transformation matrix of the rear vehicle laser radar and the rear vehicle camera; The step of obtaining the transformation matrix of the rear vehicle laser radar and the rear vehicle axle coordinate system specifically includes: Determine the XY plane of the rear axle coordinate system based on the position coordinates of the front axle, the position coordinates of the rear axle, and the direction of the rear axle coordinate system; and determine the transformation matrix between the second auxiliary laser radar and the rear axle coordinate system based on the XY plane of the rear axle coordinate system and the position coordinates of the rear axle. Obtain the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar, and obtain the transformation matrix of the rear vehicle laser radar and the rear axle coordinate system based on the transformation matrix of the second auxiliary laser radar and the rear vehicle laser radar and the transformation matrix of the second auxiliary laser radar and the rear axle coordinate system.

7. The method for calibrating external parameters of scraper sensors according to claim 1, characterized in that: The calibrating of the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system specifically includes: The calibration method of continuous transformation of transformation matrix is ​​adopted to calibrate the front vehicle sensor and the rear vehicle sensor to the rear axle coordinate system.

8. A scraper sensor external parameter calibration device, characterized in that: The device comprises: The first transformation module is used to determine the direction of the front axle coordinate system and the direction of the rear axle coordinate system, and obtain the transformation matrix between the front vehicle sensor and the front axle coordinate system according to the direction of the front axle coordinate system; and obtain the transformation matrix between the rear vehicle sensor and the rear axle coordinate system according to the direction of the rear axle coordinate system; The second transformation module is used to obtain vehicle parameters and obtain the transformation matrix of the front axle coordinate system and the rear axle coordinate system according to the vehicle parameters; A third transformation module is used to obtain a transformation matrix between the front vehicle sensor and the rear axle coordinate system according to the transformation matrix between the front vehicle sensor and the front axle coordinate system and the transformation matrix between the front axle coordinate system and the rear axle coordinate system; Calibration module: used to calibrate both the front vehicle sensor and the rear vehicle axle coordinate system according to the transformation matrix between the rear vehicle sensor and the rear vehicle axle coordinate system and the transformation matrix between the front vehicle sensor and the rear vehicle axle coordinate system, thereby realizing external parameter calibration.

9. A terminal, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images