Target three-dimensional reconstruction system and method for constructing three-dimensional contour of moving target

Abstract

The invention discloses a target three-dimensional reconstruction system and a method for constructing a three-dimensional contour of a moving target. The target three-dimensional reconstruction system comprises a portal frame, a laser radar, a linear array camera, a microwave radar, a wheel sensor and a control module, wherein an operation track of a target is arranged below the portal frame; the control module is configured to pre-process data acquired by each laser radar and each linear array camera in the process that a target runs on a running orbit, and obtain three-dimensional coordinate data of the target; calculating the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the movement speed of the target; and reconstructing a three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction and data acquired by the microwave radar and each laser radar. In the above mode, by fusing independent speed measurement or collaborative speed measurement results of multiple sensors, the scheme can realize high-precision three-dimensional contour reconstruction and dynamic analysis on a target running on a track, and the accuracy of the reconstructed contour and the reliability of the system are remarkably improved.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent detection and three-dimensional reconstruction, and specifically relates to a target three-dimensional reconstruction system and a method for constructing a three-dimensional contour of a moving target. Background Art

[0002] As railway freight transportation continues to accelerate, the safety and maintenance efficiency of freight trains are becoming increasingly important. This is not only related to the safe delivery of goods, but also to the overall efficiency and reliability of railway transportation. Traditional freight inspection relies on the two-dimensional imaging technology of a single linear array camera and requires manual visual analysis. This type of solution has the following defects: (1) Visible light imaging is easily disturbed by lighting conditions (such as low light and strong light), resulting in a decrease in the image signal-to-noise ratio; (2) Two-dimensional projection loses depth information and cannot quantify the three-dimensional deformation of the target, thus affecting the accuracy and completeness of the analysis results. In addition, the imaging mode of the linear array camera is a 2D plane mode, which can only capture the projection information of the object to be measured on the two-dimensional plane. The spatial scale information such as the depth and height of the target to be measured in three-dimensional space is completely lost in 2D imaging, resulting in the inability to perform quantitative analysis on the spatial scale. Summary of the invention

[0003] To solve the above problems, the present application provides a target three-dimensional reconstruction system and a method for constructing a three-dimensional profile of a moving target, which can perform three-dimensional reconstruction and quantitative analysis on the target to be measured.

[0004] A technical solution adopted in the present application is: to provide a target 3D reconstruction system, which includes: a gantry, laser radars and linear array cameras are arranged at the crossbeam, left and right columns of the gantry and the two angles formed by the crossbeam and left and right columns of the gantry; wherein a running track of the target is arranged under the gantry; a plurality of wheel sensors are deployed on both sides of the running track; at least one microwave radar, which is deployed on the crossbeam of the gantry; a control module, which connects each laser radar, microwave radar, each linear array camera and each wheel sensor, and the control module is configured to: pre-process the data collected by each laser radar and each linear array camera during the running of the target on the running track to obtain the three-dimensional coordinate data of the target; calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; and reconstruct the three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar and each laser radar.

[0005] In one embodiment, the control module is further configured to: perform time synchronization on the data collected by each laser radar and each linear array camera; obtain the posture information of each laser radar according to the acceleration and angular velocity of each laser radar within a preset scanning period, and pre-process the data collected by each laser radar according to the posture information to obtain the three-dimensional coordinate data of the target; wherein the posture information includes the displacement information and rotation information of the laser radar within the preset scanning period; wherein the acceleration and angular velocity are collected by the inertial measurement unit on the laser radar.

[0006] In one embodiment, the control module is further configured to: integrate the acceleration and angular velocity of each laser radar within a single scanning cycle respectively to obtain posture information; and construct a first transformation matrix based on the integration results, and compensate each point in the first transformation matrix to perform motion distortion correction processing on the data collected by each laser radar.

[0007] In one embodiment, the control module is further configured to: determine a reference coordinate system, calculate calibration parameters of each laser radar, microwave radar, each linear array camera and each wheel sensor relative to the reference coordinate system; wherein the calibration parameters include a rotation matrix and a translation vector; convert data collected by each laser radar, microwave radar, each linear array camera and each wheel sensor into the reference coordinate system according to the calibration parameters; integrate the acceleration and angular velocity of each laser radar within two scanning cycles respectively to obtain attitude information, and construct a second transformation matrix based on the integration results to obtain the attitude trajectory of the laser radar.

[0008] In one embodiment, the control module is further configured to: match the data collected by each laser radar with a preset map; construct a third transformation matrix based on the matching results, and update the attitude trajectory of the laser radar based on the third transformation matrix and the acceleration and angular velocity of each laser radar within two scanning cycles; obtain the odometer corresponding to each laser radar based on the update results to obtain the odometer of the target and obtain the three-dimensional coordinate data of the target.

[0009] In one embodiment, the control module is further configured to: obtain a first moving speed of the target based on the odometer of the target; and calculate a movement amount of the target in a three-dimensional direction according to the three-dimensional coordinate data and the first moving speed.

[0010] In one embodiment, the control module is further configured to: obtain a second moving speed based on data collected by the wheel sensor; obtain a third moving speed based on data collected by the microwave radar; fuse the first moving speed, the second moving speed and the third moving speed to obtain a fourth moving speed of the target; and calculate the movement amount of the target in the three-dimensional direction based on the three-dimensional coordinate data and the fourth moving speed.

[0011] In one embodiment, the control module is further configured to: obtain a state prediction equation, and fuse the first moving speed, the second moving speed, and the third moving speed according to the state prediction equation; obtain the odometer of the target according to the fusion result, and obtain the fourth moving speed based on the odometer of the target.

[0012] The present application also provides a method for constructing a three-dimensional contour of a moving target, wherein the target passes through a gantry while running on a running track, and laser radars and linear array cameras are arranged at the crossbeam, left and right columns of the gantry, and two angles formed by the crossbeam and left and right columns of the gantry; wherein a running track for the target is arranged below the gantry; a plurality of wheel sensors are arranged on both sides of the running track; at least one microwave radar is arranged on the crossbeam of the gantry; each laser radar, microwave radar, each linear array camera and each wheel sensor are respectively connected to a control module, and the method comprises: pre-processing data collected by each laser radar and each linear array camera to obtain three-dimensional coordinate data of the target; calculating the amount of movement of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; and reconstructing the three-dimensional contour of the target according to the amount of movement of the target in the three-dimensional direction, the data collected by the microwave radar and each laser radar.

[0013] A technical solution adopted in the present application is: to provide a target 3D reconstruction system, which includes: a gantry, laser radars and linear array cameras are arranged at the crossbeam, left and right columns of the gantry and the two angles formed by the crossbeam and left and right columns of the gantry; wherein a running track of the target is arranged under the gantry; a plurality of wheel sensors are deployed on both sides of the running track; at least one microwave radar, which is deployed on the crossbeam of the gantry; a control module, which connects each laser radar, microwave radar, each linear array camera and each wheel sensor, and the control module is configured to: pre-process the data collected by each laser radar and each linear array camera during the running of the target on the running track to obtain the three-dimensional coordinate data of the target; calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; and reconstruct the three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar and each laser radar. In the above method, the data collected by multiple sensors are fused with the results of independent speed measurement or fusion speed measurement of microwave radar, wheel sensor, and lidar, which can reconstruct the three-dimensional contour of the moving target with high precision and conduct in-depth analysis, thereby improving the accuracy and reliability of the three-dimensional contour of the reconstructed target. By fusing the independent speed measurement or coordinated speed measurement results of multiple sensors (including microwave radar, wheel sensor, and lidar), this solution can achieve high-precision three-dimensional contour reconstruction and dynamic analysis of the target running on the track, significantly improving the accuracy of the reconstructed contour and system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 is a first structural schematic diagram of a target three-dimensional reconstruction system provided in some embodiments of the present application; Figure 2 is a schematic diagram of a first process of a method for three-dimensional contouring of a target provided in some embodiments of the present application; Figure 3 Some embodiments of the present application provide Figure 2 Schematic diagram of the sub-process of step S21; Figure 4 Some embodiments of the present application provide Figure 3 Schematic diagram of the first sub-process of step S212; Figure 5 Some embodiments of the present application provide Figure 3 Schematic diagram of the second sub-process of step S212; Figure 6 Some embodiments of the present application provide Figure 3 Schematic diagram of the third sub-process of step S212; Figure 7 Some embodiments of the present application provide Figure 2 Schematic diagram of the first sub-process of step S22; Figure 8 Some embodiments of the present application provide Figure 2 Schematic diagram of the second sub-process of step S22; Fig. 9 Some embodiments of the present application provide Figure 8 Schematic diagram of the sub-process of step S225. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0016] The terms "first", "second", etc. in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0017] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0018] See also Figure 1 , Figure 1 1 is a schematic diagram of the structure of the first embodiment of the target 3D reconstruction system provided by the present application. The target 3D reconstruction system 100 comprises: a gantry 10, a laser radar 20, a linear array camera 30, a wheel sensor 40, at least one microwave radar 50 and a control module (not shown).

[0019] Among them, laser radars 20 and linear array cameras 30 are installed on the crossbeam, left and right columns of the gantry 10 and the two angles formed by the crossbeam and left and right columns of the gantry 10; a target running track is set under the gantry 10; and multiple wheel sensors 40 are deployed on both sides of the running track.

[0020] The microwave radar 50 is deployed on the crossbeam of the gantry 10 .

[0021] Among them, the control module is connected to each laser radar 20, microwave radar 50, each line array camera 30 and each wheel sensor 40, and the control module is configured to: when the target is running on the running track, pre-process the data collected by each laser radar 20 and each line array camera 30 to obtain the three-dimensional coordinate data of the target; calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; reconstruct the three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar 50 and each laser radar 20.

[0022] Specifically, multiple laser radars 20 are installed at different positions of the gantry 10, which can achieve more comprehensive spatial information acquisition and high-precision positioning of the target. After the point cloud data of the target is collected by the laser radar 20, these data are usually processed by noise filtering, de-duplication, and registration, so as to remove stray signals introduced by environmental factors (such as atmospheric scattering, equipment errors, etc.) and eliminate repeated measurement points. When the target is running on the running track, the vibration or swing of the gantry 10 will cause distortion to the collected data, so preprocessing is required to eliminate the distortion of the data; at the same time, the data collected by the laser radar 20 at different viewing angles are often independent of each other in coordinate system due to differences in collection position, angle and time. When preprocessing, it is also necessary to unify the data collected by the laser radar 20 at different viewing angles into the same coordinate system to form a complete three-dimensional point cloud map.

[0023] Specifically, when the line array camera 30 is selected as an auxiliary imaging means, it is usually necessary to process the image data collected by it, such as performing pre-processing such as image enhancement, correction and segmentation on the image data, so as to improve the visual characteristics of the image, facilitate subsequent feature extraction, eliminate lens distortion, perspective distortion and other problems, ensure the accuracy of image geometric information, extract the image area related to the target to be reconstructed, and reduce the interference of irrelevant information.

[0024] Specifically, based on the three-dimensional coordinate data of the target, the movement amount of the target in the three-dimensional direction within the continuous time interval can be calculated, for example, by comparing the three-dimensional coordinates at different time points, the displacement change of the target in the three directions of X, Y, and Z can be derived. The data collected by sensors such as the microwave radar 50, the linear array camera 30, and the laser radar 20 are fused, and the three-dimensional profile of the target is updated in real time in combination with the movement amount of the target to achieve dynamic modeling.

[0025] In the above scheme, by combining the data of multiple sensors and the dynamic information of the target, the shape and position changes of the target can be accurately captured, and high-precision and real-time three-dimensional reconstruction can be achieved. Through data fusion and dynamic modeling, the stability and reliability of the three-dimensional contour of the reconstructed target can be effectively improved.

[0026] In some embodiments, the control module is further configured to: time synchronize the data collected by each laser radar 20 and each linear array camera 30; obtain the posture information of each laser radar 20 according to the acceleration and angular velocity of each laser radar 20 within a preset scanning cycle, and pre-process the data collected by each laser radar 20 according to the posture information to obtain the three-dimensional coordinate data of the target; wherein the posture information includes the displacement information and rotation information of the laser radar 20 within the preset scanning cycle; wherein the acceleration and angular velocity are collected by the inertial measurement unit on the laser radar 20.

[0027] Specifically, since the laser radar 20 and the linear array camera 30 are two different sensors, they each collect data at different frequencies and timings. Therefore, in order to ensure that these data can accurately reflect the scene information at the same time, they need to be time synchronized. In one embodiment, time synchronization is achieved by using PTP (Precision Time Protocol). During the data collection process of the laser radar 20 and the linear array camera 30, each radar data frame and camera data frame will be accompanied by a timestamp to identify the time when the frame data is collected. To achieve multi-sensor time synchronization, the system clock or external GNSS (Global Navigation Satellite System) time can be used as a global reference time source through PTP. Specifically, based on the GNSS timing module or system clock, synchronization signals are distributed to the laser radar and the linear array camera through PTP to force the local clocks of each sensor to be aligned. Under this mechanism, both the laser radar and the linear array camera use a unified time reference to achieve sub-microsecond time alignment accuracy, thereby eliminating clock drift errors between devices and ensuring that all data are strictly aligned on the time axis.

[0028] Among them, the Inertial Navigation Unit (IMU) is an autonomous navigation system that can provide the carrier's own motion state information. In one embodiment, it uses the laser radar 20 as the carrier. Through the accelerometer and gyroscope inside the inertial measurement unit, the acceleration and angular velocity of the laser radar 20 in three-dimensional space can be measured in real time, thereby obtaining the attitude information of the laser radar 20.

[0029] In the above scheme, time synchronization is used to ensure that data from different sensors can be aligned on the same time base. The inertial measurement unit is combined with the lidar 20 to combine the posture information of the lidar 20 with the point cloud data collected by the lidar 20 to correct motion distortion, thereby improving the accuracy and reliability of the three-dimensional reconstruction of the target.

[0030] In some embodiments, the control module is also configured to: integrate the acceleration and angular velocity of each laser radar 20 within a single scanning cycle respectively to obtain posture information; and construct a first transformation matrix based on the integration results, and compensate each point in the first transformation matrix to perform motion distortion correction processing on the data collected by each laser radar 20.

[0031] Specifically, the accelerometer inside the inertial measurement unit measures the linear acceleration of the laser radar 20 in the three axes of X, Y, and Z, while the gyroscope measures the angular velocity in the three axes. By pre-integrating these acceleration and angular velocity data respectively, the linear displacement change and rotation angle change of the laser radar 20 in the preset scanning cycle, that is, the posture information, can be obtained. The first transformation matrix is ​​constructed based on the integration result. The first transformation matrix describes the displacement change and rotation angle change in a single scanning cycle of the laser radar 20. According to this change, the compensation between the remaining points and the first point (arranged in time sequence or data recording sequence) in a frame of data scanned by the laser radar 20 (that is, all point cloud data collected in one scanning cycle) is realized, and the coordinates of the remaining points in each frame are compensated to the coordinate system at the moment when the first point is collected, eliminating the distortion caused by the movement of the carrier.

[0032] In one application scenario, first, the acceleration of each laser radar 20 in a single scanning cycle is pre-integrated: the acceleration is obtained by the inertial measurement unit, and the following formula is used to obtain Transformation of velocity within a time window:

[0033] in, represents the acceleration measured by the inertial measurement unit, represents the acceleration due to gravity, and are the start time and end time of a scan of the laser radar 20 respectively. The speed can be obtained by the above formula and .

[0034] right The velocity within the time window is integrated and expressed as follows:

[0035] Assume that Within the time window, the speed changes are relatively stable or uniform. This can usually be simplified to: ,in, is the average speed, yes and The time interval between represents the average speed, usually expressed as: .

[0036] That is, the time window The internal displacement is expressed as:

[0037] in, yes The average acceleration in the time window, that is, the mean value of the linear acceleration measured by the inertial measurement unit, is the initialization speed.

[0038] Then, the angular velocity of each laser radar 20 in a single scanning cycle is pre-integrated: The angular velocity measured by the inertial measurement unit in the time window is pre-integrated and represented by quaternion. The relative rotation angle within the time window can be obtained by the following formula:

[0039] in, and The quaternion representing the rotation angle at the beginning and end of a single scan cycle, Represents angular velocity.

[0040] Finally, according to the time window The pre-integration results within and Construct the first transformation matrix, traverse each point in each frame of the laser radar 20, and then calculate the time difference between the i-th point and the initial point or the end point , perform linear interpolation to obtain the transformation from the i-th point to the first point in each frame , the coordinates of each point in the frame are compensated to the coordinate form of the first point through the following formula, thus eliminating the distortion of the point cloud:

[0041] in, is the original coordinate of the i-th point in each frame of the laser radar 20, is the transformation from the i-th point to the first point after interpolation, Represents the coordinate system of the i-th point after motion distortion correction relative to the first point.

[0042] In the above scheme, the precise compensation transformation can retain the detailed information in the point cloud data. The point cloud data after motion distortion correction by the above method can reduce error accumulation and improve the accuracy and stability of target three-dimensional reconstruction.

[0043] In some embodiments, the control module is further configured to: determine a reference coordinate system, calculate calibration parameters of each laser radar 20, microwave radar 50, each linear array camera 30 and each wheel sensor 40 relative to the reference coordinate system; wherein the calibration parameters include a rotation matrix and a translation vector; according to the calibration parameters, convert the data collected by each laser radar 20, microwave radar 50, each linear array camera 30 and each wheel sensor 40 into the reference coordinate system; integrate the acceleration and angular velocity of each laser radar 20 within two scanning cycles respectively to obtain attitude information, and construct a second transformation matrix based on the integration results to obtain the attitude trajectory of the laser radar 20.

[0044] In one embodiment, the target can be regarded as a fixed reference object, and the laser radar 20 installed on the gantry 10 itself can be used as a dynamic sensor, so as to estimate the relative motion odometer of the target by utilizing the coordinate changes and direction or orientation changes of the laser radar 20 in space relative to the reference coordinate system.

[0045] Specifically, the coordinate system of a certain laser radar 20 is used as the reference coordinate system, and the coordinate conversion relationship between different sensors can be established through calibration parameters, so that the data collected by each laser radar 20, microwave radar 50, each linear array camera 30 and each wheel sensor 40 are converted to the same reference system for fusion. For different types of sensors, the calculation method and content of the calibration parameters are also different, which is not limited here.

[0046] In one embodiment, the acceleration and angular velocity measured by the inertial measurement unit are pre-integrated respectively within two scanning times to obtain a series of attitude estimates of the laser radar 20, which can be represented as a second transformation matrix, which contains the coordinates and orientation (or direction) information of the laser radar 20 at each discrete time point within the two scanning times. Since the integration operation is based on the measurement values ​​at discrete time points, the attitude estimate obtained is also discrete, and these discrete points are connected to form a coarse discrete time trajectory that describes the attitude trajectory of the laser radar 20. Due to factors such as cumulative errors and sensor noise in the integration process, this coarse discrete time trajectory has certain errors and uncertainties, so it is necessary to perform a refinement operation.

[0047] Specifically, a set of continuous time equations are constructed (these equations may include the kinematic model, dynamic model and possible deformation model of the object, etc.). These time equations describe the continuous change process of the laser radar 20 from the previous posture to the current posture; solving these time equations can obtain a more accurate and smooth posture trajectory. In the process of refining the posture trajectory, it is also necessary to consider the deformation or tilt changes that may occur in the laser radar 20 during the relative motion. These changes may cause the position of certain points on the laser radar 20 to shift in three-dimensional space; in order to restore the accurate position of these points, it is necessary to apply a specific de-skew transformation from the most recent previous transformation (i.e., the last posture estimation) to each current point. The de-skew transformation may include rotation, translation, and possible scaling operations.

[0048] In the above scheme, by refining the trajectory and restoring the de-skew transformation, the posture trajectory of the laser radar 20 within a preset scanning period can be obtained, thereby significantly improving the accuracy of three-dimensional reconstruction of the target.

[0049] In some embodiments, the control module is also configured to: match the data collected by each laser radar 20 with a preset map; construct a third transformation matrix based on the matching results, and update the attitude trajectory of the laser radar 20 based on the third transformation matrix and the acceleration and angular velocity of each laser radar 20 within two scanning cycles; obtain the odometer corresponding to each laser radar 20 according to the update result to obtain the odometer of the target and obtain the three-dimensional coordinate data of the target.

[0050] Specifically, after the motion distortion correction processing is performed on the data collected by each laser radar 20, the data collected by each laser radar 20 is matched with the preset map, which is a high-precision three-dimensional environment model in the world coordinate system. By calculating the distance between the point cloud data collected by the laser radar 20 and the corresponding points or planes in the preset map, the optimal transformation relationship is found, and the point cloud data collected by the laser radar 20 will be converted to the world coordinate system, matched and updated with the features in the preset map. The above matching results can obtain a third transformation matrix, which reflects the three-dimensional coordinates and other information of the point cloud data collected by the laser radar 20 in the world coordinate system. The third transformation matrix is ​​integrated with the acceleration and angular velocity of the laser radar 20 in two scanning cycles to generate a more complete and accurate attitude trajectory of the laser radar 20 in the world coordinate system.

[0051] After obtaining the complete and accurate attitude trajectory of the laser radar 20 in the world coordinate system, the odometer of the laser radar 20 in the world coordinate system can be obtained according to the attitude trajectory, and then converted into the odometer of the target in the world coordinate system, that is, the accurate three-dimensional coordinate data of the target in the world coordinate system and information such as the orientation or direction of the target relative to the world coordinate system are obtained.

[0052] In the matching process, in order to improve the accuracy and efficiency of matching, the GICP (Generalized Iterative Closest Point) optimization prior can be introduced. The GICP algorithm minimizes the distance between corresponding points or planes through iterative optimization, thereby obtaining a more accurate transformation relationship. By incorporating prior knowledge (such as the previous matching result), the convergence speed of the matching process can be further accelerated, and the stability and accuracy of the matching can be improved.

[0053] In the above scheme, by matching the data collected by the laser radar 20 with the preset map, the point cloud data collected by the laser radar 20 can be converted into the world coordinate system. Combined with the acceleration and angular velocity of the laser radar 20, the attitude trajectory of the laser radar 20 can be estimated and updated in real time, thereby obtaining the odometer of the target during relative motion, thereby improving the positioning accuracy of the target's three-dimensional contour and the accuracy of constructing the target's three-dimensional contour.

[0054] In some embodiments, the control module is further configured to: obtain a first moving speed of the target based on the odometer of the target; and calculate a movement amount of the target in a three-dimensional direction according to the three-dimensional coordinate data and the first moving speed.

[0055] In one embodiment, the speed can be measured by the laser radar 20, the wheel sensor 40 and the microwave radar 50 respectively, or the speed measurement results of the three can be integrated to obtain the final moving speed. The method of measuring speed by the laser radar 20 is mainly described below.

[0056] Among them, the laser radar 20 outputs odometer information after performing the above-mentioned multiple sensor data coordinate system conversion, inertial measurement unit measurement value pre-integration, point cloud data matching, and attitude trajectory estimation and update. After the odometer output by the laser radar 20 is converted into the odometer of the target, the odometer contains the three-dimensional coordinate data of the target within the preset scanning cycle.

[0057] In an application scenario, the three-dimensional coordinate data of the target within two scanning cycles are obtained, and the first moving speed is calculated according to the three-dimensional coordinates of the target: first, the speed of the target in the three-dimensional direction is calculated, which is expressed by the following formula:

[0058] in, , , Respectively represent the speed in the X, Y, and Z directions, , , Respectively represent the changes in coordinates in the X, Y, and Z directions. Respectively represent the start and end time of the scan. Then, , , The first moving speed is obtained by merging, and the moving amount of the target in the three-dimensional direction is calculated according to the three-dimensional coordinate data and the first moving speed.

[0059] In the above scheme, the odometer output by the laser radar 20 can provide accurate position and direction information. The moving speed of the target is obtained through the laser radar 20 odometer, which can enhance the adaptability to dynamic environments and improve the accuracy and efficiency of the target's three-dimensional reconstruction.

[0060] In some embodiments, the control module is also configured to: obtain a second moving speed based on data collected by the wheel sensor 40; obtain a third moving speed based on data collected by the microwave radar 50; fuse the first moving speed, the second moving speed and the third moving speed to obtain a fourth moving speed of the target; and calculate the movement amount of the target in the three-dimensional direction based on the three-dimensional coordinate data and the fourth moving speed.

[0061] Specifically, the microwave radar 50 can directly output a speed measurement result; for the wheel sensor 40, two groups of wheel sensors 40 with known distances are used. When the target passes through the two groups of wheel sensors 40 respectively while running on the running track, the signals triggered by the first group of wheel sensors 40 and the second group of wheel sensors 40 are timed respectively to obtain two times, that is, the speed measurement result can be obtained through the distance and time difference.

[0062] Among them, for the speed measurement outputs of the wheel sensor 40 and the microwave radar 50, since their output frequencies are relatively low, it is necessary to use the similar timestamps in the speed measurement results of the laser radar 20 as a benchmark, use the uniform linear motion model, first interpolate the obtained speeds, and then integrate the speeds to obtain the final second moving speed and third moving speed.

[0063] A Kalman filter algorithm is used to fuse the data obtained from the three speed measurement methods, that is, the first moving speed, the second moving speed and the third moving speed are fused to obtain the fourth moving speed of the target, and the movement amount of the target in the three-dimensional direction is calculated according to the three-dimensional coordinate data and the fourth moving speed.

[0064] In some embodiments, the control module is further configured to: obtain a state prediction equation, fuse the first moving speed, the second moving speed and the third moving speed according to the state prediction equation; obtain the odometer of the target according to the fusion result, and obtain the fourth moving speed based on the odometer of the target.

[0065] Specifically, the state prediction equation is used to estimate the current state based on the previous state, usually including the state transfer matrix. Assuming that the target is a uniformly accelerated linear motion, the state prediction equation can be simplified as:

[0066] in, is the prediction of the state at time k based on the state at time k-1, A is the state transfer matrix, is the optimal state estimate at time k-1, that is, at time k-1, the most accurate estimate of the target speed based on all available measurement information (including the first moving speed, the second moving speed, and the third moving speed) and the dynamic model of the system. The Kalman gain determines the weights of the predicted value and the measured value in the final state estimate. The calculation of the Kalman gain is based on the process noise covariance and the measurement noise covariance. The first moving speed, the second moving speed, and the third moving speed are weighted averaged, and the weights are determined by the Kalman gain to obtain the final state estimate. , that is, the estimation of the target speed at time k, converting the state estimation into an odometer, which contains the three-dimensional coordinate data of the target within a preset scanning period, that is, obtaining the three-dimensional coordinate data of the target within a preset scanning period, calculating the speed of the target in the three-dimensional direction according to the transformation of the target three-dimensional coordinates and merging them to finally obtain the fourth moving speed.

[0067] In the above scheme, multiple optional speed measurement methods can improve the reliability of the overall speed measurement system. Fusion of the results of different speed measurement methods can improve the accuracy and real-time performance of speed measurement results in dynamic environments, thereby improving the accuracy, reliability and adaptability of target three-dimensional reconstruction.

[0068] See also Figure 2 , Figure 2 This is a flow chart of the first embodiment of the method for constructing a three-dimensional profile of a target provided by the present application. The target passes through a gantry while running on the running track. The crossbeam of the gantry, the left and right columns, and the two angles formed by the crossbeam and the left and right columns of the gantry are all provided with laser radars and linear array cameras; wherein, a running track of the target is provided below the gantry; multiple wheel sensors are deployed on both sides of the running track; at least one microwave radar is deployed on the crossbeam of the gantry; each laser radar, microwave radar, each linear array camera and each wheel sensor are respectively connected to a control module, and the method comprises: Step S21: pre-process the data collected by each laser radar and each linear array camera to obtain the three-dimensional coordinate data of the target.

[0069] In some embodiments, see Figure 3 , the above step S21 may include the following steps: Step S211: Time synchronization is performed on the data collected by each laser radar and each linear array camera.

[0070] Step S212: Acquire the posture information of each laser radar according to the acceleration and angular velocity of each laser radar within a preset scanning cycle, and pre-process the data collected by each laser radar according to the posture information to obtain the three-dimensional coordinate data of the target; wherein the posture information includes the displacement information and rotation information of the laser radar within the preset scanning cycle.

[0071] In some embodiments, see Figure 4 , the above step S212 may include the following steps: Step S2121: Integrate the acceleration and angular velocity of each laser radar in a single scanning cycle to obtain attitude information.

[0072] Step S2122: construct a first transformation matrix based on the integration result, and compensate each point in the first transformation matrix to perform motion distortion correction processing on the data collected by each laser radar.

[0073] In some embodiments, see Figure 5 , the above step S212 may further include the following steps: Step S2123: Determine a reference coordinate system, and calculate calibration parameters of each laser radar, microwave radar, each linear array camera, and each wheel sensor relative to the reference coordinate system; wherein the calibration parameters include a rotation matrix and a translation vector.

[0074] Step S2124: convert the data collected by each laser radar, microwave radar, each linear array camera and each wheel sensor into a reference coordinate system according to the calibration parameters.

[0075] Step S2125: Integrate the acceleration and angular velocity of each laser radar in two scanning cycles to obtain attitude information, and construct a second transformation matrix based on the integration results to obtain the attitude trajectory of the laser radar.

[0076] In some embodiments, see Figure 6 , the above step S212 may further include the following steps: Step S2126: Match the data collected by each laser radar with the preset map.

[0077] Step S2127: construct a third transformation matrix based on the matching results, and update the attitude trajectory of the laser radar based on the third transformation matrix and the acceleration and angular velocity of each laser radar within two scanning cycles.

[0078] Step S2128: Obtain the odometer corresponding to each laser radar according to the update result to obtain the odometer of the target and obtain the three-dimensional coordinate data of the target.

[0079] Step S22: Calculate the target's three-dimensional movement amount based on the three-dimensional coordinate data and the target's moving speed.

[0080] In some embodiments, see Figure 7 , the above step S22 may include the following steps: Step S221: Obtain a first moving speed of the target based on the odometer of the target.

[0081] Step S222: Calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the first moving speed.

[0082] In some embodiments, see Figure 8 , the above step S22 may further include the following steps: Step S223: Acquire a second moving speed based on the data collected by the wheel sensor.

[0083] Step S224: Acquire a third moving speed based on the data collected by the microwave radar.

[0084] Step S225: The first moving speed, the second moving speed and the third moving speed are merged to obtain a fourth moving speed of the target.

[0085] Step S226: Calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the fourth moving speed.

[0086] In some embodiments, see Fig. 9 , the above step S225 may include the following steps: Step S2251: Obtain a state prediction equation, and fuse the first moving speed, the second moving speed, and the third moving speed according to the state prediction equation.

[0087] Step S2252: Obtain the odometer of the target according to the fusion result, and obtain the fourth moving speed based on the odometer of the target.

[0088] Step S23: Reconstruct the three-dimensional profile of the target based on the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar and each laser radar.

[0089] A technical solution adopted in the present application is: to provide a target 3D reconstruction system, which includes: a gantry, laser radars and linear array cameras are arranged at the crossbeam, left and right columns of the gantry and the two angles formed by the crossbeam and left and right columns of the gantry; wherein a running track of the target is arranged under the gantry; a plurality of wheel sensors are deployed on both sides of the running track; at least one microwave radar, which is deployed on the crossbeam of the gantry; a control module, which connects each laser radar, microwave radar, each linear array camera and each wheel sensor, and the control module is configured to: pre-process the data collected by each laser radar and each linear array camera during the running of the target on the running track to obtain the three-dimensional coordinate data of the target; calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; and reconstruct the three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar and each laser radar. In the above method, the data collected by multiple sensors are fused with microwave radar, wheel sensor, and lidar to perform independent speed measurement or fusion speed measurement results, which can reconstruct the three-dimensional contour of the moving target with high precision and conduct in-depth analysis, thereby improving the accuracy and reliability of the three-dimensional contour of the reconstructed target.

[0090] The specific implementation of the present invention is only an example. Those skilled in the art may perform equivalent replacement or optimization of sensor types, data fusion algorithms, etc. within the scope of the claims, and such variations belong to the protection scope of the present invention. For example, the device implementation described above is only illustrative. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target three-dimensional reconstruction system, characterized in that: The target three-dimensional reconstruction system comprises: A gantry, wherein laser radars and linear array cameras are arranged at the crossbeam, left and right columns of the gantry and the two angles formed by the crossbeam and left and right columns of the gantry; wherein a running track of the target is arranged below the gantry; and a plurality of wheel sensors are deployed on both sides of the running track; at least one microwave radar, wherein the microwave radar is deployed on a crossbeam of the gantry; A control module is connected to each of the laser radars, the microwave radars, each of the linear array cameras and each of the wheel sensors, and the control module is configured to: when the target is running on the running track, pre-process the data collected by each of the laser radars and each of the linear array cameras to obtain the three-dimensional coordinate data of the target; calculate the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; and reconstruct the three-dimensional contour of the target according to the movement amount of the target in the three-dimensional direction, the data collected by the microwave radar and each of the laser radars.

2. The target three-dimensional reconstruction system according to claim 1, characterized in that: The control module is further configured to: The data collected by each of the laser radars and each of the linear array cameras are time synchronized; the posture information of each of the laser radars is obtained according to the acceleration and angular velocity of each of the laser radars within a preset scanning period, and the data collected by each of the laser radars is preprocessed according to the posture signal to obtain the three-dimensional coordinate data of the target; wherein the posture information includes the displacement information and rotation information of the laser radar within the preset scanning period; wherein the acceleration and the angular velocity are obtained by collecting the inertial measurement unit on the laser radar.

3. The target three-dimensional reconstruction system according to claim 2, characterized in that: The control module is further configured to: The acceleration and angular velocity of each of the laser radars in a single scanning cycle are respectively integrated to obtain the posture information; a first transformation matrix is ​​constructed based on the integration results, and each point in the first transformation matrix is ​​compensated to perform motion distortion correction processing on the data collected by each of the laser radars.

4. The target three-dimensional reconstruction system according to claim 2, characterized in that: The control module is further configured to: Determine a reference coordinate system, and calculate calibration parameters of each of the laser radars, the microwave radars, each of the linear array cameras, and each of the wheel sensors relative to the reference coordinate system; wherein the calibration parameters include a rotation matrix and a translation vector; Converting data collected by each of the laser radars, the microwave radars, each of the linear array cameras, and each of the wheel sensors into the reference coordinate system according to the calibration parameters; The acceleration and angular velocity of each laser radar in two scanning cycles are respectively integrated to obtain the posture information, and a second transformation matrix is ​​constructed according to the integration results to obtain the posture trajectory of the laser radar.

5. The target three-dimensional reconstruction system according to claim 4, characterized in that: The control module is further configured to: Matching the data collected by each of the laser radars with a preset map; Constructing a third transformation matrix according to the matching results, and updating the attitude trajectory of the laser radar according to the third transformation matrix and the acceleration and angular velocity of each of the laser radars within two scanning cycles; The odometer corresponding to each of the laser radars is obtained according to the update result to obtain the odometer of the target and obtain the three-dimensional coordinate data of the target.

6. The target three-dimensional reconstruction system according to claim 5, characterized in that: The control module is further configured to: Acquire a first moving speed of the target based on the odometer of the target; The movement amount of the target in the three-dimensional direction is calculated according to the three-dimensional coordinate data and the first movement speed.

7. The target three-dimensional reconstruction system according to claim 5, characterized in that: The control module is further configured to: Acquire a second moving speed based on the data collected by the wheel sensor; Acquire a third moving speed based on the data collected by the microwave radar; fusing the first moving speed, the second moving speed and the third moving speed to obtain a fourth moving speed of the target; The movement amount of the target in the three-dimensional direction is calculated according to the three-dimensional coordinate data and the fourth movement speed.

8. The target three-dimensional reconstruction system according to claim 7, characterized in that: The control module is further configured to: A state prediction equation is obtained, and the first moving speed, the second moving speed, and the third moving speed are fused according to the state prediction equation; an odometer of the target is obtained according to the fusion result, and the fourth moving speed is obtained based on the odometer of the target.

9. A method for constructing a three-dimensional contour of a target, characterized in that: The target passes through the gantry during the running track, and the crossbeam, left and right columns of the gantry and the two angles formed by the crossbeam and left and right columns of the gantry are all provided with laser radars and linear array cameras; wherein the running track of the target is arranged below the gantry; a plurality of wheel sensors are arranged on both sides of the running track; at least one microwave radar is arranged on the crossbeam of the gantry; each of the laser radars, the microwave radars, each of the linear array cameras and each of the wheel sensors are respectively connected to a control module, and the method comprises: Preprocessing the data collected by each of the laser radars and each of the linear array cameras to obtain three-dimensional coordinate data of the target; Calculating the movement amount of the target in the three-dimensional direction according to the three-dimensional coordinate data and the moving speed of the target; The three-dimensional profile of the target is reconstructed according to the movement amount of the target in the three-dimensional direction and the data collected by the microwave radar and each of the laser radars.

10. The method for constructing a three-dimensional contour of a target according to claim 9, characterized in that: The preprocessing of the data collected by each of the laser radars and each of the linear array cameras to obtain the three-dimensional coordinate data of the target includes: Performing time synchronization on the data collected by each of the laser radars and each of the linear array cameras; Acquire the attitude information of each of the laser radars according to the acceleration and angular velocity of each of the laser radars in a preset scanning cycle, and pre-process the data collected by each of the laser radars according to the attitude information to acquire the three-dimensional coordinate data of the target; wherein the attitude information includes the displacement information and rotation information of the laser radar in the preset scanning cycle; The odometer corresponding to each of the laser radars is obtained according to the update result to obtain the odometer of the target and obtain the three-dimensional coordinate data of the target.

Images