A three-dimensional target tracking method under a robot front-view perspective based on cubic matching

Abstract

The application discloses a kind of three-dimensional target tracking methods under the perspective of robot front view based on cubic matching, comprising: step 1: according to the two-dimensional target position obtained by the photographing device carried by robot, and the laser point cloud obtained by laser radar is fused to obtain three-dimensional target position;Step 2: predict the three-dimensional prediction position and two-dimensional prediction position of target;Step 3: the similarity of three-dimensional prediction position and two-dimensional prediction position and three-dimensional target position and two-dimensional target position is calculated respectively and three-dimensional similarity and two-dimensional similarity are obtained, and the final similarity is calculated;Step 4: the best matching relationship is obtained by using hungarian algorithm to optimize matching according to final similarity, and target tracking is completed by updating target trajectory according to the best matching relationship;The application is specially handled low-level target information lacking 3D information by third matching, and avoids that long-distance target cannot be tracked due to clustering failure.

Description

Technical Field

[0001] This invention belongs to the field of perception for autonomous mobile robots, and particularly relates to a three-dimensional target tracking method based on three-dimensional matching from the robot's forward-looking perspective. Background Technology

[0002] 3D target detection and tracking technology is one of the key technologies in the field of autonomous mobile robots. By detecting, identifying, and locating targets, robots can acquire semantic and positional information about obstacles in their surrounding environment. Based on this, robots can track obstacles and obtain their historical trajectories. Using this historical trajectory information, robots can understand the movement patterns of obstacles and predict their future positions, thus achieving the goal of early avoidance. Therefore, accurate target tracking is crucial for improving the autonomous navigation and obstacle avoidance performance of mobile robots.

[0003] Image and point cloud data fusion-based detection algorithms are commonly used methods for 3D target detection. Compared to single-sensor detection, the combination of LiDAR and camera can fully leverage the advantages of both sensors, reducing the computational burden of point cloud feature extraction while enabling the robot to grasp the 3D position information of the target. Compared to LiDAR-based bird's-eye view (BEV) detection methods, the camera's forward-looking view can acquire the height information of pedestrians, thus achieving better detection results for pedestrians and similar targets. However, in the forward-looking view, some targets may be occluded by other targets or obstacles, leading to a decrease in the detection algorithm's confidence in detecting such targets and ultimately a poorer matching effect between the target and the trajectory. Some algorithms simply abandon tracking such targets, resuming tracking when the target is no longer occluded; however, the tracking trajectory of these algorithms will exhibit fragmented characteristics. Furthermore, when the number of LiDAR points corresponding to a target is small, the algorithm cannot effectively cluster the target, thus failing to accurately extract the target's 3D information. Simply abandoning tracking such targets will also lead to fragmented tracking trajectories. Therefore, traditional forward-looking 3D target tracking methods cannot meet the needs of mobile robots for efficient 3D target tracking. Summary of the Invention

[0004] The purpose of this invention is to provide a three-dimensional target tracking method based on three-dimensional matching from the robot's forward-looking perspective, in order to solve the problems of poor target detection and fragmented tracking trajectory caused by target occlusion or long distance.

[0005] The present invention adopts the following technical solution: A 3D target tracking method based on cubic matching from a robot's forward-looking perspective includes: Step 1: Obtain the two-dimensional target position using the camera carried by the robot, and fuse the two-dimensional target position with the laser point cloud obtained by the LiDAR to obtain the three-dimensional target position; Step 2: Predict the target's three-dimensional and two-dimensional predicted positions based on the target's historical trajectory data; Step 3: Calculate the similarity between the 3D predicted position, the 2D predicted position and the 3D target position, and the 2D target position respectively to obtain the 3D similarity and 2D similarity. Calculate the final similarity based on the 3D similarity and 2D similarity. Step 4: Based on the final similarity, use the Hungarian algorithm to optimize the matching to obtain the best matching relationship between the target's 3D predicted position and the 3D target position. Update the target trajectory based on the best matching relationship to complete target tracking.

[0006] Further, step 1 includes: Step 101: Detect the target position in the image captured by the camera device carried by the robot and obtain the two-dimensional target position; Step 102: Map the laser point cloud detected by the lidar carried by the robot to the image captured by the camera carried by the robot, and obtain the correspondence between the two. Step 103: Segment the laser point cloud according to the correspondence and obtain all laser point clouds corresponding to the two-dimensional target position; Step 104: Cluster the obtained laser point clouds and obtain the cluster with the most laser point clouds as the target point cloud data corresponding to the two-dimensional target location; Step 105: Calculate the three-dimensional target position based on the target point cloud data.

[0007] Furthermore, in step 4, the three-dimensional target position and the two-dimensional target position are first integrated to obtain integrated target information. Then, the integrated target information is divided into high-level target information, medium-level target information and low-level target information according to the confidence level of the three-dimensional target position and the point cloud features. Finally, the high-level target information, medium-level target information and low-level target information are matched one by one in sequence.

[0008] Furthermore, the partitioning method in step 4 is as follows: In the formula, , and These are high-level target information, mid-level target information, and low-level target information, respectively. The right side of the equation represents the information contained in these three types of target information. Confidence level for the location of a two-dimensional target based on an image; The confidence threshold; This serves as a criterion for determining whether three-dimensional information is missing. This indicates that the target lacks three-dimensional information. This indicates that the target's three-dimensional information is complete; Let be the coordinates of the center of the three-dimensional target position, represented by a cylinder, in the lidar coordinate system. Let be the radius of the cylinder. The height of the cylinder; These represent the minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, and maximum y-coordinate of the two-dimensional target position represented by a rectangle in the camera coordinate system. The category of the target.

[0009] Furthermore, in step 3, the formula for calculating the similarity between the predicted 3D location and the target 3D location is as follows: In the formula, ; in, The three-dimensional similarity between the predicted three-dimensional location and the target three-dimensional location is given. The Mahalanobis distance between the predicted 3D position and the target 3D position; The error covariance is the difference between the two data points. The center point of the three-dimensional target position. The center point of the three-dimensional predicted location.

[0010] Furthermore, in step 3, the formula for calculating the similarity between the two-dimensional predicted location and the two-dimensional target location is as follows: In the formula, The two-dimensional similarity between the predicted two-dimensional location and the target two-dimensional location. This refers to the position and shape of the detection box corresponding to the two predicted positions. This refers to the position and shape of the detection box corresponding to the two-dimensional target location.

[0011] Furthermore, the formula for calculating the final similarity in step 3 based on the three-dimensional similarity and two-dimensional similarity is as follows: In the formula, These are the weighting coefficients.

[0012] The beneficial effects of this invention are: This invention uses ByteTrack for 3D target detection, which solves the problem of fragmented tracking trajectories for occluded targets. It introduces three-dimensional target positions into data association to improve the target matching success rate. This invention uses a third matching process to specifically handle low-level target information lacking 3D information, thus avoiding the inability to track distant targets due to clustering failures. This invention improves target detection by simplifying the cuboid into a cylinder. The parameters that need to be calculated are reduced from the original length, width, height, and yaw angle of the cuboid to the radius and height of the cylinder, thus reducing data dimensionality and computational complexity. In contrast, conventional 3D target detection outputs a cuboid containing the position, length, width, height, and yaw angle of the cuboid. In practical applications, accurately calculating the cuboid and yaw angle typically requires a large amount of computation. In the data association process, this invention calculates similarity using both two-dimensional and three-dimensional target positions. The final similarity matrix used in the matching process is a weighted sum of the two-dimensional and three-dimensional similarities. This invention improves tracking accuracy by introducing three-dimensional target positions. The ByteTrack detection algorithm divides targets into two categories based on their confidence level and performs two matching operations to improve the tracking effect of occluded objects. However, the ByteTrack detection algorithm only considers the detection results of two-dimensional images, so the similarity matrix in its data association process is a 2D similarity matrix. Attached Figure Description

[0013] Figure 1 For image-based two-dimensional target location; Figure 2 A diagram showing the effect of representing the three-dimensional target position as a cylinder.

[0014] Figure 3 A visualization of 3D target tracking; Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0016] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0017] This invention discloses a three-dimensional target tracking method based on three-dimensional matching from the robot's forward-looking perspective, which includes four steps. Step 1: Obtain the two-dimensional target position based on the imaging device carried by the robot, and fuse the two-dimensional target position with the laser point cloud obtained by the lidar to obtain the three-dimensional target position.

[0018] Before step 1, the sensors are first calibrated, that is, the intrinsic and extrinsic parameters of the sensors carried by the robot are calibrated and the time is synchronized, so that the measurement information of different sensors for the same target that are not synchronized can be synchronized to the same measurement coordinate system.

[0019] The time synchronization process involves acquiring data from the sensors and performing time synchronization and coordinate transformation. Based on the timestamp of each frame of LiDAR data, the camera data with the closest timestamp is searched as the sensor data at the same moment.

[0020] The camera intrinsic parameter calibration process is as follows: (1) Based on the imaging principle of a monocular camera, establish the transformation relationship between the camera coordinate system with the camera as the origin and the image pixel coordinate system: In the formula, These are the coordinates in the image pixel coordinate system. These are the coordinates of the point in the camera coordinate system.

[0021] (2) Place the calibration chessboard in front of the camera and obtain the coordinates of multiple points on the chessboard in the camera coordinate system. And the coordinates of these points in the pixel coordinate system .

[0022] (3) Based on the acquired data and the transformation formula between the camera coordinate system and the image pixel coordinate system, solve for the camera's intrinsic parameter matrix. Complete the camera's internal parameter calibration.

[0023] The steps for joint calibration of camera and lidar extrinsic parameters are as follows: (1) Establish the transformation matrix between the lidar coordinate system with the lidar as the origin and the camera coordinate system: In the formula, The coordinates are in the lidar coordinate system. This represents the transformation relationship between the camera coordinate system and the lidar coordinate system. It is a 3×3 rotation matrix. It is a 3×1 translation vector.

[0024] (2) Place the calibration plate in front of the camera and obtain the coordinates of the points on the calibration plate in the lidar coordinate system and the coordinates in the camera coordinate system.

[0025] (3) Using the autoware_camera_lidar_calibrator toolbox in Autoware, combined with the calibration board and camera intrinsic parameters, calculate the extrinsic parameter matrix of the lidar and the camera. .

[0026] Step 101: Detect the target location in the image captured by the camera device carried by the robot, and obtain the two-dimensional target location. Deep learning methods are used to detect targets in images captured by a robot's onboard camera and output the two-dimensional target locations. The detected targets include dynamic objects such as pedestrians and vehicles that can interfere with the autonomous navigation of the mobile robot. The two-dimensional target position is represented by a rectangular bounding box, and the output parameters are... .in, For the detection box position and size information, The confidence level of this object detection, The category of the target.

[0027] The steps of deep learning methods include: Training dataset construction: A dataset containing the features to be detected is constructed for the YOLOX model to be trained. In one embodiment, the training dataset used is the KITTI dataset, which contains image data of pedestrians and vehicles. The targets to be detected are labeled in the dataset.

[0028] Deep neural network model training: The neural network model is tested multiple times using a dataset until it achieves high detection accuracy for pedestrians and vehicles.

[0029] Image object detection: The generated model is used to detect objects in each frame of the image in real time, and pedestrians and vehicles are selected. In one embodiment, the result of a certain detection is as follows: Figure 1 As shown.

[0030] Step 102: Map the laser point cloud detected by the lidar on the robot to the image captured by the camera on the robot, and obtain the correspondence between the two. Specific steps include: The camera intrinsic parameter matrix obtained during the sensor intrinsic and extrinsic parameter calibration process. Sensor extrinsic matrix The transformation relationships between the pixel coordinate system and the camera coordinate system, as well as between the camera coordinate system and the LiDAR coordinate system, are used to obtain the transformation relationship between the coordinates of a point in the LiDAR coordinate system and the pixel coordinate system. in, The coordinates of the target in the lidar coordinate system. This represents the target's coordinates in the pixel coordinate system. The point cloud data from the LiDAR and image data from the camera are acquired. Based on the correspondence between the LiDAR coordinate system and the pixel coordinate system, the 3D LiDAR point cloud data is projected onto the 2D image, ensuring that the points in the point cloud correspond to the pixels in the image.

[0031] Step 103: Segment the laser point cloud according to the correspondence relationship and obtain all laser point clouds corresponding to the two-dimensional target position. Based on the position and size information of the detection frame Based on the correspondence between image pixel coordinates and laser point coordinates, the point cloud is segmented, and the point cloud within the detection box is extracted, which is the point cloud of the target area, represented as... Delete the point cloud regions that do not contain the target.

[0032] Based on the target detection results using the YOLOX algorithm, and combined with the established relative relationship between pixel coordinates and point cloud data, point cloud data within the detection bounding boxes is obtained. Each detection bounding box can be represented by pixel coordinates. and Description, in which This indicates the number of detection boxes in a frame of an image. , , and They represent the first The detection box is in the pixel coordinate system shaft and The maximum and minimum pixels on the axis, if For points within the detection bounding box, the following relationship must be satisfied: Step 104: Cluster the point clouds in the detection boxes and extract the class with the most points in each detection box as the target point cloud feature. Specific steps include: Clustering segmentation: Based on the feature information of the point cloud, the points within each bounding box are segmented. They will be divided into different clusters Each cluster represents an independent object. The specific process can be represented as follows: in, Indicates the first The original point cloud within each detection box Represents the first clustering operation. The first detection box within the first detection box Clustered cloud, This represents a clustering operation.

[0033] Combined with the obtained first The point cloud set corresponding to each detection box Using this as the input point cloud for Euclidean clustering, a KD-Tree data structure is created, and a cluster set is defined. With a queue ,queue It is mainly used to store the points to be detected. For each point Perform the following operations: 1) Place the point Add to current queue 2) Select each point According to KD-Tree at point of The neighborhood is searched for nearest neighbors to obtain the nearest points, and the distances between these points are calculated. European distance Add points that are less than the distance threshold and are not marked to the queue. In the middle, and mark it; 3) When the queue If all points are marked, then... As a category 4) For all After performing the above processing, the search within the j-th detection box ends, and the final clustering result of the j-th detection box is obtained. Add all categories to the cluster set. middle.

[0034] Target point cloud extraction: The cluster with the most point clouds within each bounding box is selected as the point cloud corresponding to the target detected by that bounding box, and finally, the point clouds corresponding to all targets are obtained. This process can be specifically represented as: in, Indicates the number of statistics The first detection madness The number of points in a point cloud This indicates taking the maximum value.

[0035] Step 105: Calculate the 3D target position based on the target point cloud data. Based on the obtained target point cloud data, the detected 3D target position is represented by a cylindrical bounding box, specifically as follows: .in Indicates the center position of the target. The cylinder's radius and height represent the target's shape. After clustering and point cloud processing for each detection box, the target point cloud is wrapped with a cylinder, and the detection result is output. The effect of a single 3D target detection is shown below. Figure 2 As shown.

[0036] Step 2: Predict the target's three-dimensional and two-dimensional predicted positions based on the target's historical trajectory data. That is, estimate the target's three-dimensional and two-dimensional predicted positions at the current moment based on the target's historical position in the trajectory and the target's motion state.

[0037] A target motion model is established based on the target's historical trajectory data. A state transition matrix is ​​then constructed to estimate the target's state in the next frame. The prediction process can be expressed by the following formula: In the formula, This refers to the three-dimensional and two-dimensional predicted positions of the target at time t+1, based on historical trajectory data. and Let A represent the true states of the target at time t and time t+1. Let A be the state transition matrix of the target. Let be the system noise at time t.

[0038] In one possible implementation, the target's acceleration and angular acceleration are estimated based on the target's historical trajectory data, and then a state transition matrix is ​​constructed for trajectory prediction.

[0039] Step 3: Calculate the similarity between the 3D predicted position, the 2D predicted position, and the 3D target position and the 2D target position, respectively, to obtain the 3D similarity and 2D similarity. Calculate the final similarity score based on the 3D and 2D similarity scores. Data association, or similarity calculation, is used to calculate the correspondence between the 3D and 2D predicted positions and the 3D and 2D target positions. Its purpose is to assign the 3D and 2D target positions to the 3D and 2D predicted positions and establish a correspondence between them.

[0040] First, a similarity matrix is ​​obtained through similarity calculation to provide a basis for matching decisions; then, the optimal matching is achieved through the Hungarian matching algorithm, the specific steps of which include: After completing the state prediction of the target trajectory for the next frame, a similarity matrix is ​​established between the three-dimensional predicted position and the two-dimensional predicted position and the three-dimensional target position and the two-dimensional target position to quantify the similarity between them.

[0041] For the two-dimensional similarity calculation process, the intersection-over-union (IOU) ratio is used to calculate the similarity. Specifically, the two-dimensional predicted location of the target is predicted based on the historical trajectory. Then, the intersection-union ratio (IUU) of each two-dimensional predicted position and each two-dimensional target position is calculated to obtain the two-dimensional similarity.

[0042] The formula for calculating two-dimensional similarity is: In the formula, The two-dimensional similarity between the predicted two-dimensional location and the target two-dimensional location. This refers to the position and shape of the detection box corresponding to the two predicted positions. This refers to the position and shape of the detection box corresponding to the two-dimensional target location.

[0043] For the 3D similarity calculation process, the similarity between the predicted 3D position and the target 3D position is calculated based on the Mahalanobis distance. Specifically, the Mahalanobis distance between the center position of the cylinder at the current 3D predicted position and the center position of the cylinder at the current 3D target position is calculated. This can be expressed by the formula: In the formula, The Mahalanobis distance between the predicted 3D position and the target 3D position; The error covariance is the difference between the two data points. The center point of the three-dimensional target position. The center point of the three-dimensional predicted location.

[0044] Then, based on the Mahalanobis distance, the three-dimensional similarity is calculated using the following formula: in, This represents the 3D similarity between the predicted 3D location and the target 3D location.

[0045] The final similarity can be calculated based on the two-dimensional and three-dimensional similarity of the same target: In the formula, These are the weighting coefficients.

[0046] The smaller the final similarity value, the closer the predicted 3D location is to the target 3D location. After calculating the similarity between all m trajectories and n targets, the final similarity matrix can be obtained. The final similarity matrix contains the similarity between each target and each trajectory.

[0047] Step 4: Based on the final similarity, use the Hungarian algorithm to optimize the matching to obtain the best matching relationship between the predicted position of the target and the 3D target position. Update the target trajectory according to the best matching relationship to complete target tracking. In step 4, the three-dimensional target position and the two-dimensional target position are first integrated to obtain integrated target information. Then, the integrated target information is divided into high-level target information, medium-level target information and low-level target information according to the confidence level of the three-dimensional target position and the point cloud features. Finally, the high-level target information, medium-level target information and low-level target information are matched one by one in sequence.

[0048] The three-dimensional and two-dimensional target positions are integrated and uniformly represented as... Regarding the test results Under normal circumstances It will include the two-dimensional target location. However, the insufficient number of point clouds may prevent clustering, leading to issues with the location of the 3D target. Missing.

[0049] Classification: The integrated target information is divided into three categories. The first category is integrated target information with good detection performance, that is, information containing 3D information and with high confidence. The first category consists of high-level target information; the second category consists of targets that may be occluded, i.e., mid-level target information containing 3D information but with low confidence; the third category consists of targets with fewer point cloud features, i.e., low-level target information that does not contain 3D information.

[0050] Specifically, it can be expressed as: In the formula, , , These are high-level, mid-level, and low-level target information, respectively. The right side of the equation represents the information contained in these three types of target information. Confidence level for the location of a two-dimensional target based on an image; The confidence threshold; This serves as a criterion for determining whether three-dimensional information is missing. This indicates that the target lacks three-dimensional information. This indicates that the target's three-dimensional information is complete; Let be the coordinates of the center of the three-dimensional target position, represented by a cylinder, in the lidar coordinate system. Let be the radius of the cylinder. The height of the cylinder; These represent the minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, and maximum y-coordinate of the two-dimensional target position represented by a rectangle in the camera coordinate system. The category of the target.

[0051] Group matching, matching refers to matching based on the final similarity matrix. By optimizing the matching algorithm, the best matching relationship between the three-dimensional target and the trajectory is found. The best matching relationship is the one with the smallest total similarity after the trajectory and the target are matched. The Hungarian algorithm is used to optimize the matching so that the target and the trajectory that meet the requirements correspond one-to-one, thus completing the data association.

[0052] Specifically, it includes: (1) First matching. The final similarity matrix of the first type of integrated target information is used. The matching optimization was performed using the Hungarian algorithm to complete the data association of the first type of integrated target information.

[0053] (2) Second matching. After the first matching is completed, the unmatched trajectories are matched with the second type of integrated target information for the second matching. The elements in the similarity matrix used in the second matching process are the final similarities corresponding to the second type of integrated target information. That is, the elements in the similarity matrix used in the second matching process are the weighted sum of the 2D and 3D similarities corresponding to the second type of integrated target information, and the weighting method is the same as that of the first matching.

[0054] (3) Third matching. After the second matching, the unmatched trajectories are matched with the third type of integrated target information. The elements in the similarity matrix used in the third matching process are the 3D similarity corresponding to the third type of integrated target information. .

[0055] Based on the data association results, all trajectory states are updated to complete target tracking, such as... Figure 3 As shown. Specifically, it includes: updating the trajectory target status, creating a new trajectory, and deleting an old trajectory.

[0056] Trajectory target state update refers to the maintenance and state update of successfully matched trajectories. Specifically, based on the predicted prior position of the trajectory target and the actual information of the detected target matching the trajectory, Kalman filtering is used to update the posterior state of the trajectory target, thereby completing target tracking.

[0057] The observation process can be expressed by the following formula: In the formula, This represents the observation of the target state at time t+1. For the observation matrix, The observation error is t+1.

[0058] New trajectory establishment refers to creating a new trajectory for newly perceived targets. First, the new target is marked as a high-scoring target of the first category that failed to match a trajectory during the initial data association matching process. Next, a new trajectory is established for this target, and its Kalman filter parameters are initialized. In the next detection and matching process, if the target is detected again and matches a trajectory, it is considered a newly perceived target, and the corresponding trajectory is retained. Conversely, if the trajectory fails to match in the next matching process, it indicates that the newly perceived target was a false detection in the previous process, and the corresponding trajectory is deleted.

[0059] Old trajectory deletion refers to deleting trajectories corresponding to targets that have moved out of the robot's perception range. Specifically, firstly, trajectories that failed to match the detected target in all three matching processes of data association are marked; further, the trajectory is detected N times, and if it fails to match in the next N matching processes, it is considered that the target corresponding to the trajectory has moved out of the robot's perception range, and the trajectory is deleted.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional target tracking method based on cubic matching from the robot's forward-looking perspective, characterized in that, include: Step 1: Obtain the two-dimensional target position using the camera carried by the robot, and fuse the two-dimensional target position with the laser point cloud obtained by the LiDAR to obtain the three-dimensional target position; Step 2: Predict the target's three-dimensional and two-dimensional predicted positions based on the target's historical trajectory data; Step 3: Calculate the similarity between the 3D predicted position, the 2D predicted position and the 3D target position, and the 2D target position respectively to obtain the 3D similarity and 2D similarity. Calculate the final similarity based on the 3D similarity and 2D similarity. Step 4: Based on the final similarity, use the Hungarian algorithm to optimize the matching to obtain the best matching relationship between the target's 3D predicted position and the 3D target position. Update the target trajectory based on the best matching relationship to complete target tracking. In step 4, the three-dimensional target position and the two-dimensional target position are first integrated to obtain integrated target information. Then, the integrated target information is divided into high-level target information, medium-level target information and low-level target information according to the confidence level of the three-dimensional target position and the point cloud features. Finally, the high-level target information, medium-level target information and low-level target information are matched one by one in sequence. The partitioning method in step 4 is as follows: ； In the formula, , and These are high-level target information, mid-level target information, and low-level target information, respectively. The right side of the equation represents the information contained in these three types of target information. Confidence level for the location of a two-dimensional target based on an image; The confidence threshold; This serves as a criterion for determining whether 3D information is missing. This indicates that the target lacks three-dimensional information. This indicates that the target's three-dimensional information is complete; Let be the coordinates of the center of the three-dimensional target position, represented by a cylinder, in the lidar coordinate system. Let be the radius of the cylinder. The height of the cylinder; These represent the minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, and maximum y-coordinate of the two-dimensional target position represented by a rectangle in the camera coordinate system. The category of the target.

2. The three-dimensional target tracking method based on cubic matching from the robot's forward-looking perspective according to claim 1, characterized in that, Step 1 includes: Step 101: Detect the target position in the image captured by the camera device carried by the robot and obtain the two-dimensional target position; Step 102: Map the laser point cloud detected by the lidar carried by the robot to the image captured by the camera carried by the robot, and obtain the correspondence between the two. Step 103: Segment the laser point cloud according to the correspondence and obtain all laser point clouds corresponding to the two-dimensional target position; Step 104: Cluster the obtained laser point clouds and obtain the cluster with the most laser point clouds as the target point cloud data corresponding to the two-dimensional target location; Step 105: Calculate the three-dimensional target position based on the target point cloud data.

3. The three-dimensional target tracking method based on cubic matching from the robot's forward-looking perspective according to claim 2, characterized in that, In step 3, the formula for calculating the similarity between the predicted 3D location and the target 3D location is as follows: ； In the formula, ； in, The three-dimensional similarity between the predicted three-dimensional location and the target three-dimensional location is given. The Mahalanobis distance between the predicted 3D position and the target 3D position; The error covariance is the difference between the two data points. The center point of the three-dimensional target position. The center point of the three-dimensional predicted location.

4. The three-dimensional target tracking method based on cubic matching from the robot's forward-looking perspective according to claim 3, characterized in that, In step 3, the formula for calculating the similarity between the two-dimensional predicted location and the two-dimensional target location is as follows: ； In the formula, The two-dimensional similarity between the predicted two-dimensional location and the target two-dimensional location. This refers to the position and shape of the detection box corresponding to the two predicted positions. This refers to the position and shape of the detection box corresponding to the two-dimensional target location.

5. A three-dimensional target tracking method based on cubic matching from the robot's forward-looking perspective, as described in claim 4, is characterized in that... The formula for calculating the final similarity in step 3 based on three-dimensional and two-dimensional similarity is as follows: ； In the formula, These are the weighting coefficients.

Images