Multi-target tracking method, device and equipment

Abstract

The invention discloses a multi-target tracking method, device and equipment. The method comprises the following steps: acquiring m pieces of obstacle information and n predicted tracking results of a plurality of tracking objects in a t-th frame; obtaining the number of first point clouds in each piece of obstacle information and the number of second point clouds in each prediction tracking result; based on the m first point cloud number and the n second point cloud number, determining the similarity between each piece of obstacle information and each prediction tracking result, and obtaining m * n similarities; associating the obstacle information of which the similarity is greater than or equal to a preset similarity threshold and the predicted tracking result into the same tracking object; and outputting the tracking information of the same tracking object in the t-th frame by combining the obstacle information of the same tracking object in the t-th frame and the predicted tracking result. According to the method and the device, the accuracy of data association matching can be improved when inter-frame data association is performed on the obstacle information and the prediction tracking result.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving technology, and in particular to a multi-target tracking method, device and equipment. Background Art

[0002] In order to realize autonomous driving functions in urban and closed scenarios, autonomous vehicles need to accurately perceive obstacles. The advantage of lidar lies in its precise ranging accuracy and strong environmental robustness, so it is usually used as one of the main sensors of autonomous vehicles.

[0003] In the related technology, in the multi-target tracking solution, the original point cloud information scanned by the laser radar is used as the input of the target detection network to detect the original point cloud, and the obstacle information of interest is output, and the output form is usually a three-dimensional directed bounding box. On this basis, each frame of obstacle information of the target detection network is received, and it is associated with the predicted tracking result of the obstacle between frames to achieve the consistency of the ID output of the same obstacle during its life cycle. Based on this, how to improve the accuracy of data association matching when performing inter-frame data association between obstacle information and predicted tracking results is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a multi-target tracking method and device thereof, which can improve the accuracy of data association matching when performing inter-frame data association between obstacle information and predicted tracking results.

[0005] In a first aspect, an embodiment of the present application provides a multi-target tracking method, the method comprising: obtaining m obstacle information and n predicted tracking results of multiple tracking objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information of the t-1th frame; obtaining the number of first point clouds in each obstacle information, and the number of second point clouds in each predicted tracking result; based on the m first point cloud numbers and the n second point cloud numbers, determining the similarity between each obstacle information and each predicted tracking result, and obtaining m*n similarities; associating the obstacle information and the predicted tracking results whose similarity is greater than or equal to a preset similarity threshold as the same tracking object; combining the obstacle information and the predicted tracking result of the same tracking object in the tth frame, outputting the tracking information of the same tracking object in the tth frame.

[0006] In some implementable embodiments of the first aspect, based on the m first point cloud numbers and the n second point cloud numbers, the similarity between each obstacle information and each predicted tracking result is determined to obtain m*n similarities, including: for each obstacle information, the first point cloud number in the obstacle information is subtracted from the n second point cloud numbers to obtain n point cloud number differences; the ratio of the n point cloud number differences to the second point cloud number is calculated to obtain the point cloud number change rate of each obstacle information and each predicted tracking result; based on the point cloud number change rate, the similarity between each obstacle information and each predicted tracking result is determined, wherein the similarity is negatively correlated with the point cloud number change rate.

[0007] In some implementations of the first aspect, based on the rate of change of the number of point clouds, the similarity between each obstacle information and each predicted tracking result is determined, including: subtracting a first preset value from the rate of change of the number of point clouds to obtain a first difference; based on the first difference, determining the similarity between each obstacle information and each predicted tracking result, wherein the similarity is positively correlated with the first difference.

[0008] In some implementations of the first aspect, based on the rate of change of the number of point clouds, the similarity between each obstacle information and each predicted tracking result is determined, including: calculating the intersection-and-union ratio of bounding boxes based on the bounding box parameters in each obstacle information and each predicted tracking result; combining the intersection-and-union ratio of bounding boxes and the rate of change of the number of point clouds to calculate the similarity between each obstacle information and each predicted tracking result; wherein the obstacle information includes the bounding box parameters of the detection bounding box, the predicted tracking result includes the bounding box parameters of the prediction bounding box, the intersection-and-union ratio of bounding boxes is the ratio of a first area to a second area, the first area is the overlapping area of ​​the detection bounding box and the prediction bounding box, the second area is the difference between the sum of the areas of the detection bounding box and the prediction bounding box and the overlapping area, and the similarity is positively correlated with the intersection-and-union ratio of bounding boxes.

[0009] In some implementations of the first aspect, the similarity between each obstacle information and each predicted tracking result is calculated in combination with the bounding box intersection-and-union ratio and the rate of change of the number of point clouds, including: based on each obstacle information and the bounding box parameters in each predicted tracking result, obtaining the Euclidean distance between the center points of the detection bounding box and the predicted bounding box; when the theoretical detection distance of the laser radar is obtained, calculating the ratio of the Euclidean distance to the theoretical detection distance to obtain the target distance; combining the bounding box intersection-and-union ratio, the target distance and the rate of change of the number of point clouds, calculating the similarity between each obstacle information and each predicted tracking result; wherein the similarity is negatively correlated with the target distance.

[0010] In some implementation methods of the first aspect, the similarity between each obstacle information and each predicted tracking result is calculated in combination with the bounding box intersection-and-union ratio, the target distance and the rate of change of the number of point clouds, including: subtracting a first preset value from the rate of change of the number of point clouds to obtain a first difference, and subtracting a second preset value from the target distance to obtain a second difference; multiplying the bounding box intersection-and-union ratio, the first difference, and the second difference by corresponding preset weights respectively to obtain three products; and calculating the sum of the three products to obtain the similarity between each obstacle information and each predicted tracking result.

[0011] In some implementations of the first aspect, after determining the similarity between each obstacle information and each predicted tracking result to obtain m*n similarities, the method further includes: when the similarity between the obstacle information in the tth frame and the n predicted tracking results is less than a preset similarity threshold, using the obstacle information in the tth frame as the first frame on the tracking trajectory of the tracking object.

[0012] In some implementations of the first aspect, after determining the similarity between each obstacle information and each predicted tracking result to obtain m*n similarities, the method further includes: when the similarities between the predicted tracking result of the first tracking object in the t frame and the m obstacle information are all less than a preset similarity threshold, obtaining a predicted tracking result of the t+1th frame obtained by predicting the predicted tracking result of the t frame; determining the similarity between the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame; and when the similarity is greater than or equal to the preset similarity threshold, comparing the predicted tracking result of the t+1th frame with the obstacle information of the t+1th frame. The obstacle information of the t+1th frame is all associated with the first tracking object, and the obstacle information and the predicted tracking result of the first tracking object in the t+1th frame are combined to output the tracking information of the first tracking object in the t+1th frame; when the similarities are all less than the preset similarity threshold, the predicted tracking result of the t+2th frame obtained by predicting the predicted tracking result of the t+1th frame is obtained, and the similarity between the predicted tracking result of the t+2th frame and the obstacle information of the t+2th frame is determined until the similarity is greater than or equal to the preset similarity threshold, or the predicted tracking result of the first tracking object in the t+Nth frame is obtained, where N is a preset value.

[0013] In a second aspect, an embodiment of the present application provides a multi-target tracking device, which includes: an acquisition module, used to acquire m obstacle information and n predicted tracking results of multiple tracking objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information of the t-1th frame; the acquisition module is also used to acquire the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result; a determination module, used to determine the similarity between each obstacle information and each predicted tracking result based on the m first point cloud numbers and the n second point cloud numbers, and obtain m*n similarities; an association module, used to associate obstacle information and predicted tracking results whose similarity is greater than or equal to a preset similarity threshold as the same tracking object; a tracking output module, used to combine the obstacle information and predicted tracking results of the same tracking object in the tth frame, and output the tracking information of the same tracking object in the tth frame.

[0014] In some implementations of the second aspect, the determination module includes: a calculation submodule, which is used to, for each obstacle information, respectively subtract the first point cloud quantity in the obstacle information from the n second point cloud quantities to obtain n point cloud quantity difference values; respectively calculate the ratio of the n point cloud quantity difference values ​​to the second point cloud quantity to obtain the point cloud quantity change rate of each obstacle information and each predicted tracking result; a determination submodule, which is used to determine the similarity between each obstacle information and each predicted tracking result based on the point cloud quantity change rate, wherein the similarity is negatively correlated with the point cloud quantity change rate.

[0015] In some implementations of the second aspect, the determination submodule includes: a calculation unit, used to subtract a first preset value from the rate of change of the point cloud quantity to obtain a first difference; a determination unit, used to determine the similarity between each obstacle information and each predicted tracking result based on the first difference, wherein the similarity is positively correlated with the first difference.

[0016] In some implementations of the second aspect, the determination unit includes: a calculation subunit, which is used to calculate the intersection-and-union ratio of bounding boxes based on the bounding box parameters in each obstacle information and each predicted tracking result; the calculation subunit is also used to calculate the similarity between each obstacle information and each predicted tracking result in combination with the intersection-and-union ratio of bounding boxes and the rate of change of the number of point clouds; wherein the obstacle information includes the bounding box parameters of the detection bounding box, and the predicted tracking result includes the bounding box parameters of the predicted bounding box, the intersection-and-union ratio of bounding boxes is a ratio of a first area to a second area, the first area is the overlapping area of ​​the detection bounding box and the predicted bounding box, and the second area is the difference between the sum of the areas of the detection bounding box and the predicted bounding box and the overlapping area, and the similarity is positively correlated with the intersection-and-union ratio of bounding boxes.

[0017] In some implementation methods of the second aspect, the computing subunit is specifically used to: obtain the Euclidean distance between the center points of the detection bounding box and the prediction bounding box based on the bounding box parameters in each obstacle information and each predicted tracking result; when the theoretical detection distance of the lidar is obtained, calculate the ratio of the Euclidean distance to the theoretical detection distance to obtain the target distance; combine the bounding box intersection ratio, target distance and point cloud quantity change rate to calculate the similarity between each obstacle information and each predicted tracking result; wherein the similarity is negatively correlated with the target distance.

[0018] In some implementations of the second aspect, the computing subunit is specifically used to: subtract a first preset value from the rate of change of the number of point clouds to obtain a first difference, and subtract a second preset value from the target distance to obtain a second difference; multiply the bounding box intersection-union ratio, the first difference, and the second difference by the corresponding preset weights, respectively, to obtain three products; and calculate the sum of the three products to obtain the similarity between each obstacle information and each predicted tracking result.

[0019] In some implementation methods of the second aspect, the device also includes: a determination module, which is also used to determine the similarity between each obstacle information and each predicted tracking result and obtain m*n similarities. When the similarity between the obstacle information of the tth frame and the n predicted tracking results is less than a preset similarity threshold, use the obstacle information of the tth frame as the first frame on the tracking trajectory of the tracking object.

[0020] In some implementations of the second aspect, the device further includes: an acquisition module, configured to, after determining the similarity between each obstacle information and each predicted tracking result and obtaining m*n similarities, obtain a predicted tracking result of the t+1th frame obtained by predicting the predicted tracking result of the tth frame when the similarity between the predicted tracking result of the first tracking object in the tth frame and the m obstacle information is less than a preset similarity threshold; a determination module, configured to determine the similarity between the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame; and an association module, configured to associate the predicted tracking result of the t+1th frame with the obstacle information of the t+1th frame when the similarity is greater than or equal to the preset similarity threshold. The measured tracking result and the obstacle information of the t+1th frame are associated as the first tracking object, and the obstacle information and the predicted tracking result of the first tracking object in the t+1th frame are combined to output the tracking information of the first tracking object in the t+1th frame; the determination module is further used to obtain the predicted tracking result of the t+2th frame obtained by predicting the predicted tracking result of the t+1th frame when the similarities are all less than a preset similarity threshold, and determine the similarity between the predicted tracking result of the t+2th frame and the obstacle information of the t+2th frame until the similarity is greater than or equal to the preset similarity threshold, or obtain the predicted tracking result of the first tracking object in the t+Nth frame, where N is a preset value.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the multi-target tracking method of the first aspect are implemented.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the steps of the multi-target tracking method of the first aspect are implemented.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a non-volatile storage medium and is executed by at least one processor to implement the steps of the multi-target tracking method of the first aspect.

[0024] In a sixth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the multi-target tracking method of the first aspect.

[0025] The present application provides a multi-target tracking method, device and equipment, which obtain m obstacle information and n predicted tracking results of multiple tracking objects in the t-th frame, wherein the n predicted tracking results of the t-th frame are obtained by predicting the n obstacle information of the t-1-th frame; obtain the number of first point clouds in each obstacle information, and the number of second point clouds in each predicted tracking result; based on the m first point cloud numbers and the n second point cloud numbers, determine the similarity between each obstacle information and each predicted tracking result, and obtain m*n similarities. Since the number of point clouds of the same obstacle in the detection process will not change dramatically between the previous and next frames, that is, the difference in the number of point clouds in the obstacle information of the t-1 frame and the t-th frame is small, and the predicted tracking result of the t-th frame is predicted by the obstacle information of the t-1 frame, and the number of point clouds is almost the same, so for the same obstacle, the difference in the number of point clouds in the obstacle information of the t-th frame and the predicted tracking result is small. Based on this, according to the judgment principle, for each obstacle information, the first point cloud quantity in the obstacle information can be subtracted from the n second point cloud quantities to obtain n point cloud quantity differences; the ratio of the n point cloud quantity differences to the second point cloud quantity can be calculated to obtain the similarity between each obstacle information and each predicted tracking result. The lower the degree of change between the first point cloud quantity and the second point cloud quantity, the higher the similarity between the two. In this way, based on the difference in the number of point clouds between the obstacle information of the t-th frame and the predicted tracking result, the similarity calculation in the point cloud feature dimension of the three-dimensional bounding box can be realized, thereby improving the accuracy of the similarity calculation. Furthermore, the obstacle information and the predicted tracking result can be matched and associated based on the similarity, and the obstacle information of the same obstacle can be accurately associated with the predicted tracking result, thereby improving the robustness of the data association method in the tracking process, reducing the mismatch and erroneous matching of the obstacle information and the predicted tracking result, and effectively suppressing the obstacle ID jump problem. By combining the obstacle information and the predicted tracking result of the same tracking object in the t-th frame, the tracking information of the same tracking object in the t-th frame is output, thereby improving the quality stability and performance reliability of the output tracking information, achieving accurate tracking of the obstacle, and maximizing the use of the original point cloud information scanned by the lidar. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application.

[0027] Figure 1 It is a flowchart of a multi-target tracking method provided by an embodiment of the present application;

[0028] Figure 2 is a flowchart of a multi-target tracking method provided by another embodiment of the present application;

[0029] Figure 3 is a flowchart of a multi-target tracking method provided by yet another embodiment of the present application;

[0030] Figure 4 is an exemplary schematic diagram of topology information provided by an embodiment of the present application;

[0031] Figure 5 is a flowchart of a multi-target tracking method provided by yet another embodiment of the present application;

[0032] Figure 6 is a flowchart of a multi-target tracking method provided by yet another embodiment of the present application;

[0033] Figure 7 It is a structural schematic diagram of a multi-target tracking device provided in an embodiment of the present application;

[0034] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0036] In the related technology, in the multi-target tracking solution, the original point cloud information scanned by the laser radar is used as the input of the target detection network to detect the original point cloud and output the obstacle information of interest, which is usually in the form of a three-dimensional directed bounding box. On this basis, each frame of obstacle information from the target detection network is received and the frame data is associated with the predicted tracking result of the obstacle to achieve the same ID outputted by the same obstacle during its life cycle.

[0037] The traditional inter-frame data association scheme only considers the geometric information of the three-dimensional directional bounding box during data association, ignoring the original point cloud information. However, due to the complexity of the scene within the actual detection range, there are many obstacles and they occlude each other, which leads to the problem of trajectory mismatch during the data association process, resulting in ID changes and affecting the use of the decision-making end. Based on this, how to improve the accuracy of data association matching when performing inter-frame data association between obstacle information and predicted tracking results is a technical problem that needs to be solved urgently.

[0038] In order to solve the above technical problems, m obstacle information and n predicted tracking results of multiple tracking objects in the tth frame are obtained, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information of the t-1th frame; the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result are obtained; based on the m first point cloud numbers and the n second point cloud numbers, the similarity between each obstacle information and each predicted tracking result is determined to obtain m*n similarities. Since the number of point clouds of the previous and next frames of the same obstacle will not change dramatically during the detection process, that is, the difference in the number of point clouds in the obstacle information of the t-1th frame and the tth frame is small, and the predicted tracking result of the tth frame is obtained by predicting the obstacle information of the t-1th frame, and the number of point clouds is almost the same, so for the same obstacle, the difference in the number of point clouds in the obstacle information of the tth frame and the predicted tracking result is small. Based on this, according to the judgment principle, for each obstacle information, the first point cloud quantity in the obstacle information can be subtracted from the n second point cloud quantities to obtain n point cloud quantity differences; the ratio of the n point cloud quantity differences to the second point cloud quantity can be calculated to obtain the similarity between each obstacle information and each predicted tracking result. The lower the degree of change between the first point cloud quantity and the second point cloud quantity, the higher the similarity between the two. In this way, based on the difference in the number of point clouds between the obstacle information of the t-th frame and the predicted tracking result, the similarity calculation in the point cloud feature dimension of the three-dimensional bounding box can be realized, thereby improving the accuracy of the similarity calculation. Furthermore, the obstacle information and the predicted tracking result can be matched and associated based on the similarity, and the obstacle information of the same obstacle can be accurately associated with the predicted tracking result, thereby improving the robustness of the data association method in the tracking process, reducing the mismatch and erroneous matching of the obstacle information and the predicted tracking result, and effectively suppressing the obstacle ID jump problem. By combining the obstacle information and the predicted tracking result of the same tracking object in the t-th frame, the tracking information of the same tracking object in the t-th frame is output, thereby improving the quality stability and performance reliability of the output tracking information, achieving accurate tracking of the obstacle, and maximizing the use of the original point cloud information scanned by the lidar.

[0039] The multi-target tracking method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0040] Figure 1 It is a flow chart of a multi-target tracking method provided by an embodiment of the present application. The executor of the multi-target tracking method may be a client or a cloud server connected to the client in communication, and the client may be a vehicle.

[0041] The following uses a vehicle as an example to illustrate the multi-target tracking method of the present application. It should be noted that the above-mentioned execution subjects and application scenarios do not constitute a limitation on the present application.

[0042] like Figure 1 As shown, the multi-target tracking method provided in the embodiment of the present application may include steps 110 to 150.

[0043] Step 110, obtaining m obstacle information and n predicted tracking results of multiple tracked objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information of the t-1th frame;

[0044] Step 120, obtaining the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result;

[0045] Step 130, based on the m numbers of first point clouds and the n numbers of second point clouds, determine the similarity between each obstacle information and each predicted tracking result to obtain m*n similarities;

[0046] Step 140, associating obstacle information and predicted tracking results with similarity greater than or equal to a preset similarity threshold as the same tracking object;

[0047] Step 150: combining obstacle information of the same tracked object in the tth frame with the predicted tracking result, outputting tracking information of the same tracked object in the tth frame.

[0048] The multi-target tracking method provided in the embodiment of the present application obtains m obstacle information and n predicted tracking results of multiple tracking objects in the t-th frame, wherein the n predicted tracking results of the t-th frame are obtained by predicting the n obstacle information of the t-1-th frame; obtains the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result; based on the m first point cloud numbers and the n second point cloud numbers, determines the similarity between each obstacle information and each predicted tracking result, and obtains m*n similarities. Since the number of point clouds of the same obstacle in the previous and next frames will not change dramatically during the detection process, that is, the difference in the number of point clouds in the obstacle information of the t-1 frame and the t-th frame is small, and the predicted tracking result of the t-th frame is obtained by predicting the obstacle information of the t-1 frame, and the number of point clouds is almost the same, so for the same obstacle, the difference in the number of point clouds in the obstacle information of the t-th frame and the predicted tracking result is small. Based on this, according to the judgment principle, for each obstacle information, the first point cloud quantity in the obstacle information can be subtracted from the n second point cloud quantities to obtain n point cloud quantity differences; the ratio of the n point cloud quantity differences to the second point cloud quantity can be calculated to obtain the similarity between each obstacle information and each predicted tracking result. The lower the degree of change between the first point cloud quantity and the second point cloud quantity, the higher the similarity between the two. In this way, based on the difference in the number of point clouds between the obstacle information of the t-th frame and the predicted tracking result, the similarity calculation in the point cloud feature dimension of the three-dimensional bounding box can be realized, thereby improving the accuracy of the similarity calculation. Furthermore, the obstacle information and the predicted tracking result can be matched and associated based on the similarity, and the obstacle information of the same obstacle can be accurately associated with the predicted tracking result, thereby improving the robustness of the data association method in the tracking process, reducing the mismatch and erroneous matching of the obstacle information and the predicted tracking result, and effectively suppressing the obstacle ID jump problem. By combining the obstacle information and the predicted tracking result of the same tracking object in the t-th frame, the tracking information of the same tracking object in the t-th frame is output, thereby improving the quality stability and performance reliability of the output tracking information, achieving accurate tracking of the obstacle, and maximizing the use of the original point cloud information scanned by the lidar.

[0049] The specific implementation of the above steps will be described in detail below in conjunction with specific embodiments.

[0050] Involving step 110, m obstacle information and n predicted tracking results of multiple tracked objects in the tth frame are obtained.

[0051] In step 110, the tracking object is an obstacle, which can be a vehicle, a pedestrian, a road obstacle, etc., and the tth frame is the current frame. A laser radar is installed on the vehicle. In the tth frame, after the laser radar scans the original point cloud information of the tth frame, the original point cloud information is input into the target detection network, so that the target detection network performs obstacle recognition on the original point cloud information of the tth frame, and recognizes the m obstacles (i.e., detection bounding boxes) existing in the tth frame, and thus outputs the m obstacle information of the tth frame, where m is a positive integer.

[0052] Similarly, in the t-1th frame, after the original point cloud information of the t-1th frame is scanned by the lidar, the original point cloud information is input into the target detection network, so that the target detection network performs obstacle recognition on the original point cloud information of the t-1th frame, recognizes the n obstacles existing in the t-1th frame (i.e., the detection bounding box), and thus outputs the n obstacle information of the t-1th frame.

[0053] Based on this, by using the information of n obstacles in the t-1th frame for prediction, the trajectories of the n obstacles in the future tth frame can be tracked and predicted, so that n predicted tracking results of the tth frame can be obtained.

[0054] The target detection network is a deep learning neural network model, which is used to detect the original point cloud information, identify obstacles of interest within the FOV, and obtain obstacle information.

[0055] In some embodiments, a Kalman filter algorithm may be used to track and predict n obstacle information in the t-1th frame to obtain predicted tracking results of multiple tracking objects in the tth frame.

[0056] It should be noted that since the surrounding environment of the vehicle is constantly changing during driving, the obstacles identified from the original point cloud information at different times are also different, so the number of obstacles identified in two adjacent frames can be the same or different. That is, m and n can be the same or different, and both m and n are positive integers.

[0057] In some embodiments of the present application, if t-1=1, all the n obstacle information output by the target detection network can be initialized as n tracks, and the Kalman filter parameters are initialized as the starting frame of the tracking trajectory of the n obstacles, that is, the first frame.

[0058] Involving step 120, the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result are obtained.

[0059] In step 120, the first point cloud number is the number of point clouds in the obstacle information, and the second point cloud number is the number of point clouds in the predicted tracking result. The obstacle information includes bounding box parameters of the detection bounding box, and the bounding box parameters include the number of point clouds in the detection bounding box, which is the first point cloud number; the predicted tracking result includes bounding box parameters of the predicted bounding box, and the bounding box parameters include the number of point clouds in the predicted bounding box, which is the second point cloud number.

[0060] In step 130 , based on the m first point cloud numbers and the n second point cloud numbers, the similarity between each obstacle information and each predicted tracking result is determined to obtain m*n similarities.

[0061] In step 130, for each obstacle information, similarity may be calculated once with each predicted tracking result to obtain n similarities corresponding to the obstacle information. Therefore, for m obstacle information, m*n similarities may be obtained.

[0062] In some embodiments of the present application, Figure 2 is a flow chart of a multi-target tracking method provided by another embodiment of the present application. The above step 130 determines the similarity between each obstacle information and each predicted tracking result based on the number of m first point clouds and the number of n second point clouds, and obtains m*n similarities, which may specifically include Figure 2 Steps 210 to 230 are shown.

[0063] Step 210, for each obstacle information, subtract the first point cloud quantity in the obstacle information from the n second point cloud quantities to obtain n point cloud quantity difference values;

[0064] Step 220, respectively calculating the ratio of the difference in the number of n point clouds to the number of the second point cloud, and obtaining the change rate of the number of point clouds of each obstacle information and each predicted tracking result;

[0065] Step 230 : Determine the similarity between each obstacle information and each predicted tracking result based on the rate of change of the number of point clouds.

[0066] Among them, the similarity is negatively correlated with the rate of change of the number of point clouds.

[0067] In some embodiments of the present application, the ratio of a first preset value to the rate of change of the number of point clouds can be calculated to obtain a first ratio, and the similarity between each obstacle information and each predicted tracking result can be determined based on the first ratio, wherein the similarity is positively correlated with the first ratio.

[0068] In some other embodiments of the present application, the above step 230 determines the similarity between each obstacle information and each predicted tracking result based on the rate of change of the number of point clouds, which may specifically include:

[0069] Subtract the first preset value from the rate of change of the point cloud quantity to obtain a first difference value;

[0070] Based on the first difference, the similarity between each obstacle information and each predicted tracking result is determined.

[0071] Among them, the similarity is positively correlated with the first difference.

[0072] In some embodiments of the present application, the obstacle information and the predicted tracking results may include bounding box parameters. Figure 3 is a flowchart of a multi-target tracking method provided by another embodiment of the present application. The above step 230 determines the similarity between each obstacle information and each predicted tracking result based on the rate of change of the number of point clouds, which may specifically include Figure 3 Steps 310 and 320 are shown.

[0073] Step 310, calculating the bounding box intersection-to-union ratio based on each obstacle information and the bounding box parameters in each predicted tracking result;

[0074] Step 320, combining the bounding box intersection-to-union ratio and the point cloud quantity change rate, calculate the similarity between each obstacle information and each predicted tracking result;

[0075] Among them, the obstacle information includes the bounding box parameters of the detection bounding box, the predicted tracking result includes the bounding box parameters of the predicted bounding box, the bounding box intersection-union ratio is the ratio of the first area to the second area, the first area is the overlapping area of ​​the detection bounding box and the predicted bounding box, the overlapping area is the area of ​​the overlapping area between the detection bounding box and the predicted bounding box, the second area is the difference between the sum of the areas of the detection bounding box and the predicted bounding box and the overlapping area, and the similarity is positively correlated with the bounding box intersection-union ratio.

[0076] For example, Figure 4 As shown, box1 is the detection bounding box, box2 is the predicted bounding box, and lou is the intersection-union ratio of the bounding boxes.

[0077] In some embodiments, the bounding box parameters may include the bounding box position, the length, width and height of the bounding box, and the number of point clouds in the bounding box. The obstacle information may also include the heading angle and obstacle category; the predicted tracking result may also include the obstacle ID and speed.

[0078] Specifically, the area of ​​the overlapped region between the detected bounding box and the predicted bounding box may be determined based on each obstacle information and the length, width, height and position of the bounding box in each predicted tracking result to obtain the overlapped area.

[0079] In some embodiments of the present application, Figure 5is a flowchart of a multi-target tracking method provided by another embodiment of the present application. The above step 320 combines the bounding box intersection-union ratio and the point cloud quantity change rate to calculate the similarity between each obstacle information and each predicted tracking result, which may specifically include Figure 5 Steps 510 - 530 are shown.

[0080] Step 510, based on each obstacle information and the bounding box parameters in each predicted tracking result, obtain the Euclidean distance between the center point of the detected bounding box and the predicted bounding box;

[0081] Step 520, when the theoretical detection distance of the laser radar is obtained, the ratio of the Euclidean distance to the theoretical detection distance is calculated to obtain the target distance;

[0082] Step 530, combining the bounding box intersection-to-union ratio, the target distance and the rate of change of the number of point clouds, calculate the similarity between each obstacle information and each predicted tracking result;

[0083] Specifically, the similarity is negatively correlated with the target distance. Based on each obstacle information and the length, width, height and position of the bounding box in each predicted tracking result, the first position coordinates of the center point of the detection bounding box and the second position coordinates of the center point of the predicted bounding box can be obtained, and then the Euclidean distance between the center points of the detection bounding box and the predicted bounding box can be calculated based on the first position coordinates and the second position coordinates.

[0084] In some embodiments of the present application, the above step 530 combines the bounding box intersection-to-union ratio, the target distance and the point cloud quantity change rate to calculate the similarity between each obstacle information and each predicted tracking result, which may specifically include the following steps:

[0085] Subtracting the first preset value from the point cloud quantity change rate to obtain a first difference, and subtracting the second preset value from the target distance to obtain a second difference;

[0086] Multiply the bounding box intersection-union ratio, the first difference, and the second difference by the corresponding preset weights to obtain three products;

[0087] The sum of the three products is calculated to obtain the similarity between each obstacle information and each predicted tracking result.

[0088] Among them, the first preset value and the second preset value can be set according to specific needs, for example, to 1, 1.5 or other values. The first preset value and the second preset value can be the same or different, and this application does not make any specific limitations on this.

[0089] For example, the similarity can be calculated using formula (1): sum .

[0090]

[0091] Among them, IoU is the intersection over union ratio of bounding boxes; distance is the Euclidean distance between the center points of the detected bounding box and the predicted bounding box, and FOV is the theoretical detection distance of the lidar. is the target distance, is the second difference; track points is the number of the second point cloud, object points is the number of the first point cloud, is the rate of change of point cloud quantity, is the first difference; α, β, and λ are three preset weights respectively, and the sum of the three preset weights is 1. The specific values ​​of the preset weights can be set according to specific needs, and this application does not make specific limitations on this. For example, α, β, and λ are 0.4, 0.2, and 0.4 respectively.

[0092] In an embodiment of the present application, a multi-dimensional data association method is proposed in the tracking process in combination with the characteristics of the lidar sensor, which comprehensively considers the shape characteristics, position characteristics and contained point cloud characteristics of the three-dimensional bounding box, improves the robustness of the data association method in the tracking process, reduces mismatches, erroneous matches, etc., and effectively suppresses the ID jump problem.

[0093] In step 140, the obstacle information and the predicted tracking result having a similarity greater than or equal to a preset similarity threshold are associated as the same tracking object.

[0094] In step 140, the higher the similarity, the greater the possibility that the obstacle information and the predicted tracking result belong to the same obstacle (or the same tracking object), and therefore the easier it is to associate and match. The preset similarity threshold can be set according to specific needs, for example, to 0.5, 0.6 or other values, and this application does not make specific limitations on this.

[0095] In some embodiments, if the similarity between the same obstacle information and at least two predicted tracking results is greater than a preset similarity threshold, it can be associated with the predicted tracking result with the highest similarity as the same tracking object.

[0096] In some embodiments of the present application, after determining the similarity between each obstacle information and each predicted tracking result in step 140 to obtain m*n similarities, the method may further include:

[0097] When the similarities between the obstacle information of the t-th frame and the n predicted tracking results are all less than a preset similarity threshold, the obstacle information of the t-th frame is used as the first frame on the tracking trajectory of the tracking object.

[0098] Specifically, for unmatched detection bounding boxes, each detection bounding box is initialized as a predicted bounding box, that is, a new tracking trajectory is opened.

[0099] In some embodiments of the present application, Figure 6 FIG. 1 is a flow chart of a multi-target tracking method provided by another embodiment of the present application. Figure 6 As shown, after determining the similarity between each obstacle information and each predicted tracking result in the above step 140 and obtaining m*n similarities, the method may also include Figure 6 Steps 610 - 640 are shown.

[0100] Step 610, when the similarity between the predicted tracking result of the first tracking object in the tth frame and the m obstacle information is less than a preset similarity threshold, obtain the predicted tracking result of the t+1th frame obtained by predicting the predicted tracking result of the tth frame;

[0101] Step 620, determining the similarity between the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame;

[0102] Step 630: When the similarity is greater than or equal to the preset similarity threshold, the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame are associated as the first tracking object, and the obstacle information and the predicted tracking result of the first tracking object in the t+1th frame are combined to output the tracking information of the first tracking object in the t+1th frame;

[0103] Step 640, when the similarities are all less than the preset similarity threshold, obtain the predicted tracking result of the t+2th frame obtained by predicting the predicted tracking result of the t+1th frame, and determine the similarity between the predicted tracking result of the t+2th frame and the obstacle information of the t+2th frame, until the similarity is greater than or equal to the preset similarity threshold, or obtain the predicted tracking result of the first tracking object in the t+Nth frame.

[0104] Among them, N is a preset value, which can be set according to specific needs, for example, set to 5, 6 or other values, and this application does not make any specific limitation on this.

[0105] Specifically, for the unmatched predicted tracking result of the tth frame, it indicates that the obstacle corresponding to the predicted tracking result has moved out of the field of view of the lidar. Therefore, we can try to retain N frames and use Kalman's predicted value as the output information of the predicted tracking result within the N frames from t+1 to t+N. If there is still no obstacle information that can match it after 5 frames, the predicted tracking result will be deleted.

[0106] In step 150, obstacle information of the same tracked object in the tth frame and the predicted tracking result are combined to output tracking information of the same tracked object in the tth frame.

[0107] In step 150, the update process of the Kalman filter is combined with the obstacle information of the same tracked object in the tth frame and the predicted tracking result, and the updated result of the Kalman filter is output as the tracking information of the same tracked object in the tth frame.

[0108] It is understandable that the multi-target tracking method provided in the embodiment of the present application can be executed by a multi-target tracking device or a control module in the multi-target tracking device for executing the multi-target tracking method. The multi-target tracking device is described in detail below.

[0109] Figure 7 Schematic diagram of a multi-target tracking device provided in an embodiment of the present application. Figure 7 As shown, the multi-target tracking device 700 may include: an acquisition module 710 , a determination module 720 , an association module 730 and a tracking output module 740 .

[0110] Among them, the acquisition module 710 is used to obtain m obstacle information and n predicted tracking results of multiple tracking objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information in the t-1th frame; the acquisition module 710 is also used to obtain the first point cloud number in each obstacle information and the second point cloud number in each predicted tracking result; the determination module 720 is used to determine the similarity between each obstacle information and each predicted tracking result based on the m first point cloud numbers and the n second point cloud numbers, and obtain m*n similarities; the association module 730 is used to associate the obstacle information and the predicted tracking results whose similarity is greater than or equal to the preset similarity threshold as the same tracking object; the tracking output module 740 is used to combine the obstacle information and the predicted tracking result of the same tracking object in the tth frame, and output the tracking information of the same tracking object in the tth frame.

[0111] The multi-target tracking device provided by the present application obtains m obstacle information and n predicted tracking results of multiple tracking objects in the t-th frame, wherein the n predicted tracking results of the t-th frame are obtained by predicting the n obstacle information of the t-1-th frame; obtains the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result; based on the m first point cloud numbers and the n second point cloud numbers, determines the similarity between each obstacle information and each predicted tracking result, and obtains m*n similarities. Since the number of point clouds in the previous and next frames of the same obstacle will not change dramatically during the detection process, that is, the difference in the number of point clouds in the obstacle information of the t-1 frame and the t-th frame is small, and the predicted tracking result of the t-th frame is obtained by predicting the obstacle information of the t-1 frame, and the number of point clouds is almost the same, so for the same obstacle, the difference in the number of point clouds in the obstacle information of the t-th frame and the predicted tracking result is small. Based on this, according to the judgment principle, for each obstacle information, the first point cloud quantity in the obstacle information can be subtracted from the n second point cloud quantities to obtain n point cloud quantity differences; the ratio of the n point cloud quantity differences to the second point cloud quantity can be calculated to obtain the similarity between each obstacle information and each predicted tracking result. The lower the degree of change between the first point cloud quantity and the second point cloud quantity, the higher the similarity between the two. In this way, based on the difference in the number of point clouds between the obstacle information of the t-th frame and the predicted tracking result, the similarity calculation in the point cloud feature dimension of the three-dimensional bounding box can be realized, thereby improving the accuracy of the similarity calculation. Furthermore, the obstacle information and the predicted tracking result can be matched and associated based on the similarity, and the obstacle information of the same obstacle can be accurately associated with the predicted tracking result, thereby improving the robustness of the data association method in the tracking process, reducing the mismatch and erroneous matching of the obstacle information and the predicted tracking result, and effectively suppressing the obstacle ID jump problem. By combining the obstacle information and the predicted tracking result of the same tracking object in the t-th frame, the tracking information of the same tracking object in the t-th frame is output, thereby improving the quality stability and performance reliability of the output tracking information, achieving accurate tracking of the obstacle, and maximizing the use of the original point cloud information scanned by the lidar.

[0112] In some embodiments of the present application, the determination module 720 includes: a calculation submodule, which is used to, for each obstacle information, respectively subtract the first point cloud quantity in the obstacle information from the n second point cloud quantities to obtain n point cloud quantity difference values; respectively calculate the ratio of the n point cloud quantity difference values ​​to the second point cloud quantity to obtain the point cloud quantity change rate of each obstacle information and each predicted tracking result; a determination submodule, which is used to determine the similarity between each obstacle information and each predicted tracking result based on the point cloud quantity change rate, wherein the similarity is negatively correlated with the point cloud quantity change rate.

[0113] In some embodiments of the present application, the determination submodule includes: a calculation unit, used to subtract a first preset value from the rate of change of the point cloud quantity to obtain a first difference; a determination unit, used to determine the similarity between each obstacle information and each predicted tracking result based on the first difference, wherein the similarity is positively correlated with the first difference.

[0114] In some embodiments of the present application, the determination unit includes: a calculation subunit, which is used to calculate the intersection-and-union ratio of bounding boxes based on the bounding box parameters in each obstacle information and each predicted tracking result; the calculation subunit is also used to calculate the similarity between each obstacle information and each predicted tracking result in combination with the intersection-and-union ratio of bounding boxes and the rate of change of the number of point clouds; wherein the obstacle information includes the bounding box parameters of the detection bounding box, and the predicted tracking result includes the bounding box parameters of the predicted bounding box, the intersection-and-union ratio of bounding boxes is the ratio of a first area to a second area, the first area is the overlapping area of ​​the detection bounding box and the predicted bounding box, and the second area is the difference between the sum of the areas of the detection bounding box and the predicted bounding box and the overlapping area, and the similarity is positively correlated with the intersection-and-union ratio of bounding boxes.

[0115] In some embodiments of the present application, the computing subunit is specifically used to: obtain the Euclidean distance between the center points of the detection bounding box and the predicted bounding box based on the bounding box parameters in each obstacle information and each predicted tracking result; when the theoretical detection distance of the lidar is obtained, calculate the ratio of the Euclidean distance to the theoretical detection distance to obtain the target distance; combine the bounding box intersection ratio, target distance and point cloud quantity change rate to calculate the similarity between each obstacle information and each predicted tracking result; wherein the similarity is negatively correlated with the target distance.

[0116] In some embodiments of the present application, the computing subunit is specifically used to: subtract a first preset value from the rate of change of the point cloud quantity to obtain a first difference, and subtract a second preset value from the target distance to obtain a second difference; multiply the bounding box intersection-union ratio, the first difference, and the second difference by the corresponding preset weights, respectively, to obtain three products; calculate the sum of the three products to obtain the similarity between each obstacle information and each predicted tracking result.

[0117] In some embodiments of the present application, the device also includes: a determination module 720, which is also used to determine the similarity between each obstacle information and each predicted tracking result, and after obtaining m*n similarities, when the similarity between the obstacle information of the tth frame and the n predicted tracking results is less than a preset similarity threshold, use the obstacle information of the tth frame as the first frame on the tracking trajectory of the tracking object.

[0118] In some embodiments of the present application, the device further includes: an acquisition module, which is used to determine the similarity between each obstacle information and each predicted tracking result, and obtain m*n similarities. When the similarity between the predicted tracking result of the first tracking object in the t frame and the m obstacle information is less than a preset similarity threshold, obtain the predicted tracking result of the t+1 frame obtained by predicting the predicted tracking result of the t frame; a determination module 720, which is used to determine the similarity between the predicted tracking result of the t+1 frame and the obstacle information of the t+1 frame; and an association module, which is used to associate the predicted tracking result of the t+1 frame with the obstacle information of the t+1 frame when the similarity is greater than or equal to the preset similarity threshold. The tracking result and the obstacle information of the t+1th frame are both associated as the first tracking object, and the obstacle information and the predicted tracking result of the first tracking object in the t+1th frame are combined to output the tracking information of the first tracking object in the t+1th frame; the determination module 720 is also used to obtain the predicted tracking result of the t+2th frame obtained by predicting the predicted tracking result of the t+1th frame when the similarities are all less than a preset similarity threshold, and determine the similarity between the predicted tracking result of the t+2th frame and the obstacle information of the t+2th frame until the similarity is greater than or equal to the preset similarity threshold, or obtain the predicted tracking result of the first tracking object in the t+Nth frame, where N is a preset value.

[0119] The multi-target tracking device provided in the embodiment of the present application can achieve Figure 1-Figure 6 The various processes implemented by the electronic device in the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0120] Figure 8 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0121] like Figure 8 As shown, the electronic device 800 includes a memory 801 , a processor 802 , and a computer program stored in the memory 801 and executable on the processor 802 .

[0122] In an example, the processor 802 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0123] The memory 801 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it can be operated to perform the operations described with reference to the card opening method in the embodiment according to the first aspect of the present application.

[0124] The processor 802 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 801, so as to implement the card activation method in the embodiment of the first aspect.

[0125] In some examples, the electronic device 800 may further include a communication interface 803 and a bus 804. Figure 8 As shown, the memory 801, the processor 802, and the communication interface 803 are connected via a bus 804 and communicate with each other.

[0126] The communication interface 803 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 803 can also be used to access input devices and / or output devices.

[0127] The bus 804 includes hardware, software, or both, coupling the components of the electronic device 800 to each other. By way of example and not limitation, the bus 804 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 804 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0128] The electronic device provided in the embodiment of the present application can realize Figure 1-Figure 6 The various processes implemented by the electronic device in the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0129] In combination with the multi-target tracking method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the steps of any multi-target tracking method in the above embodiment are implemented.

[0130] In combination with the multi-target tracking method in the above embodiment, the embodiment of the present application can provide a computer program product for implementation. The (computer) program product is stored in a non-volatile storage medium, and when the program product is executed by at least one processor, the steps of any multi-target tracking method in the above embodiment are implemented.

[0131] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned multi-target tracking method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0132] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0133] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0134] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0135] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0136] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0137] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A multi-target tracking method, characterized in that: The method comprises: Obtain m obstacle information and n predicted tracking results of multiple tracking objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information in the t-1th frame; Obtain the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result; Based on the m numbers of the first point clouds and the n numbers of the second point clouds, the similarity between each obstacle information and each predicted tracking result is determined to obtain m*n similarities; Associating the obstacle information and the predicted tracking result whose similarity is greater than or equal to a preset similarity threshold as the same tracking object; The obstacle information of the same tracked object in the tth frame and the predicted tracking result are combined to output the tracking information of the same tracked object in the tth frame.

2. The method according to claim 1, characterized in that Based on the m first point cloud quantities and the n second point cloud quantities, the similarity between each obstacle information and each predicted tracking result is determined to obtain m*n similarities, including: For each obstacle information, subtract the first point cloud quantity in the obstacle information from the n second point cloud quantities to obtain n point cloud quantity difference values; Calculate the ratio of the n point cloud quantity differences to the second point cloud quantity respectively, and obtain the point cloud quantity change rate of each obstacle information and each predicted tracking result; Based on the rate of change of the number of point clouds, the similarity between each obstacle information and each predicted tracking result is determined, wherein the similarity is negatively correlated with the rate of change of the number of point clouds.

3. The method according to claim 2, characterized in that The determining, based on the rate of change of the number of point clouds, the similarity between each obstacle information and each predicted tracking result includes: Subtracting the first preset value from the point cloud quantity change rate to obtain a first difference value; Based on the first difference, the similarity between each obstacle information and each predicted tracking result is determined, wherein the similarity is positively correlated with the first difference.

4. The method according to claim 2, characterized in that: The determining, based on the rate of change of the number of point clouds, the similarity between each obstacle information and each predicted tracking result includes: Calculating the bounding box intersection-over-union ratio based on each obstacle information and the bounding box parameters in each predicted tracking result; Calculate the similarity between each obstacle information and each predicted tracking result by combining the bounding box intersection-to-union ratio and the point cloud quantity change rate; Among them, the obstacle information includes bounding box parameters of the detection bounding box, the predicted tracking result includes bounding box parameters of the predicted bounding box, the bounding box intersection-and-union ratio is the ratio of a first area to a second area, the first area is the overlapping area of ​​the detection bounding box and the predicted bounding box, the second area is the difference between the sum of the areas of the detection bounding box and the predicted bounding box and the overlapping area, and the similarity is positively correlated with the bounding box intersection-and-union ratio.

5. The method according to claim 4, characterized in that The calculating the similarity between each obstacle information and each predicted tracking result by combining the bounding box intersection-to-union ratio and the point cloud quantity change rate includes: Based on each obstacle information and the bounding box parameters in each predicted tracking result, obtaining the Euclidean distance between the center point of the detected bounding box and the predicted bounding box; When the theoretical detection distance of the laser radar is obtained, the ratio of the Euclidean distance to the theoretical detection distance is calculated to obtain the target distance; Calculate the similarity between each obstacle information and each predicted tracking result by combining the bounding box intersection-to-union ratio, the target distance and the rate of change of the point cloud quantity; The similarity is negatively correlated with the target distance.

6. The method according to claim 5, characterized in that The calculating the similarity between each obstacle information and each predicted tracking result by combining the bounding box intersection-and-union ratio, the target distance and the rate of change of the point cloud quantity includes: Subtracting the first preset value from the point cloud quantity change rate to obtain a first difference, and subtracting the second preset value from the target distance to obtain a second difference; Multiplying the bounding box intersection-and-union ratio, the first difference, and the second difference by corresponding preset weights respectively to obtain three products; The sum of the three products is calculated to obtain the similarity between each obstacle information and each predicted tracking result.

7. The method according to claim 1, characterized in that After determining the similarity between each obstacle information and each predicted tracking result to obtain m*n similarities, the method further includes: When the similarities between the obstacle information of the t-th frame and the n predicted tracking results are both less than the preset similarity threshold, the obstacle information of the t-th frame is used as the first frame on the tracking trajectory of the tracking object.

8. The method according to claim 1, characterized in that After determining the similarity between each obstacle information and each predicted tracking result to obtain m*n similarities, the method further includes: When the similarity between the predicted tracking result of the first tracking object in the tth frame and the m obstacle information is less than the preset similarity threshold, obtaining the predicted tracking result of the t+1th frame obtained by predicting the predicted tracking result of the tth frame; Determine the similarity between the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame; When the similarity is greater than or equal to the preset similarity threshold, the predicted tracking result of the t+1th frame and the obstacle information of the t+1th frame are associated as the first tracking object, and the tracking information of the first tracking object in the t+1th frame is output in combination with the obstacle information and the predicted tracking result of the first tracking object in the t+1th frame; In the case that the similarities are all less than the preset similarity threshold, the predicted tracking result of the t+2th frame obtained by predicting the predicted tracking result of the t+1th frame is obtained, and the similarity between the predicted tracking result of the t+2th frame and the obstacle information of the t+2th frame is determined until the similarity is greater than or equal to the preset similarity threshold, or the predicted tracking result of the first tracking object in the t+Nth frame is obtained, where N is a preset value.

9. A multi-target tracking device, characterized in that: The device comprises: An acquisition module is used to acquire m obstacle information and n predicted tracking results of multiple tracked objects in the tth frame, wherein the n predicted tracking results of the tth frame are obtained by predicting the n obstacle information of the t-1th frame; The acquisition module is further used to acquire the number of first point clouds in each obstacle information and the number of second point clouds in each predicted tracking result; A determination module, used to determine the similarity between each obstacle information and each predicted tracking result based on the m first point cloud numbers and the n second point cloud numbers, to obtain m*n similarities; An associating module, used to associate the obstacle information and the predicted tracking result whose similarity is greater than or equal to a preset similarity threshold as the same tracking object; The tracking output module is used to combine the obstacle information of the same tracked object in the tth frame and the predicted tracking result to output the tracking information of the same tracked object in the tth frame.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the multi-target tracking method according to any one of claims 1 to 8 is implemented.

Images