Multi-target tracking method and device, and vehicle

Abstract

The application provides a multi-target tracking method, device and vehicle, and relates to the field of multi-target tracking.The method comprises the following steps: acquiring laser radar data and millimeter wave radar data collected by a laser radar and a millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; matching and tracking a plurality of obstacles in a preset range based on the detection target information and historical tracking target information; and the historical tracking target information comprises position information, speed information and a heading angle of each obstacle that needs to be tracked in a tracking list.The multi-target tracking method, device and vehicle provided by the application are used for enabling an unmanned working machine to have a stable tracking function for multiple targets, and improving the working efficiency of the working machine.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of multi-target tracking, and in particular to a multi-target tracking method, device and vehicle. BACKGROUND

[0002] In the open pit mine scene, the implementation of unmanned driving technology is conducive to improving the production efficiency of the mine, reducing the work risk and reducing the labor cost.

[0003] In the related art, in order to enable the working machine to accurately plan the path, the working machine needs to have the function of effectively sensing the motion of various targets in the mine. However, the technical solution of multi-target tracking in the related art is mostly for a single sensor, which has great limitations and is not suitable for multi-target tracking of the working machine in the open pit mine scene.

[0004] Therefore, there is an urgent need for a multi-target tracking method to enable the unmanned working machine to have a stable tracking function for multiple targets, thereby improving the working efficiency of the working machine. SUMMARY

[0005] The purpose of the present application is to provide a multi-target tracking method, device and vehicle for enabling the unmanned working machine to have a stable tracking function for multiple targets and improving the working efficiency of the working machine.

[0006] The present application provides a multi-target tracking method, comprising:

[0007] Obtaining target data collected by the laser radar and the millimeter wave radar at the current time, and determining detection target information at the current time according to the target data; the target data includes laser radar data and millimeter wave radar data; based on the detection target information and historical tracking target information, multiple obstacles in a preset range of the working machine are matched and tracked; wherein the detection target information includes the position and heading angle of each obstacle in the multiple obstacles in the preset range of the working machine at the current time; the historical tracking target information includes the position information, speed information and heading angle of each obstacle needing to be tracked in a tracking list; the tracking list includes multiple obstacles needing to be tracked.

[0008] Optionally, the detection target information includes first information, second information and third information; the detection target information at the current time is determined according to the target data, including: the first information is calculated according to the laser radar data, and the second information is calculated according to the millimeter wave radar data; the third information is obtained after the first information and the second information are fused; wherein the first information includes position information and a heading angle of each obstacle in the preset range at the current time; the second information includes position information, relative speed information and a heading angle of each obstacle in the preset range at the current time; and the third information includes position information, relative speed information and a heading angle of each obstacle in the preset range at the current time.

[0009] Optionally, the matching and tracking of the plurality of obstacles in the preset range of the working machine based on the detection target information and the historical tracking target information includes: a first matching weight matrix is calculated based on the first information, and a second matching weight matrix is calculated based on the third information; a Hungarian algorithm is used for target assignment according to the first matching weight matrix and the second matching weight matrix to obtain a first matching result; wherein the target assignment is used for target matching of the plurality of obstacles in the preset range at the current time and the plurality of obstacles in the tracking list; and the first matching result includes a matching success weight value of each obstacle in the preset range at the current time.

[0010] Optionally, the matching and tracking of the plurality of obstacles in the preset range of the working machine based on the detection target information and the historical tracking target information includes: a third matching weight matrix is calculated based on the third information, and a Hungarian algorithm is used for target assignment according to the third matching weight matrix to obtain a second matching result; wherein the second matching result includes a matching success weight value of each obstacle in the preset range at the current time.

[0011] Optionally, after the target assignment by the Hungarian algorithm, the method further includes: judging whether the matching success weight value of each obstacle in the preset range at the current time is less than a preset weight threshold value, if yes, it is determined that the matching is successful, otherwise, it is determined that the matching fails, and the matched obstacle is added to the tracking list; wherein the preset weight threshold value is positively correlated with a target distance; and the target distance is a distance between the obstacle and the working machine.

[0012] Optionally, the matching and tracking of the plurality of obstacles within the preset range of the working machine based on the detection target information and the historical tracking target information comprises: fusing the first matching result and the second matching result to determine whether each obstacle in the tracking list is detected at the current time; and predicting the position information of each obstacle in the tracking list by using a kinematic model to obtain predicted position information of each obstacle.

[0013] Optionally, after the position information of each obstacle in the tracking list is predicted by using the kinematic model to obtain predicted position information of each obstacle, the method further comprises: in the case that a target obstacle is detected, performing target updating based on the predicted position information of the target obstacle and actual position information of the target obstacle by using a Kalman filtering algorithm to obtain motion state information of the target obstacle; wherein the target obstacle is any one of the plurality of obstacles in the tracking list; and the motion state information comprises: position information, relative speed information, and heading angle.

[0014] Optionally, after the position information of each obstacle in the tracking list is predicted by using the kinematic model to obtain predicted position information of each obstacle, the method further comprises: in the case that a target obstacle is not detected, calculating a tracking loss number, and deleting the obstacle that loses tracking from the tracking list in the case that the continuous tracking loss number of the target obstacle is greater than a preset number threshold; wherein the target obstacle is any one of the plurality of obstacles in the tracking list.

[0015] The application also provides a multi-target tracking device, comprising:

[0016] The acquisition module is configured to acquire target data collected by a laser radar and a millimeter wave radar at a current time; the determination module is configured to determine detection target information at the current time according to the target data; the target data comprises: laser radar data and millimeter wave radar data; and the matching and tracking module is configured to match and track a plurality of obstacles within a preset range of the working machine based on the detection target information and historical tracking target information; wherein the detection target information comprises: position and heading angle of each obstacle of the plurality of obstacles within the preset range of the working machine at the current time; the historical tracking target information comprises: position information, speed information, and heading angle of each obstacle that needs to be tracked in a tracking list; and the tracking list comprises a plurality of obstacles that need to be tracked.

[0017] Optionally, the detection target information includes: first information, second information and third information; the determination module is specifically configured to determine the first information according to the laser radar data, and determine the second information according to the millimeter wave radar data; the determination module is further configured to obtain the third information by fusing the first information and the second information; the first information includes: position information and a heading angle of each obstacle in the preset range at the current moment; the second information includes: position information, relative speed information and a heading angle of each obstacle in the preset range at the current moment; and the third information includes: position information, relative speed information and a heading angle of each obstacle in the preset range at the current moment.

[0018] Optionally, the matching tracking module is specifically configured to calculate a first matching weight matrix based on the first information, and calculate a second matching weight matrix based on the third information; the matching tracking module is further configured to perform target assignment by using a Hungarian algorithm according to the first matching weight matrix and the second matching weight matrix, to obtain a first matching result; the target assignment is used for target matching of a plurality of obstacles in the preset range at the current moment and a plurality of obstacles in the tracking list; and the first matching result includes a matching success weight value of each obstacle in the preset range at the current moment.

[0019] Optionally, the matching tracking module is specifically configured to calculate a third matching weight matrix based on the third information, and perform target assignment by using a Hungarian algorithm according to the third matching weight matrix, to obtain a second matching result; the second matching result includes a matching success weight value of each obstacle in the preset range at the current moment.

[0020] Optionally, the matching tracking module is specifically configured to judge whether the matching success weight value of each obstacle in the preset range at the current moment is less than a preset weight threshold value, if yes, it is determined that the matching is successful, otherwise, it is determined that the matching fails, and the matched obstacle is added to the tracking list; the preset weight threshold value is positively correlated with a target distance; and the target distance is a distance between the obstacle and the working machine.

[0021] Optionally, the matching tracking module is specifically configured to fuse and calculate the first matching result and the second matching result; the determination module is further configured to determine whether each obstacle in the tracking list is detected at the current moment; and the matching tracking module is further configured to predict position information of each obstacle in the tracking list by using a kinematic model, to obtain predicted position information of each obstacle.

[0022] Optionally, the matching tracking module is specifically configured to, in a case where the target obstacle is detected, perform target updating based on the predicted position information of the target obstacle and the actual position information of the target obstacle by using a Kalman filtering algorithm to obtain motion state information of the target obstacle; the target obstacle is any one of the multiple obstacles in the tracking list; the motion state information includes position information, relative speed information, and a heading angle.

[0023] Optionally, the matching tracking module is specifically configured to, in a case where the target obstacle is not detected, calculate a tracking loss frequency, and delete the obstacle that is lost in tracking from the tracking list in a case where a continuous tracking loss frequency of the target obstacle is greater than a preset frequency threshold; the target obstacle is any one of the multiple obstacles in the tracking list.

[0024] The application further provides a computer program product, including computer programs / instructions, which, when executed by a processor, implement the steps of the multi-target tracking method according to any one of the above.

[0025] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the multi-target tracking method according to any one of the above when executing the program.

[0026] The application further provides a vehicle, provided with a laser radar and a millimeter wave radar, and computer programs / instructions, which, when executed by a processor, implement the steps of the multi-target tracking method according to any one of the above.

[0027] The application further provides a computer-readable storage medium, which stores a computer program, which, when executed by a processor, implements the steps of the multi-target tracking method according to any one of the above.

[0028] The multi-target tracking method, device, and vehicle provided by the application first acquire target data collected by the laser radar and the millimeter wave radar at a current time, and determine detection target information at the current time according to the target data; the target data includes laser radar data and millimeter wave radar data. Then, multiple obstacles in a preset range of the working machine are matched and tracked based on the detection target information and historical tracking target information. In this way, the unmanned working machine can have a stable multi-target tracking function, thereby improving the working efficiency of the working machine. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0030] Figure 1 is one of the flow schematic diagrams of the multi-target tracking method provided by the present application;

[0031] Figure 2 is another flow schematic diagram of the multi-target tracking method provided by the present application;

[0032] Figure 3 is a structural schematic diagram of the multi-target tracking device provided by the present application;

[0033] Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of protection of the present application.

[0035] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0036] The following explains the technical terms related to the embodiments of the present application:

[0037] Bounding box: It is an algorithm for solving the optimal enclosing space of a discrete point set. The basic idea is to use a geometric body with slightly larger volume and simple characteristics (called bounding box) to approximately replace the complex geometric object. In the field of computer graphics and computational geometry, the bounding box of a group of objects is a closed space that completely contains the object group. Enclosing complex objects in simple bounding boxes and using simple bounding box shapes to approximate complex geometric shapes can improve the efficiency of geometric operations. And it is usually easier to check the overlap between simple objects.

[0038] Laser radar: It is a radar system that detects the position, speed and other characteristic quantities of the target by emitting laser beams. Its working principle is to emit detection signals (laser beams) to the target, then compare the received signals (target echoes) reflected from the target with the transmitted signals, and after appropriate processing, the relevant information of the target such as target distance, direction, height, speed, attitude, and even shape can be obtained, so as to detect, track and identify targets such as aircraft and missiles. It is composed of laser transmitter, optical receiver, turntable and information processing system, etc. The laser converts electrical pulses into optical pulses for emission, and the optical receiver restores the optical pulses reflected from the target into electrical pulses and sends them to the display.

[0039] Millimeter wave radar: It is a radar that works in the millimeter wave band (millimeter wave) for detection. Usually millimeter wave refers to the frequency domain of 30-300GHz (wavelength of 1-10mm). The wavelength of millimeter wave is between microwave and centimeter wave, so millimeter wave radar has some advantages of microwave radar and optical radar. Compared with centimeter wave seeker, millimeter wave seeker has the characteristics of small size, light weight and high spatial resolution. Compared with infrared, laser and television optical seeker, millimeter wave seeker has strong ability to penetrate fog, smoke and dust, and has the characteristics of all-weather (except heavy rain) and all-day. In addition, the anti-interference and anti-stealth capabilities of millimeter wave seeker are better than those of other microwave seekers. Millimeter wave radar can distinguish and identify very small targets, and can identify multiple targets at the same time; it has imaging capability, small size, good mobility and concealment, etc.

[0040] There are personnel, working machinery (including: watering car, excavator, bulldozer, mine car, etc.), ruts, dust, irregular roads, etc. in the mine scene. The targets often appear to cross, separate and be occluded during movement. The existing technology is mostly for single sensor multi-target tracking, which has great limitations in tracking performance. Or the technical scheme of multi-sensor multi-target tracking is not combined with the characteristics of mine scene.

[0041] To address the aforementioned technical problems in related technologies, this application provides a multi-target tracking method. This method is based on the perception fusion of lidar and millimeter-wave radar, enabling the retention of target identity document (ID) information; target position prediction when target detection fails; filtering of sensor measurements; and generation of target motion trajectories. This allows for multi-target tracking in open-pit mining scenarios, thereby improving the working efficiency of machinery.

[0042] The multi-target tracking method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0043] like Figure 1 As shown in the figure, this application provides a multi-target tracking method applied to a work machine equipped with a lidar and a millimeter-wave radar. The method may include the following steps 101 and 102:

[0044] Step 101: Obtain the target data collected by the lidar and the millimeter-wave radar at the current moment, and determine the detection target information at the current moment based on the target data.

[0045] The target data includes: lidar data and millimeter-wave radar data; the target detection information includes: the position and heading angle of each obstacle among multiple obstacles within the preset range of the operating machinery at the current moment.

[0046] For example, the operating machinery in this application embodiment includes: cranes, pile drivers, mixers, excavators, mining trucks, and other operating equipment. The aforementioned lidar data is data collected by lidar installed on the operating machinery; the aforementioned millimeter-wave radar data is data collected by millimeter-wave radar installed on the operating machinery.

[0047] Understandably, light waves suffer significant attenuation during propagation in the atmosphere, requiring high precision in device fabrication. Compared to light waves, millimeter waves experience less attenuation when propagating through atmospheric windows (frequencies where some attenuation is minimized due to resonant absorption by gas molecules during propagation in the atmosphere), and are less affected by natural light and heat radiation sources. Based on this, multi-target tracking can be achieved by fusing laser and millimeter-wave radar, compensating for the shortcomings of single sensors and enabling machinery to perform multi-target tracking in open-pit mining environments.

[0048] For example, the operating machinery can acquire LiDAR data and millimeter-wave radar data respectively, and can generate corresponding relevant information for target detection based on the LiDAR data and the millimeter-wave radar data.

[0049] Exemplarily, the detection target information comprises: first information, second information and third information.

[0050] Specifically, the step 101 can further comprise the following step 101a1 and step 101a2.

[0051] The step 101a1 comprises: determining the first information according to the laser radar data, and determining the second information according to the millimeter wave radar data.

[0052] The step 101a2 comprises: obtaining the third information after fusing the first information and the second information.

[0053] The first information comprises: position information and heading angle of each obstacle in the preset range at the current moment; the second information comprises: position information, relative speed information and heading angle of each obstacle in the preset range at the current moment; and the third information comprises: position information, relative speed information and heading angle of each obstacle in the preset range at the current moment.

[0054] It should be noted that, unlike the laser radar which needs multiple frames of images to determine the speed of the detection target, the millimeter wave radar can directly determine the speed of the detection target, therefore, the second information and the third information fused with the first information and the second information both comprise the relative speed information of the obstacle.

[0055] Exemplarily, the position information can be used to indicate the relative position between the obstacle and the working machine; and the relative speed information can be used to indicate the relative speed between the obstacle and the working machine.

[0056] Exemplarily, the detection target in the embodiment of the present application can be an obstacle near the working machine, which can be a fixed object (for example, a stone, a building, etc.), or a moving object (for example, other working machines, personnel, etc.).

[0057] The step 102 comprises: matching and tracking multiple obstacles in the preset range of the working machine based on the detection target information and historical tracking target information.

[0058] The historical tracking target information comprises: position information, speed information and heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list comprises multiple obstacles needing to be tracked.

[0059] Exemplarily, in the embodiment of the present application, the position information of the obstacle can be represented by a bounding box, that is, the position information of the obstacle can include: the center point position information of the bounding box, and the size information of the bounding box. The size information of the bounding box is used to determine the size of the obstacle, and can also be used as a matching basis when matching.

[0060] Exemplarily, according to the collected laser radar data and millimeter wave radar data, the obstacle information corresponding to the laser radar at the current moment (i.e., the first information described above) and the obstacle information corresponding to the millimeter wave radar at the current moment (i.e., the second information described above) can be determined respectively, and the obstacle information near the working machine at the current moment determined after fusing the laser radar data and the millimeter wave radar data (i.e., the third information described above) can be determined.

[0061] Exemplarily, based on the detection target information and the historical tracking information indicating the position, relative speed and heading angle of each obstacle in the preset range, the matching and tracking of each obstacle can be realized.

[0062] It should be noted that the historical tracking information is obtained based on the detection target information determined at a moment before the current moment.

[0063] Optionally, in the embodiment of the present application, the calculation of the matching weight matrix can be based on the first information, the second information and the third information, and then the target assignment can be completed.

[0064] Specifically, based on the steps 101a1 and 101a2, the step 102 can further include the following steps 102a1 and 102a2:

[0065] Step 102a1, calculating a first matching weight matrix based on the first information, and calculating a second matching weight matrix based on the third information.

[0066] Step 102a2, performing target assignment by using the Hungarian algorithm according to the first matching weight matrix and the second matching weight matrix to obtain a first matching result.

[0067] Wherein, the target assignment is used to perform target matching between the plurality of obstacles in the preset range at the current moment and the plurality of obstacles in the tracking list; the first matching result includes: the matching success weight value of each obstacle in the preset range at the current moment.

[0068] Exemplarily, the matching weight is used to calculate the matching weight between any two obstacles in the plurality of obstacles indicated by the detection target information and the plurality of obstacles indicated by the historical tracking target information, and the two obstacles with the smallest matching weight are determined as the two targets having a correlation relationship.

[0069] Specifically, based on steps 101a1 and 101a2 above, step 102 may further include the following step 102b:

[0070] Step 102b: Calculate the third matching weight matrix based on the third information, and use the Hungarian algorithm to assign the target according to the third matching weight matrix to obtain the second matching result.

[0071] The second matching result includes: the matching success weight value of each obstacle within the preset range at the current time. The second matching result may also include: obstacles from the multiple obstacles indicated by the tracking list that match each obstacle within the preset range at the current time.

[0072] For example, such as Figure 2 As shown, after acquiring LiDAR and millimeter-wave radar data, the LiDAR and millimeter-wave radar data can be fused to obtain fused data. Then, a matching weight matrix is ​​calculated based on the LiDAR data, millimeter-wave radar data, and fused data respectively to complete target assignment. This yields the obstacle that matches each obstacle within the preset range indicated by historical target tracking information, along with its corresponding matching weight (i.e., the successful matching weight value).

[0073] It should be noted that, in the embodiments of this application, multiple obstacles within a preset range can be matched and tracked based on one or more of lidar data, millimeter-wave radar data, and fused data.

[0074] For example, after step 102b above, step 102 may further include the following step 102c:

[0075] Step 102c: Determine whether the matching success weight value of each obstacle within the preset range at the current time is less than the preset weight threshold. If yes, the matching is successful; otherwise, the matching fails, and the failed obstacle is added to the tracking list.

[0076] The preset weight threshold is positively correlated with the target distance; the target distance is the distance between the obstacle and the operating machinery.

[0077] For example, if the above-mentioned target detection information indicates that a new obstacle has been detected, the obstacle can be added to the tracking list so that subsequent steps can match and track it.

[0078] For example, such as Figure 2As shown, after target assignment is performed based on the calculation result of the matching weight matrix and the corresponding matching success weight value is obtained, it can be further judged whether the weight value is reasonable. If the matching success weight value is less than a threshold value, it indicates that the matching result is reasonable and the matching is successful. If the matching success weight value is greater than or equal to the threshold value, it indicates that the matching result is unreasonable and the matching is failed.

[0079] It can be understood that, since the detection accuracy of the obstacle is lower when the obstacle is farther away from the working machine, the preset weight threshold value corresponding to the obstacle farther away from the working machine can be appropriately increased.

[0080] Optionally, in the embodiment of the present application, after target matching is completed, target tracking of each obstacle and screening of an obstacle with too many tracking losses can be realized based on the matching result.

[0081] Specifically, for the above target tracking, after the step 102c, the step 102 can further include the following steps 102d1 and 102d2:

[0082] The step 102d1 fuses the first matching result and the second matching result to determine whether each obstacle in the tracking list is detected at the current time.

[0083] The step 102d2 predicts the position information of each obstacle in the tracking list by using a kinematic model to obtain the predicted position information of each obstacle.

[0084] Exemplarily, after the predicted position information of each obstacle is obtained, Kalman filtering update can be performed in combination with the measurement value of the sensor.

[0085] Specifically, after the step 102d3, the step 102 can further include the following step 102e1 or step 102e2:

[0086] The step 102e1, in the case that a target obstacle is detected, target update is performed by using a Kalman filtering algorithm based on the predicted position information of the target obstacle and the actual position information of the target obstacle to obtain the motion state information of the target obstacle.

[0087] The target obstacle is any one of the plurality of obstacles in the tracking list; and the motion state information includes position information, relative speed information and heading angle.

[0088] The step 102e2, in the case that a target obstacle is not detected, the number of tracking losses is calculated, and in the case that the number of continuous tracking losses of the target obstacle is greater than a preset number threshold value, the obstacle with tracking loss is deleted from the tracking list.

[0089] Exemplarily, in the case of a target obstacle that can be detected at the current moment, a corresponding kinematic model can be used to predict the prediction result of the current moment position information of the target obstacle in combination with the type of the target obstacle and the actual motion condition, and meanwhile, the Kalman filter is updated in combination with the actual position information indicated by the detection target information to determine the motion state information of the target obstacle at the current moment.

[0090] It can be understood that, since there is relative motion between the vehicle and the target obstacle, there can be an occlusion between the target obstacle and the vehicle, so that the target obstacle cannot be detected, and in this case, tracking loss occurs. If tracking loss occurs for multiple times continuously, the target obstacle is directly deleted from the tracking list.

[0091] The multi-target tracking method provided in the embodiment of the application first acquires target data collected by the laser radar and the millimeter wave radar at the current moment, and determines detection target information at the current moment according to the target data; the target data includes laser radar data and millimeter wave radar data. Then, based on the detection target information and historical tracking target information, multiple target obstacles in a preset range of the working machine are matched and tracked. In this way, the unmanned working machine can have a stable tracking function for multiple target obstacles, thereby improving the working efficiency of the working machine.

[0092] It should be noted that the multi-target tracking method provided in the embodiment of the application can be executed by a multi-target tracking device or a control module in the multi-target tracking device for executing the multi-target tracking method. In the embodiment of the application, the multi-target tracking device executes the multi-target tracking method as an example, and the multi-target tracking device provided in the embodiment of the application is described.

[0093] It should be noted that, in the embodiment of the application, the multi-target tracking method shown in each of the above method diagrams is exemplarily described by taking one of the diagrams in the embodiment of the application as an example. In the specific implementation, the multi-target tracking method shown in each of the above method diagrams can also be implemented in combination with any other diagram that can be combined as described above, which will not be described herein again.

[0094] The multi-target tracking device provided in the application is described below, and the multi-target tracking method described below can be correspondingly referred to the multi-target tracking method described above.

[0095] Figure 3 The structure diagram of the multi-target tracking device provided in an embodiment of the application is shown in FIG. 1, and specifically includes: Figure 3

[0096] ​The acquisition module 301 is configured to acquire target data collected by a laser radar and a millimeter wave radar at a current moment; the determination module 302 is configured to determine detection target information at the current moment according to the target data; the target data includes laser radar data and millimeter wave radar data; the matching and tracking module 303 is configured to perform matching and tracking on a plurality of obstacles in a preset range of the working machine based on the detection target information and historical tracking target information; wherein the detection target information includes a position and a heading angle of each obstacle in the plurality of obstacles in the preset range of the working machine at the current moment; the historical tracking target information includes position information, speed information and a heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list includes a plurality of obstacles needing to be tracked.

[0097] Optionally, the detection target information includes first information, second information and third information; the determination module 302 is specifically configured to determine the first information according to the laser radar data, and determine the second information according to the millimeter wave radar data; and the determination module 302 is specifically further configured to obtain the third information by fusing the first information and the second information; wherein the first information includes position information and a heading angle of each obstacle in the preset range at the current moment; the second information includes position information, relative speed information and a heading angle of each obstacle in the preset range at the current moment; and the third information includes position information, relative speed information and a heading angle of each obstacle in the preset range at the current moment.

[0098] Optionally, the matching and tracking module 303 is specifically configured to calculate a first matching weight matrix based on the first information, and calculate a second matching weight matrix based on the third information; and the matching and tracking module 303 is specifically further configured to perform target assignment by using a Hungarian algorithm according to the first matching weight matrix and the second matching weight matrix, to obtain a first matching result; wherein the target assignment is used for target matching between the plurality of obstacles in the preset range at the current moment and a plurality of obstacles in the tracking list; and the first matching result includes a matching success weight value of each obstacle in the preset range at the current moment.

[0099] Optionally, the matching and tracking module 303 is specifically configured to calculate a third matching weight matrix based on the third information, and perform target assignment by using a Hungarian algorithm according to the third matching weight matrix, to obtain a second matching result; wherein the second matching result includes a matching success weight value of each obstacle in the preset range at the current moment.

[0100] Optionally, the matching tracking module 303 is specifically configured to determine whether the matching success weight value of each obstacle in the preset range at the current time is less than a preset weight threshold, if yes, it is determined that the matching is successful, otherwise, it is determined that the matching fails, and the matched obstacle is added to the tracking list; wherein the preset weight threshold is positively correlated with the target distance; the target distance is the distance between the obstacle and the working machine.

[0101] Optionally, the matching tracking module 303 is specifically configured to fuse the first matching result and the second matching result; the determination module 302 is further configured to determine whether each obstacle in the tracking list is detected at the current time; the matching tracking module 303 is further configured to predict the position information of each obstacle in the tracking list by using a kinematic model to obtain the predicted position information of each obstacle.

[0102] Optionally, the matching tracking module 303 is specifically configured to, in the case that a target obstacle is detected, update the target based on the predicted position information of the target obstacle and the actual position information of the target obstacle by using a Kalman filtering algorithm to obtain the motion state information of the target obstacle; wherein the target obstacle is any one of the plurality of obstacles in the tracking list; the motion state information includes position information, relative speed information, and heading angle.

[0103] Optionally, the matching tracking module 303 is specifically configured to, in the case that a target obstacle is not detected, calculate the number of tracking losses, and in the case that the number of continuous tracking losses of the target obstacle is greater than a preset number threshold, delete the obstacle with tracking loss from the tracking list; wherein the target obstacle is any one of the plurality of obstacles in the tracking list.

[0104] The multi-target tracking device provided in the application first acquires target data collected by the laser radar and the millimeter wave radar at the current time, and determines detection target information at the current time according to the target data; the target data includes laser radar data and millimeter wave radar data. Then, based on the detection target information and historical tracking target information, a plurality of obstacles in the preset range of the working machine are matched and tracked. In this way, the unmanned working machine can have a stable multi-target tracking function, thereby improving the working efficiency of the working machine.

[0105] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 4As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 complete communication with each other through the communications bus 440. The processor 410 can invoke the logical instructions in the memory 430 to execute a multi-target tracking method, which includes: acquiring target data collected by the laser radar and the millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; the target data includes laser radar data and millimeter wave radar data; based on the detection target information and historical tracking target information, a plurality of obstacles in a preset range of the working machine are matched and tracked; wherein the detection target information includes the position and heading angle of each obstacle in the plurality of obstacles in the preset range of the working machine at the current time; the historical tracking target information includes the position information, speed information, and heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list includes a plurality of obstacles needing to be tracked.

[0106] In addition, the logical instructions in the memory 430 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0107] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the multi-target tracking method provided by any of the above methods, the method comprising: acquiring target data collected by the laser radar and the millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; the target data comprising: laser radar data and millimeter wave radar data; matching and tracking a plurality of obstacles within a preset range of the working machine based on the detection target information and historical tracking target information; wherein the detection target information comprises: a position and a heading angle of each obstacle of the plurality of obstacles within the preset range of the working machine at the current time; the historical tracking target information comprises: position information, speed information, and a heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list comprises a plurality of obstacles needing to be tracked.

[0108] In another aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-target tracking method provided by any of the above methods, the method comprising: acquiring target data collected by the laser radar and the millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; the target data comprising: laser radar data and millimeter wave radar data; matching and tracking a plurality of obstacles within a preset range of the working machine based on the detection target information and historical tracking target information; wherein the detection target information comprises: a position and a heading angle of each obstacle of the plurality of obstacles within the preset range of the working machine at the current time; the historical tracking target information comprises: position information, speed information, and a heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list comprises a plurality of obstacles needing to be tracked.

[0109] In still another aspect, the application also provides a vehicle provided with a laser radar and a millimeter wave radar, and a computer program / instruction which, when executed by a processor, performs the steps of any one of the above multi-target tracking methods, including: obtaining target data collected by the laser radar and the millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; the target data includes laser radar data and millimeter wave radar data; based on the detection target information and historical tracking target information, a plurality of obstacles in a preset range of the working machine are matched and tracked; wherein the detection target information includes the position and heading angle of each obstacle in the plurality of obstacles in the preset range of the working machine at the current time; the historical tracking target information includes the position information, speed information and heading angle of each obstacle needing to be tracked in a tracking list; and the tracking list includes a plurality of obstacles needing to be tracked.

[0110] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0111] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0112] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A multi-target tracking method characterized by, The method is applied to a working machine provided with a laser radar and a millimeter wave radar, and comprises the following steps: acquiring target data collected by the laser radar and the millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; the target data comprises laser radar data and millimeter wave radar data; matching and tracking a plurality of obstacles within a preset range of the working machine based on the detection target information and historical tracking target information; wherein the detection target information comprises a position and a heading angle of each obstacle among the plurality of obstacles within the preset range of the working machine at the current time; the historical tracking target information comprises position information, speed information and a heading angle of each obstacle needing to be tracked in a tracking list; the tracking list comprises a plurality of obstacles needing to be tracked; the detection target information comprises first information, second information and third information; the determination of the detection target information at the current time according to the target data comprises: determining the first information according to the laser radar data, and determining the second information according to the millimeter wave radar data; obtaining the third information after fusing the first information and the second information; the matching and tracking of the plurality of obstacles within the preset range of the working machine based on the detection target information and the historical tracking target information comprises: calculating a first matching weight matrix based on the first information, and calculating a second matching weight matrix based on the third information; performing target assignment by using a Hungarian algorithm according to the first matching weight matrix and the second matching weight matrix to obtain a first matching result; wherein the target assignment is used for target matching between the plurality of obstacles within the preset range at the current time and the plurality of obstacles in the tracking list; the first matching result comprises a matching success weight value of each obstacle within the preset range at the current time; the matching success weight value is used for calculating a matching weight between any two obstacles among the plurality of obstacles indicated by the detection target information and the plurality of obstacles indicated by the historical tracking target information, and determining two targets having a correlation relationship as the two obstacles with the minimum matching weight; the matching and tracking of the plurality of obstacles within the preset range of the working machine based on the detection target information and the historical tracking target information further comprises: calculating a third matching weight matrix based on the second information, and performing target assignment by using a Hungarian algorithm according to the third matching weight matrix to obtain a second matching result; wherein the second matching result comprises a matching success weight value of each obstacle within the preset range at the current time.

2. The method of claim 1, wherein, the first information comprises position information and a heading angle of each obstacle within the preset range at the current time; the second information comprises position information, relative speed information and a heading angle of each obstacle within the preset range at the current time; and the third information comprises position information, relative speed information and a heading angle of each obstacle within the preset range at the current time.

3. The method of claim 2, wherein, After the target assignment by using the Hungarian algorithm, the method further comprises: determining whether the matching success weight value of each obstacle in the preset range at the current time is less than a preset weight threshold, if yes, determining that the matching is successful, otherwise, determining that the matching is failed, and adding the matched obstacle to the tracking list; wherein the preset weight threshold is positively correlated with a target distance; the target distance is a distance between the obstacle and the working machine.

4. The method of claim 3, wherein, The matching and tracking of the plurality of obstacles in the preset range of the working machine based on the detected target information and the historical tracking target information comprises: fusing the first matching result and the second matching result to determine whether each obstacle in the tracking list is detected at the current time; predicting the position information of each obstacle in the tracking list by using a kinematic model to obtain the predicted position information of each obstacle.

5. The method of claim 4, wherein, After the prediction of the position information of each obstacle in the tracking list by using the kinematic model to obtain the predicted position information of each obstacle, the method further comprises: in the case that a target obstacle is detected, performing target updating by using a Kalman filtering algorithm based on the predicted position information of the target obstacle and the actual position information of the target obstacle to obtain the motion state information of the target obstacle; wherein the target obstacle is any one of the plurality of obstacles in the tracking list; the motion state information comprises: position information, relative speed information, and heading angle.

6. The method of claim 4, wherein, After the prediction of the position information of each obstacle in the tracking list by using the kinematic model to obtain the predicted position information of each obstacle, the method further comprises: in the case that a target obstacle is not detected, calculating the number of tracking losses, and deleting the obstacle with tracking loss from the tracking list in the case that the continuous tracking loss number of the target obstacle is greater than a preset number threshold; wherein the target obstacle is any one of the plurality of obstacles in the tracking list.

7. A multi-target tracking device, characterized by, The device comprises: an acquisition module configured to acquire target data collected by a laser radar and a millimeter wave radar at a current time; a determination module configured to determine detected target information at the current time according to the target data; the target data comprises: laser radar data and millimeter wave radar data; a matching and tracking module configured to match and track a plurality of obstacles in a preset range of a working machine based on the detected target information and historical tracking target information; wherein the detected target information comprises: position and heading angle of each obstacle in the plurality of obstacles in the preset range at the current time; the historical tracking target information comprises: position information, speed information, and heading angle of each obstacle needing to be tracked in a tracking list; the tracking list comprises a plurality of obstacles needing to be tracked; the detected target information comprises: first information, second information, and third information; The determination module is specifically configured to determine the first information according to the laser radar data and determine the second information according to the millimeter wave radar data; and the determination module is further configured to obtain the third information by fusing the first information and the second information. The matching tracking module is specifically configured to calculate a first matching weight matrix based on the first information and calculate a second matching weight matrix based on the third information; and the matching tracking module is further configured to perform target assignment by using a Hungarian algorithm according to the first matching weight matrix and the second matching weight matrix to obtain a first matching result; wherein the target assignment is used to perform target matching between a plurality of obstacles in the preset range at a current time and a plurality of obstacles in the tracking list; and the first matching result includes a matching success weight value of each obstacle in the preset range at the current time; the matching success weight value is used to calculate a matching weight between any two obstacles of a plurality of obstacles indicated by detection target information and a plurality of obstacles indicated by historical tracking target information, and determine two targets having a correlation relationship as the two obstacles with the smallest matching weight. The matching tracking module is specifically configured to calculate a third matching weight matrix based on the third information and perform target assignment by using a Hungarian algorithm according to the third matching weight matrix to obtain a second matching result; wherein the second matching result includes a matching success weight value of each obstacle in the preset range at the current time.

8. A vehicle characterized by comprising: A laser radar and a millimeter wave radar are provided, as well as computer programs / instructions which, when executed by a processor, perform the steps of the multi-target tracking method according to any one of claims 1 to 6.

Images