Multi-target trajectory tracking method and device, electronic equipment and storage medium

Abstract

The invention provides a multi-target trajectory tracking method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: determining a forward time sequence track and a reverse time sequence track; calculating a fusion index between every two tracks in a track set, wherein the track set comprises each forward time sequence track and each reverse time sequence track; performing trajectory fusion on the two trajectories, and adding the trajectory obtained by fusion into a trajectory set; and iteratively calculating a fusion index of every two trajectories in the current trajectory set until the fusion indexes do not meet a preset fusion condition, and taking the current trajectory set as a first target set. According to the invention, the forward time sequence track and the reverse time sequence track are obtained, the time sequence information is fully utilized, and the accuracy and robustness of multi-target track tracking are improved. The accuracy of the multi-target tracking algorithm is further improved through trajectory fusion. And two tracks belonging to different targets can be further prevented from being mistakenly associated by determining the fusion index.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for multi-target trajectory tracking. Background Technology

[0002] Multi-object trajectory tracking technology is widely used in autonomous driving. It can estimate the state of moving objects and identify the trajectories of different types of moving objects, such as pedestrians, bicycles, and cars. The resulting trajectories can be used to infer their subsequent motion patterns and driving behaviors, providing guidance for the motion decisions of autonomous vehicles. Therefore, the accuracy and reliability of multi-object trajectory tracking technology determine the stability and safety of autonomous vehicle operation.

[0003] However, the accuracy of multi-target trajectory tracking decreases in dense targets and complex scenarios. Furthermore, when the positions and shapes of trajectories of similar targets are close, it may lead to misassociating two trajectories belonging to different targets, resulting in lower trajectory tracking accuracy and reliability, thus affecting the stability and safety of autonomous driving.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a multi-target trajectory tracking method, apparatus, electronic device, and storage medium, which at least to some extent overcomes the problem that the accuracy of multi-target trajectory tracking decreases in dense targets and complex scenarios, and that related technologies have the problem of trajectory misassociation, which leads to a further reduction in trajectory tracking accuracy and reliability, affecting the stability and safety of autonomous driving.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a multi-target trajectory tracking method is provided, comprising: determining the forward temporal trajectory corresponding to each tracked target through the forward temporal sequence of multi-frame data, and determining the reverse temporal trajectory corresponding to each tracked target through the reverse temporal sequence of multi-frame data, wherein any trajectory is used to describe the state value of a tracked target in each frame; calculating a fusion index between every two trajectories in the trajectory set, the trajectory set including each forward temporal trajectory and each reverse temporal trajectory; performing trajectory fusion on two trajectories whose fusion index meets a preset fusion condition, and adding the fused trajectory to the trajectory set; iteratively calculating the fusion index between every two trajectories in the current trajectory set until the fusion index no longer meets the preset fusion condition, and using the current trajectory set as a first target set.

[0008] In some embodiments, determining the forward temporal trajectory corresponding to each tracking target through the forward temporal sequence of multiple frames of data includes: sequentially processing each frame of data according to the forward temporal sequence as follows until the last frame of data is processed to obtain each forward temporal trajectory: performing target detection on the current frame data to obtain the detection results of each candidate target; predicting the current frame state value of each tracking target to obtain the predicted state value; determining the association relationship between each tracking target and each candidate target based on the predicted state value and the detection results, wherein any group of tracked targets with an association relationship and the candidate targets are the same object; updating the predicted state value of the corresponding tracking target based on the detection results of the candidate targets with an association relationship to obtain the current frame state value; and obtaining the forward temporal trajectory of each tracking target corresponding to the current frame based on the forward temporal trajectory of the previous frame and each current frame state value.

[0009] In some embodiments, the multi-target trajectory tracking method provided in this disclosure further includes: if the first candidate target is not associated with any tracked target, then the detection result of the first candidate target is taken as the starting point of a new positive temporal trajectory, wherein the first candidate target is any candidate target.

[0010] In some embodiments, the detection result of any candidate target includes a confidence score. After performing target detection on the current frame data to obtain the detection results of each candidate target, the multi-target trajectory tracking method provided in this disclosure further includes: removing each candidate target whose confidence score is lower than a preset confidence threshold.

[0011] In some embodiments, the multi-target trajectory tracking method provided in this disclosure further includes: if the first tracking target is not associated with any candidate target, then determining the number of consecutive times that the first tracking target is not associated with any candidate target; if the number of consecutive times is not greater than the number threshold, then using the predicted state value as the current frame state value of the first tracking target; if the number of consecutive times meets the number threshold, then terminating the forward temporal trajectory corresponding to the first tracking target.

[0012] In some embodiments, calculating a fusion metric between every two trajectories in the trajectory set includes determining the fusion metric by using a weighted average of the first IOU (Intersection over Union) between every two trajectories of the same type in the trajectory set.

[0013] In some embodiments, the multi-target trajectory tracking method provided in this disclosure further includes: determining the first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating the detection results of candidate targets with correlation; and performing a weighted average calculation on the first target frame state values ​​of every two trajectories of the same type to obtain a first IOU weighted average value between every two trajectories of the same type.

[0014] In some embodiments, the multi-target trajectory tracking method provided in this disclosure further includes: determining each group of trajectories in a first target set, wherein any group of trajectories includes a forward temporal trajectory and a reverse temporal trajectory; calculating the splicing index of each group of trajectories in the first target set; splicing the trajectories of a group of trajectories whose splicing index meets a preset splicing condition, and adding the spliced ​​trajectory to the first target set; iteratively calculating the splicing index of each group of trajectories in the current first target set until the splicing index does not meet the preset splicing condition, and then using the current first target set as a second target set.

[0015] In some embodiments, any set of trajectories includes a forward temporal trajectory and a reverse temporal trajectory belonging to the same category; wherein, calculating the splicing index of each set of trajectories in the first target set includes: calculating the Euclidean distance between each set of trajectories in the first target set, wherein the Euclidean distance is the Euclidean distance between the termination state values ​​of two trajectories; if the Euclidean distance between the first set of trajectories meets a preset distance condition, then calculating the second IOU weighted average value between the first set of trajectories, wherein the first set of trajectories is any set of trajectories in the first target set; and determining the splicing index by the second IOU weighted average value and the Euclidean distance.

[0016] In some embodiments, calculating the second IOU weighted average among the first group of trajectories includes: determining the second target frame state value included in the first group of trajectories, wherein the second target frame state value is a predicted state value; and performing a weighted average calculation on the target frame state values ​​of the first group of trajectories to obtain the second IOU weighted average among the first group of trajectories.

[0017] According to another aspect of this disclosure, a multi-target trajectory tracking device is also provided, comprising: a trajectory determination module, configured to determine the forward temporal trajectory corresponding to each tracked target through the forward temporal sequence of multi-frame data, and to determine the reverse temporal trajectory corresponding to each tracked target through the reverse temporal sequence of multi-frame data, wherein any trajectory is used to describe the state value of a tracked target in each frame; a fusion index calculation module, configured to calculate the fusion index between every two trajectories in the trajectory set, the trajectory set including each forward temporal trajectory and each reverse temporal trajectory; a trajectory fusion module, configured to perform trajectory fusion on two trajectories whose fusion index meets a preset fusion condition, and add the fused trajectory to the trajectory set; and a first target set determination module, configured to iteratively calculate the fusion index of every two trajectories in the current trajectory set until the fusion index no longer meets the preset fusion condition, and to use the current trajectory set as the first target set.

[0018] In some embodiments, the trajectory determination module is used to process each frame of data sequentially in a forward temporal order as follows, until the last frame of data is processed, to obtain each forward temporal trajectory: Target detection is performed on the current frame data to obtain the detection results of each candidate target; the current frame state value of each tracked target is predicted to obtain a predicted state value; based on the predicted state value and the detection results, the association relationship between each tracked target and each candidate target is determined, wherein any group of tracked targets with an association relationship and the candidate target are the same object; the predicted state value of the corresponding tracked target is updated based on the detection results of the candidate targets with an association relationship to obtain the current frame state value; based on the forward temporal trajectory of the previous frame and each current frame state value, the forward temporal trajectory of each tracked target corresponding to the current frame is obtained.

[0019] In some embodiments, the trajectory determination module is further configured to, if the first candidate target is not associated with any tracked target, take the detection result of the first candidate target as the starting point of a new positive temporal trajectory, wherein the first candidate target is any candidate target.

[0020] In some embodiments, the detection result of any candidate target includes a confidence score. The trajectory determination module is also used to remove each candidate target whose confidence score is lower than a preset confidence threshold.

[0021] In some embodiments, the trajectory determination module is further configured to: if the first tracking target is not associated with any candidate target, determine the number of consecutive times the first tracking target is not associated with any candidate target; if the number of consecutive times is not greater than the number threshold, use the predicted state value as the current frame state value of the first tracking target; if the number of consecutive times meets the number threshold, terminate the forward temporal trajectory corresponding to the first tracking target.

[0022] In some embodiments, the fusion index calculation module is used to determine the fusion index by the first IOU weighted average between every two trajectories of the same type in the trajectory set.

[0023] In some embodiments, the fusion index calculation module is further configured to determine the first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating the detection results of candidate targets with correlation; and to calculate the weighted average of the first target frame state values ​​of every two trajectories of the same type to obtain the first IOU weighted average value between every two trajectories of the same type.

[0024] In some embodiments, the multi-target trajectory tracking provided in this disclosure further includes: a trajectory stitching module, configured to determine each group of trajectories in a first target set, wherein any group of trajectories includes a forward temporal trajectory and a reverse temporal trajectory; calculate the stitching index of each group of trajectories in the first target set; stitch the trajectories of a group of trajectories whose stitching index meets a preset stitching condition, and add the stitched trajectory to the first target set; iteratively calculate the stitching index of each group of trajectories in the current first target set until the stitching index does not meet the preset stitching condition, and then use the current first target set as a second target set.

[0025] In some embodiments, any set of trajectories includes a forward temporal trajectory and a reverse temporal trajectory belonging to the same category. The trajectory stitching module is used to calculate the Euclidean distance between each set of trajectories in the first target set, wherein the Euclidean distance is the Euclidean distance between the termination state values ​​of two trajectories; if the Euclidean distance between the first set of trajectories meets a preset distance condition, then the second IOU weighted average value between the first set of trajectories is calculated, wherein the first set of trajectories is any set of trajectories in the first target set; and the stitching index is determined by the second IOU weighted average value and the Euclidean distance.

[0026] In some embodiments, the trajectory stitching module is used to determine the state value of the second target frame included in the first group of trajectories, wherein the state value of the second target frame is a predicted state value; and to perform a weighted average calculation on the state values ​​of the target frames of the first group of trajectories to obtain a second IOU weighted average value among the first group of trajectories.

[0027] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the multi-target trajectory tracking method of any of the above via executing the executable instructions.

[0028] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the multi-target trajectory tracking method of any one of the above.

[0029] According to another aspect of this disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the multi-target trajectory tracking method provided in various alternative embodiments of this disclosure.

[0030] The technical solutions provided in the embodiments of this disclosure can acquire forward and reverse temporal trajectories separately, thereby fully utilizing temporal information and improving the accuracy and robustness of multi-target trajectory tracking. Furthermore, the embodiments of this disclosure can perform trajectory fusion by weighted averaging of the state values ​​of the two trajectories, which can further improve the accuracy of the multi-target tracking algorithm. In addition, by determining the fusion index, it is possible to further avoid misassociating two trajectories belonging to different targets, thereby improving the accuracy and reliability of trajectory tracking and ultimately enhancing the stability and safety of autonomous driving.

[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0033] Figure 1 A schematic diagram of a system architecture according to an embodiment of this disclosure is shown;

[0034] Figure 2 This diagram illustrates a flowchart of a multi-target trajectory tracking method according to an embodiment of the present disclosure;

[0035] Figure 3 This diagram illustrates a multi-target trajectory tracking process according to an embodiment of the present disclosure.

[0036] Figure 4 This diagram illustrates a trajectory fusion process according to an embodiment of the present disclosure.

[0037] Figure 5 This diagram illustrates a trajectory stitching process according to an embodiment of the present disclosure.

[0038] Figure 6 This diagram illustrates a multi-target trajectory tracking device according to an embodiment of the present disclosure;

[0039] Figure 7 This diagram illustrates a structural block diagram of an electronic device according to an embodiment of the present disclosure.

[0040] Figure 8 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0041] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0042] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0043] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0044] Figure 1 A schematic diagram of an exemplary application system architecture to which the multi-target trajectory tracking method of the embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include terminal device 101, network 102 and server 103.

[0045] Optionally, the terminal device 101 can send multiple frames of data that need to be tracked for multiple targets to the server 103. The server 103 can determine the forward time sequence trajectory corresponding to each tracked target through the forward time sequence of the multiple frames of data, and determine the reverse time sequence trajectory corresponding to each tracked target through the reverse time sequence of the multiple frames of data, wherein any trajectory is used to describe the state value of a tracked target in each frame.

[0046] Then, server 103 can calculate the fusion index between every two trajectories in the trajectory set, which includes each forward time-series trajectory and each reverse time-series trajectory. Next, server 103 can perform trajectory fusion on two trajectories whose fusion index meets preset fusion conditions, and add the fused trajectory to the trajectory set.

[0047] Finally, server 103 iteratively calculates the fusion index for every two trajectories in the current trajectory set until all fusion indices fail to meet the preset fusion conditions, at which point the current trajectory set is designated as the first target set. Server 103 can then send this first target set to terminal device 101.

[0048] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.

[0049] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0050] Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0051] Server 103 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.

[0052] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0053] Those skilled in the art will know that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0054] Under the above system architecture, this disclosure provides a multi-target trajectory tracking method, which can be executed by any electronic device with computing power.

[0055] In some embodiments, the multi-target trajectory tracking method provided in this disclosure can be executed by a terminal device of the system architecture described above; in other embodiments, the multi-target trajectory tracking method provided in this disclosure can be executed by a server in the system architecture described above; in still other embodiments, the multi-target trajectory tracking method provided in this disclosure can be implemented by the terminal device and the server in the system architecture described above through interaction.

[0056] Figure 2 A flowchart of a multi-target trajectory tracking method according to an embodiment of this disclosure is shown, as follows: Figure 2 As shown, the multi-target trajectory tracking method provided in this embodiment includes the following steps S202 to S208.

[0057] S202, determine the forward time sequence trajectory corresponding to each tracked target through the forward time sequence of multiple frames of data, and determine the reverse time sequence trajectory corresponding to each tracked target through the reverse time sequence of multiple frames of data, wherein any trajectory is used to describe the state value of a tracked target in each frame.

[0058] For example, the multi-frame data can be three-dimensional point cloud data or image data within a continuous time period. This disclosure does not limit the number of frames in the data; for example, the multi-frame data can be 24 frames, 48 ​​frames, etc. The multi-frame data can be used to describe the motion state of multiple tracked targets within a continuous time period. This disclosure does not limit the type and number of tracked targets included in the multi-frame data; for example, in a vehicle driving scenario, the tracked targets can include pedestrians, road signs, zebra crossings, vehicles on the road, etc.

[0059] It should be noted that each detected tracking target corresponds to a forward temporal trajectory and a reverse temporal trajectory. The forward temporal trajectory for any tracking target is obtained by detecting multiple frames of data in the forward playback order. The reverse temporal trajectory for any tracking target is obtained by detecting multiple frames of data in the reverse playback order.

[0060] In some embodiments, determining the forward temporal trajectory corresponding to each tracking target through the forward temporal sequence of multiple frames of data includes: sequentially processing each frame of data according to the forward temporal sequence as follows, until the last frame of data is processed to obtain each forward temporal trajectory:

[0061] Target detection is performed on the current frame data to obtain the detection results of each candidate target; the current frame state value of each tracked target is predicted to obtain the predicted state value; based on the predicted state value and the detection results, the correlation between each tracked target and each candidate target is determined, wherein any group of tracked targets with correlation and candidate targets are the same object; the predicted state value of the corresponding tracked target is updated based on the detection results of the candidate targets with correlation to obtain the current frame state value; based on the forward temporal trajectory of the previous frame and each current frame state value, the forward temporal trajectory of each tracked target in the current frame is obtained.

[0062] In an exemplary embodiment, any frame of data can be detected to determine each candidate target contained in the frame of data, and the detection result of each candidate target can be determined. Exemplarily, the detection result of any candidate target can be represented by a candidate bounding box. Exemplarily, the detection result of any candidate target can include the shape, position, and category obtained by detecting the candidate target. The shape describes the length, width, and height of the candidate target. The position describes the coordinates (X, Y, Z) of the candidate target. The category can be, for example, a car, a bus, a pedestrian, etc.

[0063] In one possible implementation, 3D object detection algorithms can be used to detect objects in the current frame data. By analyzing and processing 3D point cloud or image data, the position, shape, and attributes of each object in space can be identified and located. Specifically, deep learning models such as PointNet (Point Network) and VoxelNet (Voxel Network) can be used to extract features and predict the classification of each object, obtaining a series of candidate object shapes, positions, and corresponding categories.

[0064] In an exemplary embodiment, preprocessing operations such as coordinate transformation and result filtering can be performed on the obtained detection results. Specifically, the coordinate transformation can be used to convert the detection results from the radar coordinate system to the world coordinate system to facilitate subsequent data association.

[0065] In some embodiments, the detection result of any candidate target may include a confidence score. Result filtering can be used to remove candidate targets with low confidence scores from the detection results, ensuring the accuracy of the detection results. In this case, after performing target detection on the current frame data and obtaining the detection results of each candidate target, the multi-target trajectory tracking method provided in this disclosure embodiment may further include: removing each candidate target with a confidence score lower than a preset confidence threshold.

[0066] This disclosure does not limit the value of the preset reliability threshold; the value of the preset reliability threshold can be determined based on experience or application scenarios.

[0067] In an exemplary embodiment, after obtaining the detection results, the current frame state value of each tracked target can be predicted to obtain a predicted state value. Exemplarily, any current frame state value may include shape and position. Furthermore, any current frame state value may also include at least one of velocity, angle, and category.

[0068] In one possible implementation, the uniform motion model in the Kalman filter algorithm can be used to predict the state of each trajectory in the forward temporal trajectory set to obtain the predicted state value. Then, the previous frame state value of each trajectory in the forward temporal trajectory set can be substituted into the state transition equation of the uniform motion model to calculate the trajectory state of the current frame, thereby determining the current frame state value.

[0069] For example, the current frame state value of each trajectory in the forward temporal trajectory set can be obtained by predicting the state value, and the detection result set of the current frame can be obtained by the detection results. Therefore, the predicted state value corresponding to each trajectory and the detection result set can be correlated using a data association algorithm to determine the association relationship.

[0070] In one possible implementation, after determining the predicted state value and the detection result, the association between each tracking target and each candidate target can be determined based on the predicted state value and the detection result. Determining the association between each tracking target and each candidate target may include: calculating the association weight between each tracking target and each candidate target; and matching the tracking targets and candidate targets based on the association weight, wherein each successfully matched group of tracking targets and candidate targets is associated.

[0071] For example, multi-box matching can be performed using an IOU-based data association algorithm. The predicted candidate boxes and the candidate boxes corresponding to the detection results are compared using IOU calculation, and the result is used as the association weight between the two. After obtaining the association weight, the best-matching candidate target can be selected for each tracking target based on the association weight. For example, the Hungarian algorithm or other optimization algorithms can be used to solve for the best matching result. The best matching result is used to indicate each group of tracked targets and candidate targets with an association relationship. Any group of tracked targets and candidate targets with an association relationship can be considered as targets detected from the same object.

[0072] For example, after determining the correlation between each tracked target and each candidate target, the predicted state value of the corresponding tracked target can be updated using the detection results of the candidate targets with correlation, thus obtaining the current frame state value. At this point, the detection results and predicted state values ​​can be fused using the update step in the Kalman filter algorithm, thereby updating the current frame state value. This updated current frame state value can then be used as the final state value of the trajectory at the current frame time. Subsequently, taking any candidate target as an example, the forward temporal trajectory corresponding to the candidate target in the previous frame can be combined with the current frame state value of the candidate target to obtain the forward temporal trajectory of the tracked target in the current frame.

[0073] In some embodiments, the best matching result categorizes the detection result set into two types: one type of detection result can be associated with the predicted state value of a tracked target, while the other type of detection result cannot be associated with the predicted state value of the tracked target. For detection results that cannot be associated with the predicted state value of the tracked target, the multi-target trajectory tracking method provided in this disclosure may further include: if a first candidate target is not associated with any tracked target, then the detection result of the first candidate target is taken as the starting point of a new positive temporal trajectory, wherein the first candidate target is any candidate target.

[0074] For example, if the first candidate target is not associated with any tracked target, the detection result of the first candidate target can be used as the starting point of a new forward temporal trajectory, and this forward temporal trajectory can be added to the aforementioned forward temporal trajectory set. Therefore, embodiments of this disclosure can update the elements of the forward trajectory set according to the association relationship. Similarly, embodiments of this disclosure can also update the elements of the reverse trajectory set.

[0075] In some embodiments, the multi-object trajectory tracking method provided by the embodiments of the present disclosure may further include: if the first tracking target has no association relationship with any candidate target, determining the consecutive number of times that the first tracking target has no association relationship with any candidate target; if the consecutive number of times is not greater than the number threshold, using the predicted state value as the current frame state value of the first tracking target; if the consecutive number of times meets the number threshold, terminating the forward time-series trajectory corresponding to the first tracking target.

[0076] Exemplarily, let the consecutive number of times be n, and the number threshold be N, where both n and N are integers greater than zero. If the first tracking target fails to have an association relationship with any detection result for n consecutive frames of data, determine the magnitude relationship between n and N. If n < N, use the predicted state value of the current frame of the tracking target as the current frame state value of the first tracking target. If n > N, terminate the forward time-series trajectory corresponding to the first tracking target. Therefore, the embodiments of the present disclosure can implement trajectory life cycle management.

[0077] Exemplarily, during the life cycle of a trajectory, if the current frame state value is a predicted state value, it can be marked to serve subsequent trajectory fusion algorithms and trajectory stitching algorithms.

[0078] In an exemplary embodiment, each frame of data can be processed in reverse time sequence until the last frame of data is processed, obtaining each reverse time-series trajectory. It should be noted that the processing method for obtaining the reverse time-series trajectory is the same as that for obtaining the forward time-series trajectory. Exemplarily, by performing the above processing on each frame of data in the forward time sequence order, the forward time-series trajectories corresponding to each tracking target can be obtained. Similarly, by performing the above processing on each frame of data in the reverse time sequence order, the reverse time-series trajectories corresponding to each tracking target can be obtained. The embodiments of the present disclosure will not elaborate on the steps for obtaining the reverse time-series trajectory.

[0079] After obtaining the forward time-series trajectories and reverse time-series trajectories of each candidate target, they can be summarized to obtain the final trajectory set.

[0080] S204, calculating the fusion index between every two trajectories in the trajectory set, where the trajectory set includes each forward time-series trajectory and each reverse time-series trajectory.

[0081] It should be noted that since some trajectories in the forward time-series trajectory and the reverse time-series trajectory of the same candidate target are highly coincident and are essentially the same trajectory, the two trajectories can be fused to make full use of the information of the two trajectories and improve the accuracy of the trajectory. This fusion index is used to describe the coincidence degree of two trajectories. The higher the coincidence degree, the greater the possibility of trajectory fusion.

[0082] In some embodiments, calculating a fusion index between every two trajectories in the trajectory set includes determining the fusion index by using a first IOU weighted average between every two trajectories of the same type in the trajectory set.

[0083] In an exemplary embodiment, each trajectory in the trajectory set can be traversed, and trajectory fusion filtering can be performed based on category determination. Trajectories belonging to different categories cannot be merged. For example, cars and buses belong to different categories; therefore, even though the trajectories of a car and a bus highly overlap, they cannot be merged.

[0084] After filtering during trajectory fusion, trajectories belonging to the same category are obtained in each group. Each group of trajectories can consist of two forward-moving time-series trajectories, two backward-moving time-series trajectories, or one forward-moving time-series trajectory and one backward-moving time-series trajectory. Then, the first IOU weighted average is used to determine whether the trajectories in each group highly overlap. If they highly overlap, trajectory fusion can be performed.

[0085] For example, any trajectory can consist of state values ​​corresponding to multiple frames. The state values ​​of any frame can include information in multiple dimensions such as shape, position, velocity, and angle. Therefore, the information in each dimension of the state values ​​of each frame of any two trajectories can be weighted and averaged to obtain the first IOU weighted average of the trajectories. Then, the first IOU weighted average can be used as the fusion index between the two trajectories.

[0086] In some embodiments, the method provided in this disclosure further includes: determining a first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating the detection results of candidate targets with correlation; and performing a weighted average calculation on the first target frame state values ​​of every two trajectories of the same type to obtain a first IOU weighted average value between every two trajectories of the same type.

[0087] In an exemplary embodiment, if the state value of a certain frame is obtained by directly using the predicted state value as the current frame state value, then the state value data of that frame may not participate in the calculation process of the first IOU weighted average. This embodiment of the present disclosure ensures the accuracy and reliability of the state values ​​and improves the accuracy and reliability of multi-target trajectory tracking by filtering the current frame state values ​​obtained from updates that do not pass the detection results.

[0088] S206, perform trajectory fusion on two trajectories that meet the preset fusion conditions for the fusion index, and add the fused trajectory to the trajectory set.

[0089] In some exemplary embodiments, trajectory fusion can be performed by weighted averaging of the state values ​​of each frame in the trajectory. For example, the state values ​​of two trajectories that meet preset fusion conditions can be weighted and averaged across various dimensions to obtain the state value of the fused trajectory.

[0090] For example, the preset fusion condition can set different fusion index thresholds according to different types of candidate targets. For instance, a lower fusion index threshold can be set for targets with faster movement speeds, such as vehicles, while a higher fusion index threshold can be set for targets with slower movement speeds, such as pedestrians. When the fusion index between two trajectories is not lower than the corresponding type's fusion index threshold, the two trajectories meet the preset fusion condition.

[0091] In an exemplary embodiment, each trajectory may include a unique ID. The ID of the trajectory obtained by fusing two trajectories may be the smaller value of the IDs of the two trajectories.

[0092] S208, iteratively calculate the fusion index of every two trajectories in the current trajectory set until the fusion index no longer meets the preset fusion conditions, and then take the current trajectory set as the first target set.

[0093] For example, after adding the fused trajectory to the trajectory set, the two fused trajectories can be removed from the trajectory set. Then, the fusion index between every two trajectories in the current trajectory set is calculated iteratively, and trajectories of the same type that meet the preset fusion conditions are fused, until all fusion indexes no longer meet the preset fusion conditions, at which point the current trajectory set is taken as the first target set.

[0094] In one possible implementation, the first set of targets can be used as the final result of multi-target trajectory tracking. However, due to factors such as occlusion and lighting conditions, the detection results of some candidate targets may be lost at certain times during target detection, and then reappear at later times. If the detection result of a candidate target is lost for a long time, the trajectory of that candidate target will fail to be associated with the detection result multiple times in a row, thus causing the trajectory to terminate. When the candidate target reappears at a later time, a new trajectory will be generated, resulting in trajectory fragmentation. Therefore, in another possible implementation, trajectory stitching can be performed on a portion of the forward and reverse temporal trajectories in the first set of targets, thereby further reducing the number of fragmented trajectories.

[0095] In some embodiments, the multi-target trajectory tracking method provided in this disclosure may further include: determining each group of trajectories in a first target set, wherein any group of trajectories includes a forward temporal trajectory and a reverse temporal trajectory; calculating the splicing index of each group of trajectories in the first target set; splicing the trajectories of a group of trajectories whose splicing index satisfies a preset splicing condition, and adding the spliced ​​trajectory to the first target set; iteratively calculating the splicing index of each group of trajectories in the current first target set until the splicing index does not satisfy the preset splicing condition, and then using the current first target set as a second target set.

[0096] For example, the trajectories in the first target set can be traversed, and the trajectory splicing can be filtered by category judgment, so that two trajectories belonging to different categories cannot be spliced. Then, the splicing index is calculated for each group of filtered trajectories.

[0097] In some embodiments, any set of trajectories includes a forward-time series trajectory and a reverse-time series trajectory belonging to the same category. In this case, calculating the splicing index of each set of trajectories in the first target set includes: calculating the Euclidean distance between each set of trajectories in the first target set, wherein the Euclidean distance is the Euclidean distance between the termination state values ​​of two trajectories; if the Euclidean distance between the first set of trajectories meets a preset distance condition, then calculating the second IOU weighted average value between the first set of trajectories, wherein the first set of trajectories is any set of trajectories in the first target set; and determining the splicing index by using the second IOU weighted average value and the Euclidean distance.

[0098] In an exemplary embodiment, the Euclidean distance between the termination state values ​​of two trajectories can be calculated first. This termination state value can be the state value corresponding to the last frame of data in which the candidate target corresponding to the trajectory was detected. Furthermore, if the state value of a frame is obtained by directly using the predicted state value as the current frame state value, then the state value data of that frame cannot be used as the termination state value. The termination state value of a trajectory can be the state value obtained after the last update associated with the detection result.

[0099] After determining the Euclidean distance between each group of trajectories, trajectories whose Euclidean distance does not meet a preset distance condition can be filtered. A second IOU weighted average is then calculated for each filtered group of trajectories, and this second IOU weighted average is used as the stitching index. This embodiment of the present disclosure does not limit the preset distance condition; for example, the preset distance condition can be that the Euclidean distance is less than a distance threshold. This embodiment of the present disclosure uses Euclidean distance to filter the trajectories to be stitched, which can reduce the computational load of the second IOU weighted average and avoid incorrectly stitching trajectories belonging to different objects.

[0100] In some embodiments, calculating the second IOU weighted average among the first group of trajectories includes: determining the second target frame state value included in the first group of trajectories, wherein the second target frame state value is a predicted state value; and performing a weighted average calculation on the target frame state values ​​of the first group of trajectories to obtain the second IOU weighted average among the first group of trajectories.

[0101] In an exemplary embodiment, a second IOU weighted average can be calculated using the state value data of each frame in the trajectory where the predicted state value is directly used as the state value. This second IOU weighted average can then be used as a stitching index. Next, trajectories whose stitching index meets preset stitching conditions can be stitched together, with any missing state values ​​taken as the weighted average of the predicted state values ​​of the two trajectories. Finally, the stitched trajectories are added to a first target set, the two stitched trajectories are removed from the first target set, and the stitching index for each group of trajectories in the current first target set is iteratively calculated. Trajectories of the same type that meet the preset stitching conditions are then stitched together until all stitching indices no longer meet the preset stitching conditions. At this point, the current first target set is used as a second target set.

[0102] In an exemplary embodiment, each trajectory may include a unique ID. The ID of the trajectory obtained by concatenating two trajectories may be the smaller value of the IDs of the two trajectories.

[0103] The method provided in this disclosure can acquire forward and reverse temporal trajectories separately, thereby fully utilizing temporal information and improving the accuracy and robustness of multi-target trajectory tracking. Furthermore, this disclosure can perform trajectory fusion by weighted averaging of the state values ​​of the two trajectories, further improving the accuracy of the multi-target tracking algorithm. Additionally, by determining the fusion index, it is possible to further avoid misassociating two trajectories belonging to different targets, thereby improving trajectory tracking accuracy and reliability, and ultimately enhancing the stability and safety of autonomous driving.

[0104] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations. The various types of data, such as personal identity data, operational data, and behavioral data related to individuals, customers, and groups, obtained in the embodiments of this disclosure have all been authorized.

[0105] In an exemplary embodiment, a schematic diagram of a multi-target trajectory tracking process can be as follows: Figure 3 As shown in the figure, 3D target detection is performed on multiple frames of data in both forward and reverse temporal sequences to obtain detection results for each candidate target. The detection results are then preprocessed.

[0106] Furthermore, a set of forward and reverse time-series trajectories can be determined. The state values ​​of each trajectory are predicted to obtain the predicted state values ​​of each tracked target. Then, a data association algorithm can be used to determine the association weights between the tracked target and candidate targets. Based on these association weights, the best-matching candidate target can be selected for each tracked target. Finally, the predicted state values ​​are updated using the detection results to obtain the current frame state value.

[0107] Finally, the trajectory set elements can be updated, and trajectory lifecycle management can be performed. The above steps are iteratively executed until processing is complete, resulting in updated forward and reverse time-series trajectory sets. For example, the forward and reverse time-series trajectory sets can be fused and stitched together to obtain a second target set.

[0108] This disclosure provides an offline multi-target tracking method based on bidirectional correlation, which improves the accuracy and robustness of multi-target tracking algorithms by fully utilizing temporal information. Furthermore, this disclosure proposes trajectory fusion and trajectory stitching algorithms. The trajectory fusion algorithm improves the accuracy of the multi-target tracking algorithm by weighted averaging the state values ​​of two trajectories. The trajectory stitching algorithm reduces fragmented trajectories and improves the robustness of the multi-target tracking algorithm by stitching together forward and reverse temporal trajectories.

[0109] It needs to be clarified that yes, Figure 3 The implementation methods of each step can be found in steps S202 to S08 above, and will not be repeated here.

[0110] In an exemplary embodiment, a schematic diagram of a trajectory fusion process can be as follows: Figure 4 As shown.

[0111] For example, the obtained forward and reverse time-series trajectory sets can be combined to obtain a trajectory set. Then, the trajectory set can be filtered by trajectory category, and a first IOU weighted average can be calculated. The first IOU weighted average can then be used to determine if there are any trajectories that can be fused. If so, trajectory fusion can be performed accordingly, and the trajectory set can be updated. If not, a first target set can be obtained.

[0112] It needs to be clarified that yes, Figure 4 The implementation methods of each step can be found in steps S204 to S08 above, and will not be repeated here.

[0113] In an exemplary embodiment, a schematic diagram of a trajectory stitching process can be as follows: Figure 5 As shown.

[0114] The process involves summing the forward and reverse time-series trajectory sets after trajectory fusion processing to obtain a first target set. Then, the first target set is filtered by trajectory category, and the Euclidean distance and a second IOU weighted average are calculated. Finally, the Euclidean distance and the second IOU weighted average are used to determine if any trajectories can be stitched together. If so, the corresponding trajectories are stitched together, and the first target set is updated. If not, the second target set is obtained.

[0115] It needs to be clarified that yes, Figure 5 The implementation methods for each step can be found in step S08 above, and will not be repeated here.

[0116] Based on the same inventive concept, this disclosure also provides a multi-target trajectory tracking device, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the method embodiments described above, the implementation of this device embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.

[0117] Figure 6 This diagram illustrates a multi-target trajectory tracking device according to an embodiment of the present disclosure, such as... Figure 6 As shown, the device includes:

[0118] The trajectory determination module 601 is used to determine the forward time sequence trajectory corresponding to each tracking target through the forward time sequence of multi-frame data, and to determine the reverse time sequence trajectory corresponding to each tracking target through the reverse time sequence of multi-frame data, wherein any trajectory is used to describe the state value of a tracking target in each frame;

[0119] The fusion index calculation module 602 is used to calculate the fusion index between every two trajectories in the trajectory set, which includes each forward time-series trajectory and each reverse time-series trajectory.

[0120] The trajectory fusion module 603 is used to fuse two trajectories that meet the preset fusion conditions and add the fused trajectory to the trajectory set.

[0121] The first target set determination module 604 is used to iteratively calculate the fusion index of every two trajectories in the current trajectory set until the fusion index does not meet the preset fusion conditions, and then the current trajectory set is taken as the first target set.

[0122] In some embodiments, the trajectory determination module 601 is used to process each frame of data sequentially in a forward temporal order as follows until the last frame of data is processed to obtain each forward temporal trajectory: Target detection is performed on the current frame data to obtain the detection results of each candidate target; the current frame state value of each tracked target is predicted to obtain a predicted state value; based on the predicted state value and the detection results, the association relationship between each tracked target and each candidate target is determined, wherein any group of tracked targets and candidate targets with an association relationship are the same object; the predicted state value of the corresponding tracked target is updated based on the detection results of the candidate targets with an association relationship to obtain the current frame state value; based on the forward temporal trajectory of the previous frame and each current frame state value, the forward temporal trajectory of each tracked target corresponding to the current frame is obtained.

[0123] In some embodiments, the trajectory determination module 601 is further configured to, if the first candidate target is not associated with any tracked target, take the detection result of the first candidate target as the starting point of a new positive temporal trajectory, wherein the first candidate target is any candidate target.

[0124] In some embodiments, the detection result of any candidate target includes a confidence score. The trajectory determination module 601 is further used to eliminate each candidate target whose confidence score is lower than a preset confidence threshold.

[0125] In some embodiments, the trajectory determination module 601 is further configured to: determine the number of consecutive times that the first tracking target is not associated with any candidate target if the first tracking target is not associated with any candidate target; if the number of consecutive times is not greater than the number threshold, use the predicted state value as the current frame state value of the first tracking target; and if the number of consecutive times meets the number threshold, terminate the forward temporal trajectory corresponding to the first tracking target.

[0126] In some embodiments, the fusion index calculation module 602 is used to determine the fusion index by the first IOU weighted average between every two trajectories of the same type in the trajectory set.

[0127] In some embodiments, the fusion index calculation module 602 is further configured to determine the first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating the detection results of candidate targets with correlation; and to calculate the weighted average of the first target frame state values ​​of every two trajectories of the same type to obtain the first IOU weighted average value between every two trajectories of the same type.

[0128] In some embodiments, the multi-target trajectory tracking provided in this disclosure further includes:

[0129] The trajectory stitching module is used to determine each group of trajectories in the first target set, wherein each group of trajectories includes a forward time-series trajectory and a reverse time-series trajectory; calculate the stitching index of each group of trajectories in the first target set; stitch the trajectories of a group of trajectories whose stitching index meets the preset stitching conditions, and add the stitched trajectory to the first target set; iteratively calculate the stitching index of each group of trajectories in the current first target set until the stitching index does not meet the preset stitching conditions, and then use the current first target set as the second target set.

[0130] In some embodiments, any set of trajectories includes a forward temporal trajectory and a reverse temporal trajectory belonging to the same category. The trajectory stitching module is used to calculate the Euclidean distance between each set of trajectories in the first target set, wherein the Euclidean distance is the Euclidean distance between the termination state values ​​of two trajectories; if the Euclidean distance between the first set of trajectories meets a preset distance condition, then the second IOU weighted average value between the first set of trajectories is calculated, wherein the first set of trajectories is any set of trajectories in the first target set; and the stitching index is determined by the second IOU weighted average value and the Euclidean distance.

[0131] In some embodiments, the trajectory stitching module is used to determine the state value of the second target frame included in the first group of trajectories, wherein the state value of the second target frame is a predicted state value; and to perform a weighted average calculation on the state values ​​of the target frames of the first group of trajectories to obtain a second IOU weighted average value among the first group of trajectories.

[0132] The apparatus provided in this disclosure can acquire forward and reverse temporal trajectories separately, thereby fully utilizing temporal information and improving the accuracy and robustness of multi-target trajectory tracking. Furthermore, this disclosure can perform trajectory fusion by weighted averaging of the state values ​​of the two trajectories, further improving the accuracy of the multi-target tracking algorithm. Additionally, by determining the fusion index, it is possible to further avoid misassociating two trajectories belonging to different targets, thereby improving trajectory tracking accuracy and reliability, and ultimately enhancing the stability and safety of autonomous driving.

[0133] It should be noted that the trajectory determination module 601, fusion index calculation module 602, trajectory fusion module 603, and first target set determination module 604 mentioned above correspond to S202 to S208 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0134] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0135] This disclosure provides an electronic device, which, exemplarily, includes a processor and a memory. The memory can be used to store executable instructions of the processor. The processor is configured to execute the aforementioned executable instructions to perform the multi-target trajectory tracking method provided in this disclosure.

[0136] The following reference Figure 7 To describe an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0137] like Figure 7 As shown, the electronic device 700 is presented in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0138] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 710 can perform the following steps of the above method embodiments:

[0139] The forward temporal trajectory corresponding to each tracking target is determined by forward temporal sequence of multi-frame data, and the reverse temporal trajectory corresponding to each tracking target is determined by reverse temporal sequence of multi-frame data. Each trajectory is used to describe the state value of a tracking target in each frame. The fusion index between every two trajectories in the trajectory set is calculated. The trajectory set includes each forward temporal trajectory and each reverse temporal trajectory. Trajectory fusion is performed on two trajectories whose fusion index meets the preset fusion conditions, and the fused trajectory is added to the trajectory set. The fusion index between every two trajectories in the current trajectory set is iteratively calculated until the fusion index no longer meets the preset fusion conditions. The current trajectory set is then used as the first target set.

[0140] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0141] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0142] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0143] Electronic device 700 can also communicate with one or more external devices 740 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0144] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0145] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described multi-target trajectory tracking method.

[0146] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored. When executed by a processor, the computer program can implement the multi-target trajectory tracking method provided in the embodiments of this disclosure. The computer-readable storage medium may be a readable signal medium or a readable storage medium.

[0147] Figure 8 This illustration shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 8 As shown, the computer-readable storage medium 800 stores a program product capable of implementing the methods described above in this disclosure. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0148] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0150] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0151] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0152] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0153] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0154] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0155] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope of this disclosure is indicated by the appended claims.

Claims

1. A multi-target trajectory tracking method, characterized by, The method comprises the following steps: determining a forward time sequence trajectory corresponding to each tracking target through forward time sequence of multi-frame data, and determining a reverse time sequence trajectory corresponding to each tracking target through reverse time sequence of the multi-frame data, wherein any trajectory is used to describe a state value of a tracking target in each frame; calculating a fusion index between each two trajectories in a trajectory set, wherein the trajectory set comprises each forward time sequence trajectory and each reverse time sequence trajectory; performing trajectory fusion on two trajectories whose fusion index satisfies a preset fusion condition, and adding a trajectory obtained by fusion to the trajectory set; iteratively calculating the fusion index between each two trajectories in a current trajectory set until the fusion index does not satisfy the preset fusion condition, and taking the current trajectory set as a first target set.

2. The multi-target trajectory tracking method according to claim 1, characterized in that, The method comprises the following steps: sequentially performing the following processing on each frame data in a forward time sequence until the last frame data is processed, and obtaining each forward time sequence trajectory: performing target detection on the current frame data to obtain a detection result of each candidate target; predicting a current frame state value of each tracking target to obtain a predicted state value; determining an association relationship between each tracking target and each candidate target according to the predicted state value and the detection result, wherein any group of tracking target and candidate target having the association relationship is the same object; updating the predicted state value of the corresponding tracking target through the detection result of the candidate target having the association relationship to obtain a current frame state value; obtaining a current frame forward time sequence trajectory corresponding to each tracking target according to a previous frame forward time sequence trajectory and each current frame state value.

3. The multi-target trajectory tracking method according to claim 2, characterized in that, The method further comprises the following steps: if the first candidate target does not have the association relationship with any tracking target, taking the detection result of the first candidate target as a starting point of a new forward time sequence trajectory, wherein the first candidate target is any candidate target.

4. The multi-target trajectory tracking method according to claim 2 or 3, characterized by, The detection result of any candidate target comprises a confidence score, and after performing target detection on the current frame data to obtain the detection result of each candidate target, the method further comprises the following steps: eliminating each candidate target whose confidence score is lower than a preset confidence threshold.

5. The multi-target trajectory tracking method according to claim 2 or 3, characterized by, The method further comprises the following steps: if the first tracking target does not have the association relationship with any candidate target, determining a continuous number of times that the first tracking target does not have the association relationship with any candidate target; if the continuous number of times is not greater than a number threshold, taking the predicted state value as the current frame state value of the first tracking target; and if the continuous number of times satisfies the number threshold, terminating the forward time sequence trajectory corresponding to the first tracking target.

6. The multi-target trajectory tracking method of claim 1, wherein, The method further comprises the following steps: determining a first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating according to the detection result of the candidate target having the association relationship.

7. The multi-target trajectory tracking method according to claim 6, characterized by, The method further comprises the following steps: determining a first target frame state value included in each trajectory in the trajectory set, wherein the first target frame state value is obtained by updating according to the detection result of the candidate target having the association relationship. The first target frame state values of each two same type trajectories are weighted and averaged to obtain a first Jaccard weighted average value between each two same type trajectories.

8. The multi-target trajectory tracking method of claim 1, wherein, The method further comprises: determining each group of trajectories in the first target set, wherein any group of trajectories comprises one forward time sequence trajectory and one reverse time sequence trajectory; calculating a splicing index of each group of trajectories in the first target set; performing trajectory splicing on a group of trajectories whose splicing index satisfies a preset splicing condition, and adding the spliced trajectory to the first target set; iteratively calculating the splicing index of each group of trajectories in the current first target set until the splicing index of all groups of trajectories does not satisfy the preset splicing condition, and taking the current first target set as a second target set.

9. The multi-target trajectory tracking method according to claim 8, characterized by, Any group of trajectories comprises one forward time sequence trajectory and one reverse time sequence trajectory belonging to the same category; The calculation of the splicing index of each group of trajectories in the first target set comprises: calculating the Euclidean distance between each group of trajectories in the first target set, wherein the Euclidean distance is the Euclidean distance between the terminal state values of two trajectories; if the Euclidean distance between the first group of trajectories satisfies a preset distance condition, calculating a second Jaccard weighted average value between the first group of trajectories, the first group of trajectories being any group of trajectories in the first target set; determining the splicing index through the second Jaccard weighted average value and the Euclidean distance.

10. The multi-target trajectory tracking method of claim 9, wherein, The calculation of the second Jaccard weighted average value between the first group of trajectories comprises: determining the second target frame state value included in the first group of trajectories, wherein the second target frame state value is a predicted state value; performing weighted average calculation on the target frame state values of the first group of trajectories to obtain a second Jaccard weighted average value between the first group of trajectories.

11. A multi-target trajectory tracking apparatus characterized by comprising: comprise: a trajectory determination module configured to determine a forward time sequence trajectory corresponding to each tracking target through the forward time sequence of multiple frames of data, and determine a reverse time sequence trajectory corresponding to each tracking target through the reverse time sequence of the multiple frames of data, wherein any trajectory is used to describe the state value of a tracking target in each frame; a fusion index calculation module configured to calculate a fusion index between each two trajectories in a trajectory set, the trajectory set comprising each forward time sequence trajectory and each reverse time sequence trajectory; a trajectory fusion module configured to perform trajectory fusion on two trajectories whose fusion index satisfies a preset fusion condition, and add the fused trajectory to the trajectory set; a first target set determination module configured to iteratively calculate the fusion index of each two trajectories in a current trajectory set until the fusion index of all trajectories does not satisfy the preset fusion condition, and take the current trajectory set as a first target set.

12. An electronic device, comprising: comprise: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the multi-target trajectory tracking method of any one of claims 1-10 via execution of the executable instructions.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the multi-target trajectory tracking method of any one of claims 1-10. The computer program is executed by the processor to implement the multi-target trajectory tracking method of any one of claims 1-10.

14. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, a processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the multi-target trajectory tracking method according to any one of claims 1-10.

Images