Detection and tracking method, multi-target tracking method, and related devices
By acquiring and updating multi-moment information of the detected target, and using convolutional neural networks for feature decoding and association, the problem of lacking an end-to-end post-detection tracking paradigm in existing technologies is solved, and more accurate and generalized 3D multi-target tracking is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN DEEPROUTE AI CO LTD
- Filing Date
- 2023-08-29
- Publication Date
- 2026-07-24
AI Technical Summary
Existing 3D multi-target tracking methods lack an end-to-end post-detection tracking paradigm, resulting in insufficient generalization.
By acquiring detection information from multiple historical, current, and future moments of the target, feature extraction and updates are performed. Convolutional neural networks are then used for decoding to predict tracking information for future moments and correlate them, thus achieving an end-to-end post-detection tracking paradigm.
It achieves end-to-end 3D multi-target tracking, improving tracking accuracy and generalization ability.
Smart Images
Figure CN117409034B_ABST
Abstract
Description
Technical Field
[0001] The disclosed embodiments of this application relate to the field of three-dimensional multi-target tracking, and more specifically, to post-detection tracking methods, multi-target tracking methods, and related devices. Background Technology
[0002] Existing 3D multi-target tracking methods can be broadly categorized into two paradigms: tracking by detection and joint detection and tracking. Tracking by detection processes detection and tracking independently. Tracking is responsible for temporally associating the target detection results of each frame to form a target trajectory and temporally smoothing the state attributes of the target trajectory. Because tracking by detection is based solely on detected target information, it has stronger generalization capabilities and is easier to deploy. Typically, tracking by detection methods include three modules: prediction, association, and update. The prediction module infers the target state at the next detection moment based on the trajectory information formed by historical targets for each tracking trajectory. The association module pairs the predicted targets of multiple tracking trajectories with multiple detected targets at the current detection moment in a one-to-one manner. The update module updates the current tracking target state based on the matching results of the tracking trajectories and manages the lifecycle of trajectories, including adding and removing targets. Currently, end-to-end tracking by detection multi-target tracking methods are still lacking. Summary of the Invention
[0003] According to embodiments of this application, this application proposes a post-detection tracking method, a multi-target tracking method, and related equipment to solve the above-mentioned problems.
[0004] The first aspect of this application discloses a post-detection tracking method, comprising: acquiring multiple historical moment detection information, current moment detection information, and future moment detection information of a detected target, wherein the multiple historical moment detection information and the current moment detection information are used to form the current moment trajectory of the detected target; extracting features from the current moment trajectory of the detected target to obtain trajectory encoding features; updating and predicting the trajectory encoding features to obtain current moment updated tracking information of the detected target and prediction information within a preset future time period; and associating the future moment detection information and the trajectory encoding features to obtain the degree of association between the future moment detection information and the current moment trajectory of the detected target, wherein the degree of association is used to generate future moment updated tracking information.
[0005] In some embodiments, the plurality of historical moment detection information and the current moment detection information are concatenated to form the current moment trajectory of the detected target; feature extraction is performed on the current moment trajectory of the detected target to obtain trajectory encoding features, including: performing vector transformation on the plurality of historical moment detection information and the current moment detection information respectively to obtain a plurality of historical moment spatial vector representations and a current moment spatial vector representation; and abstracting the trajectory encoding features from the plurality of historical moment spatial vector representations and the current moment spatial vector representation.
[0006] In some embodiments, the step of performing vector transformation on the plurality of historical moment detection information and the current moment detection information to obtain a plurality of historical moment spatial vector representations and a current moment spatial vector representation includes: converting the plurality of historical moment detection information into a plurality of historical moment high-dimensional spatial vector representations, and converting the current moment detection information into a current moment high-dimensional spatial vector representation; adding the timestamp information corresponding to the plurality of historical moment detection information to the plurality of historical moment high-dimensional spatial vector representations to obtain the plurality of historical moment spatial vector representations, and adding the timestamp information corresponding to the current moment detection information to the current moment high-dimensional spatial vector representation to obtain the current moment spatial vector representation.
[0007] In some embodiments, updating and predicting the trajectory encoding features to obtain the current updated tracking information of the detected target and the prediction information within a future preset time period includes: decoding the trajectory encoding features using a convolutional neural network to obtain the current updated tracking information of the detected target and the prediction information within the future preset time period, wherein the future preset time period is adjacent to the current time.
[0008] In some embodiments, associating the future time-time detection information and the trajectory encoding features to obtain the degree of association between the future time-time detection information and the current time trajectory of the detected target includes: performing vector transformation on the future time-time detection information to obtain a future time-time spatial vector representation; performing feature interaction and feature extraction on the trajectory encoding features and the future time-time spatial vector representation to obtain a final feature; and performing preset processing on the final feature to obtain a matching score between the future time-time detection information and the current time trajectory, which is used as the degree of association.
[0009] In some embodiments, the step of vectorizing the future time detection information to obtain a future time spatial vector representation includes: converting the future time detection information into a future time high-dimensional spatial vector representation, wherein the future time high-dimensional spatial vector representation is used as the future time spatial vector representation.
[0010] In some embodiments, the step of performing preset processing on the final feature to obtain the matching score between the future time detection information and the current time trajectory of the detected target includes: converting the final feature to a preset dimension and processing it through preset rules to obtain the matching score between the future time detection information and the current time trajectory.
[0011] The second aspect of this application discloses a multi-target tracking method, comprising: identifying multiple targets; acquiring tracking information of each of the multiple targets to achieve multi-target tracking; wherein the tracking information of each target is obtained by the post-detection tracking method as described in the first aspect.
[0012] A third aspect of this application discloses an electronic device including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the post-detection tracking method described in the first aspect, or to implement the multi-target tracking method described in the second aspect.
[0013] The fourth aspect of this application discloses a non-volatile computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the post-detection tracking method described in the first aspect, or implement the multi-target tracking method described in the second aspect.
[0014] The beneficial effects of this application are as follows: It acquires multiple historical detection information, current detection information, and future detection information of the target. The multiple historical and current detection information are used to form the target's current trajectory. Features are extracted from the target's current trajectory to obtain trajectory encoding features. By updating and predicting these trajectory encoding features, updated tracking information and predicted information for a future preset time period are obtained. Furthermore, the future detection information and trajectory encoding features are correlated to obtain the degree of correlation between the future detection information and the target's current trajectory. This correlation degree is used to generate updated tracking information for the future. By placing the three modules—updating, predicting, and correlating—in the same network and sharing the same target trajectory encoding features, an end-to-end post-detection tracking paradigm for three-dimensional multi-target tracking is achieved. Attached Figure Description
[0015] The present application will be further described below with reference to the accompanying drawings and embodiments. In the drawings:
[0016] Figure 1 This is a schematic flowchart of the post-detection tracking method according to an embodiment of this application;
[0017] Figure 2This is a schematic diagram of the post-detection tracking process according to an embodiment of this application;
[0018] Figure 3 This is a flowchart illustrating the multi-target tracking method according to an embodiment of this application;
[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;
[0020] Figure 5 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. Detailed Implementation
[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0022] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this application means two or more. Moreover, the term "at least one" in this application means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Furthermore, the terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.
[0023] To enable those skilled in the art to better understand the technical solution of this application, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating the post-detection tracking method according to an embodiment of this application. The subject executing this method can be an electronic device with computing capabilities, such as a microcomputer, a server, and mobile devices such as laptops and tablets.
[0025] It should be noted that if substantially the same result is obtained, the method of this application is not based on... Figure 1 The sequence of processes shown is limited.
[0026] In some possible implementations, this method can be implemented by the processor calling computer-readable instructions stored in memory, such as... Figure 1 As shown, the method may include the following steps:
[0027] S11: Obtain multiple historical detection information, current detection information, and future detection information of the target, wherein the multiple historical detection information and current detection information are used to form the current trajectory of the target.
[0028] The system acquires detection information from multiple historical moments, the current moment, and future moments for a target. For example, the current moment is time t, the historical moments are from time tm to time t-1, and the future detection moment is time t+i. The detection information from multiple historical moments can be the detection information from time tm to time t-1 for at least one target. The detection information at the current moment is the detection information at time t for at least one target. The detection information at the future moment is the detection information at time t+i. The target corresponding to the detection information at the future moment can be the target corresponding to the detection information at the current moment, or it can be another target.
[0029] The detection information includes bounding box information of the detected target. The bounding box information includes 3D position, scale, orientation, detection confidence score, target features extracted by the detection module, timestamp, etc. The confidence score, with a value between 0 and 1, is used to determine the probability of correctly predicting the target. The higher the value, the greater the probability of correctly predicting the target.
[0030] Among them, multiple historical detection information and current detection information are used to form the current trajectory of the detection target. For example, if the detection target is vehicle A, the current trajectory of vehicle A can be formed by the detection information of vehicle A from time tm to time t.
[0031] S12: Extract features from the current trajectory of the detected target to obtain trajectory encoding features.
[0032] Feature extraction is performed on the current trajectory of the detected target, that is, feature extraction is performed on the detection information of the detected target from time tm to time t. For example, the spatiotemporal features of the detection information of the detected target at each time from time tm to time t can be extracted, and then the trajectory coding features can be obtained.
[0033] S13: Update and predict the trajectory encoding features to obtain the current updated tracking information of the detected target and the predicted information within a preset time period.
[0034] By extracting features from the current trajectory of the detected target, trajectory encoding features are obtained. Based on the trajectory encoding features, updates and predictions are made to obtain the current updated tracking information of the detected target and the prediction information within a preset time period. For example, the detection information of the detected target at time t is updated to the tracking information updated at time t, and the detection information of the detected target from time t+1 to time t+n is predicted. The tracking information updated at time t can be used as the tracking output target, and the detection information of the detected target from time t+1 to time t+n is the predicted target.
[0035] S14: Correlate the detection information and trajectory coding features at future time moments to obtain the degree of correlation between the detection information at future time moments and the trajectory of the detected target at the current time moment. The degree of correlation is used to generate the updated tracking information at future time moments.
[0036] The detection information at future time points is correlated with the trajectory encoding features. For example, the detection information at time t+i is correlated with the trajectory encoding features to obtain the degree of correlation between the detection information at future time points and the trajectory of the detected target at the current time. That is, it is to determine whether the trajectory at the current time t is associated with the detected target at the future time t+i. For example, it is to determine whether the obstacle corresponding to the trajectory at the current time and the obstacle corresponding to the detected target at time t+i are the same obstacle. The degree of correlation can be represented by a confidence score. The degree of correlation is used to generate updated tracking information at future time points. For example, for a detected target that is successfully matched at time t+i, the trajectory will acquire the target and concatenate the trajectory at the current time t with the detected target at time t+i. That is, the trajectory will extend to time t+i and be updated at time t+i to generate the tracking output target at time t+i. This target is then matched with a time further in the future than time t+i, and so on. The length of the trajectory increases with time.
[0037] In this embodiment, multiple historical detection information, current detection information, and future detection information of the target are acquired. The multiple historical and current detection information are used to form the current trajectory of the target. Feature extraction is performed on the current trajectory of the target to obtain trajectory encoding features. By updating and predicting the trajectory encoding features, the updated tracking information of the target at the current time and the predicted information within a preset time period are obtained. Furthermore, the future detection information and trajectory encoding features are correlated to obtain the correlation degree between the future detection information and the current trajectory of the target. The correlation degree is used to generate the updated tracking information at the future time. By placing the three modules of the post-detection tracking paradigm—updating, predicting, and correlating—in the same network and sharing the same target trajectory encoding features, a three-dimensional multi-target tracking paradigm of end-to-end post-detection tracking is realized.
[0038] In some embodiments, multiple historical detection information and current detection information are concatenated to form the current trajectory of the detected target.
[0039] For example, if the current time is time t and the historical time is from time tm to time t-1, the trajectory of the detected target at time t can be obtained by concatenating the bounding box information from time tm to time t.
[0040] Furthermore, feature extraction is performed on the current trajectory of the detected target to obtain trajectory encoding features, including: vector transformation of detection information at multiple historical moments and detection information at the current moment to obtain spatial vector representations at multiple historical moments and spatial vector representations at the current moment; and abstracting trajectory encoding features from the spatial vector representations at multiple historical moments and spatial vector representations at the current moment.
[0041] Vector transformations are performed on the detection information from multiple historical moments and the detection information from the current moment to obtain spatial vector representations of multiple historical moments and the current moment. For example, the detection information corresponding to each moment from time tm to time t is transformed into a spatial vector representation. Then, trajectory encoding features are abstracted from the spatial vector representations of multiple historical moments and the current moment, that is, the temporal interaction features of the input trajectory are abstracted from the spatial vector representations of the detected targets at different moments as trajectory encoding features.
[0042] In some embodiments, vector transformation is performed on multiple historical moment detection information and current moment detection information to obtain multiple historical moment spatial vector representations and current moment spatial vector representations, including: converting multiple historical moment detection information into multiple historical moment high-dimensional spatial vector representations, and converting current moment detection information into current moment high-dimensional spatial vector representations; adding the timestamp information corresponding to multiple historical moment detection information to the multiple historical moment high-dimensional spatial vector representations to obtain multiple historical moment spatial vector representations, and adding the timestamp information corresponding to the current moment detection information to the current moment high-dimensional spatial vector representation to obtain the current moment spatial vector representation.
[0043] Continuing with the example using the current time as time t, the detection information from multiple historical times is transformed into a high-dimensional spatial vector representation of those historical times, and the detection information from the current time is also transformed into a high-dimensional spatial vector representation of the current time. In other words, the target detection information corresponding to each time point from time tm to time t is transformed into a high-dimensional spatial vector representation. Further, the timestamp information corresponding to the detection information from multiple historical times is added to the high-dimensional spatial vector representations of those historical times, resulting in multiple historical time spatial vector representations. Similarly, the timestamp information corresponding to the detection information from the current time is added to the high-dimensional spatial vector representation of the current time, resulting in the spatial vector representation of the current time. In other words, by adding the timestamp information corresponding to the target detection at different times to the high-dimensional spatial vector representations, we obtain multiple historical time spatial vector representations and the current time spatial vector representation.
[0044] In some embodiments, updating and predicting the trajectory encoding features to obtain the current updated tracking information of the detected target and the prediction information within a preset future time period includes: decoding the trajectory encoding features using a convolutional neural network to obtain the current updated tracking information of the detected target and the prediction information within a preset future time period, wherein the preset future time period is adjacent to the current time.
[0045] Continuing with the example of time t, we use a convolutional neural network to decode the trajectory encoding features. Specifically, we use a convolutional neural network to decode the trajectory encoding from time tm to time t, obtaining the updated tracking information of the target at the current time and the prediction information within a preset future time period. The preset future time period is adjacent to the current time. For example, we obtain the updated tracking information of the target at time t and the detection information of the target from time t+1 to time t+n, where time t+1 to time t+n is adjacent to time t.
[0046] In some embodiments, the detection information at future time moments and the trajectory encoding features are correlated to obtain the degree of correlation between the detection information at future time moments and the trajectory of the detected target at the current time moment. This includes: converting the detection information at future time moments into vectors to obtain a spatial vector representation of the future time moments; performing feature interaction and feature extraction on the trajectory encoding features and the spatial vector representation of the future time moments to obtain the final features; and performing preset processing on the final features to obtain a matching score between the detection information at future time moments and the trajectory at the current time moment, which is used as the degree of correlation.
[0047] Taking time t+i as an example, the detection information at the future time is transformed into a vector representation to obtain a spatial vector representation of the future time. For example, the detection information at time t+i is transformed into a vector representation to obtain a spatial vector representation at time t+i. Feature interaction and feature extraction are performed on the trajectory encoding features and the spatial vector representation at the future time to obtain the final features. For example, feature interaction and feature extraction are performed on the trajectory encoding from time tm to time t and the spatial vector representation at time t+i to obtain the final features. Then, the final features are pre-processed to obtain a matching score between the detection information at the future time and the trajectory at the current time, which is used as a measure of the degree of association. That is, it determines whether the trajectory at the current time t is associated with the detection target at the future time t+i. For example, it determines whether the obstacle corresponding to the trajectory at the current time and the obstacle corresponding to the detection target at time t+i are the same obstacle. The degree of association can be represented by the matching score, which represents the confidence score of the detection target.
[0048] In some embodiments, the detection information at future time is vectorized to obtain a spatial vector representation at future time, including: converting the detection information at future time into a high-dimensional spatial vector representation at future time, and using the high-dimensional spatial vector representation at future time as the spatial vector representation at future time.
[0049] Continuing with the example of time t+i, the detection information at the future time is transformed into a high-dimensional spatial vector representation of the future time, and this high-dimensional spatial vector representation of the future time is used as the spatial vector representation of the future time. In other words, the detection information at time t+i is transformed into a high-dimensional spatial vector representation of time t+i, and this high-dimensional spatial vector representation of time t+i is used as the spatial vector representation of time t+i.
[0050] In some embodiments, the final features are subjected to preset processing to obtain a matching score between the detection information at a future time and the trajectory at the current time of the detected target. This includes: converting the final features to a preset dimension and processing them according to preset rules to obtain a matching score between the detection information at a future time and the trajectory at the current time.
[0051] Understandably, for example, by performing feature interaction and feature extraction on the trajectory encoding from time tm to time t and the spatial vector representation at time t+i, the final features are obtained. The final features are then transformed to a preset dimension and processed by preset rules to obtain the matching score between the detection information at future time and the trajectory at the current time. For example, the output is limited to a probability value range of 0.0 to 1.0, thereby obtaining the matching score between a certain detection target at time t+i and a certain trajectory at time t.
[0052] To facilitate understanding, we will use a deep neural network (DNN) as an example. Figure 2 As shown, Figure 2This is a schematic diagram of the post-detection tracking process according to an embodiment of this application. The deep neural network model may include an input module, an encoder module, an update and predict decoder module, an associated decoder module, and an output module.
[0053] In the input module, the acquired detection information from multiple historical moments, the current moment, and the future moment can be used as input, such as... Figure 2 As shown, the input can be a trajectory containing 3D bounding box information Det_t-m……Det_t provided by the detection module at different times. The values between 0 and 1 above the bounding box represent the confidence score of the detected target, and Det_t is below the bounding box. Det represents detection, and t represents the timestamp of the detected target.
[0054] In the encoder module, features are extracted from the current trajectory of the detected target to obtain trajectory encoding features. For example, the Input Embedding layer transforms the detected target information at each time step into a high-dimensional spatial vector representation, and the Position Encoding layer adds the timestamp information of the detected target at different times to the high-dimensional spatial vector representation. M cascaded Transformer Encoders abstract the trajectory encoding features (Encoder Features) of the current time step from the high-dimensional spatial vector representations of the detected target at different times.
[0055] In the update prediction decoder module, a convolutional neural network is used to decode the trajectory encoding features to obtain the current updated tracking information of the detected target and the prediction information within a preset future time period. The preset future time period is adjacent to the current time. For example, a convolutional neural network layer (CNN Decoder) is used to decode the trajectory encoding features and output the tracked output target after the update at time t and the predicted target from time t+1 to the farthest time t+n. The target updated at the current time t is Pd_t in the figure, where Pd represents prediction and t is the timestamp. The predicted targets at future times are Pd_t+1...Pd_t+n in the figure.
[0056] Understandably, the detection module detects the target at time t. At this time, the target's position, scale, orientation, and other attributes are subject to noise and jitter. Furthermore, it lacks attributes such as velocity. Therefore, the tracking module needs to combine the current detected target with the historical detected target information in the trajectory to output the target of the current frame after temporal smoothing. This makes the position, scale, orientation, and other attributes smoother and more accurate, and also provides information such as the target's velocity, i.e., updating the tracking information.
[0057] In the associated decoder module, the detection information at future time moments is vectorized to obtain a spatial vector representation of the future time moments. Feature interaction and feature extraction are performed on the trajectory encoding features and the spatial vector representation of the future time moments to obtain the final features. These final features undergo pre-processing to obtain a matching score between the detection information at future time moments and the trajectory at the current time moment, which serves as the degree of association. Figure 2 As shown, the Input Embedding layer transforms the target information detected at time t+i into a high-dimensional spatial vector representation. N concatenated Transformer Decoders further interact and extract features from the trajectory encoding features and the high-dimensional spatial vectors extracted from the target to be associated. The Linear layer transforms the features to a specified dimension, and the Softmax layer limits the final output to a probability value range of 0.0 to 1.0, outputting the matching score of a certain target at time t+i with a certain trajectory at time t, thus determining whether a certain target at time t+i can be matched with a certain trajectory at time t.
[0058] The output module outputs the tracking output target at time t, which is updated by the update prediction decoder module, and the one-to-one matching results of multiple tracking trajectories at time t and multiple detection targets at time t+i, obtained by matching optimization method using the matching scores of the tracking trajectory and the detection target obtained by the association decoder module.
[0059] Please see Figure 3 , Figure 3 This is a flowchart illustrating a multi-target tracking method according to an embodiment of this application. This method can be applied to vehicle-mounted devices with computational functions. It should be noted that if substantially the same result is obtained, the method of this application does not necessarily require further clarification. Figure 3 The sequence of processes shown is limited.
[0060] In some possible implementations, this method can be implemented by the processor calling computer-readable instructions stored in memory, such as... Figure 3 As shown, the method may include the following steps:
[0061] S31: Identify multiple objectives.
[0062] Identify multiple targets, which can be vehicles, pedestrians, or obstacles.
[0063] S32: Obtain tracking information for each of the multiple targets to achieve multi-target tracking.
[0064] The tracking information for each target is obtained through the post-detection tracking method described above. This includes acquiring multiple historical detection information, current detection information, and future detection information for the target. The historical and current detection information are used to form the target's current trajectory. Features are extracted from the target's current trajectory to obtain trajectory encoding features. The trajectory encoding features are updated and predicted to obtain the target's updated tracking information for the current time and predicted information for a preset future time period. The future detection information and trajectory encoding features are correlated to obtain the correlation degree between the future detection information and the target's current trajectory. This correlation degree is used to generate future updated tracking information, thereby achieving multi-target tracking, such as multiple different obstacles, with each obstacle corresponding to a trajectory.
[0065] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.
[0066] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other. The processor 42 is used to execute program instructions stored in the memory 41 to implement the steps of the above-described post-detection tracking method embodiment, or to implement the steps of the above-described multi-target tracking method embodiment. In a specific implementation scenario, the electronic device 40 may include, but is not limited to, a microcomputer or a server.
[0067] Specifically, processor 42 controls itself and memory 41 to implement the steps of the above-described post-detection tracking method embodiment, or to implement the steps of the above-described multi-target tracking method embodiment. Processor 42 can also be called a CPU (Central Processing Unit), and may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.
[0068] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. The non-volatile computer-readable storage medium 50 is used to store a computer program 501. When the computer program 501 is executed by a processor, for example by the aforementioned... Figure 4 When the processor 42 in the embodiment is executed, it is used to implement the steps of the above-described post-detection tracking method embodiment, or to implement the steps of the above-described multi-target tracking method embodiment.
[0069] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0070] In the several embodiments provided in this application, it should be understood that the disclosed methods and related devices can be implemented in other ways. For example, the related device implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication disconnection shown or discussed may be indirect coupling or communication disconnection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0071] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0072] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0073] Those skilled in the art will readily recognize that numerous modifications and variations can be made to the apparatus and method while maintaining the teachings of this application. Therefore, the above disclosure should be considered limited only by the scope of the appended claims.
Claims
1. A post-detection tracking method, characterized in that, include: The detection information includes multiple historical time detection information, current time detection information, and future time detection information of the target. The multiple historical time detection information and the current time detection information are used to form the current time trajectory of the target. The current time is time t, the multiple historical time includes time tm to time t-1, the future time is time t+i, and the target corresponding to the future time detection information includes the target corresponding to the current time detection information and other targets. Feature extraction is performed on the current trajectory of the detected target to obtain trajectory encoding features; The trajectory encoding features are updated and predicted to obtain the current updated tracking information of the detected target and the predicted information for a future preset time period. The future time detection information and the trajectory encoding features are correlated to obtain the correlation degree between the future time detection information and the current time trajectory of the detected target. The correlation degree is used to generate updated tracking information for future time, so as to extend the current time trajectory to time t+i according to the correlation degree, and update it at time t+i. The correlation degree represents the matching result of multiple tracking trajectories at time t and multiple detected targets at time t+i.
2. The method according to claim 1, characterized in that, The detection information from multiple historical moments and the detection information from the current moment are concatenated to form the current moment trajectory of the detected target; Feature extraction is performed on the current trajectory of the detected target to obtain trajectory encoding features, including: The detection information at multiple historical moments and the detection information at the current moment are respectively transformed into vectors to obtain spatial vector representations of multiple historical moments and spatial vector representations of the current moment. The trajectory encoding features are abstracted from the spatial vector representations of the multiple historical moments and the spatial vector representation of the current moment.
3. The method according to claim 2, characterized in that, The step of performing vector transformation on the multiple historical moment detection information and the current moment detection information to obtain multiple historical moment spatial vector representations and the current moment spatial vector representation includes: The detection information at multiple historical moments is transformed into a high-dimensional space vector representation of multiple historical moments, and the detection information at the current moment is transformed into a high-dimensional space vector representation of the current moment; The timestamp information corresponding to the detection information of the multiple historical moments is added to the high-dimensional space vector representation of the multiple historical moments to obtain the space vector representation of the multiple historical moments. The timestamp information corresponding to the detection information of the current moment is added to the high-dimensional space vector representation of the current moment to obtain the space vector representation of the current moment.
4. The method according to claim 1, characterized in that, The step of updating and predicting the trajectory encoding features to obtain the current updated tracking information of the detected target and the predicted information for a future preset time period includes: The trajectory encoding features are decoded using a convolutional neural network to obtain the current updated tracking information of the detected target and the prediction information within the future preset time period, wherein the future preset time period is adjacent to the current time.
5. The method according to claim 1, characterized in that, The step of associating the future time-time detection information and the trajectory encoding features to obtain the degree of correlation between the future time-time detection information and the current time trajectory of the detected target includes: The detected information at future time is transformed into a vector to obtain a spatial vector representation of the future time. The trajectory encoding features and the future time spatial vector representation are subjected to feature interaction and feature extraction to obtain the final features; The final features are subjected to preset processing to obtain the matching score between the future time detection information and the current time trajectory, which is used as the degree of correlation.
6. The method according to claim 5, characterized in that, The step of transforming the future time-time detection information into a vector representation to obtain a future time-time spatial vector representation includes: The future time detection information is transformed into a high-dimensional space vector representation of the future time, and the high-dimensional space vector representation of the future time is used as the space vector representation of the future time.
7. The method according to claim 5, characterized in that, The step of performing preset processing on the final features to obtain the matching score between the future time-time detection information and the current time trajectory of the detected target includes: The final features are transformed to a preset dimension and processed by preset rules to obtain the matching score between the future moment detection information and the current moment trajectory.
8. A multi-target tracking method, characterized in that, include: Define multiple objectives; The tracking information of each of the multiple targets is obtained to achieve multi-target tracking; The tracking information for each target is obtained by the post-detection tracking method as described in any one of claims 1 to 7.
9. An electronic device, characterized in that, It includes a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the post-detection tracking method of any one of claims 1 to 7, or to implement the multi-target tracking method of claim 8.
10. A non-volatile computer-readable storage medium storing program instructions thereon, characterized in that, When the program instructions are executed by the processor, they implement the post-detection tracking method according to any one of claims 1 to 7, or the multi-target tracking method according to claim 8.