Robust identification tracking method and system for anti-tracking target

Abstract

The invention relates to the technical field of computer vision, and provides a robust recognition tracking method and system for an anti-tracking target, and the method comprises the steps: extracting figure sequence information from a to-be-detected image; performing feature extraction on the given information of the task target based on a multi-modal feature extraction network to obtain task target features; performing feature extraction on a to-be-detected image by using the multi-modal feature extraction network based on the character sequence information to obtain character sequence information features; performing similarity matching on the task target features and the character sequence information features, if matching succeeds, locking a task target from the character sequence information, and if matching fails, regenerating the features of the task target based on a generation model through historical information of the task target and performing similarity matching again, and the task target is locked. The invention provides an effective tracking method for an anti-tracking target.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a robust identification and tracking method and system for an anti-tracking target. Background Art

[0002] In the field of computer vision, human target recognition and tracking technology has become one of the core technologies for applications such as dynamic monitoring of public places, traffic pedestrian detection, and security control. This technology can accurately lock and track specific human targets through deep image target recognition, providing strong support for various application scenarios. However, with the in-depth application of technology, the challenges it faces are becoming increasingly prominent. At present, mainstream target recognition technologies mainly rely on deep image target detection and feature matching recognition methods. These technologies first perform human body detection and image capture on the images collected by the sensor, and then compare them with the preset target reference image or multi-modal target reference information to achieve stable target recognition. After identifying and locking the target, advanced algorithms such as Kalman filtering, feature point tracking, and particle filtering are used to predict the future displacement state of the target based on the historical and current position information of the target in the image sequence, or to use the consistency of the target's features in different frame images to establish a matching relationship between feature points, thereby achieving accurate target position and velocity vector estimation, and ultimately achieving the purpose of continuous tracking of the target.

[0003] Although existing recognition and tracking technologies have shown good performance in many scenarios, they still have significant deficiencies in dealing with changes in target state. In particular, when the tracked person makes intentional or unintentional state changes such as changing clothes, camouflage, or hiding for a short period of time, these technologies are often difficult to cope with, resulting in the inability to continuously and effectively identify and track the target. Summary of the invention

[0004] In order to overcome the defect of the prior art that a target cannot be effectively identified after anti-tracking behavior occurs, the present invention provides a robust identification and tracking method and system for anti-tracking targets.

[0005] In order to achieve the above technical effects, the technical solution of the present invention is as follows: The present invention proposes a robust identification and tracking method for an anti-tracking target, comprising the following steps: Extracting character sequence information from the image to be detected; Based on the multimodal feature extraction network, feature extraction is performed on the given information of the task target to obtain the task target feature; based on the character sequence information, the multimodal feature extraction network is used to extract the feature of the image to be detected to obtain the character sequence information feature; The task target features and the character sequence information features are matched for similarity. If the match is successful, the task target is locked from the character sequence information. If the match fails, the features of the task target are regenerated based on the generation model through the historical information of the task target and similarity matching is performed again until the task target is locked.

[0006] The present invention also proposes a robust identification and tracking system for an anti-tracking target, the system comprising: Feature extraction module: It is equipped with a multimodal feature extraction network to extract features of task objectives and object sequence information; Similarity matching module: used to calculate the similarity and compare the task target features and character sequence information features; Feature reconstruction module: It is equipped with a generation model, which is used to regenerate the features of the task target based on the generation model and re-match the similarity when the similarity matching module fails to match until the task target is locked.

[0007] Compared with the prior art, the present invention has the following beneficial effects: This scheme is based on the robust target re-identification and tracking of the generative model. The historical information of the task target is used as training data. The generative model is trained and the target features after changes at the current moment are predicted and generated, thereby achieving re-identification and continuous tracking of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 The flowchart of a robust recognition and tracking method for anti-tracking targets is shown in FIG.

[0009] Figure 2 Architecture diagram of the multimodal feature extraction network.

[0010] Figure 3 This is the architecture diagram of the feature encoder.

[0011] Figure 4 Schematic diagram of the input and output of the multimodal feature extraction network.

[0012] Figure 5 This is the structural diagram of the variational autoencoder.

[0013] Figure 6 This is a simulation result diagram of a robust recognition and tracking method for anti-tracking targets to continuously track and identify targets that have changed their outfits and are obscured.

[0014] Figure 7 This is the simulation result of the traditional Kalman filter tracking method losing the target of refitting and obstruction.

[0015] Figure 8 The architecture diagram of a robust recognition and tracking system for anti-tracking targets. DETAILED DESCRIPTION

[0016] The accompanying drawings are only used for illustrative purposes and are not to be construed as limiting the present invention; It is understandable to those skilled in the art that some well-known descriptions may be omitted in the drawings.

[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0018] Example 1 This embodiment proposes a robust identification and tracking method for anti-tracking targets, such as Figure 1 , which is a flow chart of a robust identification and tracking method for an anti-tracking target according to this embodiment.

[0019] The present embodiment proposes a robust identification and tracking method for an anti-tracking target, comprising the following steps: Extracting character sequence information from the image to be detected; Based on the multimodal feature extraction network, feature extraction is performed on the given information of the task target to obtain the task target feature; based on the character sequence information, the multimodal feature extraction network is used to extract the feature of the image to be detected to obtain the character sequence information feature; The task target features and the character sequence information features are matched for similarity. If the match is successful, the task target is locked from the character sequence information. If the match fails, the features of the task target are regenerated based on the generation model through the historical information of the task target and similarity matching is performed again until the task target is locked.

[0020] In this embodiment, a person is first detected from the image currently being detected and the person sequence information is formed. Then, a multimodal feature network is used to extract the given task target information, such as pictures, text audio, etc., to obtain the features of the task target. Then, the image to be detected is extracted based on the person sequence to obtain the features of the person sequence information, and similarity matching is performed with the task target features. If the match is successful, the task target is locked. If the match fails, it means that the task target has anti-tracking behavior. The features of the task target are regenerated through the generation model and similarity matching is performed again until the task target is locked. If the feature similarity matching generated by the generation model still fails, it proves that the task target is not in the current detection image, and the robust recognition and tracking method for the anti-tracking target of this embodiment is re-performed on the next frame of the image.

[0021] By adopting a multimodal feature network, this scheme can effectively extract features from different types of task target information, thereby enhancing the flexibility and accuracy of feature extraction. When matching fails, this scheme regenerates the features of the task target through a generative model and re-matches the similarity until the task target is locked. This dynamic update and retry mechanism ensures the robustness of this scheme for tracking.

[0022] In an optional embodiment, the step of extracting character sequence information from the image to be detected includes: performing character body detection and segmentation on the current frame image acquired by a target detection algorithm to obtain the detected character sequence information.

[0023] As an example, the YOLO target detection algorithm is used to detect and segment the human figures in the current frame image to obtain the detected human figure set. ,in , , N is the number of detected people, For the detected i Character images, For the detected i The position information of a person image.

[0024] In this embodiment, the target detection algorithm can quickly identify multiple persons from the image currently being detected, and form person sequence information, providing accurate basic data for subsequent feature extraction and matching.

[0025] Secondly, the multimodal information of the task target is input into the multimodal feature extraction network to obtain the implicit feature vector ,in D is the feature dimension; the target image in the target sequence to be tested Input the multimodal feature extraction network to obtain a set of implicit image feature vector matrices .

[0026] In this embodiment, a multimodal feature extraction network is used to capture multiple modal information of the task target, and the generated feature vector can more comprehensively and accurately characterize the characteristics of the task target and character sequence, providing a basis for subsequent data analysis, recognition, classification, etc.

[0027] Further optionally, the step of extracting features from given information of the task target includes: Inputting the given information of the task target into the multimodal feature extraction network, encoding different types of information in the given information of the task target through different embedding codes to obtain an embedding vector; The embedding vector is input into the feature encoder in the multimodal feature extraction network, and the features of different modalities are mapped to a unified vector space to obtain the task target features.

[0028] like Figure 2 As shown, this is the architecture diagram of the multimodal feature extraction network.

[0029] Exemplarily, different embedded codes are used for the input information of pictures, texts and audios to pre-process the heterogeneous data.

[0030] Image preprocessing: The input image is divided into multiple small blocks, and each small block is converted into a vector of fixed length through linear projection, and the image data is converted into a sequence; Text preprocessing: The text is represented as input using a combination of three embedding vectors, including word embedding, fragment embedding, and position embedding, for feature encoding in the next step; Audio preprocessing: Mel-frequency cepstral coefficients (MFCC) are used to extract short-time spectrum features from the original audio signal.

[0031] In this embodiment, by adopting a specific embedded coding technology, it is possible to pre-process the input information of different modalities, effectively convert these heterogeneous data into a unified embedded vector format, and simplify the subsequent feature encoding and vector space mapping steps. By mapping the features of different modalities to a unified vector space and training them, it is helpful to explore the correlation and complementarity between multimodal data.

[0032] As an example, the feature encoder uses the Transformer feature encoder structure, such as Figure 3 As shown, this is the architecture diagram of the feature encoder.

[0033] The feature encoder forms a residual connection through the multi-head attention mechanism, the Norm normalization layer and the MLP mapping layer connected in sequence, and directly adds the input to the output after the multi-head attention mechanism, repeating a predetermined number of times to enhance the feature representation, and finally outputs a unified feature vector space through the residual structure MLP mapping. The multimodal feature information is projected into a unified feature vector space through the multimodal embedded feature encoder. Feature space.

[0034] The input samples of the multimodal feature extraction network are data sample pairs consisting of multimodal information and images. The feature encoders share parameters and use contrastive learning to optimize the parameters of the feature encoders. During the training process, the training samples are input and the feature extraction network forward inference outputs a sequence describing the features. and the corresponding image feature sequence ,in G is the number of matching sample pairs used for training. Calculate the matching similarity matrix , where the elements for I Middle i Elements and T Middle j The similarity between elements.

[0035] Define the label matrix , where the elements on the main diagonal are 1 (corresponding to positive samples) and the other elements are 0 (corresponding to negative samples). Using cross entropy, the loss is calculated:

[0036]

[0037]

[0038] in, L I is the image loss, L T To describe the loss.

[0039] like Figure 4 As shown in Figure 1, this is a schematic diagram of the input and output of the multimodal feature extraction network.

[0040] Further optionally, the step of performing similarity matching between the features of the task target and the character sequence information includes: calculating the similarity between the task target features and each character feature in the character sequence information features in turn, and if the maximum similarity is greater than a first preset threshold, taking the character information corresponding to the maximum similarity as the locking result, otherwise it is judged that the matching fails; the similarity calculation formula is as follows:

[0041] in, For the i The degree of familiarity between the character information and the task target. T is the feature vector of the task target, For the i The feature vector of each character’s information; N is the number of people to be identified in the image to be detected.

[0042] As an exemplary illustration, the value of the first preset threshold is 0.7.

[0043] In this embodiment, by adopting a similarity matching algorithm, the matching efficiency and accuracy of the task target and the character sequence information are improved. Specifically, by calculating the similarity between the feature vector of the task target and each vector in the feature vector matrix, the character information closest to the task target can be quickly and accurately identified. At the same time, by reasonably setting the first preset threshold, the strictness and reliability of the matching are ensured, and the situation of mismatching is avoided.

[0044] Example 2 This embodiment makes improvements based on the robust identification and tracking method for anti-tracking targets proposed in Example 1.

[0045] In an optional embodiment, the step of regenerating the characteristics of the task goal based on the generative model through the historical information of the task goal includes: based on a variational autoencoder, regenerating the task goal characteristics by learning the prior distribution of the task goal historical information and combining the continuous time characteristics of the task goal historical information.

[0046] Specifically, when the similarity matching fails in the tth frame, it indicates that the current target state has changed. At this time, the coherent temporal features of the task target saving historical information As real-time training data, the temporal features in historical information are generated by the model Perform online training to generate the current moment t Predictive features of , used to re-identify and lock the target in the new state.

[0047] In this embodiment, by adopting a variational autoencoder and combining the coherent temporal features of the historical information of the task target, this embodiment can quickly and accurately regenerate the task target features when the similarity matching fails, effectively dealing with the problem of target state changes, and generating a model through online training to achieve accurate prediction of the current moment features, thereby improving the system's target recognition and locking capabilities in the new state, significantly enhancing the overall robustness, and achieving continuous locking of targets that perform anti-tracking behavior.

[0048] Further optionally, the variational autoencoder includes a feature prior learning module and a reconstruction module; the feature prior learning module includes an encoder and a long short-term memory network, which are used to extract features of task target historical information and obtain a prior distribution of invariant features; the reconstruction module includes an encoder, a long short-term memory network and a decoder, which are used to reconstruct features based on the prior distribution of invariant features and the features of the last moment in the task target historical information.

[0049] like Figure 5 As shown in Figure 1, it is the structural diagram of the variational autoencoder.

[0050] Feature prior learning model: Input feature into , the encoder Enc generates a high-dimensional hidden state .go through Perform sequence modeling and output the prediction of the next moment latent variable Distribution , which is expressed as follows:

[0051]

[0052] Reconstructing the model: From Sampling latent variables ,joint enter Generate a new intermediate state , the decoder Dec predicts and reconstructs the features , which is expressed as follows:

[0053]

[0054]

[0055] In this embodiment, the variational autoencoder can more effectively extract the features of the historical information of the task target, and learn the prior distribution of the invariant features, which enhances the model's ability to process time series data and makes the feature reconstruction process more accurate. Among them, the feature prior learning module extracts high-dimensional hidden states through the encoder, and uses the long short-term memory network for sequence modeling to predict the distribution of potential variables at the next moment. The reconstruction module reconstructs the features through sampling and decoding processes based on these prior distributions and the features of the last moment in the historical information of the task target, significantly improving the model's ability to regenerate the features of the task target when the similarity matching fails, so that the system can adapt to the target recognition and locking requirements in the new state more quickly and accurately, further enhancing the robustness, adaptability and performance of the system.

[0056] Further optionally, the variational autoencoder also includes a training module, which is used to construct a network with the same structure as the feature prior learning module, and uses the features of the next time step as input for training, and adjusts the parameters in the variational autoencoder through a loss function.

[0057] Specifically, the Q The features of consecutive time steps , the variational autoencoder is iterated in sequence according to the time sequence, and a network with the same structure as the feature prior learning model is constructed. The encoder and the feature prior learning model parameters are consistent. The LSTM network is represented by LSTM_b. The network input is the feature of the next time step , after encoding Dec and LSTM_b network, outputs deterministic latent variables Distribution , which is expressed as follows:

[0058] In this embodiment, the training module constructs a network with the same structure as the feature prior learning module. By iteratively inputting the continuous time step features before the current moment into the variational autoencoder in sequence, the variational autoencoder model can gradually learn the temporal dependencies in the data, improve the training efficiency, and enhance the variational autoencoder model's ability to process time series data. Through the training module, the variational autoencoder model can learn a more accurate distribution of potential variables, providing more reliable prior information for the reconstruction module. In the feature reconstruction process, the model can reconstruct features more accurately based on these prior distributions and the features of the last moment in the task target historical information, improving the variational autoencoder model's ability to regenerate task target features when similarity matching fails.

[0059] In an optional embodiment, the loss function includes reconstruction loss and prior loss; the reconstruction loss evaluates the difference between the predicted features and the real features through L2 loss, and the prior loss constrains the predicted distribution to be close to the prior distribution through KL divergence; during the model training process, the reconstruction loss and prior loss in the loss function are weighted and balanced through hyperparameters.

[0060] Specifically, the loss function of the generated model is as follows:

[0061] in, is the reconstruction loss and is the prior loss, is a hyperparameter used to control the weight of the prior loss in the total loss.

[0062] Prior loss Predicting the distribution by constraining KL divergence Close to the prior distribution :

[0063] Reconstruction loss Use L2 loss to evaluate the features predicted at that moment Features with real next moment The difference between:

[0064] in, is the number of time steps of the input features, is the L2 norm.

[0065] In this embodiment, by introducing a loss function including reconstruction loss and prior loss, an accurate evaluation of the difference between the predicted features and the real features and an effective constraint on the closeness between the predicted distribution and the prior distribution are achieved. By weighting the reconstruction loss and the prior loss through hyperparameters, the model can achieve the best balance between the accuracy of feature reconstruction and the distribution characteristics of the latent space, thereby significantly improving the performance of the model in learning and reconstructing the task target features.

[0066] In an optional embodiment, the identification and tracking method also includes: after successfully locking the task target, predicting the position of the next frame of the task target based on Kalman filtering, performing IoU screening on the prediction result and the character sequence information of the current frame, and retaining the character sequence information with an IoU greater than a second preset threshold as the character sequence information of the next frame.

[0067] Specifically, the Kalman filter algorithm is used to iteratively predict the position of the target in the next frame. :

[0068] in, is the Kalman filter function, For the Kalman filter covariance matrix for the frame.

[0069] Then the Kalman prediction results were The character image position information set of the character shape detection result of the current frame Perform IoU screening and keep the IoU not less than the second preset threshold A collection of image information:

[0070] Among them, the intersection ratio for and The area of ​​the intersection of the described rectangular regions divided by the area of ​​their union.

[0071] As an exemplary illustration, the second preset threshold is 0.25.

[0072] In this embodiment, after successfully locking the task target, the Kalman filter algorithm is used to predict the position of the target in the next frame. Subsequently, the Kalman prediction result is screened with the character sequence information of the current frame by IoU, and the character image information matching the predicted position is retained, effectively eliminating interference information that is irrelevant to the task target or has a large position deviation, reducing the number of character images that need to be sent to the multimodal feature extraction network for complex feature extraction, and significantly reducing the computational load and complexity of the system. This embodiment provides a more efficient and accurate solution for target tracking tasks in actual application scenarios while ensuring the accuracy and reliability of the tracking results.

[0073] Example 3 This embodiment proposes a specific implementation of a robust identification and tracking method for anti-tracking targets in Example 1 and Example 2.

[0074] like Figure 6 As shown, this is a simulation result diagram of a robust recognition tracking method for anti-tracking targets in this scheme to continuously track and identify targets that have changed their outfits and are obscured.

[0075] like Figure 7 As shown, this is the simulation result of the traditional Kalman filter tracking method losing the target of changing equipment and obscuring the target.

[0076] from Figure 6 It can be seen that even if the target changes clothes or is blocked by obstacles such as trees, this method can still accurately identify and track the target person. Thanks to the generative model, the tracking of this method not only relies on the appearance features of the target, but can also effectively make predictions based on the historical trajectory information of the task target, and can maintain a high degree of recognition accuracy regardless of whether the person changes clothes or encounters occlusion.

[0077] In contrast, Figure 7 The limitations of the traditional Kalman filter tracking method when the target changes clothes are demonstrated. Once the target person changes his clothes, the traditional Kalman filter method cannot effectively identify the target, and it cannot effectively deal with significant changes in the target's appearance.

[0078] Example 4 This embodiment proposes a robust identification and tracking system for an anti-tracking target, and applies the robust identification and tracking method for an anti-tracking target proposed in Embodiment 1 and Embodiment 2. Figure 8 As shown, it is an architecture diagram of the robust recognition and tracking system for anti-tracking targets of this embodiment.

[0079] This embodiment proposes a robust identification and tracking system for an anti-tracking target, including: Feature extraction module: It is equipped with a multimodal feature extraction network to extract features of task objectives and object sequence information; Similarity matching module: used to calculate the similarity and compare the task target features and character sequence information features; Feature reconstruction module: It is equipped with a generation model, which is used to regenerate the features of the task target based on the generation model and re-match the similarity when the similarity matching module fails to match until the task target is locked.

[0080] It can be understood that the system of this embodiment corresponds to the method of the above-mentioned embodiment 1 and embodiment 2, and the options in the above-mentioned embodiment 1 and embodiment 2 are also applicable to this embodiment, so they are not described repeatedly here.

[0081] The terms used in the drawings are for illustrative purposes only and should not be construed as limiting this patent; Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A robust identification and tracking method for an anti-tracking target, characterized in that: The following steps are involved: Extracting character sequence information from the image to be detected; Based on the multimodal feature extraction network, feature extraction is performed on the given information of the task target to obtain the task target features; Based on the character sequence information, the multimodal feature extraction network is used to extract features from the image to be detected to obtain character sequence information features; The task target features and the character sequence information features are matched for similarity. If the match is successful, the task target is locked from the character sequence information. If the match fails, the features of the task target are regenerated based on the generation model through the historical information of the task target and similarity matching is performed again until the task target is locked.

2. A robust identification and tracking method for an anti-tracking target according to claim 1, characterized in that: The step of extracting character sequence information from the image to be detected includes: performing character body detection and segmentation on the current frame image collected by a target detection algorithm to obtain the detected character sequence information.

3. A robust identification and tracking method for an anti-tracking target according to claim 1, characterized in that: The step of extracting features from given information of the task target comprises: Inputting the given information of the task target into the multimodal feature extraction network, encoding different types of information in the given information of the task target through different embedding codes to obtain an embedding vector; The embedding vector is input into the feature encoder in the multimodal feature extraction network, and the features of different modalities are mapped to a unified vector space to obtain the task target features.

4. A robust identification and tracking method for an anti-tracking target according to claim 1, characterized in that: The step of matching the task target and the character sequence information features by similarity includes: calculating the similarity of the task target features and each character feature in the character sequence information features in turn, and if the maximum similarity is greater than a first preset threshold, taking the character information corresponding to the maximum similarity as the locking result, otherwise it is judged that the matching fails; the similarity calculation formula is as follows: in, is the familiarity between the i-th character information and the task target, T is the feature vector of the task target, For the i The feature vector of each character’s information; N is the number of people to be identified in the image to be detected.

5. The robust identification and tracking method for an anti-tracking target according to claim 1, characterized in that: The step of regenerating the features of the task target based on the generative model through the historical information of the task target includes: based on a variational autoencoder, regenerating the features of the task target by learning the prior distribution of the historical information of the task target and combining the continuous time features in the historical information.

6. A robust identification and tracking method for an anti-tracking target according to claim 5, characterized in that: The variational autoencoder includes a feature prior learning module and a reconstruction module; the feature prior learning module includes an encoder and a long short-term memory network, which are used to extract the features of the task target historical information and obtain the prior distribution of invariant features; the reconstruction module includes an encoder, a long short-term memory network and a decoder, which are used to reconstruct features based on the prior distribution of the invariant features and the features of the task target historical information at the last moment.

7. A robust identification and tracking method for an anti-tracking target according to claim 6, characterized in that: The variational autoencoder also includes a training module, which is used to construct a network with the same structure as the feature prior learning module, and uses the features of the next time step as input for training, and adjusts the parameters in the variational autoencoder through a loss function.

8. A robust identification and tracking method for an anti-tracking target according to claim 7, characterized in that: The loss function includes reconstruction loss and prior loss; the reconstruction loss evaluates the difference between the predicted features and the real features through L2 loss, and the prior loss constrains the predicted distribution to be close to the prior distribution through KL divergence; during the model training process, the reconstruction loss and prior loss in the loss function are weighted and balanced through hyperparameters.

9. A robust identification and tracking method for an anti-tracking target according to any one of claims 1 to 8, characterized in that: The identification and tracking method also includes: after successfully locking the task target, predicting the position of the next frame of the task target based on Kalman filtering, performing IoU screening on the prediction result and the character sequence information of the current frame, and retaining the character sequence information with an IoU greater than a second preset threshold as the character sequence information of the next frame.

10. A robust identification and tracking system for an anti-tracking target, applied to the robust identification and tracking method for an anti-tracking target according to any one of claims 1 to 9, characterized in that: The system comprises: Feature extraction module: It is equipped with a multimodal feature extraction network to extract features of task objectives and object sequence information; Similarity matching module: used to calculate the similarity and compare the task target features and character sequence information features; Feature reconstruction module: It is equipped with a generation model, which is used to regenerate the features of the task target based on the generation model and re-match the similarity when the similarity matching module fails to match until the task target is locked.

Images