Image-text associated target tracking network training method and system

Through the training method of graphic-text-related target tracking network, combined with the graphic-text conversion module and the Transformer feature extraction network, the problems of insufficient extraction of target features and unstable tracking in the existing technology are solved, and high-precision and stable target tracking effect are achieved .

CN118967745AInactive Publication Date: 2024-11-15WUHAN TEXTILE UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410967904.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Due to the limited image abstraction capability of the existing target tracking methods, it is difficult to extract the complete and accurate features of the target, and too much emphasis on timing information, making it difficult to track targets that undergo drastic changes in motion appearance.

Method used

Introduce a target tracking network training method for graphic and text association, and associate image features with text features through graphic and text conversion module. Use the improved Transformer feature to extract networks and feature adaptive modules to generate dynamic text features and final related features to achieve target state high-precision prediction.

Benefits of technology

It significantly improves the accuracy and stability of target tracking, can accurately track targets in complex environments, enhances the comprehensiveness and accuracy of feature extraction, meets real-time application needs, and improves the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967745A_ABST
    Figure CN118967745A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of computer vision and deep learning, and discloses an image-text associated target tracking network training method, which comprises the following steps of: acquiring two adjacent frames of images, an num-th frame of image and an nm + 1-th frame of image in a video sequence each time, respectively taking the images as a training image and a test image, and respectively cutting image blocks, namely a num frame target image block and a (m + 1) frame target image block, where a selected target is located from the images; inputting the num frame target image block and the (m + 1) frame target image block into an image-text conversion module to obtain initial text features and existing text features, and providing complete and accurate feature description of a tracking target; the method provided by the invention can solve the problems that an existing single target tracking method is insufficient in feature extraction capability, the tracking target concept is fuzzy, and the semantic context between adjacent keys is not considered and utilized in the feature extraction stage, and significantly improves the precision and efficiency of single target tracking.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning technology, and in particular relates to a target tracking network training method associated with images and texts. Background Art

[0002] Target tracking is one of the most important research directions in the field of computer vision, and has a wide range of applications in many fields such as intelligent transportation, autonomous driving, surveillance and security, and intelligent military security. Existing target tracking methods usually use image blocks as reference units for positioning and tracking. For example, the paper "MixFormer" published by CVPR 2022 and the "Single Target Tracking System and Method Based on Attention Mechanism and Improved Transformer" published by the Chinese patent with publication number CN117974722B both use image blocks as reference units and use transformers as backbone networks to extract features of the tracked target, thereby performing visual tracking tasks. However, due to the limited abstraction ability of images, a single image block cannot express the precise concept of the target in a simple and intuitive way like humans understand natural language, which makes it difficult for existing target tracking methods to extract more relevant features from images; not only that, existing target tracking methods overemphasize the timing information of video frames, ignore the possibility of large changes in the target appearance, and are difficult to track targets that move violently due to motion appearance.

[0003] To overcome these limitations, we are inspired by the field of text-to-image generation and introduce the Image encoder and Text encoder technology of the CLIP model proposed by Jong Wook Kim et al. in their paper "Learning Transferable Visual Models From Natural Language Supervision". This technology enables the trained target tracking network to understand both image and text representations and map them into a shared embedding space. We hope to provide a comprehensive reference point for tracking targets by generating text prompts from images, thereby introducing this technology into the field of target tracking, so that the target tracking network can achieve an effect similar to that of humans understanding natural language and clearly understand the specific concepts of the targets it is tracking.

[0004] Through the above analysis, the problems and defects of the prior art are as follows:

[0005] Due to the limited abstraction ability of images and the existence of ambiguity, a single image block cannot express a complete and precise concept of the target, which means that existing target tracking methods find it difficult to extract more relevant features from the image; not only that, traditional target tracking methods overemphasize the timing information of video frames and ignore the possibility of large changes in the target's appearance, making it difficult to track targets that move violently due to their motion appearance. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a method for training a target tracking network with image-text association.

[0007] The present invention is implemented in this way: a target tracking network training method for image-text association includes:

[0008] Step S101: acquiring two adjacent frames of images in a video sequence each time, using the numth frame image and the num+1th frame image as a training image and a test image respectively, and cropping out image blocks where the target is located respectively;

[0009] Step S102: inputting the target image block of the numth frame and the target image block of the num+1th frame into the image-text conversion module to obtain initial text features and existing text features;

[0010] Step S103: processing the initial text features and the existing text features through the feature adaptation module to generate dynamic text features whose label weights can be updated in real time;

[0011] Step S104: extracting features of the num+1th frame image using an improved transformer feature extraction network, wherein the improved transformer feature extraction network includes an encoder and a decoder;

[0012] Step S105: by means of the image-text association module, the related features between the dynamic text features obtained in step S103 and the features of the num+1th frame image obtained in step S104 are associated and fused into the final related features for target state prediction;

[0013] Step S106: Take the num+1th frame as a test image, perform state estimation on the tracked target on the test image through a prediction module composed of 4 stacked convn bn relu layers, and obtain the coordinates of the target bounding box through deep learning.

[0014] Furthermore, the image-text conversion module includes:

[0015] The label vocabulary is used to provide text labels to the image-text conversion module, which contains category labels, such as 'Car', 'Bike', and attribute labels, such as color attribute 'Red' and material attribute 'iron';

[0016] The feature reshaping operation is used to process the target image block feature x extracted by the image encoder Image encoder of the open source model CLIP, and decompose the feature map of x into a set of feature vectors with the shape of a two-dimensional array along the channel dimension, that is, reshape the height, width, and number of channels into height*width, number of channels, and obtain a set of related features v1, v2, ..., v of the target image block feature x of the numth frame k ;

[0017] Text embedding operation, using onehot encoding to convert the labels in the label vocabulary into word vectors c i ;

[0018] The image-text conversion module calculates the predicted probability p(c i |x), which is mathematically described as:

[0019]

[0020] Among them, for the target image block m, Q i (x) represents the text hint of the i-th label of the tracked target. This text hint consists of a set of related features v1, v2, ..., v of the target image patch feature x. k And the word vector c i Composition, i.e. Q i (x)={v1(x), v2(x),...,v k (x), c i}, t(·) represents the features processed by the text encoder of the open source model CLIP, S im (·,·) represents the cosine similarity score, which measures the similarity between the target image block feature x and the text feature t(Q i (x)), τ represents the parameter for adjusting the probability distribution predicted by the model, and N represents the number of labels;

[0021] Furthermore, the feature adaptation module can generate dynamic text features whose label weights can be updated in real time based on the initial text features and the existing text features:

[0022] The text feature generation is divided into category feature generation and attribute feature generation. C , which is mathematically described as:

[0023]

[0024] Among them, T iThe text features of the i-th category label generated by the CLIP text encoder in the image-to-text conversion module of claim 1, p i The predicted probability of the i-th label calculated by formula 1;

[0025] For each attribute feature Ta with the highest prediction probability, its mathematical description is:

[0026]

[0027] Among them, p i is the predicted probability of the i-th label calculated by formula 1, argmax(p i ) indicates taking the label with the highest predicted probability, and index indicates the number of the label with the highest predicted probability;

[0028] Since the attribute values ​​of the tracked target in different frames of images will change, it is necessary to adjust the weights according to the changes in different attribute features; for example, the changes in color, material, and texture are recorded as D color , D material , D texture , which is mathematically described as:

[0029] D color =|R color -S color |,

[0030] D material =|R material -S material |,(Formula 4)

[0031] D texture =|R texture -S texture |,

[0032] Among them, R attribute and S attribute It represents the probability of a specific attribute value calculated by formula 1 after the target in the target image block of the numth frame and the target image block of the num+1th frame is processed by the image-text conversion module. The attribute value is the attribute label such as color, material, texture, etc.; D attribute The lower the value, the more similar the objects on the target image block of the numth frame and the target image block of the num+1th frame are in this attribute; therefore, the attention weight formula for different attributes is:

[0033] W att =Softmax(-D color ,-D material ,-D texture ) (Formula 5)

[0034] Among them, Softmax(·) represents the classification function in the field of deep learning. color , D material , D texture The changes of other attributes are converted into probability distribution;

[0035] Afterwards, the dynamic text features T of different attributes are att Adjusted to:

[0036] T att =W att *T a (Formula 6)

[0037] Among them, T a is the text feature generated using Formula 3.

[0038] Furthermore, the improved transformer feature extraction network includes:

[0039] The encoder is used to convert the input image sequence into a series of context vectors containing rich semantic information. Its structure consists of 16 identical layers stacked together. Each layer contains a self-attention mechanism SelfAttention and a feedforward neural network FFN. Due to the use of residual connections, the output of each layer is a combination of its input and the calculation result of the layer. The weighted attention mechanism can be described as:

[0040]

[0041] Among them, q represents the query, k represents the key, and v represents the value. q, k, and v are all linear transformations of the input sequence. k Represents the dimension of the key, T represents the time dimension, pos represents the position information encoding, Softmax represents the classification function, α represents the scaling factor, and the weight range of the attention mechanism can be adjusted according to the actual situation. The default value is 1;

[0042] The decoder is used to convert the context vector generated by the encoder into a target sequence. Its structure is similar to that of the encoder, but it introduces an encoder-decoder attention mechanism, and the description of its attention mechanism weighting is consistent with Formula 7.

[0043] Furthermore, the image-text association module combines image features and text features to obtain the final relevant features for target state prediction:

[0044] Assume that the image feature extracted by the transformer feature extraction network is V∈R H×W×C , the text features extracted by the image-text conversion module are dimensionally adjusted to T∈R through the linear layer 1×1×c, so that it is aligned with the image features on the channel dimension, and then the correlation between the two features is realized through the convolution operation, where R represents the extracted image features, H represents the height dimension of the image, w represents the width dimension of the image, C represents the channel dimension of the image, and the text feature T∈R 1×1×C Is used as a kernel weight to add the relevant features between the image features and all text features as the final relevant features for state prediction, calculated as:

[0045] C orr (V,T c ,T co ,T m ,T t )=V+(L c (T c )

[0046] +L co (T co )+L m (T m )+L t (T t ))⊙V(Formula 8)

[0047] where ⊙ denotes a convolution operation, L(·) denotes a linear projection layer for channel adaptation, and T c represents the category feature, T co , T m and T t Respectively represent the dynamic color, material and texture characteristics of the image;

[0048] Furthermore, the prediction module estimates the target state through the final relevant features and obtains the coordinates of the target bounding box through deep learning:

[0049] Based on the final relevant features generated by the image-text association module, the prediction module composed of 4 stacked convn bn relu layers (3x3 convolution layer batch normalization layer activation function layer) processes the final relevant features V to obtain the feature map V F ; Use the classification function softmax to process the feature map V F , and obtain the classification score map C = softmax(V F ); The dimension map B describing the size of the prediction box is usually directly generated by the subsequent convn bn relu layer, and no additional function is required for conversion; then the coordinates of the target on the classification score map (x c ,y c )for:

[0050] (x c ,y c )=argmax(C ij )(Formula 9)

[0051] Among them, C ij represents any coordinate on the classification score map, argmax(·) represents the coordinate with the highest prediction score, that is, the coordinate of the tracked target on the classification score map;

[0052] The calculated target prediction box location information is:

[0053] (x,y,w,h)=(x c ,y c ,B w ,B h )(Formula 10)

[0054] Where (x c ,y c ) is the coordinate of the target on the classification score map, B w and B h Represents the width and height of the prediction box from B, and x, y, w, and h represent the horizontal coordinate, vertical coordinate, width, and height of the center point of the final target prediction box, respectively.

[0055] Another object of the present invention is to provide a target tracking network and training system associated with images and texts, including:

[0056] The image-text conversion module is used to convert the target image block of the numth frame and the target image block of the num+1th frame where the selected target is located, which are cut out from the numth frame image and the num+1th frame image, into text features, obtain initial text features and existing text features, and provide a complete and accurate feature description of the tracked target;

[0057] A feature adaptation module is used to process initial text features and existing text features to generate dynamic text features;

[0058] The improved transformer feature extraction network is used to extract the features of the num+1th frame image;

[0059] The image-text association module is used to associate the relevant features between the dynamic text features and the features of the num+1th frame image to generate the final relevant features of the target state;

[0060] The prediction module is used to estimate the state of the tracking target on the test image and obtain the prediction result.

[0061] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-text associated target tracking network training method.

[0062] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the image-text associated target tracking network training method.

[0063] Another object of the present invention is to provide an information data processing terminal, which is used to implement the target tracking network and training system associated with graphics and text.

[0064] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0065] First, the present invention enhances the ability of the target tracking network to extract image features; inspired by the open source model clip, the proposed image-text conversion module interprets the tracking target through text features, making the concept of the tracking target clearer; through the improved transformer feature extraction network composed of 16 identical layers stacked together, the feature extraction stage considers the use of semantic context between adjacent keys for feature extraction, which significantly improves the accuracy and efficiency of single target tracking.

[0066] Second, the image-to-text conversion module of the present invention solves the problems of low image-to-text conversion accuracy and inaccurate label recognition in the prior art by introducing a label vocabulary, feature reshaping operations, and text embedding operations. By decomposing the features extracted by the image encoder into channel dimensions and converting the labels in the label vocabulary into word vectors using onehot encoding, the accuracy of image-to-text conversion and recognition accuracy are effectively improved. In addition, by processing features using the text encoder of the CLIP model and combining them with cosine similarity scores for prediction, the accuracy of label prediction is improved, thereby significantly improving the overall effect of image-to-text conversion.

[0067] The feature adaptation module solves the problem of unstable tracking caused by changes in target attributes in the existing technology by dynamically updating the label weights. By dividing the text features into category features and attribute features, and using statistical methods to adjust different attribute features, real-time adaptive feature updates are achieved. In particular, for changes in attributes such as color, material, and texture, the sliding window and Zscore method are used to process abnormal points to ensure the continuity and consistency of target attribute features between different frames, greatly improving the stability and accuracy of target tracking.

[0068] The present invention introduces an improved Transformer feature extraction network, which solves the problem of insufficient context information extraction in the prior art through the self-attention mechanism and feedforward neural network of the encoder and decoder. The encoder stacks 16 layers of the same structure, each layer contains a self-attention mechanism and a feedforward neural network, and uses residual connections to ensure the integrity and stability of information transmission. The decoder effectively converts the context vector generated by the encoder into a target sequence by introducing the encoder-decoder attention mechanism, realizes the efficient conversion of the image sequence to the target state, and significantly improves the effect of feature extraction.

[0069] The image-text association module of the present invention solves the problem of poor correlation between image and text features in the prior art by fusing image features and text features. Feature fusion is achieved through convolution operation, and the correlation between image features and text features is weighted and summed using text features as kernel weights to obtain the final relevant features for target state prediction. The prediction module processes the final relevant features and combines deep learning technology to accurately estimate the coordinates of the target bounding box, thereby achieving high-precision target positioning and tracking. Overall, the present invention has made significant technical progress in feature extraction, image-text association and target prediction, and provides reliable technical support for target tracking in complex environments.

[0070] Third, the technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0071] Inspired by the open source model clip in the field of text-to-image generation, this paper takes the lead in the industry to add a new modality, namely the text modality, to the field of target tracking. It successfully uses the advantages of the text modality to alleviate the shortcomings of existing visual tracking methods in image feature extraction, and proves that the multimodal target tracking method combining images and texts is feasible and effective.

[0072] The technical solution of the present invention overcomes technical prejudice:

[0073] The present invention achieves the most advanced performance on numerous tracking datasets by combining the visual modality with the textual modality, proving that in the field of target tracking, it is not only the temporal and spatial modalities that can improve the accuracy of network recognition of targets, but that the application of textual modality is also effective in the field of target tracking.

[0074] Fourth, in the field of target tracking, especially in the training method of target tracking network with image-text association, the existing technology has the following problems and challenges:

[0075] 1) Insufficient target tracking accuracy: Existing target tracking methods are prone to tracking errors or target loss when dealing with complex scenes and fast-moving targets, resulting in low tracking accuracy.

[0076] 2) Insufficient feature extraction: Traditional feature extraction methods can often only extract limited image information and cannot fully utilize the semantic information in the image, resulting in incomplete target representation.

[0077] 3) Lack of effective image-text association mechanism: In terms of image-text association, existing methods usually lack an efficient mechanism to associate image and text features, resulting in the inability to fully utilize text information to assist image analysis during target tracking.

[0078] 4) Poor real-time performance: Many target tracking methods cannot meet the requirements of real-time applications in terms of processing speed, especially in scenarios of high-resolution video and multi-target tracking, where processing delays are large.

[0079] 5) Weak generalization ability: The existing target tracking model has weak generalization ability, inconsistent performance in different scenarios, and is difficult to adapt to the changing actual application environment.

[0080] 6) Insufficient dynamic adjustment capability: During target tracking, the appearance and attributes of the target will change. Existing methods lack the ability to dynamically adjust features, resulting in reduced tracking effect.

[0081] Significant advancement in technical solutions:

[0082] The target tracking network training method based on image-text association proposed in the present invention effectively solves the above-mentioned problems in the prior art through a series of innovative technologies and achieves significant technical progress:

[0083] 1) Improve target tracking accuracy: By introducing an improved transformer feature extraction network and combining it with the image-text association module, target features can be extracted more accurately, significantly improving the accuracy of target tracking.

[0084] 2) Enhanced feature extraction capability: The image encoder and text encoder of the CLIP model are used to fully mine the semantic information in the image, and the text features are dynamically adjusted through the feature adaptation module to ensure the comprehensiveness and accuracy of feature extraction.

[0085] 3) Efficient image-text association mechanism: The present invention establishes an efficient image-text association mechanism through an image-text conversion module and an image-text association module, which can effectively integrate image features and text features, thereby improving the accuracy of target state prediction.

[0086] 4) Meeting real-time requirements: The lightweight network architecture design and optimized computing process ensure the efficiency of the target tracking process and can meet the needs of real-time applications.

[0087] 5) Enhanced generalization capability: Through large-scale training data and diversified scenario simulation, the generalization capability of the model is improved, enabling it to maintain stable performance in different application scenarios.

[0088] 6) Dynamically adjust features: The feature adaptation module can dynamically adjust feature weights according to changes in target attributes, ensuring that efficient feature representation capabilities are always maintained during the tracking process. Even when the target appearance changes significantly, it can still maintain good tracking effects.

[0089] The technical solution of the present invention significantly improves the accuracy, real-time performance and generalization ability of target tracking by improving the feature extraction network, introducing an efficient image-text association mechanism and a module for dynamically adjusting features, and effectively solves multiple key problems in the prior art. By making full use of the semantic information of images and texts, the present invention provides an innovative and efficient solution for the field of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a flow chart of a target tracking network training method for image-text association provided by an embodiment of the present invention.

[0091] Figure 2 It is a structural block diagram of a target tracking network and training system associated with images and texts provided in an embodiment of the present invention.

[0092] Figure 3 It is a schematic diagram of a picture-text conversion module according to an implementation mode provided by an embodiment of the present invention.

[0093] Figure 4 It is a schematic diagram of a feature adaptation module of a second implementation mode provided in an embodiment of the present invention.

[0094] Figure 5 It is a structural diagram of an improved transformer feature extraction network of the third implementation mode provided in an embodiment of the present invention.

[0095] Figure 6 4 is a schematic diagram of a prediction module according to a fifth implementation manner provided by an embodiment of the present invention.

[0096] Figure 7 This is a diagram showing the effect of the present invention compared with an existing representative method. DETAILED DESCRIPTION

[0097] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0098] Example 1: Application of target tracking based on image-text association in autonomous driving

[0099] 1. Experimental Background

[0100] This embodiment aims to use the target tracking network training method of image-text association to improve the target detection and tracking accuracy in the autonomous driving system and ensure driving safety. Autonomous driving requires real-time and accurate detection and tracking of targets such as vehicles, pedestrians, and obstacles on the road. Traditional methods are prone to false detection or missed detection in complex environments.

[0101] 2. Experimental Procedure

[0102] Step S101: Acquire two adjacent frames of images in a video sequence

[0103] Each time, two adjacent frames of images are obtained from the vehicle camera video stream, the numth frame image is used as the training image, the num+1th frame image is used as the test image, and the image blocks where the target is located are cropped out respectively.

[0104] Step S102: Input image-text conversion module

[0105] The target image block of the numth frame and the target image block of the num+1th frame are input into the image-text conversion module to obtain the initial text features and the existing text features.

[0106] The image-text conversion module includes a label vocabulary (such as 'Car', 'Pedestrian', color attribute 'Red', etc.), a feature reshaping operation, and a text embedding operation.

[0107] Step S103: feature adaptation module processes text features

[0108] The initial text features and existing text features are processed by the feature adaptation module to generate dynamic text features whose label weights can be updated in real time.

[0109] Step S104: Improved transformer feature extraction network

[0110] The features of the num+1th frame image are extracted using an improved transformer feature extraction network, including an encoder and a decoder.

[0111] Step S105: Image-text association module

[0112] The dynamic text features obtained in step S103 and the features of the num+1th frame image obtained in step S104 are associated and fused into the final relevant features for target state prediction.

[0113] Step S106: The prediction module estimates the state of the tracking target on the test image

[0114] The num+1th frame is used as the test image, and the state of the tracked target on the test image is estimated through a prediction module composed of 4 stacked convn bn relu layers to obtain the coordinates of the target bounding box.

[0115] 3. Industrial Applications

[0116] In the automatic driving system, through the method of this embodiment, the system can:

[0117] Track vehicles, pedestrians and other targets in real time and with high precision to ensure driving safety.

[0118] Maintain stable tracking performance in complex urban traffic environments and reduce false detections and missed detections.

[0119] According to the dynamic characteristics of different targets, the tracking strategy is adaptively adjusted to improve the reliability and robustness of tracking.

[0120] Example 2: Application of target tracking based on image-text association in intelligent monitoring system

[0121] 1. Experimental Background

[0122] This embodiment aims to use the target tracking network training method of image-text association to improve the accuracy of tracking people and objects in the intelligent monitoring system and ensure the safety of public places. The intelligent monitoring system needs to detect and track potential threat targets in real-time video streams, such as stolen items, illegal intruders, etc.

[0123] 2. Experimental Procedure

[0124] Step S101: Acquire two adjacent frames of images in a video sequence

[0125] Each time, two adjacent frames of images are obtained from the surveillance camera video stream, the numth frame image is used as the training image, the num+1th frame image is used as the test image, and the image blocks where the target is located are cropped out respectively.

[0126] Step S102: Input image-text conversion module

[0127] The target image block of the numth frame and the target image block of the num+1th frame are input into the image-text conversion module to obtain the initial text features and the existing text features.

[0128] The image-text conversion module includes a label vocabulary (such as 'Person', 'Bag', color attribute 'Black', etc.), a feature reshaping operation, and a text embedding operation.

[0129] Step S103: feature adaptation module processes text features

[0130] The initial text features and existing text features are processed by the feature adaptation module to generate dynamic text features whose label weights can be updated in real time.

[0131] Step S104: Improved transformer feature extraction network

[0132] The features of the num+1th frame image are extracted using an improved transformer feature extraction network, including an encoder and a decoder.

[0133] Step S105: Image-text association module

[0134] The dynamic text features obtained in step S103 and the features of the num+1th frame image obtained in step S104 are associated and fused into the final relevant features for target state prediction.

[0135] Step S106: The prediction module estimates the state of the tracking target on the test image

[0136] The num+1th frame is used as the test image, and the state of the tracked target on the test image is estimated through a prediction module composed of 4 stacked convn bn relu layers to obtain the coordinates of the target bounding box.

[0137] 3. Industrial Applications

[0138] In the intelligent monitoring system, through the method of this embodiment, the system can:

[0139] Track people and objects in public places in real time, and promptly detect and alert potential security threats.

[0140] Maintain high-precision tracking performance in complex monitoring environments and reduce false positives and negatives.

[0141] Dynamically adjust tracking strategies to improve tracking flexibility and adaptability and ensure the reliability of security monitoring.

[0142] These two specific industrial application embodiments demonstrate the wide application of the target tracking network training method based on image-text association in different fields, fully reflecting the technical advantages and practical value of the present invention.

[0143] All technical terms used in this article have the same meaning as those generally understood by technicians in the technical field of this application. The terms used in this article in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The symbol "()" used in this article represents a further supplement to the content described. The "numth frame target image block" described in this article is an image block that just contains the selected tracking target cropped from the "numth frame image". "Num" represents the index number of any frame image. The word "open source" used in this article indicates that the technology described has been publicly shared, can be obtained through public channels, and is allowed to be modified by anyone.

[0144] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0145] See also Figure 1 , which is a flow chart of a target tracking network training method for image-text association proposed in an embodiment of the present application, is described in detail as follows:

[0146] Step S101: acquiring two adjacent frames of images in a video sequence each time, using the numth frame image and the num+1th frame image as a training image and a test image respectively, and cropping out image blocks where the target is located respectively;

[0147] Step S102: inputting the target image block of the numth frame and the target image block of the num+1th frame into the image-text conversion module to obtain the initial text features and the existing text features, so as to provide a complete and accurate feature description of the tracking target;

[0148] Specifically, see Figure 3 , is a schematic diagram of the image-text conversion module, which includes a label vocabulary, feature reshaping operations, and text embedding operations:

[0149] First, the image encoder Image encoder of the open source model CLIP is used to extract the feature x of the input target image block (the optional range is the target image block of the numth frame and the target image block of the num+1th frame), and then the feature reshaping operation is performed to decompose the feature map of x into a set of feature vectors with the shape of a two-dimensional array along the channel dimension, that is, (height, width, number of channels) is reshaped into (height*width, number of channels), and a set of related features v1, v2,..., v of the feature x of the target image block of the numth frame is obtained. k ; Then perform text embedding operation and use onehot encoding to convert the labels in the label vocabulary into word vectors c i (c i With v i have the same dimensions); then the predicted probability of the i-th label of the tracked target in the target image block is mathematically described as:

[0150]

[0151] Among them, Q i (x) represents the textual hint of the i-th label of the target in the target image patch, a set of relevant features v1,v2,...,v extracted from the image feature x k And the word vector c i Composition, i.e. Q i (x)={v1(x),v2(x),...,v k (x),c i}, t(·) represents the features processed by the CLIP text encoder, S im (·,·) represents the cosine similarity score, which measures the similarity between the image feature x and the text feature t(Q i (x)), τ represents the parameter that adjusts the probability distribution predicted by the model, and N represents the number of labels.

[0152] Finally, the image-text conversion module outputs the predicted probability of each label of the input image (the optional range is the target image block of the numth frame and the target image block of the num+1th frame).

[0153] It should be noted that in the field of single target tracking, the method of combining training image frames and test image frames can better understand the global context of the target object in the video sequence. The training frame contains the initial features of the tracked target, while the test frame contains the existing features of the current frame. This combination helps the algorithm understand the spatiotemporal transformation of the tracked target in the video, as well as the interaction between the target and the background. Therefore, each training of the present invention uses two adjacent frames in the video sequence as training images and test images as input into the image-text associated target tracking network, and uses the image encoder Image encoder of the open source model clip to extract features from the training images and test images, thereby obtaining deeper features of the two frames, namely the shape, appearance, color, texture and many other features of the target that help distinguish the target from the background.

[0154] Step S103: processing the initial text features (text features of the target image block of the numth frame) and the existing text features (text features of the target image block of the num+1th frame) through the feature adaptation module to generate dynamic text features whose label weights can be updated in real time;

[0155] Please refer to Figure 4 , the feature adaptation module divides text feature generation into category feature generation and attribute feature generation. For the category feature T C , its mathematical description is:

[0156]

[0157] Among them, T i is the text feature of the i-th label generated by the CLIP text encoder in the image-to-text conversion module of claim 1, i The predicted probability of the i-th label calculated by formula 1;

[0158] For each attribute feature Ta with the highest prediction probability, its mathematical description is:

[0159]

[0160] Among them, argmax(p i ) means taking the label with the highest predicted probability, and index is the number of the label with the highest predicted probability;

[0161] Since the attribute values ​​of the tracked target in different frames of images will change, it is necessary to adjust the weights according to the changes in different attribute features. For example, the changes in color, material, and texture are respectively denoted as D color , D material , D texture , which is mathematically described as:

[0162] D color =|R color -S color |,

[0163] D material =|R material -S material |,(Formula 4)

[0164] D texture =|R texture -S texture |,

[0165] Among them, R attribute and S attribute The probability of the target image block of the numth frame and the target image block of the num+1th frame having a specific attribute value calculated in step S102, where the attribute value is a color, material, texture or other attribute label. attribute The lower the value, the more similar the objects in the target image block of the numth frame and the target image block of the num+1th frame are in this attribute. Therefore, the attention weight formula for different attributes is:

[0166] W att =Softmax(-D color ,-D material ,-D texture ) (Formula 5)

[0167] Among them, Softmax(·) represents the classification function in the field of deep learning. color , D material , D texture The changes of other attributes are converted into probability distribution;

[0168] Afterwards, the dynamic text features T of different attributes are att Adjusted to:

[0169] T att =W att *T a (Formula 6)

[0170] Among them, T a is the text feature generated using Formula 3.

[0171] Step S104: extracting features of the num+1th frame image using an improved transformer feature extraction network, wherein the improved transformer feature extraction network includes an encoder and a decoder.

[0172] Specifically, see Figure 5 The improved transformer feature extraction network is mainly composed of an encoder and a decoder. The encoder takes the num+1th frame image as input and converts the input image sequence into a series of context vectors containing rich semantic information. Its structure is composed of 16 identical layers stacked together. Each layer contains a self-attention mechanism SelfAttention and a feedforward neural network FFN. Due to the use of residual connections, the output of each layer is a combination of its input and the calculation result of the layer. The weighted attention mechanism can be described as:

[0173]

[0174] Among them, q represents the query, k represents the key, and v represents the value. q, k, and v are all linear transformations of the input sequence. k Represents the dimension of the key, T represents the time dimension, pos represents the position information encoding, Softmax represents the classification function, α represents the scaling factor, and the weight range of the attention mechanism can be adjusted according to the actual situation. The default value is 1;

[0175] The decoder is used to convert the context vector generated by the encoder into a target sequence. The output is the feature of the image input to the encoder. Its structure is similar to that of the encoder, but an encoder-decoder attention mechanism is introduced. The weighted description of the attention mechanism is consistent with Formula 7.

[0176] Step S105: through the image-text association module, the relevant features between the dynamic text features obtained in step S103 and the features of the num+1th frame image obtained in step S104 are associated and fused into the final relevant features for target state prediction.

[0177] Specifically, the image-text association module is not a network structure, but a feature fusion operation that combines image features and text features to obtain the final relevant features for target state prediction:

[0178] Suppose the image feature extracted in step S104 is V∈R H×W×C The text features extracted in step S103 are dimensionally adjusted to T∈R through a linear layer. 1×1×c , so that it is aligned with the image features on the channel dimension, and then the correlation between the two features is realized through the convolution operation, where R represents the extracted image features, H represents the height dimension of the image, w represents the width dimension of the image, C represents the channel dimension of the image, and the text feature T∈R 1×1×C Is used as a kernel weight to add the relevant features between the image features and all text features as the final relevant features for state prediction, calculated as:

[0179] C orr (V,T c ,T co ,T m ,T t )=V+(L c (T c )

[0180] +L co (T co )+L m (T m )+L t (T t ))⊙V(Formula 8)

[0181] where ⊙ denotes a convolution operation, L(·) denotes a linear projection layer for channel adaptation, and T c represents the category feature, T co , T m and T t Represents dynamic color, material and texture features respectively.

[0182] Step S106: Take the num+1th frame as the test image, and estimate the state of the tracked target on the test image through a prediction module composed of 4 stacked convn bn relu layers (3x3 convolutional layer batch normalization layer activation function layer), and obtain the coordinates of the target bounding box through deep learning:

[0183] Specifically, see Figure 6Based on the final relevant features generated by the image-text association module, the prediction module composed of 4 stacked convnbn relu layers processes the final relevant features V to obtain the feature map V F ; Use the classification function softmax to process the feature map V F , and obtain the classification score map C = softmax(V F ); The dimension map B describing the size of the prediction box is usually directly generated by the subsequent convn bn relu layer, and no additional function is required for conversion. Then the coordinates of the target on the classification score map (x c ,y c )for:

[0184] (x c ,y c )=argmax(C ij )(Formula 9)

[0185] Among them, C ij represents any coordinate on the classification score map, argmax(·) represents the coordinate with the highest prediction score, that is, the coordinate of the tracked target on the classification score map;

[0186] The calculated target prediction box location information is:

[0187] (x,y,w,h)=(x c ,y c ,B w ,B h )(Formula 10)

[0188] Where (x c ,y c ) is the coordinate of the target on the classification score map, B w and B h Represents the width and height of the prediction box from B, and x, y, w, and h represent the horizontal coordinate, vertical coordinate, width, and height of the center point of the final target prediction box, respectively.

[0189] It should be noted that the above steps are not implemented only once. The numth frame and the num+1th frame image only refer to two adjacent frames of the entire video frame. The above steps must be repeated until all video frames are processed to obtain the target tracking model with the best training effect of image-text association.

[0190] See also Figure 2 , which is a structural diagram of a target tracking network training method for image-text association proposed in an embodiment of the present application, and the structure is as follows:

[0191] An image-text conversion module 1 is used to convert the target image block of the numth frame and the target image block of the num+1th frame into text features, obtain initial text features and existing text features, and provide a complete and accurate feature description of the tracking target;

[0192] Feature adaptation module 2, used for processing initial text features and existing text features to generate dynamic text features;

[0193] The improved transformer feature extraction network 3 is used to extract the features of the num+1th frame image;

[0194] The image-text association module 4 is used to associate the relevant features between the dynamic text features and the features of the num+1th frame image to generate the final relevant features for target state prediction;

[0195] The prediction module 5 is used to estimate the state of the tracking target on the test image and obtain a prediction result.

[0196] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-text associated target tracking network training method.

[0197] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the image-text associated target tracking network training method.

[0198] Another object of the present invention is to provide an information data processing terminal, which is used to implement the target tracking network and training system associated with graphics and text.

[0199] Application Example 1: Autonomous Vehicle Target Detection and Tracking System

[0200] In driverless cars, real-time and accurate detection and tracking of targets on the road (such as vehicles, pedestrians, bicycles, etc.) is an important task to ensure driving safety. Traditional target detection and tracking systems are often affected by factors such as environmental changes and target movement speed, making it difficult to ensure high accuracy and high stability.

[0201] 1) Image-text conversion module:

[0202] The label vocabulary includes category labels such as vehicles, pedestrians, and bicycles, as well as attribute labels such as color (e.g., red, blue) and material (e.g., metal, plastic).

[0203] The image encoder of the CLIP model is used to extract the features of road scene image blocks, and the features are decomposed into a two-dimensional array through feature reshaping operation.

[0204] Onehot encoding is used to convert the labels in the label vocabulary into word vectors, and the predicted probability of each label of the tracked target in the target image block is calculated.

[0205] 2) Feature Adaptation Module:

[0206] Based on the images captured by the real-time camera, dynamic text features are generated to update the category and attribute features in real time.

[0207] Handle changes in color, material, and texture to ensure the continuity and consistency of target attribute features between different frames.

[0208] 3) Improved Transformer feature extraction network:

[0209] Continuous road scene images are input into the encoder to extract context vectors containing rich semantic information.

[0210] The decoder converts the context vector generated by the encoder into a target state sequence to achieve efficient target detection and tracking.

[0211] 4) Image-text association and prediction module:

[0212] Image features and text features are fused to calculate the final relevant features for target state prediction.

[0213] Through deep learning algorithms, the target bounding box is accurately estimated, and the target's position and movement trajectory are output in real time.

[0214] The present invention realizes efficient target detection and tracking of unmanned vehicles in complex road environments by introducing a high-precision image-text conversion module and a feature adaptation module, significantly improves the stability and accuracy of the system, and ensures driving safety.

[0215] Application Example 2: Intelligent Video Surveillance System

[0216] In the field of public safety and security, intelligent video surveillance systems need to identify and track people and objects in the surveillance area in real time to prevent and respond to potential security threats. Traditional video surveillance systems are easily affected by factors such as lighting changes and occlusion in complex scenes, resulting in poor target recognition and tracking effects.

[0217] 1) Image-text conversion module:

[0218] The label vocabulary includes category labels such as people, vehicles, suspicious objects, and attribute labels such as color, shape, and material.

[0219] The image encoder of the CLIP model is used to extract the image block features of the surveillance video frame, and the features are decomposed into a two-dimensional array through feature reshaping operation.

[0220] Onehot encoding is used to convert the labels in the label vocabulary into word vectors, and the predicted probability of each label in the target image block is calculated.

[0221] 2) Feature Adaptation Module:

[0222] Based on the real-time video stream, dynamic text features are generated and different categories and attribute features are updated in real time.

[0223] Deal with issues such as lighting changes and occlusion to ensure the continuity and consistency of target attribute features between different frames.

[0224] 3) Improved Transformer feature extraction network:

[0225] Consecutive video frames are input into the encoder to extract context vectors containing rich semantic information.

[0226] The decoder converts the context vector generated by the encoder into a target state sequence to achieve efficient target recognition and tracking.

[0227] 4) Image-text association and prediction module:

[0228] Image features and text features are fused to calculate the final relevant features for target state prediction.

[0229] Through deep learning algorithms, the target bounding box is accurately estimated, and the target's position and activity trajectory are output in real time.

[0230] The present invention realizes efficient target recognition and tracking of intelligent video surveillance systems in complex scenarios through high-precision image-text conversion and feature adaptation technology, significantly improves the stability and accuracy of the system, and effectively ensures the real-time and accuracy of public safety and security monitoring.

[0231] These two application examples demonstrate the specific application of the present invention in different fields, and significantly improve the efficiency and accuracy of target detection and tracking through advanced image-text conversion and feature extraction technology.

[0232] To illustrate the effect of the technical solution described in the present invention, please refer to Figure 7We use four datasets recognized in the field of target tracking, TNL2K, LaSOT, TrackingNet, and GOT10K, as benchmarks, and compare the results with the methods OSTrack and Mixformer. Our method improves the accuracy of TNL2K, TrackingNet, and GOT10K by an average of 1.8% compared with other methods, and the improvement effect on the resolution SR indicator is also more significant.

[0233] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. A person of ordinary skill in the art will appreciate that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVDROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0234] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for training a target tracking network for image-text association, characterized in that: The following steps are involved: Step S101: acquiring two adjacent frames of images in a video sequence each time, using the numth frame image and the num+1th frame image as a training image and a test image respectively, and cropping out image blocks where the target is located respectively; Step S102: inputting the target image block of the numth frame and the target image block of the num+1th frame into the image-text conversion module to obtain initial text features and existing text features; Step S103: processing the initial text features and the existing text features through the feature adaptation module to generate dynamic text features whose label weights can be updated in real time; Step S104: extracting features of the num+1th frame image using an improved transformer feature extraction network, wherein the improved transformer feature extraction network includes an encoder and a decoder; Step S105: by means of the image-text association module, the related features between the dynamic text features obtained in step S103 and the features of the num+1th frame image obtained in step S104 are associated and fused into the final related features for target state prediction; Step S106: Take the num+1th frame as a test image, estimate the state of the tracked target on the test image through a prediction module composed of 4 stacked convn bn relu layers, and obtain the coordinates of the target bounding box through deep learning.

2. The target tracking network training method for image-text association as claimed in claim 1, characterized in that: The image-text conversion module comprises: The label vocabulary is used to provide text labels to the image-text conversion module, which contains category labels, such as 'Car', 'Bike', and attribute labels, such as color attribute 'Red' and material attribute 'iron'; The feature reshaping operation is used to process the target image block feature x extracted by the image encoder Image encoder of the open source model CLIP, and decompose the feature map of x into a set of feature vectors with the shape of a two-dimensional array along the channel dimension, that is, reshape the height, width, and number of channels into height*width, number of channels, and obtain a set of related features v1, v2, ..., v of the target image block feature x of the numth frame k ; Text embedding operation, using onehot encoding to convert the labels in the label vocabulary into word vectors c i ; The image-text conversion module calculates the predicted probability of the i-th label of the tracked target in the target image block m, which is mathematically described as: Among them, Q i (x) represents the textual hint of the i-th label of the target in the target image patch m, which is composed of a set of relevant features v1, v2, ..., v of the target image patch feature x. k And the word vector c i Composition, i.e. Q i (x)={v1(x), v2(x),...,v k (x), c i }, t(·) represents the features processed by the text encoder of the open source model CLIP, S im (·,·) represents the cosine similarity score, which measures the similarity between the target image block feature x and the text feature t(Q i (x)), τ represents the parameter that adjusts the probability distribution predicted by the model, and N represents the number of labels.

3. The target tracking network training method for image-text association as claimed in claim 1, characterized in that: The feature adaptation module can generate dynamic text features whose label weights can be updated in real time based on the initial text features and the existing text features: The text feature generation is divided into category feature generation and attribute feature generation. C , which is mathematically described as: Among them, T i The text features of the i-th category label generated by the CLIP text encoder in the image-to-text conversion module of claim 1, p i The predicted probability of the i-th label calculated by formula 1; For each attribute feature Ta with the highest prediction probability, its mathematical description is: Among them, p i is the predicted probability of the i-th label calculated by formula 1, argmax(p i ) indicates taking the label with the highest predicted probability, and index indicates the number of the label with the highest predicted probability; Since the attribute values ​​of the tracked target in different frames of images will change, it is necessary to adjust the weights according to the changes in different attribute features; for example, the changes in color, material, and texture are recorded as D color , D material , D texture , which is mathematically described as: Among them, R attribute and S attribute It represents the probability of a specific attribute value calculated by formula 1 after the target in the target image block of the numth frame and the target image block of the num+1th frame is processed by the image-text conversion module. The attribute value is the attribute label such as color, material, texture, etc.; D attribute The lower the value, the more similar the objects on the target image block of the numth frame and the target image block of the num+1th frame are in this attribute; therefore, the attention weight formula for different attributes is: W att = Softmax(-D color , -D material , -D texture ) (Formula 5) Among them, Softmax(·) represents the classification function in the field of deep learning. color , D material , D texture The changes of other attributes are converted into probability distribution; Afterwards, the dynamic text features T of different attributes are att Adjusted to: T att =W att *T a (Formula 6) Among them, T a is the text feature generated using Formula 3.

4. The target tracking network training method for image-text association as claimed in claim 1, characterized in that: The improved transformer feature extraction network includes: The encoder is used to convert the input image sequence into a series of context vectors containing rich semantic information. Its structure consists of 16 identical layers stacked together. Each layer contains a self-attention mechanism SelfAttention and a feedforward neural network FFN. Due to the use of residual connections, the output of each layer is a combination of its input and the calculation result of the layer. The weighted attention mechanism can be described as: Among them, q represents the query, k represents the key, and v represents the value. q, k, and v are all linear transformations of the input sequence. k Represents the dimension of the key, T represents the time dimension, pos represents the position information encoding, Softmax represents the classification function, α represents the scaling factor, and the weight range of the attention mechanism can be adjusted according to the actual situation. The default value is 1; The decoder is used to convert the context vector generated by the encoder into a target sequence. Its structure is similar to that of the encoder, but it introduces an encoder-decoder attention mechanism, and the description of its attention mechanism weighting is consistent with Formula 7.

5. The target tracking network training method for image-text association as claimed in claim 1, characterized in that: The image-text association module combines image features and text features to obtain the final relevant features for target state prediction: set up The image feature extracted by the transformer feature extraction network is V∈R H×W×C , the text features extracted by the image-text conversion module are dimensionally adjusted to T∈R through the linear layer 1×1×c , so that it is aligned with the image features in the channel dimension, and then the correlation between the two features is realized through the convolution operation, where the text feature T∈R 1×1×C Is used as a kernel weight to add the relevant features between the image features and all text features as the final relevant features for state prediction, calculated as: C orr (V,T c ,T co ,T m ,T t )=V+(L c (T c ) +L co (T co )+L m (T m )+L t (T t ))⊙V(Formula 8) where ⊙ denotes a convolution operation, L(·) denotes a linear projection layer for channel adaptation, and T c represents the category feature, T co 、T m and T t Represents dynamic color, material and texture features respectively.

6. The target tracking network training method for image-text association as claimed in claim 1, characterized in that: The prediction module estimates the target state through the final relevant features and obtains the coordinates of the target bounding box through deep learning: Based on the final relevant features generated by the image-text association module, the prediction module composed of 4 stacked convn bn relu layers (3x3 convolution layer batch normalization layer activation function layer) processes the final relevant features V to obtain the feature map V F ; Use the classification function softmax to process the feature map V F , and obtain the classification score map C = softmax(V F ); The dimension map B describing the size of the prediction box is usually directly generated by the subsequent convn bn relu layer, and no additional function is required for conversion; then the coordinates of the target on the classification score map (x c ,y c )for: (x c ,y c )=argmax(C ij )(Formula 9) Among them, C ij represents any coordinate on the classification score map, argmax(·) represents the coordinate with the highest prediction score, that is, the coordinate of the tracked target on the classification score map; The calculated target prediction box location information is: (x, y, w, h) = (x c , y c , B w , B h )(Formula 10) Where (x c ,y c ) is the coordinate of the target on the classification score map, B w and B h Represents the width and height of the prediction box from B, and x, y, w, and h represent the horizontal coordinate, vertical coordinate, width, and height of the center point of the final target prediction box, respectively.

7. A target tracking network and training system for image-text association that implements the image-text association target tracking network training method as claimed in any one of claims 16, characterized in that: The target tracking network and training system for image-text association includes: The image-text conversion module is used to convert the target image block of the numth frame and the target image block of the num+1th frame where the selected target is located, which are cut out from the numth frame image and the num+1th frame image, into text features, obtain initial text features and existing text features, and provide a complete and accurate feature description of the tracked target; A feature adaptation module is used to process initial text features and existing text features to generate dynamic text features; The improved transformer feature extraction network is used to extract the features of the num+1th frame image; The image-text association module is used to associate the relevant features between the dynamic text features and the features of the num+1th frame image to generate the final relevant features of the target state; The prediction module is used to estimate the state of the tracking target on the test image and obtain the prediction result.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-text associated target tracking network training method as claimed in any one of claim 16.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the image-text associated target tracking network training method as claimed in any one of claim 16.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the target tracking network and training system with image-text association as described in claim 7.

Citation Information

Cited By

  • Target tracking method and device, electronic equipment, program product and medium

    CN119625746A

  • Method and system for replacing object in video

    CN119922350A