Multi-target tracking method and system based on optical flow information and identity prediction

Abstract

The invention discloses a multi-target tracking method and system based on optical flow information and identity prediction, and the method comprises the following steps: obtaining a to-be-tracked target and an original image, carrying out the preprocessing, and obtaining a reference image marked with the tracking target; extracting an optical flow feature map, and cutting optical flow features of the to-be-tracked target; splicing and fusing the previous frame of image and the current frame of image to generate a feature enhanced image after optical flow guidance; cutting image features of a to-be-tracked target, inputting the image features into the feature extraction network, and outputting target image features; splicing the optical flow feature and the target image feature of the to-be-tracked target to obtain a target feature of the to-be-tracked target, and splicing the target feature with the identity of the to-be-tracked target to obtain a tracking target to which the identity is to be allocated; and inputting a multi-target tracking model for prediction to obtain a prediction result of the tracking target. Identity prediction is carried out by using the optical flow features and the image features of the targets, and multi-target tracking under occlusion and nonlinear motion is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more particularly to a multi-target tracking method based on optical flow information and identity prediction. Background Art

[0002] Multi-target tracking is an important problem in the field of image processing and computer vision, which aims to identify and track multiple targets from a video sequence. This technology is widely used in the fields of autonomous driving, intelligent monitoring, etc. This task assigns a unique identity to the newly appearing objects in the video sequence, and assigns identities to the targets detected in subsequent frames to achieve long-term tracking of the targets.

[0003] Among the currently popular detection-based multi-target tracking methods, most methods use Kalman filtering to estimate target motion and generate a cost matrix based on the estimation results to match the identity and the detected object. However, in actual multi-target tracking, since target motion is easily blocked and the target motion pattern is complex and often nonlinear, the Kalman filter based on the linear motion assumption is difficult to accurately predict the target motion in most cases, causing the target matching information to be disturbed, resulting in unsatisfactory multi-target tracking results.

[0004] Multi-target tracking methods based on deep learning have been proposed. These algorithms use deep learning techniques to extract historical features of targets or for target prediction, to assist in solving identity assignment problems.

[0005] Therefore, how to effectively extract features and improve the accuracy of identity assignment is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a multi-target tracking method and system based on optical flow information and identity prediction, which effectively solves the multi-target tracking problem under occlusion and nonlinear motion by extracting optical flow information to assist in image feature enhancement and using the optical flow features and image features of the target for identity prediction.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention proposes a multi-target tracking method based on optical flow information and identity prediction, comprising the following steps:

[0009] S1, obtaining the target to be tracked and the original image, performing preprocessing, and obtaining a reference image with the target to be tracked marked;

[0010] S2, using a lightweight deep optical flow network to extract an optical flow feature map of the current frame in the reference image compared with the previous frame image, and cropping the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked;

[0011] S3, using the optical flow feature map, splicing and fusing the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow;

[0012] S4, cropping the feature-enhanced image within the target frame to be tracked as the image feature of the target to be tracked, inputting the image feature into a feature extraction network, and outputting the target image feature;

[0013] S5, splicing the optical flow features and the target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity;

[0014] S6. Input the tracking target to be assigned an identity into a multi-target tracking model for prediction to obtain a prediction result of the tracking target.

[0015] In a second aspect, the present invention proposes a multi-target tracking system based on optical flow information and identity prediction, comprising the following modules:

[0016] Image acquisition module: used to acquire the target to be tracked and the original image, perform preprocessing, and obtain a reference image with the tracked target marked;

[0017] Optical flow feature extraction module: used to extract the optical flow feature map of the current frame in the reference image compared with the previous frame image by using a lightweight deep optical flow network, and to crop the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked;

[0018] Feature enhancement module: used to use the optical flow feature map to splice and fuse the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow;

[0019] Target image feature extraction module: used to crop the feature-enhanced image within the target frame to be tracked as the image feature of the target to be tracked, input the image feature into the feature extraction network, and output the target image feature;

[0020] Tracking target splicing module: used for splicing the optical flow features and target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity;

[0021] Tracking target prediction module: used for inputting the tracking target to be assigned an identity into the multi-target tracking model for prediction, and obtaining the prediction result of the tracking target.

[0022] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a multi-target tracking method and system based on optical flow information and identity prediction, which has the following beneficial effects:

[0023] 1. The present invention combines optical flow features and image features to extract target features, thereby achieving information enhancement and multi-angle feature extraction.

[0024] 2. Combine the identity information with the target to be tracked, and use the cross-attention mechanism and self-attention mechanism to better capture the target features of the historical tracking object and achieve accurate multi-target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 A flow chart of a multi-target tracking method based on optical flow information and identity prediction provided by an embodiment of the present invention.

[0027] Figure 2 This is an architecture diagram of an image feature enhancement module based on optical flow provided in an embodiment of the present invention.

[0028] Figure 3 An architecture diagram of a multi-target tracking method based on optical flow information and identity prediction provided in an embodiment of the present invention.

[0029] Figure 4 A schematic diagram of a multi-target tracking system based on optical flow information and identity prediction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Embodiment 1

[0032] The embodiment of the present invention discloses a multi-target tracking method based on optical flow information and identity prediction, referring to Figure 1 As shown, the following steps S1 to S6 are included:

[0033] S1, obtaining the target to be tracked and the original image, performing preprocessing, and obtaining a reference image with the target to be tracked marked;

[0034] In this embodiment, the successfully detected object and the original image are obtained, and the successfully detected object of each frame is used as the target to be tracked. The image and the detection target frame are uniformly adjusted to the same scale using image processing software or library. The specific operation is to limit the side length of the image that is too large, and scale the detection frame according to the image scaling result to ensure that the detection frame accurately selects the target to be tracked. This step ensures the consistency of subsequent processing and the effectiveness of feature extraction, while ensuring training efficiency.

[0035] S2. Use a lightweight deep optical flow network to extract an optical flow feature map of the current frame compared to the previous frame in the reference image, and crop the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked;

[0036] In this embodiment, a lightweight pre-trained optical flow extraction network is used. The optical flow extraction network automatically estimates the optical flow field from continuous video frames through a convolutional neural network. Two frames of images are input, motion displacement is learned through training, and the prediction result is optimized using a loss function. After training, the optical flow information can be quickly inferred on the two frames of images.

[0037] The lightweight deep optical flow network is used to quickly and efficiently extract the optical flow information of the current frame compared with the previous frame, and obtain the optical flow feature map Flow t Based on the reference image of the marked tracking target, find the center position of the target to be tracked, crop the 3x3 size optical flow feature map near the center point of the detection target, flatten it and splice the center point position coordinates as the optical flow feature Fcrop t .

[0038] S3, using the optical flow feature map, splicing and fusing the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow;

[0039] In this embodiment, refer to Figure 2 As shown, using the optical flow feature map Flow t For the previous frame image I t-1 Perform bilinear sampling to obtain the sampled optical flow features. The formula is:

[0040] F s (i,j)=BS(I t-1 ,i,j,dx,dy)

[0041] BS(I,i,j,dx,dy)=s x s y I(x,y)+(1-s x )s yI(x+1,y)

[0042] +s x (1-s y )I(x,y+1)+(1-s x )(1-s y )I(x+1,y+1)

[0043] s x =i+dx-[i+dx]

[0044] s y =j+dy-[j+dy]

[0045] BS(I,i,j,dx,dy) is a bilinear sampling operation for image I at (i,j) with an optical flow size of (dx,dy), where dx,dy are the x-axis offset and y-axis offset of the optical flow at position (i,j), I(x,y) represents the image pixel value at (x,y), x represents [i+dx], y represents [j+dy], [.] represents a rounding down operation, and s x ,s y It is an interpolation value limited to the range (0,1).

[0046] Concatenate the optical flow features and the current frame image I in the channel dimension t , pass the spliced ​​features into a 3x3 convolution to fuse the optical flow features and the current feature image, and limit the number of output channels to 3, and output the fused image F with optical flow guidance feature enhancement fuse , and pass this feature into a 1x1 convolution and limit the number of channels to 1. After normalization through the Sigmoid network layer, the attention weight F is generated. att The weights and fusion image F fuse The multiplication result is equal to the current image I t Add them together to get the feature enhanced image I after optical flow guidance aug .

[0047] The formula is:

[0048] I aug =F fuse *F att +I t

[0049] S4, cropping the feature enhanced image within the target frame to be tracked as the image feature of the target to be tracked, inputting the image feature into the feature extraction network, and outputting the target image feature;

[0050] In this embodiment, based on the target to be tracked, a detection target frame of the target to be tracked is obtained, and a feature enhanced image I in the target frame to be tracked is obtained. augThe image is cropped as the current frame image feature of the target. The image processing software or library is used to uniformly adjust the current frame image feature of the detected target to 32x32 and then passed to the convolutional neural network module. After the features are flattened after the three convolution modules, they are passed to a fully connected network to obtain a feature vector of size 256 as the target image feature F. app .

[0051] S5, splicing the optical flow features and the target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity;

[0052] In this embodiment, refer to Figure 3 As shown in the figure, for each target to be tracked, a learnable feature vector is used as the identity feature of the new object. The identity feature is generated by using one-hot encoding and input into the fully connected network to generate a feature vector of size 256. The target image feature F generated by each target to be tracked in steps S2 and S4 is app And optical flow feature Fcrop t The target features of the target to be tracked are spliced ​​in the size dimension as the target features of the target to be tracked. The identity features and target features of the new object are spliced ​​in the size dimension as the target information to be tracked that needs to be assigned an identity. All the target information to be tracked is spliced ​​in the channel dimension as the feature vector F of all the targets to be tracked. obj The formula is:

[0053] F id =f id (newborn)

[0054] F obj =concat(F id ,Fcrop t ,F app )

[0055] Among them, newbqrn is the unique hot encoded identity label of the preset new object, F id The identity characteristics of the new object.

[0056] S6. Input the tracking target to be assigned an identity into the multi-target tracking model for prediction, and obtain a prediction result of the tracking target.

[0057] This embodiment extracts optical flow motion information and image feature information as target features through an optical flow extraction module and an image feature enhancement module respectively, concatenates identity vectors as guidance for identity prediction, and inputs them into a backbone prediction model to output a target identity vector and feature vector that fuses historical information using an attention mechanism, thereby achieving identity prediction.

[0058] In this embodiment, the multi-target tracking model is composed of six cross-attention modules and five self-attention modules alternately. The tracking target to be assigned an identity is input into the model as the query part. For the cross-attention module, the identity features and target features of the tracking results of the previous T frames are spliced ​​as historical tracking information, and the historical tracking information is input as the key and value.

[0059] For the cross attention part, the formula is expressed as:

[0060]

[0061] For the self-attention part, the formula is expressed as:

[0062]

[0063] in, represents the learnable weight matrix, F c and F s Denote the feature vectors generated by cross attention and self-attention, Q obj represents the query for the object to be tracked, K his and V his Indicates the key and value corresponding to the history object, and K obj and V obj They represent the key and value of self-attention for the object to be tracked.

[0064] For the multi-object tracking model, six cross-attention modules and five self-attention modules Incoming F obj and historical information his , the multi-target tracking model formula is expressed as:

[0065]

[0066]

[0067] where F i is the output result of the middle part, F out is the output vector of the multi-target tracking model.

[0068] The following is a detailed description of the model training process in this embodiment.

[0069] Reference Figure 3 As shown, Γ t-T represents the historical information of the multi-target tracking model before the Tth frame, Γ t-1Represents the historical information of the previous frame passed into the multi-target tracking model. First, for the historical information part, an identity dictionary is preset based on the tracking targets that have been assigned identities in the historical T frames. For each historical target with a successfully assigned identity object, a learnable 256-dimensional feature vector is matched with its identity identifier as the identity feature vector of the target that has been successfully assigned an identity. The target information of the tracking objects that have been assigned identities in the historical T frames is concatenated with the corresponding identity feature vector as a 512-dimensional vector. For different periods of the historical T frames, T 512-dimensional learnable feature vectors are generated and added to the T-frame historical information respectively as the embedding of time information;

[0070] Secondly, the feature vector generated by the last layer of cross attention is used as the feature vector of the tracking target combined with historical information. The generated feature vector is split. The first 256-dimensional vector is used as identity information to input into the identity prediction model, the middle 256-dimensional vector is used as image features to input into the image feature extraction model, and the remaining features are used as optical flow features to input into the motion prediction model. All three models are composed of fully connected neural networks. The formula is expressed as:

[0071] (F ID ,F app ,F mv )=split(Γ t )

[0072] R ID =f ID (F ID )

[0073] R app =f app (F app )

[0074] R mv =f mv (F mv )

[0075] Among them, Γ t Represents the feature vector output by the multi-target tracking model, f ID 、f app 、f mv They represent the identity prediction output head, image feature output head, and motion prediction output head respectively; R ID , R app , R mv Corresponding identity prediction results, appearance feature results, and displacement prediction results.

[0076] Finally, the improved loss function is used to conduct multi-angle evaluation on the identity prediction results, image feature extraction, motion information prediction, and identity-guided feature extraction results obtained by the multi-target tracking model. Among them, the identity prediction loss uses the cross-entropy loss of the T-frame average, the image feature loss uses the cosine loss, the motion information loss uses the L1 loss of the predicted displacement and the actual displacement, and the identity feature-guided extraction loss uses the weighted sum of the identity probability multiplied by the feature loss of the corresponding identity. The weights of each loss are adjusted through hyperparameters as the overall loss.

[0077] The formula is:

[0078] Loss = λ1L id +λ2L ap +λ3L mv +λ4L id_guide_ap +λ5L id_guide_mv

[0079]

[0080] Among them, Loss represents the overall loss, L id represents the identity feature loss, L ap represents the image feature loss, L mv represents the motion feature loss, L id_guide_ap and L id_guide_mv They represent the identity-guided image feature extraction loss and the identity-guided motion feature extraction loss respectively; λ1, λ2, λ3, λ4, λ5 are the weight hyperparameters of the corresponding losses; M t represents the number of targets in the tth frame, 2≤t≤T; p_t m ={p_t m 1 ,p_t m 2 ,...,p_t m k ,p_t m newborn} represents the prediction result of the mth target in the tth frame, k is the preset number of targets to be tracked, gt_t m represents the actual identity result of the mth target in the tth frame; R t = {r t 1,r t 2,...,r t Mt} represents the image features of the tth frame, where r t m Represents the image features of the mth target in the tth frame, his_r t m = {r t -Tid＝m ,...,r t-2 id＝m r t-1 id＝m} represents all historical image features with the same identity as m in the historical trajectory, D t ={d t 1,d t 2,...,d t Mt} is the displacement prediction result of the tth frame, gt_d t m is the displacement difference of the target with the same identity m between the t-th frame and the t-1-th frame, L ap (i,his_r t i ) is the minimum image feature loss between the historical target with identity i and the current target, L mv (i,gt_d t m ) is the minimum motion information loss between the historical target with identity i and the current target.

[0081] The model is trained by evaluating the results. When the model loss converges, the model training loss and verification loss tend to be stable, and multi-target tracking can be performed well, the model training is successful. The trained neural network model is obtained. The multi-target tracking model can accurately assign the target identity and complete multi-target tracking.

[0082] The present invention combines optical flow information and deep learning. First, the optical flow information is extracted to obtain the optical flow features of the current frame. Secondly, the optical flow information is combined with the image information to achieve information enhancement. At the same time, the detection frame is used to extract the image features and optical flow features of the object to be tracked, and multi-angle feature extraction is performed. Finally, the identity information is combined with the target to be tracked, and the identity of the object to be tracked is output end-to-end to perform target tracking. The target features are extracted by combining the optical flow features and the image features, and the cross-attention mechanism is used to better capture the target features of the historical tracking objects, thereby achieving accurate multi-target tracking.

[0083] Embodiment 2

[0084] The embodiment of the present invention discloses a multi-target tracking system based on optical flow information and identity prediction, Figure 4 As shown, it includes the following modules:

[0085] Image acquisition module: used to acquire the target to be tracked and the original image, perform preprocessing, and obtain a reference image with the tracked target marked;

[0086] Optical flow feature extraction module: used to extract the optical flow feature map of the current frame in the reference image compared with the previous frame image by using a lightweight deep optical flow network, and to crop the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked;

[0087] Feature enhancement module: used to use the optical flow feature map to splice and fuse the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow;

[0088] Target image feature extraction module: used to crop the feature-enhanced image within the target frame to be tracked as the image feature of the target to be tracked, input the image feature into the feature extraction network, and output the target image feature;

[0089] Tracking target splicing module: used for splicing the optical flow features and target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity;

[0090] Tracking target prediction module: used for inputting the tracking target to be assigned an identity into the multi-target tracking model for prediction, and obtaining the prediction result of the tracking target.

[0091] The system of the present invention first includes an image acquisition module, which is used to acquire the target to be tracked and the original image, pre-process the image data and the tracked target, and scale them to the same scale to ensure that the detection result correctly selects the target; then, in the optical flow feature extraction module, two adjacent frames of images are passed into the optical flow feature network to extract the optical flow information of the current frame of the image. The optical flow information shows pixel-level motion information, and the optical flow information near the center point of the detected object is used as the rough motion information of the object for subsequent estimation of motion information.

[0092] At the same time, in the feature enhancement module, bilinear sampling is performed using the previous frame image and optical flow information, the sampling result and the current frame image are spliced, and the image features are adaptively enhanced using a neural network to facilitate the subsequent extraction of image features of the tracked target; secondly, in the target image feature extraction module, based on the target to be tracked, the target frame position of the target to be tracked is selected, and the obtained feature enhanced image is cropped as the current frame image feature of the target to be tracked. The image obtained in this way is fused with optical flow information to achieve the effect of image enhancement. After the image size is scaled to a preset size, it is passed into the convolutional neural network module, and the result after the convolution operation is flattened to ensure that each target to be tracked has image features of the same size.

[0093] Then, in the tracking target splicing module, the tracking target prediction module splices the image features and optical flow features of each target to be detected as the target features of the target. At the same time, the target features are spliced ​​with the pre-generated identity feature vector indicating the new object as the tracked target information that needs to be assigned an identity. The information obtained in this way contains the motion information and image information of the object to be assigned an identity, which is helpful for subsequent prediction operations.

[0094] Subsequently, in the tracking target prediction module, the self-attention mechanism and the cross-attention mechanism are used, the target information of the identity to be tracked is used as a guide, and the historical information of the successful identity assignment is used to adaptively generate the feature vector of the tracking target combined with the historical information, and the feature vector is passed into the prediction module to obtain the prediction result of the identity and the target feature combined with the historical information; finally, according to the improved loss function, the identity prediction result, image feature extraction, motion information prediction, and identity guidance feature extraction results are evaluated from multiple angles, and the model is trained by the evaluation results to obtain the trained neural network model, and finally the accurate assignment of the target identity is achieved to complete multi-target tracking. The system of the present invention can adaptively combine the historical target information to achieve more accurate multi-target tracking.

[0095] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0096] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-target tracking method based on optical flow information and identity prediction, characterized in that: The following steps are involved: S1, obtaining the target to be tracked and the original image, performing preprocessing, and obtaining a reference image with the target to be tracked marked; S2, using a lightweight deep optical flow network to extract an optical flow feature map of the current frame compared with the previous frame in the reference image, and cropping the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked; S3, using the optical flow feature map, splicing and fusing the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow; S4, cropping the feature-enhanced image within the target frame to be tracked as the image feature of the target to be tracked, inputting the image feature into a feature extraction network, and outputting the target image feature; S5, splicing the optical flow features and the target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity; S6. Input the tracking target to be assigned an identity into a multi-target tracking model for prediction to obtain a prediction result of the tracking target.

2. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 1, characterized in that: The preprocessing in step S1 specifically includes: Adjust the original image and the target to be tracked to the same scale.

3. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 2, characterized in that: In step S2, a lightweight deep optical flow network is used to extract an optical flow feature map of the current frame in the reference image compared with the previous frame, and the optical flow information near the center point of the target to be tracked is cropped as the optical flow feature of the target to be tracked; Specifically include: Inputting the current frame and the previous frame image in the reference image into a convolutional neural network; Learn motion displacement through training and use loss function to optimize prediction results; Infer the optical flow feature map of the current frame compared with the previous frame on two adjacent frames; The optical flow features of 3x3 size around the center point of the target to be tracked are cropped as the optical flow features of the target to be tracked.

4. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 3, characterized in that: In step S3, the optical flow feature map is used to splice and fuse the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow; specifically, the process includes: Using the optical flow feature map, bilinear sampling is performed on the previous frame image; Splice the obtained sampling result with the current frame image; The obtained spliced ​​features are passed through a convolutional network to generate a fused image and attention weights; The attention weight is multiplied by the fused image and then added to the current frame image to serve as a feature enhanced image guided by optical flow.

5. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 4, characterized in that: The obtained splicing features are used to generate a fused image and attention weights through a convolutional network; specifically including: The obtained splicing features are passed into the 3x3 convolutional network for fusion, and the number of output channels is set to 3 to obtain a fused image enhanced by optical flow guidance features; Similarly, the obtained splicing features are passed into the 1x1 convolutional network and the number of channels is set to 1. After normalization through the network layer, the attention weight is generated.

6. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 5, characterized in that: In step S4, the image features are input into a feature extraction network to output target image features; specifically, the steps include: Use image processing software or library to uniformly adjust the image features of the target to be tracked to 32x32 and pass them into the convolutional neural network module; After being acted upon by the three convolution modules of the convolutional neural network module, the obtained features are flattened and passed into the fully connected network to obtain a feature vector of size 256 as the target image feature.

7. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 6, characterized in that: In step S5, the optical flow features and target image features of the target to be tracked are spliced ​​to obtain the target features of the target to be tracked, and the target features are spliced ​​with the identity of the target to be tracked to obtain the tracked target to be assigned an identity; specifically, the following steps are included: The optical flow features and the target image features of the target to be tracked are concatenated in the size dimension to obtain the target features of the target to be tracked; A learnable feature vector is used as the identity of the target to be tracked, and is concatenated with the target feature in the size dimension to obtain the tracking target to be assigned an identity.

8. The multi-target tracking method based on optical flow information and identity prediction as claimed in claim 7, characterized in that: In step S6, the tracking target to be assigned an identity is input into a multi-target tracking model for prediction to obtain a prediction result of the tracking target; specifically, the step includes: Input the tracking targets to be assigned identities and the tracking targets with assigned identities in the historical T frames into the multi-target tracking model; Use the cross-attention mechanism and self-attention mechanism to generate the feature vector of the tracking target combined with historical information; Using a fully connected neural network, the identity prediction result, appearance feature result and displacement prediction result of the tracking target to be assigned an identity are obtained.

9. A multi-target tracking method based on optical flow information and identity prediction as claimed in claim 8, characterized in that: Step S6 further includes: Performing multi-angle evaluation on the identity prediction results, appearance feature results and displacement prediction results through a loss function; The multi-target tracking model is optimized and adjusted according to the evaluation results.

10. A multi-target tracking system based on optical flow information and identity prediction, characterized in that: Includes the following modules: Image acquisition module: used to acquire the target to be tracked and the original image, perform preprocessing, and obtain a reference image with the tracked target marked; Optical flow feature extraction module: used to extract the optical flow feature map of the current frame in the reference image compared with the previous frame image by using a lightweight deep optical flow network, and to crop the optical flow information near the center point of the target to be tracked as the optical flow feature of the target to be tracked; Feature enhancement module: used to use the optical flow feature map to splice and fuse the previous frame image with the current frame image to generate a feature enhanced image guided by the optical flow; Target image feature extraction module: used to crop the feature-enhanced image within the target frame to be tracked as the image feature of the target to be tracked, input the image feature into the feature extraction network, and output the target image feature; Tracking target splicing module: used for splicing the optical flow features and target image features of the target to be tracked to obtain the target features of the target to be tracked, and splicing them with the identity of the target to be tracked to obtain the tracking target to be assigned an identity; Tracking target prediction module: used for inputting the tracking target to be assigned an identity into the multi-target tracking model for prediction, and obtaining the prediction result of the tracking target.

Images