A multi-target tracking and recognition method based on non-local gait features

Through instance segmentation and Hungarian algorithms, and combined with the non-local block network to extract features, the difficulty of tracking and segmentation of gait recognition under multiple walking and poor lighting conditions is solved, and the recognition accuracy and fine-grainedness are improved Feature extraction.

CN114724065BActive Publication Date: 2024-11-15NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210330483.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-31
Publication Date
2024-11-15
Estimated Expiration
2042-03-31

AI Technical Summary

Technical Problem

Existing gait recognition technology is difficult to effectively track and segment targets under conditions of multiple people walking and poor lighting, especially in poor video quality or different target sizes, resulting in poor feature extraction effect and affecting the accuracy of recognition.

Method used

A multi-objective tracking algorithm based on instance segmentation is used to achieve target correlation with the Hungarian algorithm, generate gait energy maps, and extract non-local features through an improved non-local block network, and feature extraction is combined with instance labels and convolution modules to improve fine-grainedness Feature extraction and identification accuracy.

Benefits of technology

It improves multi-objective segmentation accuracy and recognition accuracy, effectively handles occlusion and lighting changes, and improves the fine-grainedness of gait features and recognition reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114724065B_ABST
    Figure CN114724065B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-target tracking and recognition method based on non-local gait features. The present invention first obtains a gait video sequence, and detects and segments the video sequence in the classification and positioning branch and the mask prediction branch; then, in the tracking branch, the instance label partitioning method and the Hungarian algorithm are used to associate the tracking targets to obtain the gait energy graphs of all targets; then, the gait energy graph is input into the improved non-local block network to generate a feature graph. Different regions are divided according to the feature graph, and different weights are assigned to obtain fine-grained gait features. Finally, the extracted gait features are input into a one-to-many support vector machine for target recognition. The present invention detects and segments the video sequence through the classification and positioning branch and the mask prediction branch, which helps to improve the accuracy of multi-target segmentation; through the instance label partitioning method and the Hungarian algorithm, the association of targets between different frames is effectively realized; through the improved non-local block network, it helps to extract fine-grained non-local features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to cross-technical fields such as computer vision, image processing, and identity recognition, and in particular to a multi-target tracking and recognition method based on non-local gait features. Background Art

[0002] As a human recognition technology, gait recognition has received increasing attention in recent years. Gait recognition can be used in intelligent video surveillance systems to identify criminal suspects or as a new means of medical diagnosis for leg rehabilitation. Compared with other biometric recognition methods such as face recognition and fingerprint recognition, gait recognition has the advantage of being easy to identify remotely, and the recognition process usually does not require the object to be identified to cooperate intentionally. For gait recognition, gait tracking, segmentation and feature extraction are very important parts. A good tracking and segmentation algorithm can handle the occlusion problem of pedestrian videos; in addition, gait recognition often requires more refined features than action recognition, so feature extraction methods are particularly important.

[0003] Feature extraction refers to the process of removing irrelevant features and retaining relevant features. It can also be considered as selecting the best feature subset from all features. According to current research, feature extraction techniques in gait recognition can be divided into five categories: body parameter-based methods, contour-based methods, solid contour-based methods, human characteristics-based methods, and fusion technology-based methods. These five feature extraction algorithms have been developed to be relatively mature, but there are still some problems. For example, when multiple people walk, the contours between people may be blocked. In addition, walking under poor lighting conditions or walking with too large an amplitude will also affect the extraction effect.

[0004] At present, most of the feature extraction methods based on external contours use target tracking algorithms to achieve feature extraction of multiple targets. Target tracking can be divided into two cases according to the target type: point target tracking and target tracking for targets that occupy a certain area with texture, contour and other features. The latter can be divided into methods based on feature point detection, frame difference method based on background subtraction, method based on segmentation ideas, method based on supervised learning, and method based on features such as target shape contour. Gait tracking not only requires short tracking of each target, but also requires segmenting each target. Most existing gait recognition algorithms assume that people have been tracked and their contours have been successfully extracted; however, due to occlusion, lighting changes, and different human shapes and joints, tracking and segmenting people is actually very difficult. In addition, if the quality of the surveillance video is too poor or the targets in the video are of different sizes, segmenting and tracking the target will be even more difficult. Summary of the invention

[0005] To solve the above problems, the present invention proposes a multi-target tracking and recognition method based on non-local gait features. The method realizes target segmentation within one cycle, adopts instance label partitioning method and Hungarian algorithm in the tracking head to realize target association, generates a gait energy map of each target, and extracts non-local features of the gait energy map to better extract fine-grained gait features and improve the accuracy of identity recognition.

[0006] The present invention provides a multi-target tracking and recognition method based on non-local gait features, comprising the following steps:

[0007] Step 1: input a gait video sequence of a single person or multiple people walking, and process the gait video sequence through the classification and positioning branch and the mask prediction branch of the multi-target tracking algorithm based on instance segmentation to obtain a segmented sequence frame;

[0008] Step 2: Process the sequence frames in the tracking branch of the multi-target tracking algorithm based on instance segmentation, add two fully connected layers in the branch to assign instance labels to all targets, realize the associated tracking of multiple targets, and thus obtain the corresponding gait energy maps of all targets;

[0009] Step 3: Define the gait energy graph generated in step 2 as Among them I t is the binary silhouette image at the tth frame, (x, y) is in I t The coordinates of each pixel, T is the number of frames in a gait cycle;

[0010] Step 4: According to the definition of GEI in step 3, preprocess the gait energy graphs corresponding to all targets obtained in step 2 to generate positive and negative GEI pairs;

[0011] Step 5, input the positive and negative GEI pairs of the gait energy map generated in step 4 into the convolution module and the corresponding non-local feature extraction module of the improved non-local block network respectively, and superimpose the features extracted from the two modules;

[0012] Step 6: Process the superimposed features of the positive and negative GEI pairs obtained in step 5 to obtain a feature map, and then divide the feature map into blocks to obtain more fine-grained features;

[0013] Step 7: Input the features obtained from step 6 into a one-to-many support vector machine for target recognition.

[0014] Furthermore, the specific steps of processing in the classification and positioning branch and the mask prediction branch of the multi-target tracking algorithm based on instance segmentation in step 1 are:

[0015] Step 11: First, input the initial gait video sequence into the ResNet-01 backbone network to extract features and obtain the region of interest;

[0016] Step 12: Send the region of interest obtained in step 11 to the region proposal network RPN, which continuously scans the region of interest to generate candidate regions by means of a sliding window;

[0017] Step 13: Determine whether the edge of the candidate region generated in step 12 is blurred or not. If it is not blurred, use the regional feature aggregation method of the traditional bilinear interpolation method to traverse each candidate region and extract candidate features for each candidate region; otherwise, use the oblique line interpolation method to improve the image edge segmentation effect; the formulas for the traditional bilinear interpolation method and the oblique line interpolation method for a candidate region are as follows:

[0018] Traditional bilinear interpolation:

[0019] Oblique interpolation method:

[0020] Where m11=(i1,j1), m12=(i1,j2), m21=(i2,j1), m22=(i2,j2), i and j are the coordinates of the pixels selected in the current candidate area, (i1,j1), (i1,j2), (i2,j1), (i2,j2) are the coordinates of the four endpoints of the current candidate area, f(p1) is the result of bilinear interpolation of the current image, and f(p2) is the result of oblique interpolation of the current image;

[0021] Step 14: Send the candidate features obtained in step 13 to the classification and positioning branch and the mask prediction branch respectively. In the classification and positioning branch, the prediction architecture of the Faster R-CNN algorithm is used to achieve classification and positioning; in the mask prediction branch, the fully convolutional network FCN is used to achieve segmentation.

[0022] Furthermore, the specific steps of processing the multi-target tracking algorithm based on instance segmentation in step 2 are:

[0023] Step 21: Process the segmented sequence frames generated in step 1 in the tracking branch to achieve matching of targets between different frames and cross-frame connection;

[0024] Step 22: After the classification and positioning branch and the mask prediction branch detect the target to be tracked, the tracking branch assigns an instance label to each candidate box through two fully connected layers. The first frame is the first label by default, and the label to which the candidate box of the current frame belongs is calculated; if it does not belong to the previous label, it is a new label; assuming that the previous frame already has N instance results, the current instance is regarded as an N+1 classification problem, and the formula for determining whether two adjacent frames belong to the same label is as follows:

[0025]

[0026] The N instance results obtained in the previous frame are passed through two fully connected layers in the tracking branch to obtain feature C j , j = 1, ..., N, i is the sampling result of the current frame, the feature is C i ;

[0027] Step 23: Based on the result of step 22, if two adjacent frames belong to the same instance label, the Hungarian algorithm is used to associate the two frames to determine the motion trajectory of the pedestrian target.

[0028] Step 24: After associating all frames of the video sequence using the Hungarian algorithm, gait energy graphs of all targets are obtained.

[0029] Further, the gait energy graphs corresponding to all targets obtained by preprocessing step 2 in step 4 include the following steps:

[0030] Step 41: input the gait energy graphs corresponding to all targets obtained in step 2 into a custom random function algorithm to generate positive and negative GEI sample pairs;

[0031] Step 42: Manually screen the two sample pairs generated in step 41 to select clear sample pairs to ensure the quality of the sample pairs.

[0032] Furthermore, the improved non-local block network in step 5 includes the following steps:

[0033] Step 51, inputting the positive and negative GEI pairs generated in step 4 into the improved non-local block network, and processing the positive and negative GEI pairs respectively;

[0034] Step 52: first input the positive and negative GEI pairs into the corresponding convolution module, perform two convolutions respectively, and obtain the corresponding convolution results;

[0035] Step 53: extract the non-local information features of the positive and negative GEI pairs in the non-local module, where the non-local operation is: i =G i +W C X i , where C i is a non-local feature, G i represents the output of GEI after the convolution module, X i represents the output after the non-local module, W C For X i Zoom in with G i Same dimensions;

[0036] Step 54: The feature extraction process of the non-local module described in step 53 is as follows: the feature map obtained by the convolution module is input into three 1x1 convolution channels, where θ and Perform feature extraction on a set of GEI pairs in two channels to extract two types of non-local information, namely, the non-local information of a single GEI and the non-local information between two GEIs; transpose the two output information matrices and calculate the similarity of the corresponding inputs. On this basis, use the Softmax function to obtain the combined attention of the channels and the normalized correlation between each pixel and all pixels in the current feature map, so as to calculate the pixel at each position;

[0037] Step 55, the channel g in the non-local module also takes the same operation, that is, first use 1x1 convolution to reduce the dimension and then perform matrix transposition operation with the calculation results of the first two channels to calculate the correlation between each pixel in the current feature map and all other position pixels;

[0038] Step 56, the final result is processed by the output channel of a 1×1 convolution kernel, so that the output size of the GEI sample obtained by the non-local module is the same as the size before input, and the corresponding non-local information feature is obtained;

[0039] Step 57: superimpose the convolution result obtained in step 52 and the non-local information feature obtained in step 56, and then superimpose the superposition results of the positive and negative GEI pairs again to obtain a superposition result.

[0040] Further, in step 6, processing the superposition features of the positive and negative GEI pairs obtained in step 5 includes the following steps:

[0041] Step 61, convolve the superposition result obtained in step 52 twice, superimpose the convolution result with the extracted non-local features, and input the superposition result into a subsequent convolution module to obtain a feature map;

[0042] Step 62: The feature map obtained by the convolution module and the non-local module is divided into static blocks, slightly dynamic blocks and strongly dynamic blocks, and higher weights are given to static blocks and slightly dynamic blocks with significant features;

[0043] Step 63: After obtaining the weights of the three feature blocks in step 62, three binary classifiers using the softmax function as the activation function are trained separately.

[0044] The beneficial effects described in the present invention are as follows: the present invention first obtains a gait video sequence, processes it using a multi-target tracking algorithm based on instance segmentation, detects and segments each target in the classification regression and mask prediction branches of the obtained initial gait video sequence, and improves the accuracy of multi-target segmentation; in the target tracking branch, the instance label partitioning method and the Hungarian algorithm are used to achieve target association, and the gait energy map of all targets is generated; in the feature extraction stage, an improved non-local block network is used, the image is input into the convolution module for two convolutions, and the convolution result is superimposed with the feature information obtained in the non-local module and then multiple convolutions are performed to extract more accurate features. The present invention uses a random function to process the gait energy map to generate positive and negative GEI pairs; the generated result is input into the improved non-local block network to generate a feature map, and the feature map is horizontally divided into a strong dynamic area, a weak dynamic area, and a micro-dynamic area; three secondary classifiers are used to perform weighted integration of the segmented gait features, and different weights are given to different feature areas to achieve optimal feature extraction, which helps to obtain the gait features of each person and effectively improves the accuracy and reliability of target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is the overall flow chart.

[0046] Figure 2 It is a flowchart of the multi-target tracking algorithm based on instance segmentation.

[0047] Figure 3 It is an improved non-local block network flow chart. DETAILED DESCRIPTION

[0048] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0049] like Figure 1 , Figure 2 and Figure 3 As shown, a multi-target tracking and recognition method based on non-local gait features includes the following steps:

[0050] Step 1: Input a video sequence of a single person or multiple people walking gait, and process it through the classification and positioning branch and mask prediction branch of the multi-target tracking algorithm based on instance segmentation to obtain the segmented sequence frames; the details are as follows:

[0051] Step 11: First, input the initial gait video sequence into the ResNet-01 backbone network to extract features and obtain the region of interest;

[0052] Step 12: Send the region of interest obtained in step 11 to the region proposal network RPN, which continuously scans the region of interest to generate candidate regions by means of a sliding window;

[0053] Step 13: Determine whether the edge of the candidate region generated in step 12 is blurred or not. If it is not blurred, use the regional feature aggregation method of the traditional bilinear interpolation method to traverse each candidate region and extract candidate features for each candidate region. Otherwise, use the oblique line interpolation method. The angle adopted here is preferably 135° linear interpolation method, which helps to improve the image edge segmentation effect. The formulas for the traditional bilinear interpolation method and the oblique line interpolation method for a candidate region are as follows:

[0054] Traditional bilinear interpolation:

[0055] Oblique line interpolation

[0056] Where m11=(i1,j1), m12=(i1,j2), m21=(i2,j1), m22=(i2,j2), i and j are the coordinates of the pixels selected in the current candidate area, (i1,j1), (i1,j2), (i2,j1), (i2,j2) are the coordinates of the four endpoints of the current candidate area, f(p1) is the result of bilinear interpolation of the current image, and f(p2) is the result of oblique interpolation of the current image;

[0057] Step 14: Send the candidate features obtained in step 13 to the classification and positioning branch and the mask prediction branch respectively. In the classification and positioning branch, the prediction architecture of the Faster R-CNN algorithm is used to achieve classification and positioning; in the mask prediction branch, the full convolutional network FCN is used to achieve segmentation; the Faster R-CNN algorithm is one of the classic algorithms in the field of target detection, and the region proposal network is used to replace the selective search algorithm.

[0058] Step 2: Process the sequence frames obtained in step 1 in the tracking branch of the multi-target tracking algorithm based on instance segmentation, add two fully connected layers in this branch and assign instance labels to all targets to achieve multi-target association tracking, thereby obtaining the corresponding gait energy graphs of all targets;

[0059] Step 21: Process the segmented sequence frames generated in step 1 in the tracking branch to achieve matching of targets between different frames and cross-frame connection;

[0060] Step 22: After the classification and positioning branch and the mask prediction branch detect the target to be tracked, the tracking branch assigns an instance label to each candidate box through two fully connected layers. The first frame is the first label by default, and the label to which the candidate box of the current frame belongs is calculated; if it does not belong to the previous label, it is a new label; assuming that the previous frame already has N instance results, the current instance is regarded as an N+1 classification problem, and the formula for determining whether two adjacent frames belong to the same label is as follows:

[0061]

[0062] The N instance results obtained in the previous frame are passed through two fully connected layers in the tracking branch to obtain feature C j , j = 1, ..., N, i is the sampling result of the current frame, the feature is C i ;

[0063] Step 23: Based on the result of step 22, if two adjacent frames belong to the same instance label, the Hungarian algorithm is used to associate the two frames to determine the motion trajectory of the pedestrian target.

[0064] The Hungarian algorithm uses the weighted intersection-and-union method to identify the detection frames in the current frame that may match the target in the previous frame, match them in order of identity, and obtain the best match of the intersection-and-union ratio. The steps of the Hungarian algorithm are as follows: first, the matrix of the two frames to be compared is reduced, then crossed, and finally reduced again until the elements are paired, and the distance between the tracking frame of the current frame and the detection frame of the current frame is calculated by the Euclidean distance to obtain the maximum match. The Euclidean distance calculation formula is as follows:

[0065]

[0066] Step 24: After associating all frames of the video sequence using the Hungarian algorithm, gait energy graphs of all targets are obtained.

[0067] Step 3: Define the gait energy graph generated in step 2 as Among them I t is the binary silhouette image at the tth frame, (x, y) is in I t The coordinates of each pixel, T is the number of frames in a gait cycle;

[0068] Step 4: According to the definition of GEI in step 3, preprocess the gait energy graphs corresponding to all targets obtained in step 2 to generate positive and negative GEI pairs; specifically:

[0069] Step 41: input the gait energy graphs corresponding to all targets obtained in step 2 into a custom random function algorithm to generate positive and negative GEI sample pairs;

[0070] Step 42: Manually screen the two sample pairs generated in step 41 to select clear sample pairs to ensure the quality of the sample pairs.

[0071] Step 5: Input the positive and negative GEI pairs of the gait energy map generated in step 4 into the convolution module and the corresponding non-local feature extraction module of the improved non-local block network respectively, and superimpose the features extracted from the two modules; specifically:

[0072] Step 51, inputting the positive and negative GEI pairs generated in step 4 into the improved non-local block network, and processing the positive and negative GEI pairs respectively;

[0073] Step 52: first input the positive and negative GEI pairs into the corresponding convolution module, perform two convolutions respectively, and obtain the corresponding convolution results;

[0074] Step 53: extract the non-local information features of the positive and negative GEI pairs in the non-local module, where the non-local operation is: i =G i +W C X i , where C i is a non-local feature, G i represents the output of GEI after the convolution module, X i represents the output after the non-local module, W C For X i Zoom in with G i Same dimensions;

[0075] Step 54: The feature extraction process of the non-local module described in step 53 is as follows: the feature map obtained by the convolution module is input into three 1x1 convolution channels, where θ and Perform feature extraction on a set of GEI pairs in two channels to extract two types of non-local information, namely, the non-local information of a single GEI and the non-local information between two GEIs; transpose the two output information matrices and calculate the similarity of the corresponding inputs. On this basis, use the Softmax function to obtain the combined attention of the channels and the normalized correlation between each pixel and all pixels in the current feature map, so as to calculate the pixel at each position;

[0076] Step 55, the channel g in the non-local module also takes the same operation, that is, first use 1x1 convolution to reduce the dimension and then perform matrix transposition operation with the calculation results of the first two channels to calculate the correlation between each pixel in the current feature map and all other position pixels;

[0077] Step 56, the final result is processed by the output channel of a 1×1 convolution kernel, so that the output size of the GEI sample obtained by the non-local module is the same as the size before input, and the corresponding non-local information feature is obtained;

[0078] Step 57: superimpose the convolution result obtained in step 52 and the non-local information feature obtained in step 56, and then superimpose the superposition results of the positive and negative GEI pairs again to obtain a superposition result.

[0079] Step 6: Process the superimposed features of the positive and negative GEI pairs obtained in step 5 to obtain a feature map, and then divide the feature map into blocks to obtain more fine-grained features; wherein the specific steps of processing the superimposed features are:

[0080] Step 61, convolve the superposition result obtained in step 52 twice, superimpose the convolution result with the extracted non-local features, and input the superposition result into a subsequent convolution module to obtain a feature map;

[0081] Step 62: The feature map obtained by the convolution module and the non-local module is divided into static blocks, slightly dynamic blocks and strongly dynamic blocks, and higher weights are given to static blocks and slightly dynamic blocks with significant features;

[0082] Step 63: After obtaining the weights of the three feature blocks in step 62, three binary classifiers using the softmax function as the activation function are trained separately.

[0083] Step 7: Input the features obtained from step 6 into a one-to-many support vector machine for target recognition.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to be a further limitation of the present invention. All equivalent changes made using the contents of the present specification and drawings are within the protection scope of the present invention.

Claims

1. A multi-target tracking and recognition method based on non-local gait features, characterized in that: The method comprises the following steps: Step 1: input a gait video sequence of a single person or multiple people walking, and process the gait video sequence through the classification and positioning branch and the mask prediction branch of the multi-target tracking algorithm based on instance segmentation to obtain a segmented sequence frame; Step 2: Process the sequence frames in the tracking branch of the multi-target tracking algorithm based on instance segmentation, add two fully connected layers in the branch to assign instance labels to all targets, realize the associated tracking of multiple targets, and thus obtain the corresponding gait energy maps of all targets; Step 3: Define the gait energy graph generated in step 2 as Among them I t is the binary silhouette image at the tth frame, (x, y) is in I t The coordinates of each pixel, T is the number of frames in a gait cycle; Step 4: According to the definition of GEI in step 3, preprocess the gait energy graphs corresponding to all targets obtained in step 2 to generate positive and negative GEI pairs; Step 5, input the positive and negative GEI pairs of the gait energy map generated in step 4 into the convolution module and the corresponding non-local feature extraction module of the improved non-local block network respectively, and superimpose the features extracted from the two modules; Step 6: Process the superimposed features of the positive and negative GEI pairs obtained in step 5 to obtain a feature map, and then divide the feature map into blocks to obtain more fine-grained features; Step 7: Input the features obtained from step 6 into a one-to-many support vector machine for target recognition.

2. The multi-target tracking and recognition method based on non-local gait features according to claim 1, characterized in that: The specific steps of processing in the classification and positioning branch and the mask prediction branch of the multi-target tracking algorithm based on instance segmentation in step 1 are: Step 11: First, input the initial gait video sequence into the ResNet-01 backbone network to extract features and obtain the region of interest; Step 12: Send the region of interest obtained in step 11 to the region proposal network RPN, which continuously scans the region of interest to generate candidate regions by means of a sliding window; Step 13: Determine whether the edge of the candidate region generated in step 12 is blurred or not. If it is not blurred, use the regional feature aggregation method of the traditional bilinear interpolation method to traverse each candidate region and extract candidate features for each candidate region; otherwise, use the oblique line interpolation method to improve the image edge segmentation effect; the formulas for the traditional bilinear interpolation method and the oblique line interpolation method for a candidate region are as follows: Traditional bilinear interpolation: Oblique interpolation method: Where m11=(i1,j1), m12=(i1,j2), m21=(i2,j1), m22=(i2,j2), i and j are the coordinates of the pixels selected in the current candidate area, (i1,j1), (i1,j2), (i2,j1), (i2,j2) are the coordinates of the four endpoints of the current candidate area, f(p1) is the result of bilinear interpolation of the current image, and f(p2) is the result of oblique interpolation of the current image; Step 14: Send the candidate features obtained in step 13 to the classification and positioning branch and the mask prediction branch respectively. In the classification and positioning branch, the prediction architecture of the Faster R-CNN algorithm is used to achieve classification and positioning; in the mask prediction branch, the fully convolutional network FCN is used to achieve segmentation.

3. The multi-target tracking and recognition method based on non-local gait features according to claim 1, characterized in that: The specific steps of processing the multi-target tracking algorithm based on instance segmentation in step 2 are: Step 21: Process the segmented sequence frames generated in step 1 in the tracking branch to achieve matching of targets between different frames and cross-frame connection; Step 22: After the classification and positioning branch and the mask prediction branch detect the target to be tracked, the tracking branch assigns an instance label to each candidate box through two fully connected layers. The first frame is the first label by default, and the label to which the candidate box of the current frame belongs is calculated; if it does not belong to the previous label, it is a new label; assuming that the previous frame already has N instance results, the current instance is regarded as an N+1 classification problem, and the formula for determining whether two adjacent frames belong to the same label is as follows: The N instance results obtained in the previous frame are passed through two fully connected layers in the tracking branch to obtain feature C j , j = 1, ..., N, i is the sampling result of the current frame, the feature is C i ; Step 23: Based on the result of step 22, if two adjacent frames belong to the same instance label, the Hungarian algorithm is used to associate the two frames to determine the motion trajectory of the pedestrian target. Step 24: After associating all frames of the video sequence using the Hungarian algorithm, gait energy graphs of all targets are obtained.

4. The multi-target tracking and recognition method based on non-local gait features according to claim 1 is characterized in that: The gait energy graphs corresponding to all targets obtained in step 2 are preprocessed in step 4, including the following steps: Step 41: input the gait energy graphs corresponding to all targets obtained in step 2 into a custom random function algorithm to generate positive and negative GEI sample pairs; Step 42: Manually screen the two sample pairs generated in step 41 to select clear sample pairs.

5. The multi-target tracking and recognition method based on non-local gait features according to claim 1, characterized in that: The improved non-local block network in step 5 includes the following steps: Step 51, inputting the positive and negative GEI pairs generated in step 4 into the improved non-local block network, and processing the positive and negative GEI pairs respectively; Step 52: first input the positive and negative GEI pairs into the corresponding convolution module, perform convolution twice respectively, and obtain the corresponding convolution results; Step 53: extract the non-local information features of the positive and negative GEI pairs in the non-local module, where the non-local operation is: i =G i +W C X i , where C i is a non-local feature, G i represents the output of GEI after the convolution module, X i represents the output after the non-local module, W C For X i Zoom in with G i Same dimensions; Step 54: The feature extraction process of the non-local module described in step 53 is as follows: the feature map obtained by the convolution module is input into three 1x1 convolution channels, where θ and Perform feature extraction on a set of GEI pairs in two channels to extract two types of non-local information, namely, the non-local information of a single GEI and the non-local information between two GEIs; transpose the two output information matrices and calculate the similarity of the corresponding inputs. On this basis, use the Softmax function to obtain the combined attention of the channels and the normalized correlation between each pixel and all pixels in the current feature map, so as to calculate the pixel at each position; Step 55, the channel g in the non-local module also takes the same operation, that is, first use 1x1 convolution to reduce the dimension and then perform matrix transposition operation with the calculation results of the first two channels to calculate the correlation between each pixel in the current feature map and all other position pixels; Step 56, the final result is processed by the output channel of a 1×1 convolution kernel, so that the output size of the GEI sample obtained by the non-local module is the same as the size before input, and the corresponding non-local information feature is obtained; Step 57: superimpose the convolution result obtained in step 52 and the non-local information feature obtained in step 56, and then superimpose the superposition results of the positive and negative GEI pairs again to obtain a superposition result.

6. The multi-target tracking and recognition method based on non-local gait features according to claim 1, characterized in that: In step 6, processing the superposition features of the positive and negative GEI pairs obtained in step 5 includes the following steps: Step 61, convolve the superposition result obtained in step 52 twice, superimpose the convolution result with the extracted non-local features, and input the superposition result into a subsequent convolution module to obtain a feature map; Step 62: The feature map obtained by the convolution module and the non-local module is divided into static blocks, slightly dynamic blocks and strongly dynamic blocks, and higher weights are given to static blocks and slightly dynamic blocks with significant features; Step 63: After obtaining the weights of the three feature blocks in step 62, three binary classifiers using the softmax function as the activation function are trained separately.