A context-aware, noise-robust pedestrian search method

Abstract

The present invention relates to the field of computer vision, and proposes a context-aware, noise-robust pedestrian search method, which adopts a deep learning framework, including: 1) using an end-to-end pedestrian search neural network to simultaneously complete pedestrian detection and pedestrian identification; 2) modeling the uncertainty of pedestrian features during the training process of the neural network and additionally using the context information of pedestrians in the same scene to constrain the feature representation of pedestrians; 3) after the first stage of training, performing a similarity comparison for each pedestrian identity feature center predicted by the network to analyze potential data noise; 4) performing a second stage of training, and suppressing data noise during the training process of this stage; 5) using the trained pedestrian search network to perform pedestrian retrieval tasks and visualize the results according to the similarity of pedestrians. This algorithm has the characteristics of making full use of the context information in the scene image, being robust to noise from sources such as occlusion and erroneous data annotation, and being more accurate than existing related methods.

Description

Technical Field

[0001] The invention belongs to the field of image retrieval in computer vision. Background Art

[0002] Tens of thousands of video surveillance devices are being deployed in our daily lives, forming a huge video surveillance network. Faced with a huge amount of data, how to analyze and use this data has become a problem that must be faced. The current widely used manual monitoring or retrieval methods are obviously inefficient and will consume a lot of manpower and financial resources.

[0003] Due to the particularity of surveillance video images, the existing face recognition method cannot well meet the needs of this scenario. Pedestrian re-identification and pedestrian search are a research direction to solve this problem. It aims to establish a connection between different surveillance images by utilizing the body shape, clothing and other characteristics of pedestrians, so as to complete the recognition and retrieval of pedestrians across cameras. The difference is that pedestrian search combines the work of detection and recognition and uses the complete camera image as input. Optimizing both detection and recognition can achieve better results. In addition, this is closer to the actual landing environment. Pedestrian search is an important part of retrieval and analysis of massive surveillance videos, and has attracted the attention of many researchers in recent years.

[0004] CN111209799A proposes a pedestrian search method based on a partially shared network and a cosine interval loss function. This method solves the conflict between the two subtasks of pedestrian detection and pedestrian identification through partial sharing. This method also uses a cosine interval loss function to obtain a more discriminative pedestrian feature representation. However, this method does not take into account the characteristics of the pedestrian search task and fully utilize the spatial context information of the scene image, which limits the final performance of the neural network to a certain extent.

[0005] CN112446340A proposes a pedestrian search method that combines local features of pedestrians and classification of clothing attributes. This method cascades two models of target detection and pedestrian recognition to ultimately complete the pedestrian search task; in the pedestrian recognition subtask, the clothing attributes and local features of pedestrians are optimized to improve the recognizability of pedestrian features. However, in this method, the two subtasks of pedestrian search are independent of each other and cannot be adapted to each other, which limits the final performance of the method to a certain extent. Summary of the invention

[0006] In order to overcome the defects of the above-mentioned prior art, the present invention proposes a context-aware and noise-robust end-to-end pedestrian search method. The present invention fully exploits the context information in the scene image and constructs comparison pairs using pedestrians appearing in the same picture, so that the pedestrian search neural network is more robust in crowded scenes and occluded scenes. In addition, the present invention proposes a method for discovering noise in training data and a method for suppressing the negative impact of noise in training data. Finally, this invention introduces data uncertainty learning into pedestrian search and achieves good results. In general, compared with previous pedestrian search methods, the present invention is more robust in complex scenes such as occlusion and congestion, thereby achieving better results in pedestrian search tasks.

[0007] The objective of the present invention can be achieved by the following technical solution: Step 1: construct an end-to-end pedestrian search neural network, which uses the complete surveillance video screen as input and uses a single network to simultaneously complete the two tasks of pedestrian detection and pedestrian recognition;

[0008] Step 2 is to train the neural network. The neural network training process proposed in the present invention is divided into two stages. First, the first stage of neural network training is performed. During the training process, the data uncertainty module is used to model the uncertainty in the data, and the scene context information is used to constrain the pedestrian feature representation predicted by the network.

[0009] Step 3 uses the neural network that has completed the first stage of training to analyze the noise in the training data and find possible noise in the training data.

[0010] Step 4 is to carry out the second phase of neural network training. Except for using the noise suppression module to suppress the noise in step 3, the process of this training phase and the performance of the remaining modules are consistent with the first phase of training.

[0011] Step 5: Use the trained pedestrian search neural network to perform the task of retrieving pedestrians in the scene image.

[0012] The step 1 specifically comprises the following steps:

[0013] Step 1.1 Use the first four parts (conv1 to conv4) of the general model ResNet-50 as the pedestrian search network Backbone network , used to extract scene image features;

[0014] Step 1.2 uses several convolutional layers to build Region Proposal Extraction Network (Region Proposal Network (RPN)), used to extract the areas where pedestrians may exist in the scene image;

[0015] Step 1.3 Use Region of Interest Pooling (RoI Pooling) module extracts features of candidate regions in scene images;

[0016] Step 1.4 uses the last part (conv5) of the general model ResNet-50 as the pedestrian search network Identify the network , used to further extract features of the candidate area;

[0017] Step 1.5 Use several fully connected layers to build the detection subtask Classification and regression modules , used to classify and regress the features of the candidate regions;

[0018] Step 1.6 Use Data uncertainty module Model the data uncertainty for the candidate regions.

[0019] The step 2) specifically comprises the following steps:

[0020] Step 2.1 normalizes the image by subtracting the pre-calculated image mean from the image and dividing it by the pre-calculated image variance;

[0021] Step 2.2: Scale the normalized image and the bounding box of the pedestrian on the image to a specified size while ensuring that the aspect ratio remains unchanged; horizontally flip the image and its annotation with a certain probability;

[0022] Step 2.3: Send the image processed in step 2.2 to the backbone network to extract the feature map f of the scene image. backbone ;

[0023] Step 2.4: f backbone Send it to the candidate region extraction network to obtain the candidate region where pedestrians may exist;

[0024] Step 2.5 uses the region of interest pooling module to extract the backbone Extract the feature map f of the candidate region roi ;

[0025] Step 2.6: Convert the feature map f roi Send it to the recognition network to extract the feature vector z representing the characteristics of the candidate region;

[0026] Step 2.7: Send the feature vector z to the classification and regression modules of the detection subtask to obtain the foreground and background prediction results v of the post-selected area. fgbg and v reg ;

[0027] Step 2.8: Send the feature vector z to the data uncertainty module to obtain the mean x representing the sample characteristics. μ and standard deviation x σ, sample from the Gaussian distribution composed of the above mean and standard deviation to obtain the pedestrian feature representation x for training;

[0028] Step 2.9 uses classification loss and regression loss to calculate the loss function of RPN and classification and regression modules, using Online instance matching (Online Instance Matching (OIM)) loss function and the proposed Context Ratio loss function Calculate the loss function of pedestrian feature representation;

[0029] Step 2.10 performs directional propagation operation to update the parameters of the network;

[0030] Step 2.11 Repeat steps 2.1 to 2.10 for a predefined number of iterations.

[0031] The step 3 specifically includes the following steps

[0032] Step 3.1 Use the network trained in step 2 to predict the data in the training set and calculate the feature center of each pedestrian ID;

[0033] Step 3.2 Calculate the similarity of all ID feature centers in the training set and obtain the similarity matrix M sim ;

[0034] Step 3.3: M sim The values ​​in the non-diagonal area are sorted from large to small, the identity IDs corresponding to the similarity values ​​are visualized, and a suitable similarity threshold is selected so that the threshold can better distinguish the noise and difficult samples in the data;

[0035] Step 3.4 records the identity ID tuples corresponding to the similarity above the threshold as noise in the data.

[0036] The step 4 specifically comprises the following steps:

[0037] Perform the second phase of neural network training, based on step 2, and replace step 2.9 with the following steps:

[0038] Step 4.1 uses classification loss and regression loss to calculate the loss functions of RPN and classification and regression modules;

[0039] Step 4.2 uses the noise-robust online instance matching loss function to suppress the noise binary obtained in step 3.4, and uses the Context Contrastive Loss Function Calculate the loss function for pedestrian feature representation.

[0040] The step 5 specifically comprises the following steps:

[0041] Step 5.1: input the image of the pedestrian to be found and its position on the image into the neural network to obtain the feature representation of the pedestrian to be found;

[0042] Step 5.2: Input the image in the image library into the neural network to obtain the location of the pedestrian in the image library and the corresponding feature representation;

[0043] Step 5.3 calculates the cosine distance between the feature representation of the pedestrian to be found and the feature representation of the pedestrian in the image library;

[0044] Step 5.4 Sort by distance from near to far and display relevant information.

[0045] Compared with the previous solutions, the technical solution of the present invention has the following advantages:

[0046] The present invention makes full use of the context information in the scene image by constructing a context contrast loss. Pedestrians appearing in the same scene image are used as context information to construct contrast pairs and train the pedestrian search neural network to distinguish them, so that the network can have better overall performance when pedestrians overlap or crowd each other. This improves the robustness and discrimination of the pedestrian features extracted by the network in crowded and occluded scenes. At the same time, the present invention introduces data uncertainty learning into pedestrian search. By modeling the uncertainty of the data, the feature representation of the pedestrian sample is modeled as a probability distribution. This makes the network more robust to low-quality samples.

[0047] At the same time, the present invention proposes a training strategy to alleviate the negative impact of incorrect labeling in the training data and the noise of severe overlapping pedestrians on the training process. The training strategy includes a two-stage training process. The first stage is to conduct normal pedestrian search network training. After the first stage of training is completed, reasoning analysis is performed on the training data, and the feature centers of the pedestrian identity IDs are compared to analyze the incorrect data labeling and severe overlapping data noise in the training set. The second stage of training retrains the neural network, and during the training process, the above-mentioned incorrect labeling and data noise are suppressed to alleviate the negative impact of incorrect data labeling and data noise on the training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of the process of the present invention

[0049] Figure 2 Schematic diagram of the neural network structure of the present invention: (a) Main network structure (b) Data uncertainty modeling module structure (c) Context contrast loss schematic diagram

[0050] Figure 3 Step 3 of the present invention is the calculation process of the noise suppression method

[0051] Figure 4 Schematic diagram comparing the proposed method with other methods in terms of loss function constraints during training

[0052] Figure 5 Examples of noise in pedestrian search training data: (a) Noise caused by pedestrian overlap (b) Noise caused by incorrect labeling

[0053] Figure 6 Schematic diagram of pedestrian search and visualization in the implementation example DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The overall process of the present invention is as follows: Figure 1 It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in each implementation method of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] Step 1: Build an end-to-end pedestrian search neural network

[0056] The structure of the neural network is as follows Figure 2 (a) is shown. The neural network has two branches: pedestrian detection branch and pedestrian recognition branch, so that the two subtasks of pedestrian detection and pedestrian recognition are completed in the same neural network. The pedestrian detection branch is responsible for locating pedestrians in the monitored scene image and obtaining the boundary bounding box of the area where the pedestrians are located; the pedestrian recognition branch is responsible for extracting the pedestrian feature representation. The distance between the feature representations reflects the similarity between pedestrians. Subsequently, by comparing the distance between the features, pedestrians similar to the target pedestrian are retrieved.

[0057] The pedestrian search neural network is based on the Faster R-CNN target detection framework and adds a pedestrian recognition branch on this basis. Specifically, the pedestrian search neural network includes a Backbone Network) , used to extract areas where pedestrians may exist Candidate region extraction network Region Proposal Network) as well as Region of Interest Pooling (RoI Pooling module , used to further extract pedestrian feature representation Pedestrian Recognition Network , used to predict the location of pedestrians in scene images Pedestrian Bounding Box Classification as well as Regression Module , Data uncertainty moduleIn order to make the connection between modules clearer and more understandable, the following will further describe each component, specifically:

[0058] (1) For the backbone network, the first four parts (conv1 to conv4) of the general model ResNet-50 are used as the backbone network part of the pedestrian search network. The backbone network uses the image preprocessed by the general method as input and outputs the scene image feature map.

[0059] (2) For the candidate region extraction network, a convolutional layer with input and output channels of 1024 and 512 and a convolution kernel size of 3×3 is used to further extract the scene image features extracted by the backbone network. Subsequently, a convolutional layer with input and output channels of 512 and 18 and a convolution kernel size of 1×1 is used to classify and predict the foreground and background of the 9 anchors at each spatial position on the feature map; a convolutional layer with input and output channels of 512 and 36 and a convolution kernel size of 1×1 is used to regress and predict the coordinate position of the bounding box of the 9 anchors at each spatial position on the feature map as well as the spatial position of width and height. The scale of the anchor is set to [32×32~512×512], and the aspect ratio of the anchor is set to [1:1,1:2,2:1].

[0060] (3) For the pedestrian recognition network, the last part (conv5) of the general model ResNet-50 is used as the pedestrian recognition network. The pedestrian recognition network uses the output result of the region of interest pooling module as input. The output is a pedestrian feature vector for pedestrian detection and pedestrian recognition.

[0061] (4) For the pedestrian bounding box classification and regression module, the pedestrian feature vector output by the pedestrian recognition network is used as input. A fully connected layer with input and output channels of 2048 and 2 is selected to classify the corresponding foreground / background; a fully connected layer with input and output channels of 2048 and 8 is selected to regress the coordinate position of the pedestrian and the transformation parameters of the width and height of the bounding box. Thus, the subtask of pedestrian detection is completed.

[0062] (5) For the data uncertainty module, its structure is as follows Figure 2 (b) shows the pedestrian feature vector output by the pedestrian recognition network as input. Two fully connected layers with input and output channels of 2048 and 256 are selected to predict the mean and standard deviation of the pedestrian features. This completes the subtask of pedestrian recognition.

[0063] Step 2: Perform the first phase of training on the neural network (innovative step)

[0064] The training process of the neural network is divided into two stages. The training process and behavior of the two stages are consistent, except for the differences in the performance of some modules. In this step, the first stage of neural network training is performed.

[0065] The specific steps are as follows:

[0066] (1) Map the brightness of each channel of the image to the range of 0 to 1. Then normalize the image by subtracting [0.485, 0.456, 0.406] from the three RGB channels of the image and dividing by [0.229, 0.224, 0.225];

[0067] (2) Scale the normalized image and the bounding box of the pedestrian on the image to ensure that the image is as large as possible while ensuring that the aspect ratio remains unchanged, the long side does not exceed 1500 pixels, and the short side does not exceed 900 pixels. Flip the image and its annotations horizontally with a probability of 0.5. Finally, pad the left and bottom edges of the image so that the width and height of the image are both integer multiples of 32, and finally obtain an image with a height of H and a width of W;

[0068] (3) Send the image processed in (2) to the backbone network to extract the feature map f of the scene image backbone , f backbone The width and height are W / 16 and H / 16 respectively, and the number of channels is 1024;

[0069] (4) f backbone Send it to the candidate region extraction network, for f backbone The 9 anchors at each spatial position on the image are predicted to obtain candidate areas where pedestrians may exist;

[0070] (5) Using the region of interest pooling module, the candidate regions are sampled according to the ratio of positive to negative samples to obtain 256 candidate regions, and the cropped regions are backbone The features of the corresponding position are scaled to the target size of 14×14. The feature map f of the candidate region is obtained roi ;

[0071] (6) f roi Send it to the recognition network to extract the 2048-dimensional feature vector z representing the characteristics of the candidate region;

[0072] (7) Send z to the classification and regression modules of the detection subtask to obtain the foreground and background prediction results v of the selected area respectively. fgbg and v reg ;

[0073] (8) Send z to the data uncertainty module, Modeling the uncertainty in pedestrian data.In the data uncertainty module, the mean and standard deviation of pedestrian features are calculated. Making predictions , we get the sample feature mean x μ , standard deviation x σ The feature representation of pedestrian samples is modeled as a normal distribution (rather than a fixed vector in the usual practice), and the mean and standard deviation of the distribution corresponding to the pedestrian feature representation are predicted through two fully connected layers:

[0074]

[0075]

[0076] where z i is the pedestrian feature vector corresponding to the i-th sample obtained from the pedestrian recognition network, and is the mean and standard deviation of the feature representation predicted by the fully connected layer, θ μ and θ σ Respectively represent the parameters of the fully connected layer. In this way, the probability distribution corresponding to the pedestrian sample can be obtained based on the mean and standard deviation. During the training of the pedestrian search network, the pedestrian feature representation used for loss function calculation is sampled from the multivariate Gaussian distribution; during the reasoning of the pedestrian search network, only the mean of the multivariate Gaussian distribution predicted by the network is used for the final pedestrian retrieval.

[0077] Due to the non-differentiability of the sampling operation, a reparameterization method is used to sample from a probability distribution consisting of the predicted mean and standard deviation:

[0078]

[0079] where ∈~N(0,I), x i represents the feature representation of the i-th pedestrian sample used to calculate the loss function and pedestrian matching, N(·) represents Gaussian distribution, and I is the unit matrix;

[0080] At the same time, KL divergence is used to represent the pedestrian features predicted by the network as x i , constrained to a standard normal distribution:

[0081]

[0082] (9) Use Softmax cross entropy loss and smooth L1 loss function to calculate the loss function of RPN and classification and regression modules: L rpn_cls ,L rpn_reg ,L cls ,L reg, calculate the online instance matching loss function: L OIM .

[0083] (10) Figure 2 (c) Using contextual information, construct comparison pairs . The context contrast loss proposed by the present invention is used to constrain the representation of pedestrian features in the scene image so that the features corresponding to the samples belonging to the same pedestrian are as consistent as possible, and the features belonging to different pedestrian samples are as identifiable as possible. More specifically, the context contrast loss includes the context contrast of the candidate area and the context contrast of the manually annotated area. After adding the context contrast loss, the constraints of this method in the training process are compared with other methods. Figure 4 Compared with other methods, this method additionally constrains the feature discrimination between pedestrians without identity labels.

[0084] Among them, the context contrast loss of the candidate region is expressed as:

[0085]

[0086] in, represents the candidate region features assigned to the i-th manually annotated region. G represents the number of manually annotated regions, g j represents the feature corresponding to the jth manually labeled region, T represents the transpose of the matrix, and s p is the scaling factor used for the loss function, which is a hyperparameter. The context contrast loss function of the manually annotated area that acts simultaneously with the above candidate area context contrast loss function is expressed as:

[0087]

[0088] Among them, s g is the scaling factor used for the loss function and is a hyperparameter; the rest of the symbols are the same as the candidate region context contrast loss function.

[0089] In order to solve the problem of limited number of candidate regions in the early stage of training, the present invention proposes to shake the manually marked regions up and down to obtain more candidate regions, and the operation process is as follows:

[0090]

[0091]

[0092] Where N(·) represents Gaussian distribution, and They represent the coordinates of the upper left corner and lower right corner of the manually annotated rectangular area, k∈[1,2]. and Represents the new candidate area obtained after up and down jittering.

[0093] (11) Calculate the overall loss of the neural network L = L rpn_cls +L rpn_reg +L cls +L reg +L OIM +L gt_cc +L p_cc , perform back propagation operations on the neural network and update the parameters of the neural network.

[0094] Step 3: Analyze the noise in the training data (innovation step)

[0095] (1) Use the network trained in step 2 to predict the data in the training set. Due to the characteristics of the neural network, the weight of the classifier can be regarded as the feature center of each pedestrian ID. The classifier weight in the loss function of pedestrian classification in the pedestrian search neural network is extracted. As the identity feature center of each pedestrian, L is the number of identity IDs in the training set and D is the dimension of pedestrian feature representation.

[0096] (2) Based on the above ID center, calculate the similarity M between the IDs sim =VV T , get the similarity matrix between pedestrian IDs Among them, M sim is a symmetric matrix. The i-th row and j-th column of the matrix represent the distance between the feature centers of identity IDs i and j. (1) and the specific schematic diagram of this step are as follows Figure 3 (1) as shown.

[0097] (3) M sim The values ​​in the non-diagonal area are sorted from large to small, and the identity IDs corresponding to the similarity values ​​are visualized. 0.75 is selected as the threshold based on experience, so that the threshold can better distinguish the noise and difficult samples in the data.

[0098] (4) Record the identity ID binary T corresponding to the similarity above the threshold noise =[(a0,b0),(a1,b1),…,(a n ,b n )] as noise in the data

[0099] Step 4: Conduct the second phase of neural network training (innovative step)

[0100] During the neural network training process at this stage, the data noise obtained by the analysis in step 3 is suppressed. Based on step 2, the following steps are used to replace (9) in step 2:

[0101] (1) Use Softmax cross entropy loss and smooth L1 loss function to calculate the loss function of RPN and classification and regression modules: L rpn_cls ,L rpn_reg ,L cls ,L reg .

[0102] (2) Calculate the loss function L using the modified noise-robust online instance matching loss function OIM This loss function uses the noise ID binary obtained in step 3 (4) to suppress the noise in the data set. The specific process is shown in the figure below. Figure 3 (2) As shown in Figure 2. For a pair of pedestrian identities (a, b) with high similarity, when the sample with identity a is input into the network, the subscript b in the logit predicted by the network is multiplied by 0 to reduce the impact of noise samples on the training process:

[0103]

[0104] Where B{·} is an indicator function. When the condition is true, its value is 1, and when the condition is false, its value is 0. a is the true label of the sample, logit c Output the prediction result of category c for the classifier; The corresponding result prediction result after noise suppression.

[0105] Step 5: Pedestrian retrieval task in scene images

[0106] Use the trained pedestrian search neural network to perform the task of retrieving pedestrians in scene images. The overall process is as follows: Figure 6 As shown, specifically:

[0107] (1) An image containing the pedestrian to be queried and its corresponding position is input into the pedestrian search neural network. The network outputs the feature representation corresponding to the pedestrian to be queried.

[0108] (2) Input the scene image data set to be searched into the pedestrian search neural network. The neural network predicts the location of pedestrians in the scene image and their corresponding deep feature representation. The above is stored for subsequent pedestrian feature similarity comparison.

[0109] (3) The distance between the features of the target pedestrian and the features of the pedestrian to be searched is measured, and the similarity between the features is calculated. The similarity between the pedestrian feature representations is measured using the cosine distance.

[0110] (4) For each pedestrian to be queried, similar pedestrians in the scene image to be searched are displayed in descending order of similarity, and related information is also displayed.

Claims

1. A context-aware, noise-robust pedestrian search method, characterized in that: Includes steps: Step 1: Build an end-to-end pedestrian search neural network. The neural network uses the complete surveillance video as input and uses a single network to simultaneously complete the two tasks of pedestrian detection and pedestrian recognition. Step 2: training the neural network. The neural network training process proposed in the present invention is divided into two stages. First, the first stage of neural network training is performed. During the training process, a data uncertainty module is used to model the uncertainty in the data, and scene context information is used to constrain the pedestrian feature representation predicted by the network. Step 3: Use the neural network that has completed the first stage of training to analyze the noise in the training data and find out possible noise in the training data; Step 4 is to train the second phase of the neural network. Except for using the noise suppression module to suppress the noise in step 3, the process of this training phase and the performance of the remaining modules are consistent with the first phase of training. Step 5: Use the trained pedestrian search neural network to perform the task of retrieving pedestrians in the scene image; The step 2 specifically includes the following steps: Step 2.1 normalizes the image by subtracting the pre-calculated image mean from the image and dividing it by the pre-calculated image variance; Step 2.2: Scale the normalized image and the bounding box of the pedestrian on the image to a specified size while ensuring that the aspect ratio remains unchanged; horizontally flip the image and its annotation with a certain probability; Step 2.3: Send the processed image in step 2.2 to the backbone network to extract the feature map of the scene image. ; Step 2.4 Send it to the candidate region extraction network to obtain the candidate region where pedestrians may exist; Step 2.5 Use the region of interest pooling module to Extract feature maps of candidate regions ; Step 2.6: Feature map Send it to the recognition network to extract the feature vector representing the characteristics of the candidate region ; Step 2.7: Transform the feature vector The results are sent to the classification and regression modules of the detection subtask to obtain the foreground and background prediction results of the selected areas. as well as ; Step 2.8: Transform the feature vector Send it to the data uncertainty module to get the mean value representing the sample characteristics and standard deviation , sample from the Gaussian distribution composed of the above mean and standard deviation to obtain the pedestrian feature representation used for training ; Step 2.9: Use the classification loss and regression loss to calculate the loss functions of the RPN and the classification and regression modules, and use the online instance matching loss function and the context contrast loss function to calculate the loss function of the pedestrian feature representation; The context contrast loss function includes the context contrast loss of the candidate region and the context contrast loss function of the manually marked region; Among them, the context contrast loss of the candidate region is expressed as: in, Indicates that it is assigned to The candidate region features corresponding to the manually annotated regions; represents the number of manually annotated regions, Indicates The features corresponding to the manually annotated regions are represents the transpose of a matrix, is the scaling factor used for the loss function, which is a hyperparameter; Among them, the context contrast loss function of the manually annotated area is expressed as: in, is the scaling factor used for the loss function and is a hyperparameter; the rest of the symbols are the same as the context contrast loss function of the candidate region; Step 2.10 performs directional propagation operation to update the parameters of the network; Step 2.11 Repeat steps 2.1 to 2.10 for a predefined number of iterations.

2. A context-aware, noise-robust pedestrian search method according to claim 1, characterized in that: The step 1 specifically comprises the following steps: Step 1.1 uses the first four parts of the general model ResNet-50 as the backbone network in the pedestrian search network to extract scene image features; Step 1.2 uses several convolutional layers to build a candidate region extraction network to extract the areas where pedestrians may exist in the scene image; Step 1.3 uses the region of interest pooling module to extract features of the candidate region in the scene image; Step 1.4 uses the last part of the general model ResNet-50 as the recognition network in the pedestrian search network to further extract the features of the candidate area; Step 1.5 uses several fully connected layers to build the classification and regression modules of the detection subtask, which are used to classify and regress the features of the candidate regions; Step 1.6 uses the data uncertainty module to model the data uncertainty of the candidate region.

3. The context-aware, noise-robust pedestrian search method according to claim 1, characterized in that: The step 3 specifically includes the following steps Step 3.1 Use the network trained in step 2 to predict the data in the training set and calculate the feature center of each pedestrian ID; Step 3.2 Calculate the similarity of all ID feature centers in the training set to obtain the similarity matrix ; Step 3.3 The values ​​in the non-diagonal area are sorted from large to small, the identity IDs corresponding to the similarity values ​​are visualized, and a suitable similarity threshold is selected so that the threshold can better distinguish the noise and difficult samples in the data; Step 3.4: Record the identity ID tuples corresponding to the similarity higher than the threshold as the noise in the data.

4. The context-aware, noise-robust pedestrian search method according to claim 1, characterized in that: The step 4 specifically comprises the following steps: Perform the second phase of neural network training, based on step 2, and replace step 2.9 with the following steps: Step 4.1 uses classification loss and regression loss to calculate the loss functions of RPN and classification and regression modules; Step 4.2 uses the noise-robust online instance matching loss function to suppress the noise binary obtained in step 3.4, and at the same time uses the context contrast loss function to calculate the loss function of pedestrian feature representation.

5. The context-aware, noise-robust pedestrian search method according to claim 1, characterized in that: The step 5 specifically comprises the following steps: Step 5.1 Input the image of the pedestrian to be found and its position on the image into the neural network to obtain the feature representation of the pedestrian to be found; Step 5.2 Input the images in the image library into the neural network to obtain the location of the pedestrian in the image library and the corresponding feature representation; Step 5.3: Calculate the cosine distance between the feature representation of the pedestrian to be found and the feature representation of the pedestrian in the image library; Step 5.4 Sort by distance from near to far and display relevant information.

Images