A pedestrian retrieval method for vehicle-road collaboration based on deep learning

Abstract

The present invention discloses a pedestrian retrieval method based on deep learning for vehicle-road collaboration, which belongs to the technical field of pedestrian re-identification, and comprises the following steps: establishing a training set; setting an unsupervised discriminator network structure in the ReID model; S3: using the training set to perform iterative training on the ReID model for a preset number of rounds through warm-up learning; after completing the preset number of rounds of training, deploying the trained ReID model; inputting the image to be identified into the trained ReID model, selecting the top n% of the results according to an adaptive threshold n, determining them as the target person, and obtaining a detection result. The present application introduces an unsupervised discriminator network structure in the ReID model, which not only improves the representativeness and distinguishing ability of the features, but also significantly improves the accuracy of pedestrian recognition. The present application also sets the determination of the pedestrian threshold n to ensure the efficiency and accuracy of the retrieval.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian re-identification, and specifically relates to a pedestrian retrieval method based on deep learning for vehicle-road collaboration. Background Art

[0002] Pedestrian retrieval is a computer vision technology that aims to identify and retrieve specific individuals from images or videos. For example, in intelligent monitoring, pedestrian retrieval can identify and track target persons in real time to help maintain public safety. This technology has important practical significance in many fields. Especially in the field of smart transportation, pedestrian retrieval, as important information provided to vehicles by the road side of vehicle-road collaboration, plays a very important role in the application of vehicle-road collaboration such as accident warning, ensuring traffic safety, improving traffic management, and optimizing traffic flow. It provides important technical support for realizing dynamic real-time data interaction between vehicles and roads, active vehicle safety control and road collaborative management, and jointly creating digital transportation intelligent control services integrating vehicles, roads, sides, networks, clouds and maps.

[0003] Deep learning is a machine learning technology that simulates the human brain's neural network. It automatically extracts features and makes predictions from data by building a multi-level neural network. Compared with traditional algorithms, deep learning can process complex data patterns, identify high-dimensional features, and has stronger self-learning capabilities. This makes deep learning perform well in processing visual tasks such as image classification and object detection.

[0004] Applying deep learning technology to pedestrian retrieval can significantly improve the accuracy and robustness of the system, but it also suffers from problems of feature imbalance and low retrieval efficiency. Summary of the invention

[0005] In order to solve the problems raised in the above background technology, the present invention provides a pedestrian retrieval method based on deep learning for vehicle-road collaboration, so as to solve the problems of feature imbalance and low retrieval efficiency in the prior art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A pedestrian retrieval method for vehicle-road collaboration based on deep learning includes the following steps:

[0008] S1: Use YOLOv8n to detect pedestrians, obtain images, enhance the images by randomly rotating them, and randomly erase the enhanced images to obtain training sets;

[0009] S2: Set up a ReID model. The ReID model consists of two branches. The first branch is a standard classification network structure including a residual network, and the second branch is an unsupervised discriminator network structure. The r-layer convolutional layer of the first branch is an adversarial layer.

[0010] S3: Use the training set to perform a preset number of rounds of iterative training on the ReID model through warm-up learning. The training process for each round is:

[0011] The images in the training set are input into the residual network of the ReID model. After r layers of convolutional layers, feature maps are obtained. The feature maps are input into the subsequent modules of the residual network for subsequent training. According to the set parameters, the discriminator in the unsupervised discriminator network structure is enabled with a certain probability. The ReID model uses the smoothed labels to calculate the ternary loss function and perform gradient updates on the residual network.

[0012] When the discriminator is called, the feature map and the original image cropped to the same size as the feature map are input into the discriminator. The discriminator determines the image category and calculates the discriminator's loss function. The discriminator's own weights are gradient updated through the discriminator's loss function, and back-propagated to the first r convolutional layers for gradient update.

[0013] S4: After completing the preset number of rounds of training, deploy the trained ReID model;

[0014] S5: Input the image to be identified into the trained ReID model, select the top n% of the results according to the adaptive threshold n, determine them as the target person, and obtain the detection result.

[0015] Preferably, in S1, the detection image is enhanced by randomly rotating 90°, 180° and 270°.

[0016] Preferably, in S3, the preset number of rounds is 120, and the value of r is 2.

[0017] Preferably, in the warm-up learning of S3, the learning rate lr(t) of the tth round is as follows:

[0018] .

[0019] Preferably, in S3, before the feature map is input into the fully connected layer of the residual network, the last step size of the residual network is changed.

[0020] Preferably, during the training of the ReID model in S3, two different loss functions are used to train the first branch and the second branch;

[0021] The loss function of the first branch is recorded as , the loss function of the second branch is recorded as , With a given probability P c Update the adversarial layer and the discriminator;

[0022] Total loss functionF total As shown below:

[0023] ;

[0024] Among them, x represents the input image, y represents the true label of x, and θ C represents the parameters of the standard classification network structure, optimized using the cross entropy loss function, θ d represents the parameters of the discriminator, n is the number of people, i is the image category, P r is the data distribution, P ƒ is the hidden feature distribution, D(x) is the output of the discriminator under input x, and G(x) is the hidden feature output of the adversarial layer. and .

[0025] Preferably, in S3, when performing gradient update, the ID loss function L(ID) is:

[0026] ;

[0027] in, ;

[0028] ε is a small constant, p i Represented as ID prediction logic for class i;

[0029] The ternary loss function L Tri for:

[0030] ;

[0031] Among them, d p and d n are the feature distances of the positive sample pair and the negative sample pair, α is the offset of the ternary loss, and a[z] ＋ Refers to max(z,0), α is set to 0.3;

[0032] Center loss function for:

[0033] ;

[0034] where ƒ tj is the feature vector of the jth sample, y j is the label of the jth image in each mini-batch, c yj The yth one represents the deep features i class centers, B represents the batch size;

[0035] The final loss function is:

[0036] ;

[0037] Where β is a parameter used to balance the center loss and is set to 0.0005.

[0038] Preferably, in S5, n is 30.

[0039] Preferably, the residual network uses ResNet50.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. First, this application introduces an unsupervised discriminator network structure in the ReID model to optimize feature balance and fully coordinate global features with local details. This mechanism not only improves the representativeness and distinguishing ability of features, but also significantly improves the accuracy of pedestrian recognition with almost no additional time and space overhead.

[0042] 2. In addition, this application also sets the pedestrian threshold n to ensure the efficiency and accuracy of the retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the overall process of this application;

[0044] Figure 2 It is a structural diagram of the ReID model;

[0045] Figure 3 Schematic diagram of the unsupervised discriminator network structure. DETAILED DESCRIPTION

[0046] In order to facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] Embodiment 1:

[0048] A pedestrian retrieval method based on deep learning for vehicle-road collaboration, such as Figure 1 , Figure 2 , Figure 3 As shown, the following steps are included:

[0049] S1: Use YOLOv8n to detect pedestrians and obtain images. Enhance the detected images by randomly rotating them by 90°, 180°, and 270°. Randomly erase the enhanced images to obtain a training set.

[0050] S2: Set up a ReID model. The ReID model consists of two branches. The first branch is a standard classification network structure including a residual network, and the second branch is an unsupervised discriminator network structure. The r-layer convolutional layer of the first branch is an adversarial layer.

[0051] S3: The ReID model is iteratively trained for 120 rounds using the training set through warm-up learning. The learning rate lr(t) of the tth round is as follows:

[0052] ;

[0053] The training process for each round is:

[0054] The images in the training set are input into the residual network of the ReID model. ResNet50 is selected as the residual network. After two convolutional layers, the feature map is obtained. The feature map is input into the subsequent modules of the residual network for subsequent training. Before the feature map is input into the fully connected layer of the residual network, the last step size of the residual network is changed. According to the set parameters, the discriminator in the unsupervised discriminator network structure is enabled with a certain probability. The ReID model uses the label after label smoothing to calculate the ternary loss function and perform gradient update on the residual network.

[0055] Changing the last step of the backbone specifically means: the higher the spatial resolution, the richer the feature hierarchy. The last spatial downsampling operation in ResNet50 is represented as the last step. The last step of ResNet50 is set to 2. When an image of size 256×128 is input, the backbone of ResNet50 outputs a feature map with a spatial size of 8×4. If the last step is changed from 2 to 1, we can get a feature map with a larger spatial size (16×8). This operation only adds a small computational cost and does not require additional training parameters, but the higher spatial resolution brings significant improvements;

[0056] When the discriminator is called, the feature map and the original image cropped to the same size as the feature map are input into the discriminator. The discriminator determines the image category and calculates the discriminator's loss function. The discriminator's own weights are gradient updated through the discriminator's loss function, and back-propagated to the first two convolutional layers for gradient update.

[0057] During training, two different loss functions are used to train the first branch and the second branch;

[0058] The loss function of the first branch is recorded as , the loss function of the second branch is recorded as , Using cross entropy loss, Update the adversarial layer and the discriminator with given probabilities to improve representation and balance low-level details with high-level features;

[0059] Total loss function F total As shown below:

[0060] ;

[0061] Where x represents the input image, which is input to the classification network, y represents the true label of x, which is used for the classification task, and θ C represents the parameters of the standard classification network structure, which is optimized using loss functions such as cross entropy to improve the performance of classification tasks, θ d represents the parameters of the discriminator, which is part of the adversarial component and helps balance the distance between the model hidden features and the real data mapping in the hidden feature space. n is the number of people, i is the image category, and P r is the data distribution, P ƒ is the hidden feature distribution, D(x) is the output of the discriminator under input x, and G(x) is the hidden feature output of the adversarial layer. We perform and ;

[0062] When performing gradient updates, the ID embedding network IDE is the basic baseline for personal identification. The last layer of IDE outputs the ID prediction logic of the image, which is a fully connected layer with a hidden size equal to the number of people n. Given an image, we represent y as the real ID label, p i Expressed as the ID prediction logistic of class i, the cross entropy loss is calculated as:

[0063] ;

[0064] Since the classification category is determined by the person's ID, this loss function is called ID loss. However, since the person ID of the test set does not appear in the training set, the person ID can be regarded as a one-time learning task. Therefore, it is very important to prevent the ReID model from overfitting the training ID. Label smoothing (LS) is used to reduce q i The structure is changed to:

[0065] ;

[0066] Where ε is a small constant, LS can significantly improve the performance of the model;

[0067] The ternary loss function L Tri for:

[0068] ;

[0069] Among them, dp and d n are the feature distances of the positive sample pair and the negative sample pair, α is the offset of the ternary loss, and a[z] ＋ Refers to max(z,0), α is set to 0.3;

[0070] Center loss function for:

[0071] ;

[0072] where ƒ tj is the feature vector of the jth sample, y j is the label of the jth image in each mini-batch, c yj The yth one represents the deep features i class centers, B represents the batch size;

[0073] The final loss function is:

[0074] ;

[0075] Where β is a parameter used to balance the center loss and is set to 0.0005.

[0076] S4: After completing the preset number of rounds of training, deploy the trained ReID model;

[0077] S5: Input the image to be identified into the trained ReID model, select the top 30% of the results according to the adaptive threshold 30, determine them as the target person, and obtain the detection result.

[0078] The adaptive threshold is specifically to improve the matching accuracy by calculating the cosine similarity between query images and gallery images. The main steps include:

[0079] 1. Calculate cosine similarity: For each pair of query image and gallery image, calculate the cosine similarity between them, which is a commonly used method to measure the similarity between image feature vectors.

[0080] 2. Sorting: Sort all calculated cosine similarities, usually from the smallest to the largest value.

[0081] 3. Select the top n%: Based on the sorting results, select the top 30% of similarity values ​​as the objects for further analysis. This method ensures that the closest matches are selected regardless of the overall similarity level.

[0082] The advantage of this method is its adaptability, that is, it does not rely on a fixed threshold to determine whether it matches, but makes selections based on relative positions, so that it can maintain good performance in different data sets or scenarios.

[0083] In this way, pedestrians can be effectively identified even when their appearance captured by different cameras changes, improving the robustness and accuracy of the pedestrian re-identification system.

[0084] In this embodiment, the Market1501 dataset is used for the person re-identification task, that is, identifying the same pedestrian under different camera perspectives. It is one of the standard datasets for evaluating the performance of pedestrian re-identification algorithms, which can help researchers test the matching ability and robustness of the algorithm under different camera perspectives. We applied this application to the Market1501 dataset for pedestrian re-identification experiments.

[0085] The method proposed in the paper "Bag of Tricks and A Strong Baseline for Deep Person Re-identification" (2019CVPR) is used as the original method for comparison, and the model with added threshold operation and unsupervised discriminator is called this method. In the experiment, we trained both methods on the Market1501 dataset for 120 rounds. After the training, we tested the model to evaluate its performance and effect. The experimental results are shown in Table 1.

[0086]

[0087] Table 1 Verification results of Market1501 dataset

[0088] After adopting the pedestrian re-identification algorithm based on feature balance, the performance of the model has been significantly improved. Specifically, the convergence speed of this application has been improved by 10 epochs. This means that the model can reach the optimal state faster during training, thereby shortening the training time. In addition, the accuracy of the model has also been improved, with the accuracy increased by about 1 percentage point. This improvement shows that the algorithm based on feature balance not only speeds up the learning process of the model when dealing with pedestrian re-identification tasks, but also further improves the accuracy of recognition, providing a more reliable guarantee for the performance of pedestrian re-identification in practical applications.

Claims

1. A pedestrian retrieval method based on deep learning for vehicle-road collaboration, characterized in that: The following steps are involved: S1: Use YOLOv8n to detect pedestrians, obtain images, enhance the images by randomly rotating them, and randomly erase the enhanced images to obtain training sets; S2: Set up a ReID model. The ReID model consists of two branches. The first branch is a standard classification network structure including a residual network, and the second branch is an unsupervised discriminator network structure. The r-layer convolutional layer of the first branch is an adversarial layer. S3: Use the training set to perform a preset number of rounds of iterative training on the ReID model through warm-up learning. The training process for each round is: The images in the training set are input into the residual network of the ReID model. After r layers of convolutional layers, feature maps are obtained. The feature maps are input into the subsequent modules of the residual network for subsequent training. According to the set parameters, the discriminator in the unsupervised discriminator network structure is enabled with a certain probability. The ReID model uses the smoothed labels to calculate the ternary loss function and perform gradient updates on the residual network. When the discriminator is called, the feature map and the original image cropped to the same size as the feature map are input into the discriminator. The discriminator determines the image category and calculates the discriminator's loss function. The discriminator's own weights are gradient updated through the discriminator's loss function, and back-propagated to the first r convolutional layers for gradient update. S4: After completing the preset number of rounds of training, deploy the trained ReID model; S5: Input the image to be identified into the trained ReID model, select the top n% of the results according to the adaptive threshold n, determine them as the target person, and obtain the detection result.

2. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: In S1, the detection image is enhanced by randomly rotating 90°, 180°, and 270°.

3. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: In S3, the preset number of rounds is 120 and the value of r is 2.

4. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 3 is characterized in that: In the warm-up learning of S3, the learning rate lr(t) of the tth round is as follows: 。 5. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: In S3, the feature map is changed before entering the fully connected layer of the residual network, and the last step size of the residual network is changed.

6. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: During the training of the ReID model in S3, two different loss functions are used to train the first and second branches; The loss function of the first branch is recorded as , the loss function of the second branch is recorded as , With a given probability P c Update the adversarial layer and the discriminator; Total loss function F total As shown below: ； Among them, x represents the input image, y represents the true label of x, and θ C represents the parameters of the standard classification network structure, optimized using the cross entropy loss function, θ d represents the parameters of the discriminator, n is the number of people, i is the image category, P r is the data distribution, P ƒ is the hidden feature distribution, D(x) is the output of the discriminator under input x, and G(x) is the hidden feature output of the adversarial layer. and .

7. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 6, characterized in that: In S3, when performing gradient update, the ID loss function L(ID) is: ； in, ; ε is a small constant, p i Represented as ID prediction logic for class i; The ternary loss function L Tri for: ； Among them, d p and d n are the feature distances of the positive sample pair and the negative sample pair, α is the offset of the ternary loss, and a[z] ＋ Refers to max(z,0), α is set to 0.3; Center loss function for: ； where ƒ tj is the feature vector of the jth sample, y j is the label of the jth image in each mini-batch, c yj The yth one represents the deep features i class centers, B represents the batch size; The final loss function is: ； Where β is a parameter used to balance the center loss and is set to 0.0005.

8. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: In S5, n is 30.

9. The pedestrian retrieval method based on deep learning for vehicle-road collaboration according to claim 1, characterized in that: ResNet50 is used as the residual network.

Images