Nighttime pedestrian re-identification method, device, equipment and storage medium

Abstract

The present invention discloses a method, apparatus, device and storage medium for nighttime pedestrian re-identification. The method comprises: training a nighttime pedestrian re-identification network to obtain a trained network; sending an original image into a DCE-Net to obtain a mapping map, iteratively performing brightness enhancement on the original image using the mapping map to obtain a brightness-enhanced image, sending the image into a first ResNet50 to obtain a feature map of the brightness-enhanced image; sending the original image into a second ResNet50 to obtain a feature map of the original image; cascading the feature map of the original image and the feature map of the brightness-enhanced image to obtain a fused feature, sending the feature map into a brightness distillation module to obtain a final fused feature; comparing the final fused feature with the features corresponding to each image in a gallery, and sorting them according to similarity scores to obtain a recognition result. The advantage of the present invention is that identity discrimination information is retained, thereby having a strong ability to re-identify pedestrians at night.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to a method, apparatus, device and storage medium for nighttime pedestrian re-identification. Background Art

[0002] Person re-identification (PIR) uses computer vision techniques to determine whether a specific person exists in an image or video sequence. The goal is to find images of the same person as the query image from a large image library. As a key task in computer vision, it has widespread applications in many fields, such as intelligent security, smart person search systems, and smart business.

[0003] In recent years, thanks to advances in deep learning technology, pedestrian re-identification (PRE) in daytime scenes has made significant progress. However, research on nighttime PRE is relatively limited and remains in its early stages. Recent reports suggest that violent crimes are more likely to occur at night, as darkness provides a natural camouflage for suspects. Under low-light conditions, the quality of images or videos captured at night is significantly lower than that of images or videos captured during the day, significantly degrading the performance of traditional PRE methods.

[0004] Therefore, there are many problems to be solved in terms of how to improve the performance of pedestrian re-identification at night. Because there is sufficient brightness during the day and insufficient brightness at night, low brightness is a major challenge for pedestrian re-identification at night. Some works study the impact of brightness changes on pedestrian re-identification by learning brightness-invariant features. Some studies simulate night scenes by using images with good brightness during the day to generate low-brightness images, and then use the paired image sets of good brightness images and generated low-brightness images as input to learn a brightness-invariant feature through a generative adversarial network to improve the feature representation of pedestrians in low-brightness conditions. However, the method used in these works to generate low-brightness images is usually the gamma correlation method. For example, the document "Jiang Xuesong, Yao Hongxun. A review of night image enhancement methods [J]. Intelligent Computers and Applications, 2020, 10(3):395." introduces the application of gamma function in image processing. The gamma correlation method uses random seeds to synthesize images. First, it is difficult to simulate the real low-light conditions at night in the real world, and the simulated scenes are quite different from the real night scenes. Secondly, this method of learning brightness-invariant features is explicitly designed for the task of pedestrian re-identification with brightness changes. It mainly focuses on the challenges of low light levels and has difficulty retaining the discriminative identity information of pedestrians, resulting in poor ability to re-identify pedestrians at night. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that the existing nighttime pedestrian re-identification method is difficult to retain the pedestrian's discriminative identity information, and thus the ability to re-identify pedestrians at night is not strong.

[0006] The present invention solves the above technical problems through the following technical means: a method for nighttime pedestrian re-identification, the method comprising:

[0007] Step 1: Train a nighttime person re-identification network to obtain a trained network. The nighttime person re-identification network includes a brightness enhancement branch, a backbone branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with symmetrical skip connection structures. The backbone branch consists of a second ResNet50. The brightness distillation module consists of a bottleneck fusion sub-network, which is an encoder-decoder structure.

[0008] Step 2: The original image is fed into the trained network. First, a pixel-level curve parameter map of the same size as the original image is obtained through DCE-Net. The original image is iteratively enhanced using the map to obtain a brightness-enhanced image. The brightness-enhanced image is fed into the first ResNet50 to obtain a feature map of the brightness-enhanced image.

[0009] Step 3: Send the original image to the second ResNet50 to obtain the feature map of the original image;

[0010] Step 4: The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fused features, which are then fed into the brightness distillation module to obtain the final fused features.

[0011] Step 5: Compare the final fusion features with the features corresponding to each image in the gallery, and sort them according to the similarity score to obtain the recognition results.

[0012] This method uses a brightness enhancement branch to improve the quality of the initial low-brightness nighttime image and obtain the features of the brightness-enhanced image. The bottleneck fusion subnetwork in the brightness distillation module then fuses the feature maps of the initial low-brightness image with those of the brightness-enhanced image, reducing redundant features while preserving identity information. This results in better re-identification (RID) and enhances nighttime pedestrian RID capabilities.

[0013] Furthermore, the step 1 includes constructing a loss function for the brightness enhancement branch:

[0014] The original image is sent to DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image. The original image is iteratively enhanced in brightness using the mapping map. After reaching the preset number of iterations, the image with enhanced brightness is obtained. The formula for iterative brightness enhancement is I n =LE(I n-1 ; A n), and LE(I0,A1)=I0+A1*I0*(1-I0), where In is the pixel value of the brightness enhanced image obtained after n iterations, A n The pixel value of the map used in the nth iteration;

[0015] The brightness enhanced image is fed into the first ResNet50 to obtain the feature map of the brightness enhanced image, which is fed into the first classifier to predict the pedestrian ID. The loss function of the brightness enhancement branch is L IEB =L ID +λ1L DCE and Among them, λ1 is the balance coefficient, L DCE is the illumination enhancement loss, B is the number of images to be trained sent to the first classifier, is the predicted probability that the bth image to be trained belongs to the real image ID label.

[0016] Furthermore, the step 1 also includes constructing a loss function for the trunk branch:

[0017] The original image is fed into the second ResNet50 to obtain the feature map of the original image, which is then fed into the second classifier. The loss function of the main branch is L MB =L ID .

[0018] Furthermore, the step 1 also includes constructing a loss function for the brightness distillation module:

[0019] The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fusion features, which are sent to the brightness distillation module to obtain the final fusion features. The final fusion features are sent to the third classifier. The loss function of the brightness distillation module is L IDM =L ID +λ2L rec , where λ2 is the smoothing parameter, L rec =||z out -z in || 2 , z out and z in They are the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the bottleneck fusion sub-network;

[0020] Then the prediction results of the brightness distillation module are compared with the prediction results of the main branch and the brightness enhancement branch respectively to calculate the loss. The loss function is: Among them, P t is the predicted identity probability of the brightness distillation module, P sis the predicted identity probability of the main branch or the brightness enhancement branch, S is the student model, and t is the teacher model. The student model includes the main branch and the brightness enhancement branch, and the teacher model is the brightness distillation module.

[0021] Furthermore, the step 1 also includes training the entire nighttime person re-identification network, and the training process is as follows:

[0022] Use the stochastic gradient descent algorithm to train the nighttime pedestrian re-identification network, adjust the network parameters, and use the joint loss function When the value of is the smallest, stop training and fix the network parameters to get a trained network, where θ mb ,θ ieb ,θ idm They represent the parameters of the main branch, brightness enhancement branch, and brightness distillation module respectively.

[0023] Furthermore, the step five includes: inputting the final fusion feature into the third classifier, comparing it with the features corresponding to each image in the gallery, and sorting them according to the similarity score to obtain the recognition result.

[0024] The present invention also provides a nighttime pedestrian re-identification device, comprising:

[0025] A training module is used to train a nighttime person re-identification network to obtain a trained network. The nighttime person re-identification network includes a brightness enhancement branch, a backbone branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with a symmetrical skip connection structure. The backbone branch consists of a second ResNet50. The brightness distillation module consists of a bottleneck fusion subnetwork, which has an encoding and decoding structure.

[0026] The brightness enhancement module is used to feed the original image into the trained network, first pass it through the DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image, use the mapping map to iteratively enhance the brightness of the original image, and then feed the brightness-enhanced image into the first ResNet50 to obtain the feature map of the brightness-enhanced image;

[0027] The feature extraction module is used to feed the original image into the second ResNet50 to obtain the feature map of the original image;

[0028] The fusion module is used to obtain fusion features by cascading the feature maps of the original image and the image after brightness enhancement, and then send them to the brightness distillation module to obtain the final fusion features;

[0029] The pedestrian re-identification module is used to compare the final fusion features with the features corresponding to each image in the library, and sort them according to the similarity score to obtain the recognition results.

[0030] Furthermore, the training module is also used to construct the loss function of the brightness enhancement branch:

[0031] The original image is sent to DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image. The original image is iteratively enhanced in brightness using the mapping map. After reaching the preset number of iterations, the image with enhanced brightness is obtained. The formula for iterative brightness enhancement is I n =LE(I n-1 ; A n ), and LE(I0,A1)=I0+A1*I0*(1-I0), where In is the pixel value of the brightness enhanced image obtained after n iterations, A n The pixel value of the map used in the nth iteration;

[0032] The brightness enhanced image is fed into the first ResNet50 to obtain the feature map of the brightness enhanced image, which is fed into the first classifier to predict the pedestrian ID. The loss function of the brightness enhancement branch is L IEB =L ID +λ1L DCE and Among them, λ1 is the balance coefficient, L DCE is the illumination enhancement loss, B is the number of images to be trained sent to the first classifier, is the predicted probability that the bth image to be trained belongs to the real image ID label.

[0033] Furthermore, the training module is also used to construct the loss function of the trunk branch:

[0034] The original image is fed into the second ResNet50 to obtain the feature map of the original image, which is then fed into the second classifier. The loss function of the main branch is L MB =L ID .

[0035] Furthermore, the training module is also used to construct the loss function of the brightness distillation module:

[0036] The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fusion features, which are sent to the brightness distillation module to obtain the final fusion features. The final fusion features are sent to the third classifier. The loss function of the brightness distillation module is L IDM =L ID +λ2L rec , where λ2 is the smoothing parameter, L rec=||z out -z in || 2 , z out and z in They are the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the bottleneck fusion sub-network;

[0037] Then the prediction results of the brightness distillation module are compared with the prediction results of the main branch and the brightness enhancement branch respectively to calculate the loss. The loss function is: Among them, P t is the predicted identity probability of the brightness distillation module, P s is the predicted identity probability of the main branch or the brightness enhancement branch, S is the student model, and t is the teacher model. The student model includes the main branch and the brightness enhancement branch, and the teacher model is the brightness distillation module.

[0038] Furthermore, the training module is also used to train the entire nighttime person re-identification network. The training process is as follows:

[0039] Use the stochastic gradient descent algorithm to train the nighttime pedestrian re-identification network, adjust the network parameters, and use the joint loss function When the value of is the smallest, stop training and fix the network parameters to get a trained network, where θ mb ,θ ieb ,θ idm They represent the parameters of the main branch, brightness enhancement branch, and brightness distillation module respectively.

[0040] Furthermore, the pedestrian re-identification module is further used to: input the final fusion features into the third classifier, compare them with the features corresponding to each image in the gallery, and sort them according to the similarity score to obtain the recognition result.

[0041] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the above method steps are implemented.

[0042] The present invention also provides a computer-readable storage medium storing computer program instructions, which implement the above method steps when called and executed by a processor.

[0043] The advantages of the present invention are:

[0044] (1) The present invention uses a brightness enhancement branch to improve the quality of the initial low-brightness nighttime image and obtain the image features after brightness enhancement. The bottleneck fusion subnetwork in the brightness distillation module then fuses the feature map of the initial low-brightness image with the feature map of the brightness-enhanced image, reducing redundant features and retaining identity discrimination information, achieving better re-identification results and enhancing the ability to re-identify pedestrians at night.

[0045] (2) During the network training process of the present invention, the prediction results of the brightness distillation module are respectively compared with the prediction results of the main branch and the prediction results of the brightness enhancement branch for loss calculation. The distillation loss is used to promote further learning of the network on night images and brightness enhanced images, improve the ability of the main branch and the brightness enhancement branch to extract features, and obtain a more robust feature representation, thereby achieving better pedestrian re-identification effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of a nighttime pedestrian re-identification method disclosed in Example 1 of the present invention;

[0047] Figure 2 This is a network architecture diagram of a nighttime pedestrian re-identification method disclosed in Example 1 of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of DCE-Net in a nighttime pedestrian re-identification method disclosed in Example 1 of the present invention;

[0049] Figure 4 This is a comparison chart of experimental data between a nighttime pedestrian re-identification method disclosed in Example 1 of the present invention and other prior art methods. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] Example 1

[0052] like Figure 1 and Figure 2As shown, embodiment 1 of the present invention provides a method for nighttime pedestrian re-identification. The method is implemented based on a nighttime pedestrian re-identification network. The method includes two processes: training and inference. The training mainly trains the nighttime pedestrian re-identification network, and the inference mainly uses the trained nighttime pedestrian re-identification network to perform pedestrian re-identification. The nighttime pedestrian re-identification network includes a brightness enhancement branch, a trunk branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with a symmetrical skip connection structure. The structure diagram is shown in FIG. Figure 3 As shown in the figure, the first six convolutional layers each consist of 32 3×3 kernels, and the final convolutional layer consists of 24 3×3 kernels. This method sets the number of iterations n to 8 during brightness enhancement, resulting in 24 convolutional layers in the final layer (8 iterations require 8 curve maps, each with 3 channels). The backbone branch consists of a second ResNet50, and the brightness distillation module consists of a bottleneck fusion subnetwork, which is an encoder-decoder structure. The following details the method.

[0053] The training process of the nighttime pedestrian re-identification network is as follows:

[0054] Step 1: Set the details of the brightness enhancement branch as follows

[0055] Step 1.1) First, input the nighttime pedestrian image to be identified and send it to the brightness enhancement branch of DEC-Net. DEC-Net is a convolutional neural network composed of 7 convolutional layers with a symmetrical skip connection structure. It outputs a set of pixel-level curve parameter mappings with the same size as the input image. The obtained mappings are used to iteratively enhance the original nighttime pedestrian image to obtain a brightness enhanced image. The iterative enhancement process is I n =LE(I n-1 ; A n ), where I n is the pixel value of the brightness enhanced image obtained after n iterations, A n The pixel value of the map used in the nth iteration is LE(I0,A1)=I0+A1*I0*(1-I0). In this embodiment, n, i.e., the number of iterations, is set to 8. The map A is generated by the illumination enhancement loss (L DCE) is self-supervised, and the loss consists of four types of losses: spatial consistency loss, exposure control loss, brightness smoothing loss, and color constancy loss. Specifically, the spatial consistency of the brightness enhanced image is promoted by maintaining the difference in local areas between the night image and its brightness enhanced image; the exposure level is controlled by measuring the distance between the average intensity value of the brightness enhanced image and the exposed constant; the brightness smoothing loss is used to avoid generating more noise on the brightness enhanced image; and the color constancy loss is used to control the color shift problem in the brightness enhancement process.

[0056] After step 1.2), the brightness-enhanced image is fed into the ResNet50 in the brightness enhancement branch to obtain the brightness-enhanced image features, and then the extracted brightness-enhanced features are fed into the first classifier to predict the pedestrian ID. This process is achieved by minimizing the loss function L ID To constrain, Where B is the number of images in a small batch, is the predicted probability of the brightness enhanced image belonging to the real image ID label. The loss function of the brightness enhancement branch is L IEB =L ID +λ1L DCE , set the balance coefficient λ1 to 1. The first classifier is a softmax classifier, and the second and third classifiers mentioned below are also softmax classifiers.

[0057] Step 2: The details of the main branch are set as follows

[0058] The original nighttime pedestrian image is sent to the main branch, which is composed of ResNet50. The convolution operation is performed to obtain the original image features to help reduce the interference of noise or excessive brightness enhancement during the brightness enhancement process, and then sent to the second classifier. This process also uses L ID Constraints are applied. The loss function of the trunk branch is L MB =L ID .

[0059] Step 3: The details of the brightness distillation module are set as follows

[0060] The brightness distillation module plays two roles during training: first, it serves as a teacher network to guide the brightness enhancement branch and the main branch in learning to extract more discriminative identity features; second, it reduces redundant information in the features obtained after the two features are fused. The training details of the distillation module are as follows:

[0061] Step 3.1) The features output by the brightness enhancement branch and the main branch are cascaded and sent to the brightness distillation module. The brightness distillation module consists of a bottleneck fusion sub-network, which is a codec structure (the encoder and decoder have a symmetrical structure, and both the encoder and the decoder include two fully connected layers and a nonlinear ReLU activation function). The cascaded features are sent to the encoder to reduce redundant features to obtain reduced-dimensional features, and then sent to the decoder to perform the reconstruction task. The reconstruction task is used to guide the reduced-dimensional features to retain the basis vectors (linearly independent vectors) in the original features, avoiding the loss of discriminative information due to the reduction in the dimension of the feature map. At the same time, the features after dimensionality reduction by the encoder are sent to the third classifier of the fusion branch for classification. The loss function of the brightness distillation module is L IDM =L ID +λ2L rec , where λ2 is the smoothing parameter, set to 0.1. L rec =||z out -z in || 2 , where z out and z in They are respectively the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the fusion sub-network.

[0062] Step 3.2) The prediction results of the brightness distillation module are respectively compared with the prediction results of the main branch and the brightness enhancement branch to calculate the loss. By forcing the distribution of the prediction results of the two branches to simulate the distribution of the prediction results in the distillation module, the two branches are guided to learn more effective identity discrimination information and improve the robustness of the learned features. This process is constrained by minimizing the KL divergence between the output of the teacher model (brightness distillation module) and the output of the student model (main branch, brightness enhancement branch). The loss function is Among them, P t and P s are the identity probabilities predicted by the teacher model and the student model respectively, S is the student model, and t is the teacher model. The student model includes the backbone branch and the brightness enhancement branch, and the teacher model is the brightness distillation model.

[0063] Step 4: The training of the entire nighttime pedestrian re-identification network is as follows

[0064] The network of the present invention is trained using the stochastic gradient descent algorithm (SGD). The entire network is trained end-to-end using a joint loss function (the loss function of the brightness enhancement branch, the backbone branch, and the brightness distillation module). The network parameters are adjusted. When the joint loss function value is minimized, training is stopped and the network parameters are fixed to obtain a trained network. The joint loss function expression is:

[0065]

[0066] where θ mb ,θ ieb ,θ idm They represent the parameters of the main branch, brightness enhancement branch, and brightness distillation module respectively.

[0067] In the subsequent reasoning process, only the main branch, brightness enhancement branch, brightness distillation module and the third classifier are used for pedestrian re-identification. The first and second classifiers only participate in training and do not participate in subsequent pedestrian re-identification.

[0068] The reasoning process is:

[0069] Step a: The original image is fed into the trained network, first passing it through DCE-Net to obtain a pixel-level curve parameter mapping of the same size as the original image. The mapping is used to iteratively enhance the brightness of the original image to obtain a brightness-enhanced image. The brightness-enhanced image is fed into the first ResNet50 to obtain a feature map of the brightness-enhanced image; the original image is a nighttime pedestrian image to be identified.

[0070] Step b: Send the original image to the second ResNet50 to obtain the feature map of the original image. The obtained feature map is a nighttime low-brightness image, which serves as a supplement to the feature map of the brightness-enhanced image to reduce the noise generated during the brightness enhancement process or the interference caused by excessive brightness enhancement.

[0071] Step c: The feature map of the original image and the feature map of the brightness enhanced image are concatenated to obtain fused features, which are then fed into the encoder of the brightness distillation module to reduce the redundant features between the two feature maps, retain the robust feature representation with identity discrimination information, and obtain the final fused features.

[0072] Step d: The final fusion features are sent to the third classifier, compared with the features corresponding to each image in the gallery, and sorted according to the similarity score to obtain the recognition results.

[0073] like Figure 4 As shown in FIG, the experimental results of the method of the present invention are compared with those of the prior art method. Since the backbone of the main branch and the brightness enhancement branch in the method of the present invention can be replaced by the backbone of the existing Re-ID method, the method of the present invention can be used as a general framework. Figure 3As can be seen from the results, our method IDFIDE+ improves rank-1 (R-1) by 5.88% and mAP by 3.25 compared to IDE+, demonstrating its effectiveness in real-world nighttime person re-ID. Similarly, our method IDFAGW and IDFTran's'ReID also improve upon existing state-of-the-art AGW and TransReID methods, demonstrating the versatility of our method.

[0074] Through the above technical solution, the present invention uses a brightness enhancement branch to improve the quality of the initial low-brightness nighttime image and obtain the features of the brightness-enhanced image. The bottleneck fusion subnetwork in the brightness distillation module then fuses the feature map of the initial low-brightness image with the feature map of the brightness-enhanced image, reducing redundant features and preserving identity discriminant information, achieving better re-identification results and enhancing the ability to re-identify pedestrians at night.

[0075] Example 2

[0076] Based on Example 1, Example 2 of the present invention further provides a nighttime pedestrian re-identification device, the device comprising:

[0077] A training module is used to train a nighttime person re-identification network to obtain a trained network. The nighttime person re-identification network includes a brightness enhancement branch, a backbone branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with a symmetrical skip connection structure. The backbone branch consists of a second ResNet50. The brightness distillation module consists of a bottleneck fusion subnetwork, which has an encoding and decoding structure.

[0078] The brightness enhancement module is used to feed the original image into the trained network, first pass it through the DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image, use the mapping map to iteratively enhance the brightness of the original image, and then feed the brightness-enhanced image into the first ResNet50 to obtain the feature map of the brightness-enhanced image;

[0079] The feature extraction module is used to feed the original image into the second ResNet50 to obtain the feature map of the original image;

[0080] The fusion module is used to obtain fusion features by cascading the feature maps of the original image and the image after brightness enhancement, and then send them to the brightness distillation module to obtain the final fusion features;

[0081] The pedestrian re-identification module is used to compare the final fusion features with the features corresponding to each image in the library, and sort them according to the similarity score to obtain the recognition results.

[0082] Specifically, the training module is also used to construct the loss function of the brightness enhancement branch:

[0083] The original image is sent to DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image. The original image is iteratively enhanced in brightness using the mapping map. After reaching the preset number of iterations, the image with enhanced brightness is obtained. The formula for iterative brightness enhancement is I n =LE(I n-1 ; A n ), and LE(I0,A1)=I0+A1*I0*(1-I0), where In is the pixel value of the brightness enhanced image obtained after n iterations, A n The pixel value of the map used in the nth iteration;

[0084] The brightness enhanced image is fed into the first ResNet50 to obtain the feature map of the brightness enhanced image, which is fed into the first classifier to predict the pedestrian ID. The loss function of the brightness enhancement branch is L IEB =L ID +λ1L DCE and Among them, λ1 is the balance coefficient, L DCE is the illumination enhancement loss, B is the number of images to be trained sent to the first classifier, is the predicted probability that the bth image to be trained belongs to the real image ID label.

[0085] More specifically, the training module is also used to construct the loss function of the trunk branch:

[0086] The original image is fed into the second ResNet50 to obtain the feature map of the original image, which is then fed into the second classifier. The loss function of the main branch is L MB =L ID .

[0087] More specifically, the training module is also used to construct the loss function of the brightness distillation module:

[0088] The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fusion features, which are sent to the brightness distillation module to obtain the final fusion features. The final fusion features are sent to the third classifier. The loss function of the brightness distillation module is L IDM =L ID +λ2L rec , where λ2 is the smoothing parameter, L rec =||z out -z in || 2 , zout and z in They are the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the bottleneck fusion sub-network;

[0089] Then the prediction results of the brightness distillation module are compared with the prediction results of the main branch and the brightness enhancement branch respectively to calculate the loss. The loss function is: Among them, P t is the predicted identity probability of the brightness distillation module, P s is the predicted identity probability of the main branch or the brightness enhancement branch, S is the student model, and t is the teacher model. The student model includes the main branch and the brightness enhancement branch, and the teacher model is the brightness distillation module.

[0090] More specifically, the training module is also used to train the entire nighttime person re-identification network. The training process is as follows:

[0091] Use the stochastic gradient descent algorithm to train the nighttime pedestrian re-identification network, adjust the network parameters, and use the joint loss function When the value of is the smallest, stop training and fix the network parameters to get a trained network, where θ mb ,θ ieb ,θ idm They represent the parameters of the main branch, brightness enhancement branch, and brightness distillation module respectively.

[0092] More specifically, the pedestrian re-identification module is further used to: input the final fusion features into the third classifier, compare them with the features corresponding to each image in the gallery, and sort them according to the similarity score to obtain the recognition result.

[0093] Example 3

[0094] Embodiment 3 of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the method steps described in embodiment 1 are implemented.

[0095] Example 4

[0096] Embodiment 4 of the present invention further provides a computer-readable storage medium storing computer program instructions, which implement the method steps described in embodiment 1 when called and executed by a processor.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A nighttime pedestrian re-identification method, characterized in that: The method comprises: Step 1: Train the nighttime pedestrian re-identification network to obtain a trained network. The nighttime pedestrian re-identification network includes a brightness enhancement branch, a backbone branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with a symmetrical skip connection structure; the backbone branch consists of a second ResNet50; the brightness distillation module consists of a bottleneck fusion sub-network, which is an encoding and decoding structure; the loss function construction process for the brightness distillation module is as follows: The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fusion features, which are sent to the brightness distillation module to obtain the final fusion features. The final fusion features are sent to the third classifier. The loss function of the brightness distillation module is ,in, is the smoothing parameter, , and They are the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the bottleneck fusion sub-network; Then the prediction results of the brightness distillation module are compared with the prediction results of the main branch and the brightness enhancement branch respectively to calculate the loss. The loss function is: ,in, is the predicted identity probability of the brightness distillation module, is the predicted identity probability of the main branch or brightness enhancement branch, For student models, is the teacher model, where the student model includes the trunk branch and the brightness enhancement branch, and the teacher model is the brightness distillation module; Step 2: The original image is fed into the trained network. First, a pixel-level curve parameter map of the same size as the original image is obtained through DCE-Net. The original image is iteratively enhanced using the map to obtain a brightness-enhanced image. The brightness-enhanced image is fed into the first ResNet50 to obtain a feature map of the brightness-enhanced image. Step 3: Send the original image to the second ResNet50 to obtain the feature map of the original image; Step 4: The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fused features, which are then fed into the brightness distillation module to obtain the final fused features. Step 5: Compare the final fusion features with the features corresponding to each image in the gallery, and sort them according to the similarity score to obtain the recognition results.

2. The method for nighttime pedestrian re-identification according to claim 1, characterized in that: The first step includes constructing a loss function for the brightness enhancement branch: The original image is fed into DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image. The mapping map is used to iteratively enhance the brightness of the original image. After reaching the preset number of iterations, the brightness-enhanced image is obtained. The formula for iterative brightness enhancement is ,and , where In is the pixel value of the brightness enhanced image obtained after n iterations, The pixel value of the map used in the nth iteration; The brightness-enhanced image is fed into the first ResNet50 to obtain the feature map of the brightness-enhanced image, which is then fed into the first classifier to predict the pedestrian ID. The loss function of the brightness enhancement branch is: and ,in, is the balance coefficient, For light enhancement loss, is the number of images to be trained that are fed into the first classifier, is the predicted probability that the bth image to be trained belongs to the real image ID label.

3. The method for nighttime pedestrian re-identification according to claim 2, characterized in that: The step 1 also includes constructing a loss function for the main branch: The original image is fed into the second ResNet50 to obtain the feature map of the original image, which is then fed into the second classifier. The loss function of the main branch is .

4. The method for nighttime pedestrian re-identification according to claim 1, characterized in that: The step 1 also includes training the entire nighttime pedestrian re-identification network. The training process is as follows: Use the stochastic gradient descent algorithm to train the nighttime pedestrian re-identification network, adjust the network parameters, and use the joint loss function When the value of is the smallest, stop training and fix the network parameters to get a trained network, where 、 、 They represent the parameters of the main branch, brightness enhancement branch, and brightness distillation module respectively.

5. The method for nighttime pedestrian re-identification according to claim 4, characterized in that: The step five includes: inputting the final fusion feature into the third classifier, comparing it with the features corresponding to each image in the gallery, and sorting them according to the similarity score to obtain the recognition result.

6. A nighttime pedestrian re-identification device, characterized in that: The device comprises: The training module is used to train a nighttime pedestrian re-identification network to obtain a trained network. The nighttime pedestrian re-identification network includes a brightness enhancement branch, a backbone branch, and a brightness distillation module. The brightness enhancement branch consists of a DCE-Net and a first ResNet50. The DCE-Net consists of seven convolutional layers with a symmetrical skip connection structure; the backbone branch consists of a second ResNet50; the brightness distillation module consists of a bottleneck fusion sub-network, which is an encoding and decoding structure; the loss function construction process for the brightness distillation module is as follows: The feature map of the original image and the feature map of the image after brightness enhancement are concatenated to obtain fusion features, which are sent to the brightness distillation module to obtain the final fusion features. The final fusion features are sent to the third classifier. The loss function of the brightness distillation module is ,in, is the smoothing parameter, , and They are the features reconstructed after the bottleneck fusion sub-network and the cascade features before inputting the bottleneck fusion sub-network; Then the prediction results of the brightness distillation module are compared with the prediction results of the main branch and the brightness enhancement branch respectively to calculate the loss. The loss function is: ,in, is the predicted identity probability of the brightness distillation module, is the predicted identity probability of the main branch or brightness enhancement branch, For student models, is the teacher model, where the student model includes the trunk branch and the brightness enhancement branch, and the teacher model is the brightness distillation module; The brightness enhancement module is used to feed the original image into the trained network, first pass it through the DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image, use the mapping map to iteratively enhance the brightness of the original image, and then feed the brightness-enhanced image into the first ResNet50 to obtain the feature map of the brightness-enhanced image; The feature extraction module is used to feed the original image into the second ResNet50 to obtain the feature map of the original image; The fusion module is used to obtain fusion features by cascading the feature maps of the original image and the image after brightness enhancement, and then send them to the brightness distillation module to obtain the final fusion features; The pedestrian re-identification module is used to compare the final fusion features with the features corresponding to each image in the library, and sort them according to the similarity score to obtain the recognition results.

7. The nighttime pedestrian re-identification device according to claim 6, characterized in that: The training module is also used to construct the loss function of the brightness enhancement branch: The original image is fed into DCE-Net to obtain a pixel-level curve parameter mapping map of the same size as the original image. The mapping map is used to iteratively enhance the brightness of the original image. After reaching the preset number of iterations, the brightness-enhanced image is obtained. The formula for iterative brightness enhancement is ,and , where In is the pixel value of the brightness enhanced image obtained after n iterations, The pixel value of the map used in the nth iteration; The brightness-enhanced image is fed into the first ResNet50 to obtain the feature map of the brightness-enhanced image, which is then fed into the first classifier to predict the pedestrian ID. The loss function of the brightness enhancement branch is: and ,in, is the balance coefficient, For light enhancement loss, is the number of images to be trained that are fed into the first classifier, is the predicted probability that the bth image to be trained belongs to the real image ID label.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the method steps described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that Computer program instructions are stored, and when the computer program instructions are called and executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images