Construction method of cross-camera proxy contrast loss based on adaptive weights

Abstract

The present invention relates to the field of image retrieval technology, and in particular, provides a method for constructing a cross-camera proxy contrast loss based on adaptive weights. The method includes obtaining a preprocessed training data set; inputting the preprocessed training data set into a convolutional neural network to obtain global features; constructing a cross-camera proxy contrast loss function through the global features; constructing an adaptive weight of the cross-camera proxy contrast loss according to the cross-camera proxy contrast loss function; training a network using a cross-camera proxy contrast loss fused with adaptive weights, optimizing network parameters through back propagation, and optimizing the network. Through the adaptive weights, the network model can adjust the contribution of each sample to the loss according to the relationship between the similarity between samples and the center of mass of camera features, and samples with similar features will be given greater weights, thereby prompting the model to pay more attention to these important samples during the training process and improving the efficiency of feature learning.

Description

Technical Field

[0001] The present invention relates to the field of image retrieval technology, and in particular to a method for constructing a cross-camera proxy contrast loss based on adaptive weights. Background Art

[0002] With the rapid development of artificial intelligence technology, the task of person re-identification in image retrieval technology has attracted much attention. By giving a single picture, using the pedestrian re-identification system for retrieval, it can quickly and accurately identify the person. The image library is usually composed of photos taken by multiple cameras.

[0003] In the pedestrian re-identification task, the perspective, lighting conditions and background changes between different cameras lead to significant differences in the feature performance of the same pedestrian under different cameras. This difference will increase the intra-class variance and reduce the recognition performance. Therefore, it is necessary to introduce a cross-camera proxy contrast loss to mitigate the impact. The traditional contrast loss will give the same weight to all samples, but in cross-camera learning, the similarity differences between samples from the same class but different cameras are often large. Summary of the invention

[0004] In view of this, the present invention provides a method for constructing a cross-camera proxy contrast loss based on adaptive weights, so as to improve the learning ability of the model, adjust the contribution of each sample to the loss, and improve the efficiency of feature learning.

[0005] In a first aspect, the present invention provides a method for constructing a cross-camera proxy contrast loss based on adaptive weights, the method comprising:

[0006] Step 1: preprocess pedestrian image datasets from different cameras to obtain a preprocessed training dataset;

[0007] Step 2: Input the preprocessed training data set into the convolutional neural network to obtain global features;

[0008] Step 3: Construct a cross-camera proxy comparison loss function through global features;

[0009] Step 4: According to the cross-camera proxy contrast loss function, construct an adaptive weight of the cross-camera proxy contrast loss;

[0010] Step 5: Use the cross-camera proxy contrast loss with fused adaptive weights to train the network, optimize the network parameters through back propagation, and optimize the network.

[0011] Optionally, step 1 includes:

[0012] In form, represents the unlabeled training dataset, where is an image, is the number of images; first extract the feature map representation , where C, H and W are the channel size, height and width of the feature map respectively; the collected pedestrian image dataset is preprocessed, and the preprocessing includes horizontal flipping, padding, and random cropping.

[0013] Optionally, the step 2 includes:

[0014] First, the preprocessed training dataset is passed to a convolutional neural network, which consists of multiple convolutional layers, activation layers, pooling layers, and fully connected layers;

[0015] In the convolution layer, the network extracts local features in the image by sliding the convolution kernel to capture the visual information of different areas; in each layer, after the convolution operation, a nonlinear activation function is applied to introduce nonlinear characteristics and enhance the expressiveness of the model; then, after the pooling layer, the network will further downsample to reduce the size of the feature map, thereby reducing the amount of calculation and the risk of overfitting; after multiple layers of convolution and pooling operations, the final feature map is flattened into a one-dimensional vector and input to the fully connected layer to further integrate the feature information; in the last layer of the fully connected layer, the output feature vector is the global feature ,The global features represent the overall information of the input image and are used in the subsequent person re-identification task.

[0016] Optionally, step 3 includes:

[0017] First, perform cluster analysis on each sample feature in the training data and divide similar features into the same cluster; obtain the centroid of the cluster by calculating the mean of all features in each cluster , then, the cross-camera proxy contrast loss is calculated through the centroid to optimize the model, which is expressed as:

[0018] ;

[0019] in and For global features The index set of positive samples and hard negative samples; is the transpose of the feature centroid of the camera perception agent associated with positive sample j, representing the mean of features from the same camera label a and belonging to the same cluster b; is the transpose of the feature centroid of the camera perception agent, representing the centroid associated with the negative sample k; is the temperature parameter that controls the degree of smoothing in the loss calculation.

[0020] Optionally, step 4 includes:

[0021] In order to dynamically adjust the cross-camera proxy contrast loss, an adaptive weight is constructed, which is expressed as:

[0022] ;

[0023] in Expressed as a cosine similarity calculation, used to measure global features With camera centroid The higher the value, the greater the similarity between the sample and the centroid; is the temperature parameter, which is used to control the influence of similarity on weight; threshold It is expressed as an offset, which is used to adjust the threshold of similarity; finally, the Sigmoid function is used to ensure the weight Between (0, 1).

[0024] Optionally, step 5 includes:

[0025] The cross-camera proxy contrast loss fused with adaptive weights is expressed as:

[0026] ;

[0027] in, is the number of images; and For global features The index set of positive samples and hard negative samples; is the weight; is the transpose of the feature centroid of the camera perception agent associated with positive sample j; is the temperature parameter; is the transpose of the feature centroid of the camera perception agent.

[0028] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for constructing a cross-camera proxy contrast loss based on adaptive weights in the first aspect or any possible implementation of the first aspect.

[0029] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the method for constructing a cross-camera proxy contrast loss based on adaptive weights in the first aspect or any possible implementation of the first aspect.

[0030] In the technical solution provided by the present invention, the method includes preprocessing pedestrian image data sets from different cameras to obtain a preprocessed training data set; inputting the preprocessed training data set into a convolutional neural network to obtain global features; constructing a cross-camera proxy contrast loss function through the global features; constructing an adaptive weight of the cross-camera proxy contrast loss according to the cross-camera proxy contrast loss function; using a cross-camera proxy contrast loss training network fused with adaptive weights, optimizing network parameters through back propagation, and optimizing the network. Through the adaptive weights, the network model can adjust the contribution of each sample to the loss according to the relationship between the similarity between samples and the camera feature centroid. Samples with similar features will be given greater weights, thereby prompting the model to pay more attention to these important samples during the training process and improving the efficiency of feature learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A flowchart of a method for constructing a cross-camera proxy contrast loss based on adaptive weights provided in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0037] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0038] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0039] Figure 1 A flowchart of a method for constructing a cross-camera proxy contrast loss based on adaptive weights provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0040] Step 1: Preprocess the pedestrian image datasets from different cameras to obtain the preprocessed training dataset.

[0041] In the embodiment of the present invention, step 1 includes:

[0042] In form, represents the unlabeled training dataset, where is an image, is the number of images; first extract the feature map representation , where C, H and W are the channel size, height and width of the feature map respectively; the collected pedestrian image dataset is preprocessed, and the preprocessing includes horizontal flipping, padding, and random cropping.

[0043] Step 2: Input the preprocessed training data set into the convolutional neural network to obtain global features.

[0044] In the embodiment of the present invention, step 2 includes:

[0045] First, the preprocessed training dataset is passed to a convolutional neural network, which consists of multiple convolutional layers, activation layers, pooling layers, and fully connected layers;

[0046] In the convolution layer, the network extracts local features in the image by sliding the convolution kernel to capture the visual information of different areas; in each layer, after the convolution operation, a nonlinear activation function is applied to introduce nonlinear characteristics and enhance the expressiveness of the model; then, after the pooling layer, the network will further downsample to reduce the size of the feature map, thereby reducing the amount of calculation and the risk of overfitting; after multiple layers of convolution and pooling operations, the final feature map is flattened into a one-dimensional vector and input to the fully connected layer to further integrate the feature information; in the last layer of the fully connected layer, the output feature vector is the global feature ,The global features represent the overall information of the input image and are used in the subsequent person re-identification task.

[0047] Step 3: Construct a cross-camera proxy comparison loss function through global features.

[0048] In the embodiment of the present invention, step 3 includes:

[0049] First, perform cluster analysis on each sample feature in the training data and divide similar features into the same cluster; obtain the centroid of the cluster by calculating the mean of all features in each cluster , then, the cross-camera proxy contrast loss is calculated through the centroid to optimize the model, which is expressed as:

[0050] ;

[0051] in and For global features The index set of positive samples and hard negative samples; is the transpose of the feature centroid of the camera perception agent associated with positive sample j, representing the mean of features from the same camera label a and belonging to the same cluster b; is the transpose of the feature centroid of the camera perception agent, representing the centroid associated with the negative sample k; is the temperature parameter that controls the degree of smoothing in the loss calculation.

[0052] In the embodiment of the present invention, the loss function brings agents in the same cluster but from different cameras closer together, thereby reducing the intra-class variance caused by the disjoint camera views.

[0053] Step 4: According to the cross-camera proxy contrast loss function, construct the adaptive weight of the cross-camera proxy contrast loss.

[0054] In the embodiment of the present invention, step 4 includes:

[0055] In order to dynamically adjust the cross-camera proxy contrast loss, an adaptive weight is constructed, which is expressed as:

[0056] ;

[0057] in Expressed as a cosine similarity calculation, used to measure global features With camera centroid The higher the value, the greater the similarity between the sample and the centroid; is the temperature parameter, which is used to control the influence of similarity on weight; threshold It is expressed as an offset, which is used to adjust the threshold of similarity; finally, the Sigmoid function is used to ensure the weight Between (0, 1).

[0058] In the embodiment of the present invention, the larger This will make the weight more sensitive to changes in similarity and more accurately reflect the impact of similarity on loss; when the cosine similarity exceeds The weight will increase when , and vice versa. This can help set a similarity benchmark, making samples more influential when they are more similar; use the Sigmoid function to ensure that the weight Between (0, 1) to control the contribution of weights to the loss, making the training more stable. It decreases with decreasing similarity, which reflects that when feature similarity is low, the impact of samples from different categories on the loss should be reduced, thereby avoiding interference with model training.

[0059] Step 5: Use the cross-camera proxy contrast loss with fused adaptive weights to train the network, optimize the network parameters through back propagation, and optimize the network.

[0060] In the embodiment of the present invention, step 5 includes:

[0061] The cross-camera proxy contrast loss fused with adaptive weights is expressed as:

[0062] ;

[0063] in, is the number of images; and For global features The index set of positive samples and hard negative samples; is the weight; is the transpose of the feature centroid of the camera perception agent associated with positive sample j; is the temperature parameter; is the transpose of the feature centroid of the camera perception agent.

[0064] In the embodiment of the present invention, through this adaptive weight design, the model can dynamically adjust its weight according to the similarity between samples, so that when the similarity is high, more attention is paid to the influence of these similar samples, and when the similarity is low, the influence of these samples is reduced. This method can not only enhance the learning ability of the model in cross-camera tasks, but also effectively reduce the noise impact caused by different perspectives and improve the overall performance.

[0065] Traditional loss functions usually use fixed weights and fail to fully consider the similarity differences between different samples, resulting in increased noise influence and intra-class variance during training. By introducing adaptive weights, the present invention can dynamically adjust the contribution of each sample to the loss according to the similarity between features, reducing the interference of samples of different categories on model training. This not only improves the robustness of the model in cross-camera recognition, but also effectively reduces the noise influence caused by perspective differences, thereby significantly improving the overall performance of pedestrian re-identification.

[0066] Each step of the embodiment of the present invention may be performed by an electronic device, which includes but is not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, etc.

[0067] In the technical solution provided by the present invention, the method includes preprocessing pedestrian image data sets from different cameras to obtain a preprocessed training data set; inputting the preprocessed training data set into a convolutional neural network to obtain global features; constructing a cross-camera proxy contrast loss function through the global features; constructing an adaptive weight of the cross-camera proxy contrast loss according to the cross-camera proxy contrast loss function; using a cross-camera proxy contrast loss training network fused with adaptive weights, optimizing network parameters through back propagation, and optimizing the network. Through the adaptive weights, the network model can adjust the contribution of each sample to the loss according to the relationship between the similarity between samples and the camera feature centroid. Samples with similar features will be given greater weights, thereby prompting the model to pay more attention to these important samples during the training process and improving the efficiency of feature learning.

[0068] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the electronic device where the computer-readable storage medium is located is controlled to execute an embodiment of the method for constructing the cross-camera proxy contrast loss based on adaptive weights.

[0069] Figure 2 A schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 2As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the method for constructing a cross-camera proxy contrast loss based on adaptive weights in the embodiment is implemented. To avoid repetition, they are not described one by one here.

[0070] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will appreciate that Figure 2 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0071] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0072] The memory 212 may be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 may also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 21. Further, the memory 212 may also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 may also be used to temporarily store data that has been output or is to be output.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a cross-camera proxy contrast loss based on adaptive weights, characterized in that: The method comprises: Step 1: preprocess pedestrian image datasets from different cameras to obtain a preprocessed training dataset; Step 2: Input the preprocessed training data set into the convolutional neural network to obtain global features; Step 3: Construct a cross-camera proxy comparison loss function through global features; Step 4: According to the cross-camera proxy contrast loss function, construct an adaptive weight of the cross-camera proxy contrast loss; Step 5: Use the cross-camera proxy contrast loss with fused adaptive weights to train the network, optimize the network parameters through back propagation, and optimize the network; The step 4 comprises: In order to dynamically adjust the cross-camera proxy contrast loss, an adaptive weight is constructed, which is expressed as: ； in Expressed as a cosine similarity calculation, used to measure global features With camera centroid The higher the value, the greater the similarity between the sample and the centroid; is the temperature parameter, which is used to control the influence of similarity on weight; threshold It is expressed as an offset, which is used to adjust the threshold of similarity; finally, the Sigmoid function is used to ensure the weight Between (0, 1); The step 5 comprises: The cross-camera proxy contrast loss fused with adaptive weights is expressed as: ； in, is the number of images; and For global features The index set of positive samples and hard negative samples; is the weight; is the transpose of the feature centroid of the camera perception agent associated with positive sample j; is the temperature parameter; is the transpose of the feature centroid of the camera perception agent.

2. The method according to claim 1, characterized in that The step 1 comprises: In form, represents the unlabeled training dataset, where is an image, is the number of images; first extract the feature map representation , where C, H and W are the channel size, height and width of the feature map respectively; the collected pedestrian image dataset is preprocessed, and the preprocessing includes horizontal flipping, padding, and random cropping.

3. The method according to claim 1, characterized in that The step 2 comprises: First, the preprocessed training dataset is passed to a convolutional neural network, which consists of multiple convolutional layers, activation layers, pooling layers, and fully connected layers; In the convolution layer, the network extracts local features in the image by sliding the convolution kernel to capture the visual information of different areas; in each layer, after the convolution operation, a nonlinear activation function is applied to introduce nonlinear characteristics and enhance the expressiveness of the model; then, after the pooling layer, the network will further downsample to reduce the size of the feature map, thereby reducing the amount of calculation and the risk of overfitting; after multiple layers of convolution and pooling operations, the final feature map is flattened into a one-dimensional vector and input to the fully connected layer to further integrate the feature information; in the last layer of the fully connected layer, the output feature vector is the global feature ,The global features represent the overall information of the input image and are used in the subsequent person re-identification task.

4. The method according to claim 1, characterized in that The step 3 comprises: First, perform cluster analysis on each sample feature in the training data and divide similar features into the same cluster; obtain the centroid of the cluster by calculating the mean of all features in each cluster , then, the cross-camera proxy contrast loss is calculated through the centroid to optimize the model, which is expressed as: ； in and For global features The index set of positive samples and hard negative samples; is the transpose of the feature centroid of the camera perception agent associated with positive sample j, representing the mean of features from the same camera label a and belonging to the same cluster b; is the transpose of the feature centroid of the camera perception agent, representing the centroid associated with the negative sample k; is the temperature parameter that controls the degree of smoothing in the loss calculation.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is run, the computer-readable storage medium is controlled to execute the method for constructing a cross-camera proxy contrast loss based on adaptive weights as described in any one of claims 1 to 4.

6. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the method for constructing a cross-camera proxy contrast loss based on adaptive weights as described in any one of claims 1 to 4.

Images