Pedestrian re-identification method and device for image change, equipment and storage medium

Abstract

The invention provides an image change-oriented pedestrian re-identification method and device, and relates to the technical field of computer vision. The image change-oriented pedestrian re-identification method mainly comprises the following steps of: constructing a pedestrian re-identification model by using ResNet50 as a backbone network, wherein a multi-granularity attention mechanism is added into the pedestrian re-identification model; a classification loss function, a triple loss function and a component decorrelation loss function are combined to train the pedestrian re-identification model, and local pedestrian features with discrimination are learned; and executing a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian identification result. By implementing the pedestrian re-identification method and device facing the image change provided by the invention, the robustness of the pedestrian re-identification model to the image change in a real scene can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a pedestrian re-identification method and device facing image changes, equipment and storage medium. BACKGROUND

[0002] Pedestrian re-identification is a technology for searching for the identity of a specific pedestrian in an image or video library using computer vision methods. In the prior art, a pedestrian re-identification model constructed by a neural network is used to identify images or videos.

[0003] In the prior art, various image change problems (such as pedestrian posture, camera angle, body occlusion, background noise, light intensity, image resolution, etc.) exist in the pedestrian data captured by the camera, which brings great challenges to the application and promotion of real scene pedestrian re-identification technology. These complex and changing pedestrian images will reduce the matching performance of pedestrian re-identification, making it very difficult to achieve an efficient re-identification model. Current mainstream methods mostly use additional human part labeling or uniform horizontal feature division to learn pedestrian features with robustness, but cannot fully meet the needs of application scenarios where image changes are not controlled in terms of robustness and accuracy. Therefore, there is an urgent need to design an efficient pedestrian re-identification algorithm facing image changes to learn pedestrian features with robustness and discriminability. SUMMARY

[0004] The present application provides a pedestrian re-identification method and device facing image changes, equipment and storage medium, aiming to improve the performance of the pedestrian re-identification model.

[0005] The present application provides a pedestrian re-identification method and device facing image changes, equipment and storage medium, aiming to improve the performance of the pedestrian re-identification model.

[0006] S01: using ResNet50 as a backbone network to construct a pedestrian re-identification model, wherein the pedestrian re-identification model is added with a multi-granularity attention mechanism;

[0007] S02: training the pedestrian re-identification model by combining a classification loss function, a triplet loss function and a part decorrelation loss function to learn local pedestrian features with discriminability;

[0008] S03: performing a pedestrian re-identification task by using the pedestrian re-identification model to obtain a pedestrian identification result. Optionally, the construction step of the pedestrian re-identification model in step S01 of the pedestrian re-identification method facing image changes comprises:

[0009] S011: extracting pedestrian convolutional features from the backbone network;

[0010] S012: convert the pedestrian convolutional features into query features, key features, and value features, and calculate semantic correlation and spatial similarity;

[0011] S013: calculate value space feature weighting values using the semantic correlation and the first spatial similarity, and fuse value space features using the value space feature weighting values to obtain first pedestrian features;

[0012] S014: downsample the pedestrian convolutional features into second pedestrian features using a plurality of different downsamplers, input the second pedestrian features into a semantic alignment layer and an upsample layer, restore the second pedestrian features to the original size to obtain third pedestrian features;

[0013] S015: fuse all third pedestrian features to learn a low-dimensional pedestrian feature, and use multi-granularity attention to weight and sum the third pedestrian features to obtain fused pedestrian features;

[0014] S016: uniformly divide the fused pedestrian features into horizontal stripe convolutional features, and input the horizontal stripe convolutional features into a global average pooling layer to obtain local feature vectors;

[0015] S017: convert the local feature vectors into query feature vectors through a fully connected layer, convert the fused pedestrian features into key features and value features, and calculate semantic correlation and spatial correlation;

[0016] S018: fuse value features by weighting attention values of value features at different positions to obtain pedestrian part features.

[0017] Optionally, step S012 of the image change-oriented pedestrian re-identification method comprises: converting the pedestrian convolutional features into the query features, the key features, and the value features using different 1x1 convolutional layers; given query space features and key space features, a two-dimensional Gaussian kernel is used to calculate the spatial similarity, as shown in the formula:

[0018]

[0019] wherein, represents the query space features at the i-th position, represents the key space features at the j-th position, p(·) represents the position coordinates of the space features, σ represents the standard deviation, and ||·||2 represents the two-norm.

[0020] Optionally, the first pedestrian features, the second pedestrian features, the third pedestrian features, and the fused pedestrian features in steps S013-S015 of the image change-oriented pedestrian re-identification method are as shown in the formula:

[0021]

[0022]

[0023] where softmax(·) denotes the Softmax function. Since the Softmax function is very sensitive to larger or smaller inputs (easily mapped to 1 or 0), it is scaled by a factor g denotes the pedestrian feature of the g-th granularity, N g denotes the number of granularities of the pedestrian feature, denotes the semantic alignment function, upsample(·, H, W) denotes an up-sampling function to up-sample the input feature to a spatial size of H x W, a g denotes the attention value of the g-th granularity.

[0024] Optionally, the step S017 of the image change-oriented pedestrian re-identification method specifically comprises: converting the local feature vector into a query feature vector by using a fully connected layer; converting the pedestrian convolutional feature into a key feature and a value feature by using different 1 x 1 convolutional layers; modeling the semantic correlation by using the inner product of the query feature and the key feature; given the query spatial feature and the key spatial feature, measuring the spatial correlation by using a piecewise Gaussian kernel function; when the key feature and the query feature are in the same region, setting the spatial correlation as 1; when the key feature and the query feature are in different regions, using the spatial correlation to help exclude the interference of edge position features. The semantic correlation and the spatial correlation are as follows:

[0025]

[0026] where q l denotes the query feature, K j denotes the key feature, p h (·) denotes the height coordinate of the key feature, denotes the upper bound of the height coordinate of the region where the query feature is located, denotes the lower bound of the height coordinate of the region where the query feature is located.

[0027] Optionally, the step S018 of the image change-oriented pedestrian re-identification method specifically comprises: multiplying the semantic correlation and the spatial correlation to obtain the attention weighting value of the value feature of the different positions; fusing the value features of the different positions according to the obtained weighting value; further obtaining an iterative updating mode of the pedestrian part feature, and the key feature and the value feature are shared in the iterative updating mode process, as follows:

[0028]

[0029] where, denotes the part feature after the n-th iterative updating.​ denotes the query feature updated in the n-th iteration, The initialization value of is denoted as ψ (n) denotes a fully connected layer mapping function for updating in the n-th iteration, which converts the part feature into the query feature.

[0030] Optionally, the part de-correlation loss function in step S02 of the image change-oriented pedestrian re-identification method comprises: reducing the correlation between different part features, so that the cosine similarity between the part features is less than the similarity interval; and ensuring that each attention mask focuses on different regions of a given pedestrian image.

[0031] The second aspect of the present application provides a pedestrian re-identification device oriented to image changes, the device comprising:

[0032] A global semantic alignment module is configured to align the global semantics, convert the pedestrian feature into a query feature, a key feature and a value feature, and model the global semantic correlation of the human body parts.

[0033] A two-dimensional Gaussian kernel module is configured to calculate the spatial similarity by using a two-dimensional Gaussian kernel, and fuse the value space features at different positions by using the obtained weighted values to obtain the corrected pedestrian feature.

[0034] A feature fusion module is configured to effectively fuse the multi-granularity global semantic information to improve the generalization ability of the pedestrian re-identification model.

[0035] A judgment module is configured to represent the local human body part information by the query feature, calculate the semantic correlation between all query features and key features, and determine whether the key feature at the current position belongs to the human body part represented by the query feature.

[0036] A positioning module is configured to accurately measure the contribution degree of the value feature to the part feature, and define the attention weighted value of the value feature at the different positions as the product value of the semantic correlation and the spatial correlation.

[0037] An attention focusing module is configured to make the cosine similarity between the part features less than the similarity interval according to the part de-correlation loss function, and reduce the correlation between different part features.

[0038] A model training module is configured to train the pedestrian re-identification model by using the classification loss function, the triplet loss function and the part de-correlation loss function, and learn the discriminative local pedestrian feature.

[0039] A pedestrian re-identification module is configured to perform the pedestrian re-identification task by using the pedestrian re-identification model, and obtain the pedestrian recognition result.

[0040] Optionally, the constructing step of the pedestrian re-identification model comprises:

[0041] The pedestrian re-identification model extracts the fused pedestrian features by adding a multi-granularity attention mechanism;

[0042] The fused pedestrian features are evenly divided into horizontal stripe convolution features and input into a global average pooling layer to obtain a local feature vector;

[0043] The local feature vector is converted into a query feature vector by a fully connected layer, the fused pedestrian features are converted into key features and value features, and semantic correlation and spatial correlation are calculated;

[0044] The attention weighting values of the value features at different positions are fused to obtain pedestrian part features;

[0045] The pedestrian re-identification model is trained by combining a classification loss function, a triplet loss function and a part decorrelation loss function to learn discriminative local pedestrian features;

[0046] The pedestrian re-identification model is used to perform a pedestrian re-identification task to obtain a pedestrian identification result.

[0047] Optionally, the constructing step of the pedestrian re-identification model further comprises:

[0048] Pedestrian convolution features are extracted from the backbone network of the pedestrian re-identification model;

[0049] The pedestrian convolution features are converted into query features, key features and value features, and semantic correlation and spatial similarity are calculated;

[0050] The semantic correlation and the first spatial similarity are used to calculate value space feature weighting values, and the value space feature weighting values are used to fuse value space features to obtain first pedestrian features;

[0051] The pedestrian convolution features are down-sampled into second pedestrian features by using a plurality of different down-samplers, and the second pedestrian features are input into a semantic alignment layer and an up-sampling layer, and then the second pedestrian features are restored to the original size to obtain third pedestrian features;

[0052] All third pedestrian features are fused to learn a low-dimensional pedestrian feature, and the third pedestrian features are weighted and summed by using a multi-granularity attention to obtain the fused pedestrian features.

[0053] Optionally, the global semantic alignment constructing module comprises:

[0054] A pedestrian feature extraction submodule is configured to extract pedestrian convolution features by the pedestrian re-identification model and convert the pedestrian convolution features into query features, key features and value features;

[0055] The feature correlation calculation sub-module is configured to calculate semantic correlation between query features and key features of all pairs of positions by using the pedestrian re-identification model.

[0056] Optionally, the two-dimensional Gaussian kernel construction module further comprises:

[0057] The spatial similarity calculation sub-module is configured to calculate spatial similarity given two spatial features.

[0058] The spatial feature weighting value calculation sub-module is configured to calculate the weighting value of the value spatial feature of different positions by using the semantic correlation and the spatial similarity of all pairs of position features.

[0059] The modified feature sub-module is configured to fuse the value spatial features of different positions by using the obtained weighting value to obtain a modified pedestrian feature.

[0060] Optionally, the feature fusion module comprises:

[0061] The sampling sub-module is configured to sample the pedestrian convolutional features into pedestrian features of different granularities by using multiple different down-samplers.

[0062] The feature input sub-module is configured to input the pedestrian features of different granularities into the semantic alignment layer and then into the up-sampling layer.

[0063] The weighted summation sub-module is configured to fuse the global semantic information of multiple granularities, and perform weighted summation on the pedestrian features of different granularities by using the multi-granularity attention to obtain the fused pedestrian features.

[0064] Optionally, the judgment module comprises:

[0065] The correlation modeling sub-module is configured to model the semantic correlation by using the inner product of the query features and the key features.

[0066] The segmented two-dimensional Gaussian kernel sub-module is configured to design a segmented two-dimensional Gaussian kernel for calculating the similarity of spatial positions in a local region.

[0067] The third aspect of the present application provides a readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method of the first aspect of the present application.

[0068] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the steps of the method of the first aspect of the present application.

[0069] The image change-oriented pedestrian re-identification method provided by the application designs a global semantic alignment method, converts the pedestrian features into query features, key features and value features, and models the global semantic correlation of the human body parts; a two-dimensional Gaussian kernel is used to calculate the spatial similarity, and the obtained weighted values are used to fuse the value space features at different positions to obtain the corrected pedestrian features; a multi-granularity attention mechanism is designed to effectively fuse the multi-granularity global semantic information, so as to improve the generalization ability of the pedestrian re-identification model; the query features represent local human body part information, the semantic correlation between all query features and key features is calculated, and it is judged whether the key features at the current position belong to the human body parts represented by the query features; the contribution degree of the value features to the part features is accurately measured, and the attention weighted values of the value features at different positions are defined as the product values of the semantic correlation and the spatial correlation; according to the part decorrelation loss function, the cosine similarity between the part features is less than the similarity interval, and the correlation between different part features is reduced; the pedestrian re-identification model is trained by combining the classification loss function, the triplet loss function and the part decorrelation loss function, and the discriminative local pedestrian features are learned; the pedestrian re-identification task is performed through the pedestrian re-identification model, and the pedestrian recognition result is obtained.

[0070] In the application, the pedestrian re-identification model with the multi-granularity attention mechanism effectively fuses the multi-granularity global semantic information, and then the re-identification model adaptively adjusts the semantic alignment granularity according to the input content, globally averages all the granularities, generates multi-granularity attention through a fully connected layer and Softmax, and obtains the final features through weighted summation; the adaptive region pooling learning method is introduced to preliminarily pool the extracted local features to generate a preliminary local feature vector. The local feature vector is converted into corresponding query features through a fully connected layer, and the original convolutional features are respectively converted into key features and value features through multiple independent convolutional layers, and then the semantic correlation between the query features and the key features is calculated. In order to further enhance the spatial focusing ability of the local features, a segmented two-dimensional Gaussian kernel is designed to calculate the similarity of the spatial positions in the local region, and the part positioning accuracy is gradually improved through iterative optimization. The pedestrian re-identification model is trained by combining the classification loss function, the triplet loss function and the part decorrelation loss function, so as to learn the discriminative local pedestrian features. Sufficient pedestrian images with human semantic annotations are obtained from the global features and the local features. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the description of the application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0072] Figure 1 is a global feature learning schematic diagram based on multi-granularity semantic alignment;

[0073] Figure 2 is a flowchart of a pedestrian re-identification method for image changes;

[0074] Figure 3 is a local feature learning schematic diagram based on adaptive region pooling;

[0075] Figure 4 is a schematic diagram of a pedestrian re-identification device for image changes according to an embodiment of the present application;

[0076] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0078] Since pedestrians are non-rigid, the shape of the pedestrian and the position of each part of the body change, and the appearance of the pedestrian is also affected by factors such as camera angle, lighting conditions, body occlusion, and camera parameter differences, which increase the difficulty of feature description of the appearance of the pedestrian. Existing methods can be summarized into two categories: component prior information learning and component supervised information learning.

[0079] The component prior information learning method only uses pedestrian class labels to train the network, and does not use any additional annotation information of pedestrian components, and divides the human body region based on the prior information of the spatial distribution of the components. In order to alleviate the problem of non-alignment of pedestrian components, considering that the spatial distribution of the human body components does not change much in the pedestrian image, these methods generally use a horizontal feature segmentation method to learn the component features of the pedestrian image.

[0080] The component supervised information learning method uses additional pedestrian component supervision information to assist the pedestrian re-identification model to learn discriminative pedestrian component features.

[0081] The component prior information learning method can only realize coarse-grained component alignment, but cannot realize fine-grained semantic alignment of local features. The real environment in which the pedestrian is located often makes the pedestrian image captured by the camera have rich image changes, so the method may be difficult to adapt to the real scene with complex image changes, which will seriously affect the retrieval performance of the pedestrian re-identification model. In addition, the component supervision information learning method seriously depends on component labeling information, but it is difficult to obtain sufficient pedestrian images with component labeling and high-precision component detection network in the real environment. Low-precision component labeling and component detection will introduce additional errors, affecting the quality of the extracted local features, so it may not be able to be well generalized to pedestrian images with new changes.

[0082] The image change-oriented pedestrian re-identification method of the present application comprises the following steps:

[0083] S01: using ResNet50 as a backbone network to construct a pedestrian re-identification model, wherein the pedestrian re-identification model is added with a multi-granularity attention mechanism;

[0084] S011: extracting pedestrian convolutional features from the backbone network;

[0085] Specifically, step S011 comprises: in this embodiment, the re-identification model is a model constructed based on a deep learning model, and the backbone network thereof is ResNet50. The network can extract features from images, and the pedestrian convolutional features are convolutional features extracted from the input pedestrian image by the backbone network. It should be understood that the backbone network in the pedestrian re-identification model of the present application is preferably a deep residual network ResNet50, and it should be understood that this is only a preferred embodiment of the backbone network in the pedestrian re-identification model of the present application. The backbone network in the pedestrian re-identification model of the present application can also be other backbone networks.

[0086] In this embodiment, the reference Figure 1 , Figure 1 is a global feature learning diagram based on multi-granularity semantic alignment, as shown in Figure 1 When constructing the re-identification model, first construct an attention mechanism module, and extract pedestrian convolutional features of the image from the input image based on a deep learning network. Attention mechanism is an algorithm in deep learning that simulates human cognitive resource allocation. By dynamically assigning different weights to different input information, it helps the model focus on key information and improves the ability to handle complex tasks.

[0087] For example, the pedestrian convolutional features extracted from the backbone network are wherein C represents the number of channels, H represents the height, and W represents the width.

[0088] S012: convert the pedestrian convolutional features into query features, key features, and value features, and calculate semantic relevance and spatial similarity;

[0089] Specifically, step S012 includes converting pedestrian features F into query features key features and value features

[0090] In this embodiment, the query, key, and value three achieve information filtering and focusing through synergistic effect, and the core relationship can be summarized as: the query initiates demand actively, the key matches demand passively, and the value provides specific content.

[0091] In this embodiment, feature pairs with close distances in human body components have higher semantic relevance, and feature pairs with far distances have lower relevance, so it is necessary to limit the interference of feature pairs with far distances on global semantic alignment. The two-dimensional Gaussian kernel makes the pose alignment pay more attention to local areas.

[0092] The semantic relevance is calculated given the query spatial features and the key spatial features, and the two-dimensional Gaussian kernel is used to calculate the spatial similarity, as shown in the formula:

[0093]

[0094] wherein, denotes the query spatial features of the i-th position, denotes the key spatial features of the j-th position, p(·) denotes the position coordinates of the spatial features, σ denotes the standard deviation, and ||·||2 denotes the two-norm.

[0095] S013: calculating value spatial feature weighting values using the semantic relevance and the first spatial similarity, and fusing value spatial features using the value spatial feature weighting values to obtain a first pedestrian feature;

[0096] S014: down-sampling the pedestrian convolutional features into second pedestrian features using a plurality of different down-samplers, inputting the second pedestrian features into a semantic alignment layer and an up-sampling layer, restoring the second pedestrian features to the original size to obtain third pedestrian features;

[0097] S015: fusing all third pedestrian features to learn a low-dimensional pedestrian feature, and performing weighted summation on the third pedestrian features using multi-granularity attention to obtain the fused pedestrian features.

[0098] Specifically, the first pedestrian feature, the second pedestrian feature, the third pedestrian feature, and the fused pedestrian feature described in steps S013-S015 specifically include using the obtained weighted values to fuse the value space features at different positions to obtain corrected pedestrian features, using multiple different down-samplers to sample the pedestrian convolutional features into pedestrian features of different granularities, resampling the pedestrian features back to the original size, and using multi-granularity attention to weight-sum the pedestrian features of different granularities to obtain the fused pedestrian features, as shown in the following formula:

[0099]

[0100] wherein softmax(·) represents a Softmax function. Since the Softmax function is very sensitive to large or small inputs (easily mapped to 1 or 0), the input is scaled by a factor of g represents the pedestrian feature of the gth granularity, N g represents the number of granularities of the pedestrian feature, represents a semantic alignment function, upsample(·, H, W) represents an up-sampling function for up-sampling the input feature to a spatial size of HxW, and a g represents the attention value of the gth granularity.

[0101] In this embodiment, in order to improve the calculation efficiency, a low-dimensional pedestrian feature is learned through a fully connected layer to learn the global information of the pedestrian of all granularities. Under the guidance of the compact pedestrian feature, the multi-granularity attention a is learned through a Softmax mapping function to adaptively select the feature information of different granularities.

[0102] In this embodiment, without using human body part annotation information (such as human key points, human semantic segmentation, etc.), the semantic alignment of the global feature is realized by using the semantic correlation between each pair of position features in the global convolutional feature; and according to the semantic alignment of all the current granularity features, the global convolutional feature of the best granularity can be dynamically selected as the output result.

[0103] It should be noted that after the second pedestrian feature is obtained, all the pedestrian features of different granularities need to be fused to obtain global information. In this embodiment, the pedestrian re-identification model can adaptively adjust the semantic alignment granularity according to the input content, and the basic idea is to use a gate to control the information flow from multiple granularities, and these branches carry different scale information into the next layer of neurons of the re-identification model.

[0104] For example, all the pedestrian features of different granularities are fused to obtain global information as follows:

[0105]

[0106] ​where GAP(·) denotes a global average pooling function.

[0107] S016: uniformly segmenting the fused pedestrian feature into horizontal stripe convolution features and inputting the horizontal stripe convolution features into a global average pooling layer to obtain a local feature vector;

[0108] Specifically, step S016 includes: in this embodiment, referring to Figure 3 , Figure 3 is a local feature learning scheme based on adaptive region pooling, as shown in Figure 3 , through the local feature learning based on adaptive region pooling, the human body component region can be adaptively located. The convolution feature F is uniformly segmented into N l horizontal stripe convolution features to obtain local visual clues. Then, all N l stripe convolution features are input into a global average pooling layer to obtain N l local feature vectors

[0109] For example, the calculation of the local feature vector is as follows:

[0110]

[0111] S017: converting the local feature vector into a query feature vector through a fully connected layer, converting the fused pedestrian feature into a key feature and a value feature, and calculating semantic relevance and spatial relevance;

[0112] Specifically, step S017 includes: converting the local feature vector into a query feature vector through a fully connected layer; converting the pedestrian convolution feature into a key feature and a value feature through different 1×1 convolution layers; modeling the semantic relevance through the inner product of the query feature and the key feature; given the query spatial feature and the key spatial feature, measuring the spatial relevance through a piecewise Gaussian kernel function; when the key feature and the query feature are in the same region, setting the spatial relevance as 1; when the key feature and the query feature are in different regions, using the spatial relevance to help exclude the interference of edge position features. The semantic relevance and the spatial relevance are as follows:

[0113]

[0114] where q l denotes a query feature, K j denotes a key feature, p h (·) denotes the height coordinate of the key feature, h denotes the upper bound of the height coordinate of the region where the query feature is located, denotes the lower bound of the height coordinate of the region where the query feature is located.

[0115] In this embodiment, the spatial correlation degree is measured by using a segmented Gaussian kernel function, because the position features in the same region have similar semantic information, and the semantic information of the position features between different regions may have certain differences. Considering the spatial distance between the local features and the position features, the segmented Gaussian kernel function is used to calculate the spatial correlation degree, so that the local features can be more focused on the limited local region.

[0116] S018: fuse the attention weight values of the value features at different positions to obtain pedestrian part feature values;

[0117] Specifically, step S018 includes: multiplying the semantic correlation degree and the spatial correlation degree to obtain the attention weight values of the value features at different positions; fusing the value features at different positions according to the obtained weight values; and further obtaining an iterative updating mode of the pedestrian part feature values, in which the key features and the value features are shared, as shown in the following formula:

[0118]

[0119] In this embodiment, the part positioning accuracy is gradually improved through iterative optimization:

[0120]

[0121] wherein, denotes the part feature after the n th iterative update; denotes the query feature after the n th iterative update, The initial value of is denoted as ψ (n) denotes a full connection layer mapping function for the n th iterative update, which converts the part feature into the query feature.

[0122] S02: jointly train the pedestrian re-identification model by using the classification loss function, the triplet loss function and the part decorrelation loss function, and learn the discriminative local pedestrian features;

[0123] Specifically, the part decorrelation loss function in step S02 includes: reducing the correlation between different part features, so that the cosine similarity between the part features is less than the similarity interval; and ensuring that each attention mask focuses on different regions of a given pedestrian image.

[0124] In this embodiment, in order to sample human part features with different semantics, the attention mask of the part feature should have a discriminative spatial distribution. Since multiple attention masks can easily learn to detect the same human part, it is necessary to ensure that each attention mask focuses on different regions of a given pedestrian image.

[0125] In this embodiment, the local pedestrian features can be effectively learned without using human part annotation information (such as human key points, human semantic segmentation, etc.).

[0126] S03: performing a pedestrian re-identification task by using the pedestrian re-identification model to obtain a pedestrian re-identification result.

[0127] In this embodiment, as shown in Figure 1 , the global feature learning scheme can perform semantic alignment on the global features, and Figure 3 , the local feature learning scheme can perform region positioning on the local features, so that the semantic alignment of the global features promotes the region positioning of the local features.

[0128] Based on the same inventive concept, an embodiment of the present application provides a pedestrian re-identification device for image changes. Referring to Figure 4 , Figure 4 is a schematic diagram of the pedestrian re-identification device 400 provided by an embodiment of the present application. As shown in Figure 4 , the device comprises:

[0129] a global semantic alignment module 401 configured to perform global semantic alignment, convert pedestrian features into query features, key features, and value features, and model global semantic correlation of human parts;

[0130] a two-dimensional Gaussian kernel module 402 configured to calculate spatial similarity by using a two-dimensional Gaussian kernel, and use the obtained weighted values to fuse value space features at different positions to obtain corrected pedestrian features;

[0131] a feature fusion module 403 configured to effectively fuse multi-granularity global semantic information to improve the generalization ability of the pedestrian re-identification model;

[0132] a judgment module 404 configured to determine whether the key features at the current position belong to the human parts represented by the query features, by using the query features to represent local human part information and calculating semantic correlation between all query features and key features;

[0133] a positioning module 405 configured to accurately measure the contribution degree of the value features to the part features, and define attention weighted values of the value features at different positions as a product value of semantic correlation and spatial correlation;

[0134] an attention focusing module 406 configured to make the cosine similarity between the part features less than a similarity interval according to a part de-correlation loss function, and reduce the correlation between different part features;

[0135] a model training module 407 configured to train the pedestrian re-identification model by using a classification loss function, a triplet loss function, and a part de-correlation loss function, and learn discriminative local pedestrian features.

[0136] The pedestrian re-identification module 408 is configured to perform a pedestrian re-identification task by using the pedestrian re-identification model to obtain a pedestrian re-identification result.

[0137] Optionally, the pedestrian re-identification model is constructed by:

[0138] extracting pedestrian convolutional features from the pedestrian image by using a backbone network of the pedestrian re-identification model;

[0139] converting the pedestrian convolutional features into query features, key features, and value features;

[0140] calculating semantic relevance between the query features and the key features at all pairs of positions;

[0141] calculating weighted values of the value space features at different positions to obtain modified pedestrian features;

[0142] fusing pedestrian features of all granularities to obtain global information;

[0143] The query features represent local body part information, and the semantic relevance between all query features and key features is calculated.

[0144] The attention weighted values of the value features at different positions are defined as the product values of the semantic relevance and spatial relevance.

[0145] According to the obtained weighted values, the value features at different positions are fused to obtain pedestrian part features with more accurate positioning.

[0146] Optionally, the pedestrian re-identification model is constructed by:

[0147] a plurality of different down-samplers are used to sample the pedestrian convolutional features into pedestrian features of different granularities, the pedestrian features of different granularities are input into a semantic alignment layer, and then input into an up-sampling layer;

[0148] The pedestrian re-identification model is trained by using a joint classification loss function, a triplet loss function, and a part decorrelation loss function to learn discriminative local pedestrian features.

[0149] Optionally, the global semantic alignment construction module includes:

[0150] a pedestrian feature extraction submodule configured to extract pedestrian convolutional features by using the pedestrian re-identification model and convert the pedestrian convolutional features into query features, key features, and value features;

[0151] a feature relevance calculation submodule configured to calculate semantic relevance between query features and key features at all pairs of positions by using the pedestrian re-identification model.

[0152] Optionally, the two-dimensional Gaussian kernel modeling module further comprises:

[0153] a spatial similarity calculation submodule, configured to calculate spatial similarity given two spatial features;

[0154] a spatial feature weighting value calculation submodule, configured to calculate the weighting value of the value spatial feature of different positions by using the semantic correlation degree and the spatial similarity of all pairs of position features;

[0155] a correction feature submodule, configured to fuse the value spatial features of different positions by using the calculated weighting value to obtain the corrected pedestrian feature.

[0156] Optionally, the feature fusion module comprises:

[0157] a sampling submodule, configured to sample the pedestrian convolutional feature into pedestrian features of different granularities by using multiple different down-samplers;

[0158] a feature input submodule, configured to input the pedestrian features of different granularities to the semantic alignment layer and then to the up-sampling layer;

[0159] a weighted sum submodule, configured to fuse the global semantic information of multiple granularities, and perform weighted sum on the pedestrian features of different granularities by using the attention of multiple granularities to obtain the fused pedestrian feature.

[0160] Optionally, the judgment module comprises:

[0161] a correlation degree modeling submodule, configured to model the semantic correlation degree by using the inner product of the query feature and the key feature;

[0162] a segmented two-dimensional Gaussian kernel submodule, configured to design a segmented two-dimensional Gaussian kernel for calculating the similarity of spatial positions in a local region.

[0163] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the pedestrian re-identification method according to any one of the above embodiments of the present application.

[0164] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Figure 5 is a schematic diagram of an electronic device 500 according to an embodiment of the present application, which comprises a memory 502, a processor 501, and a computer program stored in the memory and executable on the processor, and the processor implements the pedestrian re-identification method according to any one of the above embodiments of the present application when executed.

[0165] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts are described in the part of the method embodiment.

[0166] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be understood by referring to each other.

[0167] Those skilled in the art will appreciate that the present application can be provided as a method, device, or computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0168] The present application is described with reference to flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device that implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more blocks or flows.

[0169] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus that implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more blocks or flows.

[0170] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more blocks or flows.

[0171] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they have the benefit of the present disclosure. Therefore, it is to be understood that the appended claims are intended to cover all such modifications and changes as can come within the scope of the application. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0172] Finally, it is to be understood that the phraseology or terminology employed herein, such as "first" and "second", etc., are for descriptive purposes and should not be construed as limiting. Also, the use of "including" and "comprising" as well as other like terms is used herein is meant for the purpose of interpretation of the description and claims to include not only stated components but also other components not specifically listed or implied to be present in the compositions, methods, articles, or apparatuses. Further, it is to be understood that the use of relational terms, if any, such as "first", "second", and the like are used solely to distinguish one from another entity or action without necessarily implying a relationship or order between these entities or actions. Also, the use of the terms "including", "comprising", or "having" and variations thereof, are meant to encompass the inclusion of other integers, not exclusively those listed.

[0173] The above describes in detail the image change-oriented pedestrian re-identification method provided by the present application, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A pedestrian re-identification method for image changes, characterized in that: The method comprises: S01: Use ResNet50 as the backbone network to build a pedestrian re-identification model, and the pedestrian re-identification model also incorporates a multi-granularity attention mechanism; S02: training the person re-identification model by combining the classification loss function, the triplet loss function, and the component decorrelation loss function to learn discriminative local pedestrian features; S03: Execute the pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian recognition result.

2. The method for person re-identification oriented to image changes according to claim 1, characterized in that: The steps of constructing the person re-identification model in step S01 specifically include: S011: extracting pedestrian convolution features from the backbone network; S012: Convert the pedestrian convolution features into query features, key features and value features, and calculate semantic relevance and spatial similarity; S013: Calculating a value space feature weighted value using the semantic relevance and the first spatial similarity, and fusing the value space feature weighted value with the value space feature to obtain a first pedestrian feature; S014: using a plurality of different downsamplers to downsample the pedestrian convolution feature into a second pedestrian feature and input it into a semantic alignment layer and an upsampling layer, and then restoring the second pedestrian feature to its original size to obtain a third pedestrian feature; S015: Fuse all third-party pedestrian features to learn a low-dimensional pedestrian feature, and use multi-granularity attention to perform weighted summation on the third-party pedestrian features to obtain the fused pedestrian feature; S016: Evenly divide the fused pedestrian features into horizontal stripe convolution features and input them into the global average pooling layer to obtain local feature vectors; S017: Convert the local feature vector into a query feature vector through a fully connected layer, convert the fused pedestrian features into key features and value features, and calculate the semantic relevance and spatial relevance; S018: The attention weighted values ​​of the value features at different positions are fused to obtain the pedestrian component features.

3. The person re-identification algorithm for image changes according to claim 2, characterized in that: Step S012 specifically includes: using different 1×1 convolution layers to convert the pedestrian convolution features into the query features, key features and value features, and calculating the semantic relevance; given the query space features and key space features, using a two-dimensional Gaussian kernel to calculate the spatial similarity, such as the formula: in, represents the query space feature of the i-th position, represents the key space feature at the jth position, p(·) represents the position coordinate of the spatial feature, σ represents the standard deviation, and ||·||2 represents the two-norm.

4. The person re-identification algorithm for image changes according to claim 2, characterized in that: In steps S013-S015, the first pedestrian feature, the second pedestrian feature, the third pedestrian feature and the fused pedestrian feature are calculated as follows: Among them, softmax(·) represents the Softmax function. Since the Softmax function is very sensitive to large or small inputs (easy to map to 1 or 0), the factor To zoom, F g represents the pedestrian features of the g-th granularity, N g The number of granularities representing pedestrian features, represents the semantic alignment function, upsample(·, H, W) represents the upsampling function that upsamples the input features to the spatial size H×W, a g Represents the attention value of the g-th granularity.

5. The method for person re-identification oriented to image changes according to claim 2, characterized in that: Step S017 specifically includes: using a fully connected layer to convert the local feature vector into a query feature vector; using different 1×1 convolutional layers to convert the pedestrian convolution features into key features and value features; using the inner product of the query feature and the key feature to model the semantic relevance; given the query spatial feature and the key spatial feature, using a piecewise Gaussian kernel function to measure the spatial relevance; when the key feature and the query feature are in the same area, setting the spatial relevance to 1; when the key feature and the query feature are in different areas, using spatial relevance helps to eliminate the interference of edge position features. The semantic relevance and spatial relevance are as follows: Among them, q l represents the query feature, K j represents the key feature, p h (·) indicates the height coordinate of the key feature, Indicates the upper bound of the height coordinates of the area where the query feature is located. Indicates the lower bound of the height coordinates of the area where the query feature is located.

6. The method for person re-identification oriented to image changes according to claim 2, characterized in that: Step S018 specifically includes: multiplying the semantic relevance and spatial relevance to obtain the attention weighted values ​​of the value features at different positions; fusing the value features at different positions according to the obtained weighted values; and further obtaining an iterative update method for pedestrian component features, wherein the key features and value features are shared during the iterative update method, as shown in the formula: in, Represents the component features after the nth iteration update; represents the query features after the nth iteration update, The initialization value is expressed as ψ (n) (·) represents the fully connected layer mapping function used for the nth iteration update, which converts the part features into query features.

7. The method for person re-identification oriented to image changes according to claim 1, characterized in that: The component decorrelation loss function in step S02 specifically includes: reducing the correlation between different component features so that the cosine similarity between the component features is less than the similarity interval; and ensuring that each attention mask focuses on different areas of a given pedestrian image.

8. A pedestrian re-identification device for image changes, characterized in that: The device comprises: The global semantic alignment module is used for global semantic alignment, converting pedestrian features into query features, key features, and value features, and modeling the global semantic correlation of human body parts; The two-dimensional Gaussian kernel module is used to calculate the spatial similarity using a two-dimensional Gaussian kernel, and the obtained weighted values ​​are used to fuse the spatial features of values ​​at different positions to obtain the corrected pedestrian features; Feature fusion module, used to effectively fuse multi-granularity global semantic information to improve the generalization ability of the pedestrian re-identification model; A judgment module is used to query the information of the local human body part represented by the query feature, calculate the semantic relevance between all query features and key features, and judge whether the key feature at the current position belongs to the human body part represented by the query feature; A positioning module is used to accurately measure the contribution of the value feature to the component feature, and define the attention weight value of the value feature at different positions as the product of the semantic relevance and the spatial relevance; The attention focusing module is used to make the cosine similarity between component features smaller than the similarity interval based on the component decorrelation loss function, thereby reducing the correlation between different component features; The model training module is used to train the pedestrian re-identification model by combining the classification loss function, the triplet loss function and the component decorrelation loss function to learn the discriminative local pedestrian features; The pedestrian re-identification module is used to perform the pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian recognition result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images