Pedestrian re-identification method and system based on multi-state perception consistency constraint and enhancement

Abstract

The invention discloses a multi-state perception consistency constraint and enhanced pedestrian re-identification method and system, and the method comprises the steps: designing a multi-state perception consistency constraint model, enabling a backbone network to take an original pedestrian image, an overturned image and a zoomed image as the input data of three states, and obtaining three single-channel feature maps; a perception consistency constraint module coordinates the consistency of the feature maps of the three states in spatial positions and local details; the block attention module applies a channel attention mechanism to each part to obtain attention-enhanced features; designing a loss training strategy; the loss function comprises label smooth cross entropy loss and triple loss, and constraining each state feature; and the parameters are optimized. According to the method, the network is guided to extract consistent features in different states, and the foreground and background feature distinguishing capability is enhanced. The model is forced to learn consistent feature representation in different states through the module, and the perception consistency of key parts of pedestrians in different states is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and relates to a pedestrian re-identification method and system with multi-state perception consistency constraints and enhancements. Background Art

[0002] In application scenarios such as smart monitoring and smart security, it is often necessary to identify and track pedestrians in surveillance videos. Pedestrian re-identification technology can more conveniently process a large number of complex pedestrian images across cameras by using artificial intelligence, thereby reducing the consumption of human resources. Unlike face recognition, which requires detailed facial features, pedestrian re-identification focuses on capturing the overall appearance features of pedestrians to achieve pedestrian recognition across cameras and scenes.

[0003] Due to the differences between different camera devices and the diversity of pedestrian appearance, pedestrian re-identification also faces many technical challenges. The different distances between pedestrians and the camera will result in different resolutions of pedestrian images, and the different orientations of the camera will result in different perspectives of the captured pedestrian images. In addition, due to changes in pedestrian postures, the same pedestrian may appear very different in different images, and even the influence of the background and the occlusion of objects may make it impossible for the camera to capture a complete pedestrian image. These factors bring many difficulties and challenges to pedestrian re-identification.

[0004] Traditional deep learning methods extract global features from pedestrian images and use them to identify pedestrians, but the results are often not ideal. These methods are sensitive to factors such as occlusion, viewpoint changes, illumination and scale changes, resulting in reduced re-identification accuracy; on the other hand, the method of extracting global features may ignore the key details of non-salient areas in the image, which are often important for distinguishing different pedestrians. In order to fully express the semantic information and spatial relationships in the image, many studies use local features to improve the accuracy of pedestrian re-identification technology. Commonly used local feature-based methods include horizontal block-based, pose estimation-based, and attention-based methods, but these methods are not robust to viewpoint changes and pose changes or increase a lot of computational overhead. Recognizing the complementary advantages of global and local features, researchers are increasingly paying attention to methods that fuse these two types of features to obtain more robust pedestrian representations.

[0005] In 2022, Gong et al. proposed a LAG-Net, which constructed a multi-granularity network by generating regional interest maps and adopting local attention blocks with high semantic relevance to effectively capture the local features of various parts of the human body. In 2022, the method proposed by Chen et al. introduced two branches, namely the global attention branch and the local attention branch, to guide the network to focus on global pedestrian features and local pedestrian features respectively. In 2023, Li et al. proposed a method based on generative adversarial networks, combining global and local features, extracting and enhancing key local features in pedestrian images through an adaptive CAM-enhanced local appearance encoder, and using pedestrian segmentation technology to separate the foreground and background. In 2020, Li et al. effectively combined global and local features through multi-scale spatial transformation and nonlinear activation, and used a self-attention mechanism to evaluate the importance of different human regions.

[0006] Although these methods have made progress in dealing with pose changes and scale changes in pedestrian ReID, these approaches that combine extracting global features with extracting local features only consider feature extraction from a single perspective, ignoring the rich pedestrian features in different perspectives, and limiting the model's exploration of the correlation between different perspectives. Summary of the invention

[0007] The purpose of the present invention is to solve the problems that the existing technology is difficult to process massive image data in actual scenes, and faces the interference of background clutter on feature extraction and inaccurate recognition caused by feature misalignment, and to provide a pedestrian re-identification method and system with multi-state perception consistency constraints and enhancement.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: A multi-state perception consistency constraint and enhanced pedestrian re-identification method includes the following steps: A multi-state perceptual consistency constraint model is designed, which includes a multi-state fusion backbone network, a perceptual consistency constraint module and a block attention module; the backbone network uses the original pedestrian image, the flipped image and the scaled image as input data of the three states, extracts the features of the three states, and obtains three single-channel feature maps; the perceptual consistency constraint module includes inter-state consistency constraints and intra-state consistency constraints, which are used to coordinate the consistency of the feature maps of the three states in spatial position and local details; the block attention module divides the input feature map into multiple parts along the height dimension, applies the channel attention mechanism to each part, and obtains the features enhanced by attention; Design a loss training strategy, including setting a separate classifier for each state, calculating the loss of each state and averaging to optimize the entire network; loss functions include label smoothed cross entropy loss and triple loss, which are used to constrain the features of each state; The network model structure parameters are optimized and tested on multiple person re-identification datasets to obtain the best model performance.

[0009] The multi-state fusion backbone network uses resnet-50 as the backbone network to transform the original pedestrian image , flipped image And the scaled image As input data of three states respectively, are the height, width and number of channels of the original pedestrian image, is the height and width of the scaled pedestrian image; after feature extraction, the feature maps of the three states are mean fused in the channel dimension to generate three single-channel feature maps for subsequent pedestrian image recognition processing.

[0010] The perceptual consistency constraints are specifically divided into inter-state consistency constraints and intra-state consistency constraints: Intra-state consistency constraint: Calculate the mean square error loss MSE of the feature map averaged in each state channel dimension and the corresponding attention map, and average the MSE losses of the three states to ensure the consistency of the feature maps of each state in the global features and reduce feature offset; the loss calculation formula for the intra-state consistency constraint is: (1) in m represents the number of states, n represents the number of elements in the graph, Indicates i The attention map corresponding to the state j elements, Indicates i The channel dimension corresponding to the state is averaged in the feature map j elements; Consistency constraints between states: Calculate the original image attention map Attention map of flipped image , and the original image attention map Attention map of the scaled image The mean square error loss between them is calculated and these losses are minimized to maintain the consistency of the attention maps in different states; the loss calculation formula for the consistency constraint between states is: (2) in n represents the number of elements in the graph, is the first i elements, is the firsti elements, is the first i elements; After applying the classifier to the feature map, a single-channel map in which each channel represents the pedestrian prediction score is obtained. The single-channel map of the corresponding channel is selected according to the pedestrian label for normalization to generate an attention map associated with the corresponding state.

[0011] The block attention module divides the input feature map into several parts along the height dimension, each part corresponds to a different body part of the pedestrian image; given a feature map , where C is the number of channels, H is the height, and W is the width; the feature is divided into N parts in the height dimension, and each part is represented as ; Apply channel attention branches to each part, including: Perform global maximum pooling and global average pooling on each part to obtain two pooled features; Map each pooled feature to a higher dimension through a linear layer, and concatenate the mapped features in the channel dimension; Apply 1*1 convolution operation to the concatenated features to calculate the weights, and process them through activation function to obtain the channel attention weights of this part; (3) in, and They are k The features of the global average pooling and the features after the global maximum pooling, and They are k The parameters of the feature fully connected layer after global average pooling and the parameters of the feature fully connected layer after global maximum pooling, Operations represent cascade operations in the channel dimension. Represents a 1*1 convolution operation, ReLU activation function. Each feature And the corresponding channel attention weight Multiply and add to the original feature Add together to achieve adaptive weighting of each channel feature; Reconstruct each partial feature weighted by attention in the height dimension to obtain the feature enhanced by attention, which is expressed as: (4) in, Indicates K Block part feature map, Indicates K The channel attention weights corresponding to the partial feature maps are Represents a splicing operation in the height dimension.

[0012] The loss training strategy is specifically: Classifier design: During the training phase, a separate classifier with no shared parameters is set for each state of the model, the loss of each classifier is calculated, and the losses of the three classifiers are averaged to optimize the entire network; Loss function construction, for the characteristics of each state, the loss function used for training consists of two parts, namely the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability and the triple loss, which is used to constrain the distance difference between the same identity sample pairs and different identity sample pairs; the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability is as follows: (5) in N Indicates the number of pedestrian images in each mini-batch, Indicates i The true label distribution after smoothing the labels corresponding to pedestrians can be expressed as formula (6): The prediction for the network is i The probability of a pedestrian; (6) in, K represents the total number of pedestrian identities, y represents the true pedestrian label, is a very small hyperparameter; (7) The triplet loss is shown as follows: in is the feature extraction function, They are the feature maps of the extracted anchor samples, positive samples, and negative samples, respectively. Indicates the calculation of the two norms, that is, the calculation of the distance between two feature maps. is a hyperparameter used to control the distance difference between pairs of samples with the same identity and pairs of samples with different identities; The final loss function is obtained by summing the loss functions of the two branches, as shown below: (8) in, is the joint loss calculated from the feature map of the original image, is the joint loss calculated from the feature map of the flipped image, is the joint loss calculated on the feature map of the scaled image; The total loss is calculated by combining the different feature details extracted from the three states and averaging the losses of the three states in formula (8) to obtain the loss value: (9) in, It is i The combined loss of the states; The total loss is formalized as: (10) in, is the mean of the joint losses of the three states, It is used to adjust the balance between the basic loss and the constraints. is the mean of the consistency constraint losses in the three states, It is the sum of the consistency constraint losses after scale transformation and the original state.

[0013] The optimization of the network model structure parameters is specifically as follows: Select the hyperparameters to be optimized, including the regularization parameters and the number of blocks N; On at least three different person re-identification datasets, according to the regularization parameter In order from large to small, the model is trained and its performance is tested; At each regularization parameter value, fix the Value, and further train the model and test its performance in the order of the number of blocks N from small to large; Record the model performance under each parameter combination and select the parameter combination with the best performance as the current regularization parameter The optimal number of blocks N under all regularization parameters is found. The regularization parameter that makes the model performance optimal and the corresponding number of blocks N; The optimal regularization parameter is And the optimal number of blocks N are used as the final optimization parameters of the network model.

[0014] A multi-state perception consistency constraint and enhanced pedestrian re-identification system, including the following modules: The state fusion backbone network is used to receive the original pedestrian image, the flipped image, and the scaled image as input data of three states, and extract the features of each state to generate three single-channel feature maps; A perceptual consistency constraint module, including inter-state consistency constraints and intra-state consistency constraints, is used to coordinate the consistency of the feature maps of the three states in terms of spatial position and local details to ensure that the features extracted in different states are consistent and robust; A block attention module is used to split the input feature map into multiple parts along the height dimension, and apply a channel attention mechanism to each part to obtain attention-enhanced features, thereby improving the accuracy and discrimination ability of feature representation; A loss training strategy includes setting a classifier for each state separately, calculating the loss of each state and averaging it to optimize the entire network; the loss function includes label smoothed cross entropy loss and triple loss, which are used to constrain the features of each state and improve the classification performance and feature discrimination ability of the model; The optimization module is used to optimize the structural parameters of the network model and obtain the best model performance by training and testing on multiple person re-identification datasets.

[0015] It also includes a preprocessing module, which is used to preprocess the input original pedestrian image, including image denoising, grayscale and size adjustment, to ensure that the quality of the input image meets the requirements of the subsequent processing module.

[0016] A device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method as described in any one of the preceding items when executing the computer program.

[0017] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the preceding items.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The multi-state perceptual consistency constraint and enhanced pedestrian re-identification method in the present invention integrates a multi-state fusion backbone network, a perceptual consistency constraint module and a block attention module, thereby significantly improving the recognizability and consistency of features. This method not only utilizes the advantages of CNN in capturing local information of the image, but also combines the powerful capture ability of VisionTransformer for global contextual information of the image, thereby realizing the effective interaction of heterogeneous network features. Through the perceptual consistency constraint module, the model can learn consistent feature representations in different states, ensuring the consistency of perception of key parts of pedestrians. At the same time, the application of the block attention module further enhances the model's ability to distinguish foreground and background features, and improves the accuracy and discrimination of feature representation. During the training process, the present invention adopts a reasonable loss training strategy, including label smoothing cross entropy loss and triple loss, to fully constrain the features of each state and optimize the entire network. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a diagram of a pedestrian re-identification network model with multi-state perception consistency constraints and enhancements according to the present invention; Figure 2 This is a block attention module diagram of the present invention; Figure 3 This is a visualization of the re-identification results of the example graph of the present invention. (a) represents the ranking of the baseline network recognition results, and (b) represents the recognition results of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0024] A multi-state perception consistency constraint and enhanced pedestrian re-identification method in the present invention comprises the following steps: S1, design a multi-state perceptual consistency constraint model, which includes a multi-state fusion backbone network, a perceptual consistency constraint module and a block attention module.

[0025] The backbone network uses the original pedestrian image, the flipped image, and the scaled image as input data of three states, extracts the features of the three states, and obtains three single-channel feature maps; the multi-state fusion backbone network uses resnet-50 as the backbone network, and converts the original pedestrian image , flipped image And the scaled image As input data of three states respectively, are the height, width and number of channels of the original pedestrian image, is the height and width of the scaled pedestrian image; after feature extraction, the feature maps of the three states are mean fused in the channel dimension to generate three single-channel feature maps for subsequent pedestrian image recognition processing.

[0026] The perceptual consistency constraint module includes inter-state consistency constraints and intra-state consistency constraints, which are used to coordinate the consistency of the feature maps of the three states in terms of spatial position and local details; the perceptual consistency constraints are specifically divided into inter-state consistency constraints and intra-state consistency constraints: Intra-state consistency constraint: Calculate the mean square error loss (MSE) of the feature map averaged in each state channel dimension and the corresponding attention map, and average the MSE losses of the three states to ensure the consistency of the feature maps of each state in the global features and reduce feature offset; the loss calculation formula for the intra-state consistency constraint is: (1) in m represents the number of states, n represents the number of elements in the graph, Indicates i The attention map corresponding to the state j elements, Indicates i The channel dimension corresponding to the state is averaged in the feature map j elements; Consistency constraints between states: Calculate the original image attention map Attention map of flipped image , and the original image attention map Attention map of the scaled image The mean square error loss between them is calculated and these losses are minimized to maintain the consistency of the attention maps in different states; the loss calculation formula for the consistency constraint between states is: (2) in n represents the number of elements in the graph, is the first i elements, is the first i elements, is the first i elements; After applying the classifier to the feature map, a single-channel map in which each channel represents the pedestrian prediction score is obtained. The single-channel map of the corresponding channel is selected according to the pedestrian label for normalization to generate an attention map associated with the corresponding state.

[0027] The block attention module divides the input feature map into multiple parts along the height dimension, applies the channel attention mechanism to each part, and obtains the features enhanced by attention. The block attention module divides the input feature map into several parts along the height dimension, each part corresponds to a different body part of the pedestrian image; given a feature map , where C is the number of channels, H is the height, and W is the width; the feature is divided into N parts in the height dimension, and each part is represented as ; Apply channel attention branches to each part, including: Perform global maximum pooling and global average pooling on each part to obtain two pooled features; Map each pooled feature to a higher dimension through a linear layer, and concatenate the mapped features in the channel dimension; Apply 1*1 convolution operation to the concatenated features to calculate the weights, and process them through activation function to obtain the channel attention weights of this part; (3) in, and They are k The features of the global average pooling and the features after the global maximum pooling, and They are kThe parameters of the feature fully connected layer after global average pooling and the parameters of the feature fully connected layer after global maximum pooling, Operations represent cascade operations in the channel dimension. Represents a 1*1 convolution operation, ReLU activation function. Each feature And the corresponding channel attention weight Multiply and add to the original feature Add together to achieve adaptive weighting of each channel feature; Reconstruct each partial feature weighted by attention in the height dimension to obtain the feature enhanced by attention, which is expressed as: (4) in, Indicates K Block part feature map, Indicates K The channel attention weights corresponding to the partial feature maps are Represents a splicing operation in the height dimension.

[0028] S2, design a loss training strategy, including setting a separate classifier for each state, calculating the loss of each state and averaging to optimize the entire network; the loss function includes label smoothed cross entropy loss and triple loss, which are used to constrain the features of each state.

[0029] The loss training strategy is as follows: Classifier design: During the training phase, a separate classifier with no shared parameters is set for each state of the model, the loss of each classifier is calculated, and the losses of the three classifiers are averaged to optimize the entire network; Loss function construction, for the characteristics of each state, the loss function used for training consists of two parts, namely the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability and the triple loss, which is used to constrain the distance difference between the same identity sample pairs and different identity sample pairs; the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability is as follows: (5) in N Indicates the number of pedestrian images in each mini-batch, Indicates i The true label distribution after smoothing the labels corresponding to pedestrians can be expressed as formula (6): The prediction for the network is i The probability of a pedestrian; (6) in, K represents the total number of pedestrian identities, y represents the true pedestrian label, is a very small hyperparameter; The triplet loss is shown as follows: (7) in is the feature extraction function, They are the feature maps of the extracted anchor samples, positive samples, and negative samples, respectively. Indicates the calculation of the two norms, that is, the calculation of the distance between two feature maps. is a hyperparameter used to control the distance difference between pairs of samples with the same identity and pairs of samples with different identities; The final loss function is obtained by summing the loss functions of the two branches, as shown below: (8) in, is the joint loss calculated from the feature map of the original image, is the joint loss calculated from the feature map of the flipped image, is the joint loss calculated on the feature map of the scaled image; The total loss is calculated by combining the different feature details extracted from the three states and averaging the losses of the three states in formula (8) to obtain the loss value: (9) in, It is i The combined loss of the states; The total loss is formalized as: (10) in, is the mean of the joint losses of the three states, It is used to adjust the balance between the basic loss and the constraints. is the mean of the consistency constraint losses in the three states, It is the sum of the consistency constraint losses after scale transformation and the original state.

[0030] S3, optimizes the network model structure parameters and tests them on multiple person re-identification datasets to obtain the best model performance.

[0031] Specifically: Select the hyperparameters to be optimized, including the regularization parameters and the number of blocks N; On at least three different person re-identification datasets, according to the regularization parameter In order from large to small, the model is trained and its performance is tested; At each regularization parameter value, fix the Value, and further train the model and test its performance in the order of the number of blocks N from small to large; Record the model performance under each parameter combination and select the parameter combination with the best performance as the current regularization parameter The optimal number of blocks N under all regularization parameters is found. The regularization parameter that makes the model performance optimal and the corresponding number of blocks N; The optimal regularization parameter is And the optimal number of blocks N are used as the final optimization parameters of the network model. Example

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings: Step 1: Design of multi-state-aware consistency constraint model The designed CNN-based multi-state perception consistency constraint model pedestrian re-identification network model is as follows Figure 1 As shown in the figure, the model consists of a multi-state fusion backbone network, perceptual consistency constraints, and a block attention module.

[0033] The multi-state fusion backbone network uses resnet-50 as the backbone network to transform the original pedestrian image , flipped image And the scaled image As input data of three states respectively, are the height, width and number of channels of the original pedestrian image, is the height and width of the scaled pedestrian image. After the feature extractor, three features are obtained, which capture the information contained in the pedestrian image in different states. The three feature maps are averaged in the first dimension (channel dimension) to obtain three single-channel feature maps.

[0034] The perceptual consistency constraint coordinates the consistency of the feature maps of the three states in terms of spatial position and local details. It is divided into two parts: inter-state consistency constraint and intra-state consistency constraint. Specifically, after the feature map passes through the classifier, each channel represents the predicted score of the pedestrian. The single channel map of the corresponding channel is selected according to the pedestrian label, and then normalized to obtain the attention map of the corresponding state. The intra-state consistency constraint module calculates the feature map averaged in the channel dimension of each state. And the corresponding attention map The mean square error loss of , makes the extracted feature map consistent in global features and reduces feature offset. Then the MSE losses of these three states are averaged, which can avoid a certain state (such as inversion or scaling) from having too much impact on the total loss. Instead, the consistency of the features of each state is given equal attention and optimization, so that the goal of model optimization is consistent, and the features learned by the network are globally consistent in different states. Therefore, formula (1) can be obtained.

[0035] The consistency constraint between states calculates the mean square error loss of the attention map under different states. Attention map with flipped image , the attention map of the original image Attention map with scaled image Calculate the mean square error loss and minimize it. The formula is as shown in formula (2).

[0036] The block attention module focuses on extracting the body part information contained in the pedestrian image in the vertical direction (height dimension). The input feature map is divided into multiple parts along the height dimension, which correspond to different body parts of the pedestrian image (such as head, torso, legs, etc.). For a given feature map , where C is the number of channels, H is the height, and W is the width. In order to learn the features of different height regions and better capture the detailed features of pedestrians at different scales, we divide the features into N parts in the height dimension, and each part can be expressed as . For each part, it is fed into the channel attention branch. Specifically, the features of each part are mapped to a higher dimension through a linear layer after global maximum pooling and global average pooling, and they are concatenated in the channel dimension. The concatenated features are weighted using 1*1 convolution and passed through an activation function to introduce nonlinearity. The weight obtained through this operation can be expressed as formula (3).

[0037] After obtaining the attention weight, each feature With corresponding weight Multiply and then This method realizes the adaptive weighting of each channel feature and dynamically adjusts the attention to the feature. Each processed feature block is reassembled in the height dimension to obtain the feature enhanced by attention, which can be expressed as formula (4).

[0038] Step 2: Loss training strategy design The three states in the model have different perceived positions of pedestrian key points. In order to ensure the network training effect, the present invention constrains the three states respectively.

[0039] (1) Classifier design During the training phase, a separate classifier is set for each state of the model, that is, the classifier parameters are not shared. The losses calculated by the three classifiers are averaged and then the entire network is optimized.

[0040] (2) Loss Function For the features of each state, the training loss function consists of two parts, as shown in formulas (5) and (7).

[0041] The final loss function is obtained by summing the loss functions of the two branches, as shown in formula (8).

[0042] In order to combine the different feature details extracted from the three states, reduce the impact of noise, and more comprehensively consider the differences and similarities of pedestrian features under different transformations, the loss value of the three states in formula (8) is averaged to obtain the loss value, as shown in formula (9).

[0043] In the training phase, a multi-state backbone network is used to extract discriminative features, and the focus consistency constraint and the block attention module are used to enhance the robustness of the model. Accordingly, the total loss can be formalized as Equation (10).

[0044] Step 3: Network model structure parameter optimization The effects of the hyperparameter λ and the number of blocks N on the model performance were compared and tested on three person re-identification datasets. The optimization was carried out in the order of decreasing λ from large to small and increasing the number of blocks N from small to large. After each parameter was optimized, the optimization result was fixed and the next parameter optimization experiment was started.

[0045] The multi-state perception consistency constraint and enhanced pedestrian re-identification method proposed in this invention learns more identifiable features, guides the network to extract consistent features in different states, and enhances the ability to distinguish foreground and background features. The module forces the model to learn consistent feature representations in different states, ensuring the consistency of perception of key parts of pedestrians in different states. Figure 3 An example of an image re-identified by the present invention is shown in Figure 1. From the recognition results, it can be seen that the number of erroneous images retrieved by the baseline network based on ResNet50 and the baseline network based on Vision Transformer is greater than that of the model proposed by the present invention. In addition, among the images retrieved by the present invention, the accuracy of the first few images (especially rank-5) is higher than that of the baseline network based on ResNet50 and the baseline network based on Vision Transformer. The average accuracy rate reaches 66.6%, which shows the effectiveness of the pedestrian re-identification method of the heterogeneous network feature interaction model proposed by the present invention.

[0046] The pedestrian re-identification method of the heterogeneous network feature interaction model provided by the present invention fully utilizes the ability of CNN to capture local information of the image and the ability of Vision Transformer to capture global contextual information in the image, and establishes an effective implicit relationship between the global information and local significant features of the pedestrian body area, so that more correct people can be identified. Figure 3 An example of an image re-identified by the present invention is shown in Figure 1. From the recognition results, it can be seen that the number of wrong images retrieved by the baseline network is greater than that of the model proposed by the present invention. In addition, among the images retrieved by the present invention, the accuracy of the first few images (especially rank-5) is higher than that of the baseline network. This shows the effectiveness of the multi-state perception consistency constraint and enhanced pedestrian re-identification method proposed by the present invention.

[0047] An embodiment of the present invention is a multi-state perception consistency constraint and enhanced pedestrian re-identification system, comprising the following modules: The state fusion backbone network is used to receive the original pedestrian image, the flipped image, and the scaled image as input data of three states, and extract the features of each state to generate three single-channel feature maps.

[0048] The perceptual consistency constraint module includes inter-state consistency constraints and intra-state consistency constraints, which are used to coordinate the consistency of the feature maps of the three states in terms of spatial position and local details to ensure that the features extracted in different states are consistent and robust.

[0049] The block attention module is used to divide the input feature map into multiple parts along the height dimension, and apply the channel attention mechanism to each part to obtain attention-enhanced features, thereby improving the accuracy and discrimination ability of feature representation.

[0050] The loss training strategy includes setting a classifier for each state separately, calculating the loss of each state and averaging it to optimize the entire network; the loss function includes label smoothed cross entropy loss and triplet loss, which are used to constrain the features of each state and improve the classification performance and feature discrimination ability of the model.

[0051] The optimization module is used to optimize the structural parameters of the network model and obtain the best model performance by training and testing on multiple person re-identification datasets.

[0052] Preprocessing module,The preprocessing module is used to preprocess the input original pedestrian image, including image denoising, grayscale and size adjustment, to ensure that the quality of the input image meets the requirements of the subsequent processing module.

[0053] An embodiment of the present invention provides a terminal device. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0054] The computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to accomplish the present invention.

[0055] The device / terminal equipment may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The device / terminal equipment may include, but is not limited to, a processor and a memory.

[0056] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0057] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the apparatus / terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0058] If the module / unit integrated in the device / terminal equipment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0059] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-state perception consistency constraint and enhancement pedestrian re-identification method, characterized in that: The following steps are involved: Design a multi-state perceptual consistency constraint model, the model includes a multi-state fusion backbone network, a perceptual consistency constraint module and a block attention module; The backbone network uses the original pedestrian image, the flipped image, and the scaled image as input data of three states, extracts the features of the three states, and obtains three single-channel feature maps; the perceptual consistency constraint module includes inter-state consistency constraints and intra-state consistency constraints, which are used to coordinate the consistency of the feature maps of the three states in spatial position and local details; the block attention module divides the input feature map into multiple parts along the height dimension, applies the channel attention mechanism to each part, and obtains the features enhanced by attention; Design a loss training strategy, including setting up a separate classifier for each state, calculating the loss for each state and averaging it to optimize the entire network; The loss functions include label smoothed cross entropy loss and triplet loss, which are used to constrain the features of each state; The network model structure parameters are optimized and tested on multiple person re-identification datasets to obtain the best model performance.

2. The method for pedestrian re-identification with multi-state perception consistency constraint and enhancement as claimed in claim 1, characterized in that: The multi-state fusion backbone network uses resnet-50 as the backbone network to transform the original pedestrian image , flipped image And the scaled image As input data of three states respectively, are the height, width and number of channels of the original pedestrian image, is the height and width of the scaled pedestrian image; after feature extraction, the feature maps of the three states are mean fused in the channel dimension to generate three single-channel feature maps for subsequent pedestrian image recognition processing.

3. The method for pedestrian re-identification with multi-state perception consistency constraint and enhancement as claimed in claim 1, characterized in that: The perceptual consistency constraints are specifically divided into inter-state consistency constraints and intra-state consistency constraints: Intra-state consistency constraint: Calculate the mean square error loss MSE of the feature map averaged in each state channel dimension and the corresponding attention map, and average the MSE losses of the three states to ensure the consistency of the feature maps of each state in the global features and reduce feature offset; the loss calculation formula for the intra-state consistency constraint is: （1） in m represents the number of states, n represents the number of elements in the graph, Indicates i The attention map corresponding to the state j elements, Indicates i The channel dimension corresponding to the state is averaged in the feature map j elements; Consistency constraints between states: Calculate the original image attention map Attention map of flipped image , and the original image attention map Attention map of the image after scaling The mean square error loss between them is calculated and these losses are minimized to maintain the consistency of the attention maps in different states; the loss calculation formula for the consistency constraint between states is: （2） in n represents the number of elements in the graph, is the first i elements, is the first i elements, is the first i elements; After applying the classifier to the feature map, a single-channel map in which each channel represents the pedestrian prediction score is obtained. The single-channel map of the corresponding channel is selected according to the pedestrian label for normalization to generate an attention map associated with the corresponding state.

4. The method for pedestrian re-identification with multi-state perception consistency constraint and enhancement as claimed in claim 1, characterized in that: The block attention module divides the input feature map into several parts along the height dimension, each part corresponds to a different body part of the pedestrian image; given a feature map , where C is the number of channels, H is the height, and W is the width; the feature is divided into N parts in the height dimension, and each part is represented as ; Apply channel attention branches to each part, including: Perform global maximum pooling and global average pooling on each part to obtain two pooled features; Map each pooled feature to a higher dimension through a linear layer, and concatenate the mapped features in the channel dimension; Apply 1*1 convolution operation to the concatenated features to calculate the weights, and process them through activation function to obtain the channel attention weights of this part; （3） in, and They are k The features of the global average pooling and the features after the global maximum pooling, and They are k The parameters of the feature fully connected layer after global average pooling and the parameters of the feature fully connected layer after global maximum pooling, Operations represent cascade operations in the channel dimension. Represents a 1*1 convolution operation, ReLU activation function. Each feature and The corresponding channel attention weight is multiplied and added to the original feature Add together to achieve adaptive weighting of each channel feature; Reconstruct each partial feature weighted by attention in the height dimension to obtain the feature enhanced by attention, which is expressed as: （4） in, Indicates K Block part feature map, Indicates K The channel attention weights corresponding to the partial feature maps are Represents a splicing operation in the height dimension.

5. The method for pedestrian re-identification with multi-state perception consistency constraint and enhancement as claimed in claim 1, characterized in that: The loss training strategy is specifically: Classifier design: During the training phase, a separate classifier with no shared parameters is set for each state of the model, the loss of each classifier is calculated, and the losses of the three classifiers are averaged to optimize the entire network; Loss function construction, for the characteristics of each state, the training loss function consists of two parts, namely the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability and the triple loss, which is used to constrain the distance difference between the same identity sample pairs and different identity sample pairs; the cross entropy loss between the real label distribution based on label smoothing and the network prediction probability is as follows: （5） in N Indicates the number of pedestrian images in each mini-batch, Indicates i The true label distribution after smoothing the labels corresponding to pedestrians is expressed as formula (6): The prediction for the network is i The probability of a pedestrian; （6） in, K represents the total number of pedestrian identities, y represents the true pedestrian label, is a very small hyperparameter; The triplet loss is shown as follows: （7） in is the feature extraction function, They are the feature maps of the extracted anchor samples, positive samples, and negative samples, respectively. Indicates the calculation of the two norms, that is, the calculation of the distance between two feature maps. is a hyperparameter used to control the distance difference between pairs of samples with the same identity and pairs of samples with different identities; The final loss function is obtained by summing the loss functions of the two branches, as shown below: （8） in, is the joint loss calculated from the feature map of the original image, is the joint loss calculated from the feature map of the flipped image, is the joint loss calculated on the feature map of the scaled image; The total loss is calculated by combining the different feature details extracted from the three states and averaging the losses of the three states in formula (8) to obtain the loss value: （9） in, It is i The combined loss of the states; The total loss is formalized as: （10） in, is the mean of the joint loss of the three states, It is used to adjust the balance between the basic loss and the constraints. is the mean of the consistency constraint losses in the three states, It is the sum of the consistency constraint losses after scale transformation and the original state.

6. The method for pedestrian re-identification with multi-state perception consistency constraint and enhancement as claimed in claim 1, characterized in that: The optimization of the network model structure parameters is specifically as follows: Select the hyperparameters to be optimized, including the regularization parameters and the number of blocks N; On at least three different person re-identification datasets, according to the regularization parameter In order from large to small, the model is trained and its performance is tested; At each regularization parameter value, fix the Value, and further train the model and test its performance in the order of the number of blocks N from small to large; Record the model performance under each parameter combination and select the parameter combination with the best performance as the current regularization parameter The optimal number of blocks N under all regularization parameters is found. The regularization parameter that makes the model performance optimal and the corresponding number of blocks N; The optimal regularization parameter is And the optimal number of blocks N are used as the final optimization parameters of the network model.

7. A multi-state perception consistency constraint and enhanced pedestrian re-identification system, characterized in that: Includes the following modules: The state fusion backbone network is used to receive the original pedestrian image, the flipped image, and the scaled image as input data of three states, and extract the features of each state to generate three single-channel feature maps; A perceptual consistency constraint module, including inter-state consistency constraints and intra-state consistency constraints, is used to coordinate the consistency of the feature maps of the three states in terms of spatial position and local details to ensure that the features extracted in different states are consistent and robust; A block attention module is used to split the input feature map into multiple parts along the height dimension, and apply a channel attention mechanism to each part to obtain attention-enhanced features, thereby improving the accuracy and discrimination ability of feature representation; A loss training strategy includes setting a classifier for each state separately, calculating the loss of each state and averaging it to optimize the entire network; the loss function includes label smoothed cross entropy loss and triple loss, which are used to constrain the features of each state and improve the classification performance and feature discrimination ability of the model; The optimization module is used to optimize the structural parameters of the network model and obtain the best model performance by training and testing on multiple person re-identification datasets.

8. The multi-state perception consistency constraint and enhanced pedestrian re-identification system according to claim 7, characterized in that: It also includes a preprocessing module, which is used to preprocess the input original pedestrian image, including image denoising, grayscale and size adjustment, to ensure that the quality of the input image meets the requirements of the subsequent processing module.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images