Multi-correlation pedestrian attribute identification method

Abstract

The invention relates to a multi-correlation pedestrian attribute identification method. The method comprises the following steps: obtaining and preprocessing a pedestrian picture; constructing a multi-correlation pedestrian attribute recognition model, wherein the multi-correlation pedestrian attribute recognition model comprises a super-resolution structure calibration module, a region correlation causal object sensing module and a space attribute correlation sensing module; the super-resolution structure calibration module comprises a pre-trained VIT visual encoder and a global super-resolution module GSR, and the output of the global super-resolution module GSR is processed by the pre-trained VIT visual encoder to obtain the output # imgabs0 # of the super-resolution structure calibration module; and training a multi-correlation pedestrian attribute recognition model by using a polynomial contrast loss function, and performing pedestrian attribute recognition. According to the method, a super-resolution improvement module, a region correlation module, an attribute correlation module and the like are fully integrated in a unified model framework, and the quality problem and the attribute imbalance problem of pedestrian images are considered by utilizing mutual guidance training of comparative learning.

Description

Technical Field

[0001] The present invention relates to the technical field of processing recording media for identifying graphics, in particular to a method for identifying attributes of pedestrians in images, and more particularly to a method for identifying attributes of pedestrians with multiple correlations. Background Art

[0002] As an important task in the field of computer vision, pedestrian attribute recognition (PAR) aims to accurately infer the attribute information of pedestrians by deeply analyzing and understanding pedestrian images. These attribute information include but are not limited to gender, age, clothing style and behavioral characteristics. This task has broad application prospects in the fields of intelligent monitoring, human-computer interaction and intelligent security, and has promoted the research of other human-related visual tasks, including pedestrian re-identification, pedestrian search and pedestrian detection. With the development of society and the acceleration of urbanization, the management and safety issues of crowded areas are increasingly concerned. Through continuous innovation, pedestrian attribute recognition technology provides a powerful tool to solve these problems.

[0003] Traditional manual monitoring methods are difficult to meet the needs of large-scale and high efficiency, while the pedestrian attribute recognition system based on deep learning can automatically extract rich semantic features, thereby realizing accurate and efficient recognition of pedestrian attributes in complex scenes. The challenges faced by pedestrian attribute recognition mainly stem from the diversity and complexity of pedestrian images. First, pedestrian images may be affected by factors such as shooting conditions, lighting, and background interference, resulting in uneven image quality. Secondly, pedestrians may behave very differently in different scenes, angles, and distances, which increases the difficulty of attribute recognition. In addition, factors such as occlusion and posture changes in the image also pose challenges to accurate recognition.

[0004] The current PAR algorithms mainly focus on CNN-based models, RNN-based models, and Transformer-based models. For example, CN111052126B uses convolutional neural networks to extract features and pooling technology to improve the pedestrian attribute recognition and positioning effect in video surveillance. However, this method may face challenges in accuracy and real-time performance in complex environments such as complex backgrounds, lighting changes, perspective problems, pedestrian overlap and occlusion, and motion blur. CN115700830A proposes a pedestrian attribute recognition method based on pedestrian body structure. By dividing pedestrian features into three parts: head, upper body, and lower body, and combining non-local module enhancement features, the attribute recognition effect is optimized. However, this method relies heavily on the construction and annotation quality of the data set. When the model encounters a data set with low annotation quality or different data distribution, the effect may drop significantly, affecting the generalization ability of the model. CN110222636B proposes a pedestrian attribute recognition method based on background suppression, which extracts mask images of pedestrians and backgrounds, combines a multi-layer feature extraction network and a dual loss function, but the method's reliance on background suppression may reduce recognition performance in complex background environments. When the background is complex, such as there are multiple similar interfering objects or dynamic background elements, the background suppression effect of the model will deteriorate, thereby affecting the overall recognition accuracy. Complex background environments may make it difficult for the model to effectively distinguish the boundary between pedestrians and backgrounds, resulting in misidentification or missed recognition. CN114792426B uses rare attribute recognition and image enhancement, which may deviate from the true pedestrian attribute distribution during the rare attribute recognition and image enhancement process, such as the elements added by the enhancement or the color feature changes unnaturally, thereby introducing noise data. In addition, the model may perform well on the enhanced data, but when the enhanced data is too dependent on certain specific styles or attributes, the model may perform well on the training data, but poorly on the actual data, resulting in overfitting.

[0005] At the same time, the problem of pedestrian attribute imbalance is also a challenge. Pedestrian attribute imbalance refers to the problem of uneven distribution of different attribute labels in the training data set. For example, in the pedestrian attribute data set, the samples with the attribute of "wearing a hat" may be far less than the samples of "not wearing a hat", or the pedestrians wearing dark clothes are much more than those wearing light clothes. Such imbalance will cause the model to be more inclined to recognize those attributes with larger sample sizes, ignore or misjudge the attributes with fewer samples, and thus affect the overall recognition effect. Common solutions include weighting the loss function and using batch normalization before classification. These methods can alleviate attribute imbalance to a certain extent. However, the degree of imbalance of pedestrian attributes in different data sets varies, and a unified weighting strategy is often difficult to achieve ideal results.

[0006] In addition, factors such as low resolution, partial body occlusion, and complex appearance changes are also important problems that must be overcome in pedestrian attribute recognition. Summary of the invention

[0007] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a multi-correlation pedestrian attribute recognition method. The method solves the technical problems existing in the existing pedestrian attribute recognition methods, such as inaccurate multi-correlation attribute recognition, low calculation efficiency, and incomplete feature extraction, and can comprehensively consider the correlation between attributes, improve recognition accuracy, and ensure efficient calculation performance.

[0008] The technical solution adopted by the present invention to solve the technical problem is: A multi-correlation pedestrian attribute recognition method, the recognition method includes the following contents: Get pedestrian images and preprocess them; Constructing a multi-correlated pedestrian attribute recognition model: The multi-correlated pedestrian attribute recognition model includes a super-resolution structure calibration module, a region-related causal object perception module, and a spatial attribute-related perception module; The super-resolution structure calibration module includes a pre-trained VIT visual encoder and a global super-resolution module GSR. The output of the global super-resolution module GSR is processed by the pre-trained VIT visual encoder to obtain the output of the super-resolution structure calibration module. ; The region-related causal object perception module includes a global adaptive average pooling operation, a dimension transposition operation, a dimension expansion operation, a pooling branch, and an attention branch; After global adaptive average pooling, dimension transposition, dimension expansion, and Sigmoid activation function processing, Multiply to obtain intermediate features ; Enter the pooling branch and the attention branch respectively. In the pooling branch, the intermediate features are first The output is obtained after global adaptive average pooling Avgpool, dimension transformation, and batch normalization layer processing. ; In the attention branch, After n times of multisampling, the output is obtained by processing it through the transformer block and calculating the regional correlation score of the attribute. ; Will and Combined with the above, the final output of the regional correlation causal object perception module RCOP module is obtained rcop ; The spatial attribute related perception module includes a hybrid attention block composed of channel attention and spatial attention. The output of the hybrid attention block is , Pass 1 The convolution operation generates the initial attribute score; the initial attribute score is normalized; the normalized score is flattened into a one-dimensional vector, and the arithmetic mean of all elements in the one-dimensional vector is calculated to obtain the basic attribute features. ; At the same time, the softmax function is used to adjust the flattened one-dimensional vector, and then the weighted sum is calculated with the flattened one-dimensional vector to obtain the attribute features after the relevant information is adjusted ; Perform weighted summation of basic attribute features and attribute features adjusted with relevant information to obtain the final attribute prediction ; A polynomial contrast loss function is used to train a multi-correlated pedestrian attribute recognition model, and pedestrian attribute recognition is performed using the trained multi-correlated pedestrian attribute recognition model.

[0009] Furthermore, the global super-resolution module includes two groups of global convolution modules. Each group of global convolution modules firstly processes the input image through two convolution layers. Perform horizontal and vertical convolution operations; then, combine the outputs of the two convolutional layers into a tensor by concatenating them in the channel dimension; next, add the outputs of the two global convolutional modules element-wise to get a fused output ;right Execute the steps to reduce the number of channels. Convolution operation, feature extraction Convolution operation, adjusting the number of channels 1 After the convolution operation, the input image Add together to get the output of the global super-resolution module .

[0010] Furthermore, the total loss function during training is: (18) in, is the self-supervised contrastive learning loss, is an asymmetric polynomial loss; is the weight of the self-supervised contrastive learning loss; is the polynomial contrast loss function; (17), Indicates The self-supervised contrastive learning loss of samples, I represents the total number of samples, Indicates The embedded feature vector of samples, Indicates samples come from the embedded feature vector of another enhanced sample of the same source sample; features and is considered a positive sample pair. represents the temperature parameter, which is used to adjust the sensitivity of the similarity calculation between feature vectors; In addition to The set of all other samples in the batch is called negative samples; Represents negative samples The embedded feature vector of Used with Calculate similarity and form negative sample pairs; (16), in, It is The predicted value of an attribute, It is The true value of the attribute, , , is the focus parameter, is the weight calculated based on asymmetric focusing; Indicates the total number of attribute categories.

[0011] The specific steps of a multi-correlation pedestrian attribute recognition method are: The first step is to preprocess the pedestrian image: Get images of pedestrians and augment the training images by random horizontal flipping, hue and saturation, rotation, translation, cropping, erasing, and adding random Gaussian blur; The second step is super-resolution structure calibration: Before the image is sent to the backbone, it passes through a global super-resolution module GSR to improve the resolution and capture more context information. GSR receives and Parameters, where are the number of output filters, is the kernel size, GSR consists of multiple large separable convolutional layers and a series of subsequent convolutional layers. The large separable convolutional layers capture greater contextual information from the input feature map, and the final residual operation can eliminate the similarity between the upsampled low-resolution image and the converted high-resolution image; The global super-resolution module GSR contains two sets of global convolution modules. Each set of global convolution modules first performs horizontal and vertical convolution operations on the input image through two two-dimensional convolution layers. Then, the outputs of the two convolution layers are integrated into a tensor by splicing in the channel dimension to capture a wider range of local features and contextual information; the final integrated output is used to provide richer image information; next, the outputs of the two global convolution modules are element-wise added to obtain a fused output. , The output of two global convolution modules is processed through a 1×1 convolution The number of channels is compressed, and then the compressed feature map is input into a standard k×k convolution layer. Finally, a 1×1 convolution layer is used to adjust the number of channels of the feature map and compare it with the original input. Perform residual connection to get the final output ; Pre-train the VIT visual encoder on the LUPerson large-scale human dataset, introduce mask image modeling MIM for pre-training, incorporate intuitive human structure priors into pre-training, and mask the image blocks corresponding to human body parts with high priority before guiding the mask sampling process. During pre-training, more attention is paid to human structure information, and the visual encoder of pre-trained VIT is used as the backbone. After backbone processing, the output feature tensor ; The third step is regional related causal object perception: After the global adaptive average pooling operation GAP is processed, the dimension is transposed and the spatial or channel dimension is expanded through the one-dimensional convolution operation. Then, the Sigmoid activation function is applied to generate the channel attention weight, which has an effect on the input feature tensor. Weighted to get the intermediate features , intermediate features The calculation formula for the part that will also be used for contrastive learning is: (4), in, is the sigmoid function; GAP represents the global adaptive average pooling operation; Indicates that the convolution kernel is 1 Convolution operation; It is then divided into two branches: pooling branch and attention branch. Pooling branch: First, the intermediate features Perform global adaptive average pooling Avgpool, then flatten the pooled feature tensor into a one-dimensional representation by linearizing it; finally, transform the flattened features through a linear logic layer (Batch Norm) to generate the module output : (5), in, represents a batch normalization layer; represents a fully connected layer; It is a global adaptive average pooling operation; Attention branch: Perform n multiple samplings to obtain sample-specific region-aware features, which are then sent to the transformer encoder for reweighting to obtain the self-attention tensor. ; Will Multiply the weight matrices of the convolutional layers in each sampling element by element, and sum them up in the spatial dimension to obtain the final pedestrian attribute prediction result : Will and Combined, the final output of the RCOP module is obtained rcop ; Step 4: Spatial attribute related perception SARP module: Spatial attribute-related perception includes channel attention and spatial attention, specifically: For the input feature tensor Perform channel attention processing, the formula is: (9), in, Represents the Sigmoid activation function; represents the activation function used to remove negative values; Represents a 1×1 convolution operation; and Respectively represent the input feature tensor Perform average pooling and maximum pooling operations to extract the global average and maximum values ​​in the input tensor; and Represents the learnable parameters in the convolution operation; is the output of channel attention; In spatial attention, the output of channel attention is used as input, including average pooling and maximum pooling operations, convolutional layers, and Sigmoid activation functions; first, by Perform average pooling and maximum pooling operations to obtain the average and maximum values ​​respectively. Then, concatenate the two pooling results in the channel dimension to obtain a new feature tensor containing the average and maximum information. Finally, process the concatenated features through a convolutional layer to generate an output with spatial attention. ; The whole process is normalized by the Sigmoid activation function to obtain a weight tensor ranging from [0,1]. The calculation formula is: (10), in Represents the Sigmoid activation function; Represents a 1×1 convolution operation; Represents a splicing operation; Represents the averaging operation; It represents the maximum value operation; Represents the learnable parameters of the convolutional layer; Input feature tensor through channel attention Perform channel weighting, perform spatial weighting on the channel-weighted features through spatial attention, and then use the feature tensor weighted by spatial attention as the output of the first half of SARP ; Through a convolution operation, the initial attribute score is obtained, and the initial attribute score is standardized, in which the denominator uses the norm of the weight matrix; then, through the flattening operation, the score is mapped to a one-dimensional vector, and the basic attribute feature is obtained by calculating the average value. ; Then the softmax function is used to adjust the standardized scores and calculate the weighted sum to obtain the attribute features adjusted by relevant information. : The basic attribute characteristics Attribute characteristics adjusted with relevant information Perform weighted summation to obtain the final attribute prediction : (14), in, represents the weight coefficient; Step 5, loss function: The total loss function during training is: (18) in, is the self-supervised contrastive learning loss, is an asymmetric polynomial loss; is the weight of the self-supervised contrastive learning loss; is the polynomial contrast loss function; (17), Indicates The self-supervised contrastive learning loss of samples, I represents the total number of samples, Indicates The embedded feature vector of samples, Indicates samples come from the embedded feature vector of another enhanced sample of the same source sample; features and is considered a positive sample pair. represents the temperature parameter, which is used to adjust the sensitivity of the similarity calculation between feature vectors; In addition to The set of all other samples in the batch is called negative samples; Represents negative samples The embedded feature vector of Used with Calculate similarity and form negative sample pairs; (16), in, It is The predicted value of an attribute, It is The true value of the attribute, , , is the focus parameter, is the weight calculated based on asymmetric focusing; Indicates the total number of attribute categories; Step 6: Network training After VIT is pre-trained on LUPerson, it is then fine-tuned on a pedestrian attribute dataset using the complete network and then trained using the total loss function for pedestrian attribute recognition.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The significant improvements of the present invention are as follows: In order to solve the problems of low quality of pedestrian images, cluttered background and no restrictions under different observation conditions, the present invention adopts a super-resolution structure calibration module. Through the global super-resolution module, it aims to capture more contextual information while restoring the image texture details, where "calibration" refers to the optimization of image resolution and structural information so that the model can more accurately identify and extract key features in pedestrian images while reducing errors caused by low resolution or structural distortion. In terms of processing regional correlation and attribute-related information of pedestrian images, the present invention proposes two key modules (region-related causal object perception module and spatial attribute-related perception module). First, in order to extract regional correlation information of pedestrian images to solve the contextual bias caused by cluttered background, a region-related causal object perception module is introduced to make full use of two attention layers with multi-sampling intervention ( Figure 4 The Transformer Encoder in is a two-layer attention structure) to extract regional related information, thereby optimizing the performance of human attribute recognition. Secondly, in order to capture the different spatial regions occupied by objects of different categories in the image, the present invention proposes a spatial attribute related perception module, which generates category-specific features for each category through spatial attention scores, and combines them with category-independent average pooling features to more effectively capture the attribute related information of the object. In addition, considering that the traditional binary cross entropy loss cannot be personalized for different tasks, resulting in poor performance of different models, and the imbalance between redundant negative samples and rare positive samples will reduce model performance, the present invention proposes polynomial contrast loss Loss supervised training, which aims to improve the adaptability of the model to multiple tasks by personalizing the loss function, and further optimize the model performance by solving the sample imbalance problem.

[0013] The outstanding essential features of the present invention are: 1) The method of the present invention proposes an end-to-end pedestrian attribute recognition framework, which aims to address problems such as poor pedestrian image quality, fully considers regional correlation and attribute-related information, and introduces contrastive learning between two correlation modules (regional correlation causal object perception module and spatial attribute correlation perception module) to guide each other's training. The method of the present invention uses a new loss function that combines asymmetric polynomial loss and contrastive learning loss for supervised training, thereby enhancing the robustness and performance of the model. Here, "end-to-end" means that the entire process of the framework from the original input (low-quality pedestrian image) to the final output (identified pedestrian attributes) is a continuous, automated, and integrated processing process, and no manual intervention is required at each step from input to output, ensuring the automatic flow of data and the continuity of feature extraction and recognition tasks.

[0014] 2) The present invention fully integrates super-resolution enhancement, regional correlation, attribute correlation and other modules in a unified model framework, and uses contrastive learning to guide mutual training. The invention systematically considers the quality problem and attribute imbalance problem of pedestrian images in the design, and integrates them in a unified network architecture, so it is called an end-to-end framework.

[0015] 3) The method of the present invention proposes region-correlated causal object perception, which aims to learn the regional correlation between pedestrian attributes. This module infers the connection between pedestrian attributes by using the causal relationship between the region of interest (ROI) and the object in the image, which can not only effectively reduce noise interference, but also improve the accuracy and robustness of attribute recognition, making the model more generalizable. The operation in the RCOP (Region-correlated Causal Object Perception) module refers to analyzing certain key regions (ROIs) and their relative positions in the pedestrian image through adaptive pooling and multi-head self-attention to infer the association of pedestrian attributes. For example, the model may identify the color of the pedestrian's clothing through the upper body area and infer the clothing category through the lower body area. The causal relationship between ROI and other regions enables the model to identify the correlation inside and outside these regions, and pass the attributes associated with specific regions to different modules of the model. By analyzing the causal relationship between regions in the pedestrian image, the regions that are helpful for recognition can be screened out, while suppressing the interference information of the background and irrelevant regions, reducing the interference of noise on recognition, and improving the accuracy and stability of the model in complex background recognition. This avoids the problem that the existing technology often ignores the correlation between different attributes when processing pedestrian attributes, resulting in insufficient reliability and accuracy of recognition results and limited improvement in volume recognition performance.

[0016] 4) The method of the present invention proposes spatial attribute correlation perception, which is used to capture the relevant information between different attribute categories and the different spatial regions they occupy. The spatial relationship between attributes is considered when processing images, and the attribute recognition effect is enhanced through the attribute correlation of spatial features, so that the network can more accurately understand the spatial position and distribution patterns between different attributes in the image, thereby improving the performance and generalization ability of pedestrian attribute recognition tasks.

[0017] 5) Compared with CN113239820B, the present invention uses a multi-branch attribute localization network in CN113239820B, and extracts feature maps respectively by paying attention to the height and width dimensions, thereby enhancing the ability to locate pedestrian attributes. This method emphasizes the spatial distribution of attributes, but may still face recognition difficulties when dealing with complex backgrounds or low-resolution images. The present method performs comprehensive feature extraction through region-related and attribute-related perception networks, uses super-resolution processing modules to effectively improve image details, reduces recognition errors caused by blur and distortion, and can better capture the relationship between different regions and the correlation of attributes by dynamically adjusting the feature extraction process. In contrast, although CN113239820B performs well in some aspects, it may not have the depth and flexibility of the present method in contextual learning and multi-attribute association, and therefore may face more challenges in actual application scenarios.

[0018] 6) Compared with CN111199212B: CN111199212B uses an attention module for attention feature extraction, focusing on selective attention to important features. This method enables the network to better focus on specific areas of pedestrian attributes through a combination of channel attention and spatial attention. However, this selective attention may cause contextual confusion effects, affecting the accurate positioning and recognition of attributes. In contrast, this method introduces a regional correlation causal object perception module (RCOP) to dynamically capture the correlation between different regions in pedestrian images through adaptive pooling and multi-head self-attention. This design enables the model to effectively cope with contextual confusion effects and enhances the ability to recognize attributes in complex backgrounds. Unlike traditional methods that directly perform classification through fully connected layers, this method handles the imbalance problem in the dataset by optimizing the polynomial contrast loss function in the feature extraction stage, ensuring that even rare attributes can be effectively recognized.

[0019] 7) CN114792426A enhances rare attribute images. This method increases the number of samples of rare attributes in the training set and effectively avoids the problem of rare attributes being ignored by the model. By setting the ratio of the training set to the test set and controlling the content of the training set in combination with the number of rare attribute enhancements, the reasonable distribution of training and test data can be better guaranteed. However, the multiple enhancement processes of rare attributes may require a lot of computing resources, especially when there are few rare attribute samples and a large number of enhancements, which may significantly increase data processing and storage costs. This method is more suitable for pedestrian recognition scenarios where the attribute categories are clear and can be labeled. If the attributes are difficult to label or the image quality is poor, it may be difficult to obtain accurate enhancement effects.

[0020] 8) CN116563167A uses an adaptive texture perception module and multi-level attention to refine complex areas, thereby enhancing the restoration of detailed textures. It also separates and fuses high, medium, and low frequency information through wavelet transform and inverse transform, which can effectively avoid high-frequency noise interference during reconstruction, retain important medium and low frequency features, and enhance the overall clarity of the image. However, the multiple self-attention calculations in the adaptive texture perception module, especially the multi-head self-attention operations on complex texture areas, greatly increase the amount of calculation. In addition, the wavelet transform may not be able to accurately extract effective frequency domain information in some cases. For example, in images with low quality or complex backgrounds, the reconstruction effect may be less than expected.

[0021] 9) This application solves the problem of pedestrian attribute recognition in complex environments by introducing feature correlation analysis. The present invention introduces a super-resolution structure calibration module, a region-related causal object perception module, and a spatial attribute correlation perception module. These modules are significantly innovative in structural design and functional implementation. Significant performance improvements have been achieved in pedestrian attribute recognition tasks. By designing a polynomial contrast loss function and combining contrastive learning, the sample imbalance problem has been effectively solved, and the performance of the model in a multi-task environment has been improved. Through an end-to-end framework design, the various innovative modules are organically combined to achieve a comprehensive consideration of attribute correlation and region correlation, significantly improving the accuracy and efficiency of pedestrian attribute recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0023] Figure 1 It is a schematic diagram of the network structure of the pedestrian attribute recognition method of the present invention.

[0024] Figure 2 It is a schematic diagram of the overall structure of the method of the present invention.

[0025] Figure 3 It is a structural schematic diagram of the global super-resolution module GSR in the method of the present invention.

[0026] Figure 4 It is a schematic diagram of the structure of the attention branch in the region-related causal object perception in the method of the present invention.

[0027] Figure 5 It is a schematic diagram of the key structure of spatial attribute related perception in the method of the present invention. DETAILED DESCRIPTION

[0028] Low resolution: The resolution of surveillance videos or pedestrian images taken from a distance is low, and details are difficult to be extracted by the model, resulting in reduced recognition effect.

[0029] Partial body occlusion: Pedestrians are partially blocked by other objects (such as pillars and vehicles), resulting in incomplete features obtained by the model, affecting recognition.

[0030] Complex appearance changes: refers to the frequent changes in pedestrians’ clothing, posture, accessories, etc. Due to these changes, it is difficult for the model to extract stable and recognizable features, thus affecting the recognition effect.

[0031] "Correlation between different attributes" refers to the possible mutual influence or dependency between different attributes of pedestrians (such as gender, age, clothing color, and items carried). For example, a person's age may affect the style of clothing or whether they carry certain items, and gender and hairstyle may also have a certain correlation. Considering the correlation between attributes helps improve recognition accuracy, because these attributes are not completely independent, but may influence each other, so using these relationships can enhance the performance of the recognition model.

[0032] Most current methods do not effectively consider this correlation, or even if they do, they fail to fully utilize these relationships to improve recognition results. This may mean that existing methods are insufficient in modeling the correlation between attributes and fail to deeply explore the interactions or potential dependencies between attributes.

[0033] Figure 1 The illustrated embodiment shows that the process of the method of the present invention is: pedestrian image preprocessing → super-resolution structure calibration → region-related causal object perception, spatial attribute-related perception → polynomial contrast loss. Assume that the training set contains samples, and expressed as ,in Representing images of The label of the attribute, is the number of attributes. The proposed overall network structure is as follows Figure 2 As shown, in the super-resolution structure calibration module, VIT (such as VIT-Base version) is used as the backbone network, and the preprocessed image First, it passes through the global super-resolution module (GSR) to improve the resolution and capture more context information, and then it is sent to the pre-trained VIT to obtain the output image feature vector ; Eigenvector The information is sent to two branches (region-related causal object perception and spatial attribute-related perception) to capture region-related and attribute-related information. The two branches share the first half of the network and guide each other through contrastive learning. The loss function used is called polynomial contrastive loss, which is based on APLloss and contrastive learning, effectively integrating the two types of correlation information and alleviating the imbalance problem of the data set.

[0034] The VIT (Vision Transformer) visual encoder in the present invention can be ViT-Base, ViT-Small, ViT-Large, etc.

[0035] The "super-resolution structure calibration module" in the present invention is a technical module that improves image resolution and enhances image details. Its main purpose is to reduce background blur or interference by enlarging pedestrian images and improving image quality. This module is particularly suitable for pedestrian image scenes with low quality or complex backgrounds, and its design combines two parts: a global super-resolution module and a VIT visual encoder. These components can capture more contextual information while improving image clarity, providing clearer and more informative image input for subsequent pedestrian attribute recognition tasks. "Calibration" refers to the optimization of image resolution and structural information so that the model can more accurately identify and extract key features in pedestrian images while reducing errors caused by low resolution or structural distortion. "Super-resolution" refers to the use of machine learning or deep learning methods to improve image resolution, which can enlarge low-resolution images and restore more details to make the image texture clearer.

[0036] Region-related causal object perception module: It is mainly used to capture relevant information of different regions in the image and solve the background noise problem in pedestrian images. This module can extract the region-related features of pedestrian images, thereby reducing the impact of background interference on pedestrian attribute recognition.

[0037] Spatial attribute related perception module: It is used to identify the distribution of different attributes (such as different categories of objects or pedestrian attributes) in the image. It helps to identify various attributes in pedestrian images by generating specific category spatial features and combining other general features, making the recognition more accurate.

[0038] "Two attention layers with multi-sampling intervention" refers to the attention mechanism in the region-related causal object perception module, which consists of two layers of attention structure and uses multi-sampling intervention technology. Multi-sampling intervention refers to strengthening or weakening the feature information of different regions through multiple sampling operations. This method helps to screen out the regional information that is more important for pedestrian attribute recognition and reduce the interference of other irrelevant regions, thereby improving the module's perception of regional correlation.

[0039] The present invention introduces the spatial attribute correlation perception module (SARP) to generate attribute-related features with strong discriminative power (feature representations that can significantly distinguish different attribute categories, making the features of each category more independent and more discriminative), capture the spatial relationship between different attribute categories, and use the attention mechanism: channel attention , spatial attention etc. to generate a complete and accurate feature representation and optimize the combination of feature representations (the attribute features after adjusting the basic attributes and related information are weighted fused: ).

[0040] Figure 3 The structure diagram of the global super-resolution module GSR, which consists of multiple large separable convolutional layers and a series of subsequent convolutional layers, pedestrian image The global super-resolution module GSR is provided to extract input feature maps through multiple large separable convolutional layers to capture larger context information, and extract more effective key features through subsequent convolutional layer processing. Specifically, the global super-resolution module GSR contains two groups of global convolutional modules (GCMs). Each group of global convolutional modules performs horizontal and vertical convolution operations on the input image, using different convolution kernels. The convolution kernels for the horizontal convolution operation are , , the convolution kernels of the vertical convolution operation are , , where the convolution operations are all large separable convolutions. The outputs of the two sets of global convolution modules are concatenated in the channel dimension to generate a fused output . For the fused output The 1×1 convolution channel reduction operation is used to reduce the computational complexity; the output after the channel number reduction is subjected to convolution with a convolution kernel of k×k and a convolution kernel of 1×1 to extract more effective key features. The features processed by the three convolutions are compared with the input image. Add together to form the final output .

[0041] The "large" in "large separable convolution layer" refers to the fact that the size of the convolution kernel is larger than that of a normal convolution kernel. Larger convolution kernels are used here to obtain richer contextual information, which is especially suitable for tasks that require extracting more details and background information from low-resolution images. The design of separable convolution can also reduce computational overhead, so that even if the convolution kernel is larger, it will not significantly increase the complexity of the model.

[0042] Figure 4 The embodiment shown shows that for the input feature tensor Perform n multiple samplings to obtain sample-specific region-aware features: Split into a set of sub-tensors according to the specified dimension; for each sub-tensor, perform linear transformation and normalization (Conv&Norm) to ensure model stability; flatten the output tensor after linear transformation and normalization (Flatten1), and use the softmax function to calculate the attention score; use the calculated attention score to directly flatten each sub-tensor (Flatten2) and perform weighted summation to obtain the attribute feature tensor. Combine all attribute feature tensors obtained by n multiple samplings into one tensor, and perform shape transformation to adapt to subsequent self-attention operations; process the combined tensor through a multi-layer self-attention structure (Transformer Encoder) to obtain a self-attention tensor ; Output the self-attention tensor It is multiplied element by element with the normalized convolutional layer (Conv) weight matrix, and finally summed in the spatial dimension (dimension 0: batch_size, indicating the number of input samples. Dimension 1: num_classes, indicating the number of categories, each category has a feature map. Dimension 2: height×width, indicating the flattened spatial dimension of the feature map, each spatial position has a response value. The purpose is to calculate the attention weight of each category in the spatial dimension), and then the pedestrian attribute prediction result is obtained through Classifier .

[0043] n times of multiple sampling, processed by n convolutional layers, each convolutional layer Conv corresponds to a "head". These "heads" are responsible for calculating the feature scores of each category. Each head uses a convolutional layer to extract local features and generate outputs, and these outputs will be integrated and weighted in subsequent calculations to achieve the final classification.

[0044] "Multi-layer self-attention structure" refers to a series of stacked self-attention layers, which allows the model to process input features layer by layer and gradually learn features and relationships at different levels. Figure 4 The TransformerEncoder in .

[0045] Figure 5 The embodiment shown shows that the input feature Pass 1 The convolution operation generates the initial attribute score; the initial attribute score is standardized by Norm, using the norm of the weight matrix as the denominator to introduce relevant information; the standardized score is flattened into a one-dimensional vector; the arithmetic mean of all elements in the one-dimensional vector is calculated to obtain the basic attribute features ; Use the softmax function to adjust the flattened one-dimensional vector, calculate the weighted sum, and obtain the attribute features after the relevant information is adjusted ; Perform weighted summation of basic attribute features and attribute features adjusted with relevant information to obtain the final attribute prediction It can effectively capture the relevant information between pedestrian attributes, thereby improving the recognition ability of different attributes and improving the performance of pedestrian attribute recognition tasks.

[0046] The present invention is a multi-module architecture that comprehensively considers attribute correlation and regional correlation. Region Correlated Causal Object Perception Module (RCOP): Through causal relationships and multi-head self-attention, it effectively captures the correlation between different regions in the image, reduces noise interference, and improves the accuracy and robustness of attribute recognition. The model uses multiple sampling and adaptive pooling operations, combined with Transformer blocks for feature reweighting, to enhance the model's ability to capture regional correlations, and through multi-head self-attention, ensures that the association information between different attributes is fully utilized. Through causal object perception and multi-head self-attention structure, the RCOP module can dynamically capture the correlation between different regions in pedestrian images, reduce the interference of background noise, and improve the accuracy of attribute recognition.

[0047] Spatial Attribute Relationship Perception Module (SARP): By calculating the attribute score, specific spatial features are generated for each attribute category, and combined with category-independent overall features, the model's understanding of the spatial distribution of different attributes is enhanced, thereby improving the recognition effect. By calculating the attribute score, specific spatial features are generated for each attribute category, and combined with category-independent overall features, accurate positioning and optimized recognition of the attribute spatial distribution are achieved. By combining channel attention and spatial attention, the input features are double-weighted in channels and space, highlighting important areas and key attributes, and improving the model's generalization ability for complex images. Combining channel attention and spatial attention, spatial weight tensors are generated through average pooling, maximum pooling, and convolution operations to achieve accurate weighting of spatial features.

[0048] The super-resolution structure calibration module (GSR) significantly improves the quality of pedestrian images in low resolution and complex backgrounds by increasing image resolution and capturing more contextual information, providing more expressive feature representation for subsequent attribute recognition.

[0049] Creativity: This module combines the global convolution module to improve the image resolution while capturing more contextual information, which not only improves the image quality, but also provides richer information for subsequent feature extraction. It introduces multiple sets of global convolution modules, performs convolution operations in the horizontal and vertical directions, and optimizes feature fusion through channel splicing and residual connection, which significantly improves the modeling ability of the global image structure.

[0050] Loss function design (polynomial contrastive loss), this paper combines contrastive learning and polynomial loss functions to effectively solve the problem of sample imbalance, especially for the recognition of rare attributes, and improves the adaptability and performance of the model in a multi-task environment through personalized customization of loss functions. Through Taylor expansion and asymmetric focusing, the traditional binary cross entropy loss function is transformed to make it more flexible to adapt to multi-task requirements, and the performance of the model is improved through personalized customization of loss functions.

[0051] The present invention adopts an end-to-end training method as a whole, combines multiple key modules and new loss functions, and realizes collaborative optimization between modules through self-supervised contrastive learning and polynomial contrastive loss functions, so that the model can maintain efficient computing performance and superior recognition accuracy while comprehensively considering regional and attribute correlations. It organically combines super-resolution processing, regional correlation, spatial attribute correlation and innovative loss functions to form an end-to-end pedestrian attribute recognition framework, which is highly innovative and practical as a whole.

[0052] Embodiment 1: The multi-correlation pedestrian attribute recognition method of this embodiment is a pedestrian attribute recognition method that uses two types of correlation, namely, region correlation and attribute correlation, and combines them with polynomial contrast loss. The specific steps are as follows: The first step is to preprocess the pedestrian image: Pedestrian images are used as model input with an image size of 256×192. Data augmentation is added during the training phase to improve performance. Training images are augmented by random horizontal flipping, hue and saturation, rotation, translation, cropping, erasing, and adding random Gaussian blur.

[0053] The second step is super-resolution structure calibration SHC module: In this method, the visual encoder of pre-trained VIT is used as the backbone. The pre-training of the visual encoder of VIT is crucial for the perception of pedestrian attributes. First, Mask Image Modeling (MIM) is introduced as a pre-training method, and the intuitive human structure prior is further incorporated into the pre-training. Before guiding the mask sampling process, the image patches corresponding to the human body part area have high priority to be masked, which can make VIT pay more attention to the human structure information during pre-training. Pre-training is performed on the LUPerson large-scale human dataset, and the encoder part of VIT is retained for subsequent experiments. The role of the encoder is to convert the input image into a high-level feature representation so that the subsequent modules can further process and identify the attributes. In this framework, after pre-training on the LUPerson dataset, the encoder part can effectively extract the basic features of pedestrian images and is retained for subsequent feature processing and attribute recognition tasks.

[0054] The height and width of the VIT output image are 8 and 6 respectively, and the number of channels is 768 channels. Initially, VIT was pre-trained on LUPerson, and then fine-tuned on the pedestrian attribute dataset using the full network (the full network includes the pre-trained VIT visual encoder, and subsequent modules such as the super-resolution structure calibration module, the region-related causal object perception module, the spatial attribute-related perception module, etc.), and all networks of VIT are optimized by the Adam optimizer with the hyperparameter WEIGHT_DECAY=5e-4. The number of epochs is set to 30, and this embodiment uses a cosine decay learning rate scheme, with the learning rate initially starting at 1.9e-4. The batch size is fixed to 64.

[0055] Before the image is sent to the backbone, it passes through a global super-resolution module (GSR) to improve the resolution and capture more context information. and Parameters, where is the number of output filters, is the kernel size, is the weight and bias term of each convolutional layer in GSR, and the calculation formula is as follows: (1), in, Output of super-resolution structural calibration; Represents the global super-resolution module operation; Represents a pre-trained visual encoder.

[0056] GSR consists of multiple large separable convolutional layers and a series of subsequent convolutional layers. The large separable convolutional layers can capture greater contextual information from the input feature map, and the residual operation performed at the end of GSR can eliminate the similarity between the upsampled low-resolution image and the converted high-resolution image.

[0057] The global super-resolution module contains two sets of global convolution modules GCM. Each set of global convolution modules first performs horizontal and vertical convolution operations on the input image through two convolution layers. Then, the outputs of the two convolution layers are integrated into a tensor by splicing in the channel dimension to capture a wider range of local features and contextual information. The final integrated output can be used to provide richer image information, which helps to optimize the performance of pedestrian attribute recognition. Next, the outputs of the two global convolution modules are element-wise added to obtain a fused output. , the calculation is shown as follows: (2), in is the input image; and is the horizontal convolution kernel; and is the vertical convolution kernel; The convolution kernel size is Convolution operation; The convolution kernel size is 1 Convolution operation; For splicing operation.

[0058] Fusion output The information of two global convolution modules GCM is combined to improve the modeling ability of the global image structure. In order to reduce the computational complexity and the number of channels, the fused output is processed by applying a channel reduction operation, which helps to ensure that key features can be extracted more effectively in subsequent processing. Specifically, the output of the two global convolution modules is firstly processed by a 1×1 convolution. The number of channels is compressed, which reduces the dimension while retaining important features. Then, the compressed feature map is input into a standard k×k convolutional layer to further refine the spatial features and extract higher-level information. Finally, a 1×1 convolutional layer is used to adjust the number of channels of the feature map and compare it with the original input. Perform residual connection to get the final output This fusion of input and transformed feature maps helps preserve the details of the original image while integrating the features learned by the network. The calculation formula is as follows: (3), in, Indicates that the convolution kernel is 1 Convolution operation; The convolution kernel is Convolution operation; , , are the learning parameters in the convolution operation, i.e., the weight and bias of the convolution kernel; k represents the size of the convolution kernel, which has the same value as k in the previous GCM (global convolution module), which can simplify the code logic while maintaining the consistency of the network structure.

[0059] This step aims to retain some original information of the input image while integrating the high-level features obtained after the convolution operation, so that the global super-resolution module can significantly improve the image quality and provide a more expressive feature representation for pedestrian attribute recognition tasks.

[0060] The third step is the region-related causal object perception RCOP module: To learn causal object features that are robust to contextual biases, specifically: Input feature tensor After the global adaptive average pooling operation GAP, global information is obtained. Subsequently, the correlation between channels is effectively captured through a one-dimensional convolution operation. In order to maintain the spatial size of the output, a new feature tensor is generated after the one-dimensional convolution operation, and then the dimension of the feature tensor is transposed, that is, a dimension transposition operation is performed. The expand operation in the figure refers to the expansion of the spatial or channel dimension of the data tensor after the dimension transposition to ensure that it is compatible with subsequent operations or meets specific dimensional requirements. Next, by applying the Sigmoid activation function, a channel attention weight is generated, which has an effect on the input feature tensor. Weighted to get the intermediate features , intermediate features The calculation formula for the contrastive learning part is as follows: (4), in, is the sigmoid function used to obtain the attention weight; GAP represents the global adaptive average pooling operation; Indicates that the convolution kernel is 1 convolution operation; in formula (4), the 1×1 convolution kernel is used to generate the weight tensor of the attention weight (convolution is used to model the correlation between channels and generate the attention weight of each channel. The final output is a feature tensor adjusted by the channel weight. The shape is consistent with the input, which is used to enhance the feature selection ability of the model).

[0061] A weighted multiplication operation is performed on each channel by multiplying the original input with the channel attention weight to encourage the model to learn the spatial correlations between pedestrian attributes in a more detailed manner.

[0062] It is then divided into two branches, the pooling branch and the attention branch.

[0063] Pooling branch: First, the intermediate features Perform global adaptive average pooling Avgpool to obtain the overall feature representation. Then, the pooled feature tensor is flattened into a one-dimensional representation by performing a linear transformation (Linear). Finally, the flattened features are transformed through a linear logic layer (Batch Norm) to generate the module output : (5), in, represents a batch normalization layer; represents a fully connected layer; It is a global adaptive average pooling operation.

[0064] Attention branch: Multi-sampling is crucial for attention intervention. Multiple sampling is performed n times to obtain sample-specific region-aware features, which are then sent to the transformer encoder for re-weighting.

[0065] The process of n-times multisampling is: Split into a set of sub-tensors according to the specified dimension; for each sub-tensor, perform linear transformation and normalization (Conv&Norm) to ensure model stability; flatten the output tensor after linear transformation and normalization (Flatten1), and use the softmax function to calculate the attention score; use the calculated attention score to perform weighted summation on the tensor after direct flattening (Flatten2) of each sub-tensor to obtain the attribute feature tensor.

[0066] These attribute feature tensors are then processed by multiple attention heads in the transformer encoder to calculate the regional relevance scores of the attributes.

[0067] First, the input tensor Cut according to the specified dimension to get a list of sub-tensors. For each attention head, the cut sub-tensors are processed by linear transformation and normalized. The normalization here uses the norm of the weight matrix divided by a set norm scale to ensure the stability of the model. The output tensor after the linear transformation is flattened in the second dimension and the attention score is obtained through the softmax function , the softmax function is transformed into a probability distribution form for subsequent weighted calculation: (6), in, Indicates flattening the tensor; It is a normalization operation; Represents a 1×1 convolution operation; Represents the learnable parameters of the convolutional layer; Represents the input tensor To split; is the norm scale.

[0068] At the same time, for tensors Flatten and transform the dimensions, then use the attention score The attribute feature tensor is obtained by weighted summation. The attribute feature tensors calculated by all attention heads are combined into a tensor, which is transformed in shape to adapt to the subsequent self-attention operation and processed through a multi-layer self-attention structure. Finally, the obtained tensor is further processed by linear projection to obtain the module output : (7), in, represents a fully connected layer; represents multi-head self-attention; Represents a dimension transformation operation; For splicing operation; Indicates flattening the tensor; Represents the input tensor To split; Represents the weight coefficient.

[0069] The self-attention tensor output by the model Element-wise multiplication is performed with the weight matrices of the convolutional layers in each sampling. Prior to this, the weight matrix of each convolutional layer is processed and normalized by dividing its norm and adding a predefined scale parameter. This step is designed to ensure the stability and effectiveness of the model. The processed convolutional layer weight matrices are then concatenated and spliced ​​on the last dimension. Finally, this spliced ​​tensor and the self-attention tensor are combined. Perform element-by-element multiplication and sum operation on the spatial dimension to obtain the final pedestrian attribute prediction result : (8), in, For splicing operation; Represents the weight matrix of the attention head; is the norm scale; It represents the number of attention heads in the multi-head self-attention, not the number of data set samples. It is used to refer to the different spatial features or attribute correlation distributions learned by each attention head. Therefore, the final result will perform a weighted summation of the weight matrix of each head to obtain the final pedestrian attribute prediction.

[0070] In this way, the region-related information can be effectively combined with self-attention to improve the performance and accuracy of pedestrian attribute recognition tasks. and Combined, the final output of the RCOP module is obtained rcopBy introducing multiple sampling, the capture of regional correlation is further enhanced, and multi-head self-attention is used for reweighting. Multi-head self-attention plays a key role in this process. It can process sub-tensors through multiple attention heads and calculate the regional correlation scores of attributes, ensuring that the model fully captures the correlation information between attributes in different dimensions. It can effectively capture the relevant information between attributes and regions, providing a robust and efficient feature learning and regional correlation modeling method for pedestrian attribute recognition tasks, significantly improving the performance of pedestrian attribute recognition tasks.

[0071] Step 4: Spatial attribute related perception SARP module: In order to effectively capture the relevant information between different attribute categories and the different spatial regions they occupy, a category-specific feature is generated for each attribute category by calculating an attribute score, which is then combined with the overall feature that is unrelated to the category. This not only locates the attribute region but also assists in the identification of related attributes. Spatial attribute-related perception includes channel attention and spatial attention. Specifically: For the input feature tensor Perform channel attention processing: the channel attention includes average pooling and max pooling operation . Then, a series of 1×1 convolutions and activation functions are used to achieve channel compression and recovery. Specifically, the tensor after average pooling undergoes channel compression and ReLU activation function, and then undergoes channel recovery to form a reconstructed representation of the average information. Similarly, the result of maximum pooling is also processed in the same way to obtain a reconstructed representation of the maximum information. Finally, these two reconstructed representations are added together and the output of the channel attention is obtained through the Sigmoid activation function. : (9), in, Represents the input feature tensor; Represents the Sigmoid activation function; represents the activation function used to remove negative values; Represents a 1×1 convolution operation; and Respectively represent the input feature tensor Perform average pooling and maximum pooling operations to extract the global average and maximum values ​​in the input tensor; and Represents the learnable parameters in the convolution operation.

[0072] Such an output weights the information of different channels and emphasizes the channels with important features in the input tensor. Overall, channel attention improves the network's sensitivity to important features by capturing the channel correlation in the input features, which helps optimize the performance of image representation.

[0073] In spatial attention, the output of channel attention is used as input to improve the attention to spatial positions, including average pooling and maximum pooling operations, convolutional layers, and sigmoid activation functions. First, by Perform average pooling and maximum pooling operations to obtain the average and maximum values, respectively. Then, by concatenating the two pooling results in the channel dimension, a new feature tensor containing the average and maximum information is obtained. Finally, the concatenated feature is processed through a convolutional layer to generate an output with spatial attention. The whole process is normalized by the Sigmoid activation function to obtain a weight tensor ranging from [0,1]. The calculation formula is as follows: (10), in Represents the Sigmoid activation function; Represents a 1×1 convolution operation; Represents a splicing operation; Represents the averaging operation; It represents the maximum value operation; Represents the learnable parameters of the convolutional layer.

[0074] This weight tensor is used to weight the input features, emphasizing locations with important information in space. Therefore, spatial attention improves the network's attention to important locations by capturing the spatial and attribute correlations in the input features, helping to optimize the performance of image representation.

[0075] The representation of input features is strengthened by integrating channel attention and spatial attention. First, the input feature tensor is Channel weighting is performed to emphasize the part of the input with important channel information. Subsequently, the channel-weighted features are spatially weighted through spatial attention to highlight the areas with important information in spatial position. Finally, the feature tensor weighted by spatial attention is used as the output of the first half of SARP (channel attention + spatial attention) output This process is achieved through multiplication, that is, by converting the input feature With channel attention ( ) weights are element-wise multiplied and then combined with the spatial attention ( ) is multiplied element by element, realizing the joint regulation of the channel and spatial information of the input features. This helps the network to capture the key features in the image more flexibly and improves the performance of image expression. The calculation formula is as follows: (11), The features obtained Through a convolution operation, the initial attribute scores are obtained. In order to normalize these scores and introduce relevant information, the initial attribute scores are standardized, where the denominator uses the norm of the weight matrix. Then, through a flattening operation, the scores are mapped to a one-dimensional vector. The basic attribute features are obtained by calculating the average value. .

[0076] (12), in, Represents the average operation, which calculates the average value of all elements in the flattened vector; Represents a flattening operation, which converts the feature from a multi-dimensional structure to a one-dimensional vector to facilitate subsequent average calculation; Represents a 1×1 convolution operation; Represents the learnable weight parameters of the 1×1 convolution.

[0077] The softmax function is then used to adjust the standardized scores and calculate the weighted sum: (13), in, It processes the flattened feature vector to make the relative importance of each attribute feature more significant; Represents a 1×1 convolution operation; Represents the learnable weight parameters of the 1×1 convolution.

[0078] Finally, the basic attribute features Attribute characteristics adjusted with relevant information Perform weighted summation to obtain the final attribute prediction : (14), in, Represents the weight coefficient.

[0079] This enables the model to effectively capture relevant information between pedestrian attributes and improves the recognition ability of different attributes through category-specific features. This is expected to improve performance in pedestrian attribute recognition tasks and enable the model to better understand and distinguish the performance of different attributes.

[0080] Step 5, loss function: The method of the present invention uses an effective asymmetric polynomial loss to alleviate the problem that the existing standard binary cross entropy (BCE) loss is affected by the imbalance between redundant negative samples and rare positive samples, resulting in reduced model performance. Specifically: first, the binary cross entropy (BCE) loss function is Taylor expanded to improve its flexibility and optimization ability. Through Taylor expansion, the shape of the loss function can be better adjusted to make it more adaptable to the characteristics of different tasks and data sets. Then, the coefficients of the polynomial function are improved, and asymmetric focusing is further used to decouple the gradient contributions of negative samples and positive samples to deal with the imbalance problem between positive and negative samples. By setting different focusing parameters, the loss function can pay more attention to rare positive samples, thereby improving the effect of feature learning. The specific Loss formula is as follows: (16), in, It is The predicted value of an attribute, It is The true value of the attribute, , , is the focus parameter, is the weight calculated based on asymmetric focusing; Indicates the total number of attribute categories.

[0081] Through this loss function, the performance of the model in the "pedestrian attribute recognition" task is improved, especially better results are achieved in the presence of sample imbalance.

[0082] The asymmetric focusing method is not a classic loss calculation method. It is proposed to address the problem of sample imbalance. It controls the gradient contribution of positive and negative samples by focusing on the loss weights of positive and negative samples, so that the model pays more attention to those rare and difficult-to-distinguish positive samples. In asymmetric focusing, by setting different focusing coefficients for positive and negative samples, the impact of each on the overall loss can be adjusted separately. This decoupling allows the gradients of negative and positive samples to be focused separately, thereby alleviating the model bias caused by data imbalance and enhancing the learning ability for rare positive samples.

[0083] It is the weight calculated based on the asymmetric focusing method, which is used to control the weight distribution of positive and negative samples in the loss. It will be set according to the number or category distribution of positive and negative samples, so that the model is more inclined to give higher weights to rare positive samples when calculating the loss, thereby improving its performance during training. The specific value of can be set through prior knowledge or dynamically after statistics in the data set.

[0084] In order to effectively combine region correlation and attribute correlation, this method introduces self-supervised contrastive learning to enable the two branches to guide each other in learning. is the index of any enhanced sample, and let is a source sample with the same index as another augmented sample. In self-supervised contrastive learning, the loss takes the following form: (17), in, Indicates The self-supervised contrastive learning loss for samples, Indicates The embedded feature vector of samples, Indicates The sample is the embedded feature vector of another enhanced sample from the same source sample. These two features and is considered a positive sample pair. Represents the temperature parameter, which is used to adjust the sensitivity of the similarity calculation between feature vectors. This will make the similarity distribution more sharp. In addition to In addition, the set of all other samples in the batch is called negative samples, which are used to calculate the contrast term in the formula. Represents negative samples The embedded feature vector of Used with Calculate the similarity and form negative sample pairs.

[0085] First, a diagonal matrix mask is created, whose elements are 1 on the diagonal and 0 elsewhere, which is used for masking when calculating contrastive loss. Next, a series of processing is performed on the input features, including calculating the number of features, merging features, selecting anchor features, and calculating the similarity matrix. After that, numerical stability techniques are used to process by calculating the logits (log probability ratio) of the contrastive loss. Next, mask is used for masking operations, self-contrast cases are excluded, and log probabilities are calculated. Finally, the final loss value is calculated by averaging the log probabilities of the positive samples. In this process, the code effectively compares the similarities between different features by introducing mask for masking operations in contrastive loss, thereby guiding the learning process of the model. This method helps to improve the network's representation learning effect on pedestrian attributes and optimize feature learning in pedestrian attribute recognition tasks.

[0086] The total loss of training the network is as follows: (18); in, is the self-supervised contrastive learning loss, is an asymmetric polynomial loss; is the weight of the self-supervised contrastive learning loss.

[0087] This embodiment conducts experiments on the PETA, PA100K, RAPv1 and RAPv2 databases respectively.

[0088] The PETA dataset contains multiple small, publicly available datasets. The dataset contains 19,000 images with resolutions ranging from 17×39 to 169×365 pixels. Each image is labeled with 61 binary attributes and 4 multi-class attributes. Nevertheless, according to the established protocol, some attributes will not be used. Only 35 attributes are selected for evaluation based on the attributes with a positive label ratio of more than 5%. In addition, the dataset is randomly divided into 9,500 for training, 1,900 for validation, and 7,600 for testing.

[0089] The PA-100K dataset is the largest dataset for pedestrian attribute recognition, which has 100,000 pedestrian images collected using 598 real outdoor surveillance cameras, with resolutions ranging from 50×100 to 758×454. The dataset is randomly divided into training, validation, and test sets in a ratio of 8:1:1. Each image in the dataset contains 26 attributes, with binary labels 0 or 1 indicating the absence or presence of the corresponding attribute.

[0090] The RAPv1 dataset comes from a real indoor surveillance scene. 26 cameras were selected for image acquisition, including 41,585 samples with resolutions ranging from 36×92 to 344×554, including 33,268 training images and 33,268 test images. Only 51 binary attributes were selected for evaluation based on the attributes with a positive label ratio of more than 5%. The dataset clearly annotates three environmental and contextual factors, namely: viewpoint, occlusion pattern, and body part. Attribute annotation considers six parts: spatiotemporal information, body attributes, accessories, posture and action, occlusion, and part attributes.

[0091] The RAPv2 dataset is collected from uncontrolled multi-camera surveillance scenes. RAPv2 is an extended version of RAP. There are 84,928 pedestrian images and 69 binary attributes, of which 67,943 are used for training. 54 attributes are selected to train and evaluate the proposed method.

[0092] Five criteria are used to evaluate the models on the PETA, PA100K, RAPv1, and RAPv2 datasets, including the label-based criterion Average Accuracy (mA) and four instance-based criteria Accuracy (Accu), Precision (Prec), Recall, and F1.

[0093] The method proposed in the present invention is compared with several relatively classic methods, including DeepMAR, HPNet, VeSPA, JRL, GRL, MsVAA, PGDM, RA, CoCNN, ALM, JLAC, SCRL, CAS-SAL-FR, MCFL, DRFormer, DAFL, VTB, ASCB, EALC, PARFormer, VAC, ALM, UniHCP, IAA-Caps and SOFAFormer. Table 1, Table 2, Table 3 and Table 4 respectively list the accuracy comparison of the method proposed in the present invention and other methods on the PETA, PA100K, RAPv1 and RAPv2 datasets, where ours is the method of the present invention.

[0094]

[0095]

[0096]

[0097]

[0098] As can be seen from the table, in the label-based metric mA of all datasets, the method of the present invention achieves State-of-the-Art (SOTA) performance; in the four instance-based indicators, the method of the present invention achieves the highest accuracy, recall and F1 on the PETA dataset and the PA100K dataset; and the highest recall is also achieved on the RAP and RAPv2 datasets, which shows that the method of the present invention has good adaptability to different types of datasets. The method of the present invention learns attribute correlation and region correlation, so that the model can better understand the attribute information in the image and make accurate predictions. Secondly, the method of the present invention optimizes the loss function, which can effectively handle the imbalance between different attributes and improve the generalization ability and performance stability of the model. The combination of these technologies enables the method of the present invention to achieve good results on different datasets, showing its versatility and scalability.

[0099] Some suboptimal results may be due to the fact that the network structure of the present invention has many complex modules ("complex modules" refers to multiple sub-modules with different functions contained in the network structure of the present invention. For example, region-related causal object perception module, spatial attribute-related perception module, super-resolution structure calibration module. These modules increase the complexity of the network, such as different attention mechanisms introduced to capture finer spatial and channel information, loss function adjustments to handle imbalances between attributes, etc. Therefore, although these modules make the model perform better, they also lead to more network parameters, requiring more resources and time for training.), which leads to a large number of network parameters and a more challenging training process. The specific loss function (polynomial contrast loss), sample enhancement technology and optimization algorithm adopted during the training process, asymmetric focus on adjusting the weights of positive and negative samples, effectively address the problem of sample imbalance; the model uses a specific optimizer (such as Adam) to improve convergence efficiency and reduce the risk of overfitting, and still achieves excellent performance. For example, Among them, DeepMAR, HPNet, VeSPA, JRL, GRL, MsVAA, PGDM, RA, CoCNN, ALM, JLAC, SCRL, CAS-SAL-FR, MCFL, DRFormer, DAFL, VTB, ASCB, EALC, PARFormer, VAC, ALM, UniHCP, IAA-Caps and SOFAFormer are all well known in the art.

[0100] In the pedestrian attribute recognition task, the distribution of positive and negative samples of each attribute is usually seriously unbalanced. For example, the positive samples of some attributes (such as "wearing glasses") are far less than the negative samples, resulting in the standard Accu and Prec being calculated higher for most attributes with more negative samples, but it is difficult to show the learning effect of rare positive samples. The method of the present invention uses asymmetric polynomial loss and asymmetric focus to specifically strengthen the learning of rare positive samples, resulting in a higher Recall, because the recall rate mainly focuses on the proportion of positive samples correctly identified by the model, and is better adapted to sample imbalance.

[0101] mA calculates the accuracy on each attribute separately and takes the average, giving each attribute equal weight, so it is not dominated by attributes with a large number of negative samples like Accu. This means that even if an attribute with a large number of negative samples performs poorly, it will not affect the mA value too much. Since the method of the present invention can better identify rare positive samples, the higher mA reflects that the recognition effect of each attribute is generally more balanced.

[0102] Reasons for low precision and high recall: The combination of high recall and low precision usually indicates that the model tends to predict more positive samples, causing some negative samples to be misclassified as positive samples. Although this prediction strategy increases the recognition of positive samples (increases Recall), it also causes some negative samples to be misidentified (reduces Prec), resulting in a low F1 score. For application scenarios where the goal is to maximize the recognition of positive samples, the deviation of this method is the result of intentional design to ensure that recall is prioritized.

[0103] In summary, in the RAPv1 and RAPv2 datasets, the high values ​​of mA and Recall indicate that the proposed method maintains a high average accuracy rate in all attributes, especially in the unbalanced samples. The ability to identify positive samples is stronger. However, due to the emphasis on positive samples, the accuracy, precision and F1 scores are low. In the pedestrian attribute recognition task, the reason for focusing on positive samples is mainly to solve the problem of sample imbalance, especially to improve the recognition effect of rare attributes, with the intention of improving the practical applicability of the model and its expressiveness in key scenarios.

[0104] This application provides a new solution for the field of pedestrian attribute recognition. Through a multi-module architecture that comprehensively considers attribute correlation and regional correlation, super-resolution structure calibration, and innovative loss function design, multiple modules are organically combined to form an end-to-end pedestrian attribute recognition framework, which successfully addresses the challenges faced by existing methods. Future research directions can further optimize the network structure and explore more effective super-resolution and related information extraction methods to further promote the development of pedestrian attribute recognition technology.

[0105] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A multi-correlation pedestrian attribute recognition method, the recognition method comprising the following contents: Get pedestrian images and preprocess them; Constructing a multi-correlated pedestrian attribute recognition model: The multi-correlated pedestrian attribute recognition model includes a super-resolution structure calibration module, a region-related causal object perception module, and a spatial attribute-related perception module; The super-resolution structure calibration module includes a pre-trained VIT visual encoder and a global super-resolution module GSR. The output of the global super-resolution module GSR is processed by the pre-trained VIT visual encoder to obtain the output of the super-resolution structure calibration module. ; The region-related causal object perception module includes a global adaptive average pooling operation, a dimension transposition operation, a dimension expansion operation, a pooling branch, and an attention branch; After global adaptive average pooling, dimension transposition, dimension expansion, and Sigmoid activation function processing, Multiply to obtain intermediate features ; Enter the pooling branch and the attention branch respectively. In the pooling branch, the intermediate features are first The output is obtained after global adaptive average pooling Avgpool, dimension transformation, and batch normalization layer processing. ; In the attention branch, After n times of multisampling, the output is obtained by processing it through the transformer block and calculating the regional correlation score of the attribute. ; Will and Combined with the above, the final output of the regional correlation causal object perception module RCOP module is obtained rcop ; The spatial attribute related perception module includes a hybrid attention block composed of channel attention and spatial attention. The output of the hybrid attention block is , Pass 1 The convolution operation generates the initial attribute score; the initial attribute score is normalized; the normalized score is flattened into a one-dimensional vector, and the arithmetic mean of all elements in the one-dimensional vector is calculated to obtain the basic attribute features. ; At the same time, the softmax function is used to adjust the flattened one-dimensional vector, and then the weighted sum is calculated with the flattened one-dimensional vector to obtain the attribute features after the relevant information is adjusted ; Perform weighted summation of basic attribute features and attribute features adjusted with relevant information to obtain the final attribute prediction ; A polynomial contrast loss function is used to train a multi-correlated pedestrian attribute recognition model, and pedestrian attribute recognition is performed using the trained multi-correlated pedestrian attribute recognition model.

2. The method according to claim 1, characterized in that The global super-resolution module contains two groups of global convolution modules. Each group of global convolution modules first passes two convolution layers on the input image. Perform horizontal and vertical convolution operations; then, combine the outputs of the two convolutional layers into a tensor by concatenating them in the channel dimension; next, add the outputs of the two global convolutional modules element-wise to get a fused output ;right Execute the steps to reduce the number of channels. Convolution operation, feature extraction Convolution operation, adjusting the number of channels 1 After the convolution operation, the input image Add together to get the output of the global super-resolution module .

3. The method according to claim 1, characterized in that The total loss function during training is: （18） in, is the self-supervised contrastive learning loss, is an asymmetric polynomial loss; is the weight of the self-supervised contrastive learning loss; is the polynomial contrast loss function; （17）， Indicates The self-supervised contrastive learning loss of samples, I represents the total number of samples, Indicates The embedded feature vector of samples, Indicates samples come from the embedded feature vector of another enhanced sample of the same source sample; features and is considered a positive sample pair. represents the temperature parameter, which is used to adjust the sensitivity of the similarity calculation between feature vectors; In addition to The set of all other samples in the batch is called negative samples; Represents negative samples The embedded feature vector of Used with Calculate similarity and form negative sample pairs; （16）， in, It is The predicted value of an attribute, It is The true value of the attribute, , , is the focus parameter, is the weight calculated based on asymmetric focusing; Indicates the total number of attribute categories.

4. A multi-correlation pedestrian attribute recognition method, characterized in that: The specific steps of the method are: The first step is to preprocess the pedestrian image: Get images of pedestrians and augment the training images by random horizontal flipping, hue and saturation, rotation, translation, cropping, erasing, and adding random Gaussian blur; The second step is super-resolution structure calibration: Before the image is sent to the backbone, it passes through a global super-resolution module GSR to improve the resolution and capture more context information. GSR receives and Parameters, where are the number of output filters, is the kernel size, GSR consists of multiple large separable convolutional layers and a series of subsequent convolutional layers. The large separable convolutional layers capture greater contextual information from the input feature map, and the final residual operation can eliminate the similarity between the upsampled low-resolution image and the converted high-resolution image; The global super-resolution module GSR contains two sets of global convolution modules. Each set of global convolution modules first performs horizontal and vertical convolution operations on the input image through two two-dimensional convolution layers. Then, the outputs of the two convolution layers are integrated into a tensor by splicing in the channel dimension to capture a wider range of local features and contextual information; the final integrated output is used to provide richer image information; next, the outputs of the two global convolution modules are element-wise added to obtain a fused output. , The output of two global convolution modules is processed through a 1×1 convolution The number of channels is compressed, and then the compressed feature map is input into a standard k×k convolution layer. Finally, a 1×1 convolution layer is used to adjust the number of channels of the feature map and compare it with the original input. Perform residual connection to get the final output ; Pre-train the VIT visual encoder on the LUPerson large-scale human dataset, introduce mask image modeling MIM for pre-training, incorporate intuitive human structure priors into pre-training, and mask the image blocks corresponding to human body parts with high priority before guiding the mask sampling process. During pre-training, more attention is paid to human structure information, and the visual encoder of pre-trained VIT is used as the backbone. After backbone processing, the output feature tensor ; The third step is regional related causal object perception: After the global adaptive average pooling operation GAP is processed, the dimension is transposed and the spatial or channel dimension is expanded through the one-dimensional convolution operation. Then, the Sigmoid activation function is applied to generate the channel attention weight, which has an effect on the input feature tensor. Weighted to get the intermediate features , intermediate features The calculation formula for the part that will also be used for contrastive learning is: （4）， in, is the sigmoid function; GAP represents the global adaptive average pooling operation; Indicates that the convolution kernel is 1 Convolution operation; It is then divided into two branches: pooling branch and attention branch. Pooling branch: First, the intermediate features Perform global adaptive average pooling Avgpool, then flatten the pooled feature tensor into a one-dimensional representation by transforming its dimension (Linear); finally, transform the flattened features through a linear logic layer (BatchNorm) to generate the module output : （5）， in, represents a batch normalization layer; represents a fully connected layer; It is a global adaptive average pooling operation; Attention branch: Perform n multiple samplings to obtain sample-specific region-aware features, which are then sent to the transformer encoder for reweighting to obtain the self-attention tensor. ; Will Multiply the weight matrices of the convolutional layers in each sampling element by element, and sum them up in the spatial dimension to obtain the final pedestrian attribute prediction result : Will and Combined, the final output of the RCOP module is obtained rcop ; Step 4: Spatial attribute related perception SARP module: Spatial attribute-related perception includes channel attention and spatial attention, specifically: For the input feature tensor Perform channel attention processing, the formula is: （9）， in, Represents the Sigmoid activation function; represents the activation function used to remove negative values; Represents a 1×1 convolution operation; and Respectively represent the input feature tensor Perform average pooling and maximum pooling operations to extract the global average and maximum values ​​in the input tensor; and Represents the learnable parameters in the convolution operation; is the output of channel attention; In spatial attention, the output of channel attention is used as input, including average pooling and maximum pooling operations, convolutional layers, and Sigmoid activation functions; first, by Perform average pooling and maximum pooling operations to obtain the average and maximum values ​​respectively. Then, concatenate the two pooling results in the channel dimension to obtain a new feature tensor containing the average and maximum information. Finally, process the concatenated features through a convolutional layer to generate an output with spatial attention. ; The whole process is normalized by the Sigmoid activation function to obtain a weight tensor ranging from [0,1]. The calculation formula is: （10）， in Represents the Sigmoid activation function; Represents a 1×1 convolution operation; Represents a splicing operation; Represents the averaging operation; It represents the operation of finding the maximum value; Represents the learnable parameters of the convolutional layer; Input feature tensor through channel attention Perform channel weighting, perform spatial weighting on the channel-weighted features through spatial attention, and then use the feature tensor weighted by spatial attention as the output of the first half of SARP ; Through a convolution operation, the initial attribute score is obtained, and the initial attribute score is standardized, in which the denominator uses the norm of the weight matrix; then, through the flattening operation, the score is mapped to a one-dimensional vector, and the basic attribute feature is obtained by calculating the average value. ; Then the softmax function is used to adjust the standardized scores and calculate the weighted sum to obtain the attribute features adjusted by relevant information. : The basic attribute characteristics Attribute characteristics adjusted with relevant information Perform weighted summation to obtain the final attribute prediction : （14）， in, represents the weight coefficient; Step 5, loss function: The total loss function during training is: （18） in, is the self-supervised contrastive learning loss, is an asymmetric polynomial loss; is the weight of the self-supervised contrastive learning loss; is the polynomial contrast loss function; （17）， Indicates The self-supervised contrastive learning loss of samples, I represents the total number of samples, Indicates The embedded feature vector of samples, Indicates samples come from the embedded feature vector of another enhanced sample of the same source sample; features and is considered a positive sample pair. represents the temperature parameter, which is used to adjust the sensitivity of the similarity calculation between feature vectors; In addition to The set of all other samples in the batch is called negative samples; Represents negative samples The embedded feature vector of Used with Calculate similarity and form negative sample pairs; （16）， in, It is The predicted value of an attribute, It is The true value of the attribute, , , is the focus parameter, is the weight calculated based on asymmetric focusing; Indicates the total number of attribute categories; Step 6: Network training After VIT is pre-trained on LUPerson, it is then fine-tuned on a pedestrian attribute dataset using the complete network and then trained using the total loss function for pedestrian attribute recognition.

Images