A heading-aware pedestrian attribute representation and learning method

Abstract

The application provides a pedestrian attribute representation and learning method based on orientation perception, wherein the method comprises the following steps: (A) extracting image features by using a Swin Transformer neural network, associating feature maps with feature parts, and converting the feature maps into attribute feature vectors with strong association with attributes; (B) generating a pedestrian orientation vector by using an MEBOW module according to the attribute feature vectors generated in (A); (C) generating a class activation map of the structured description of the pedestrian by using a GradCAM method according to the attribute feature vectors generated in (A), and calculating a spatial attention correction sub-module by using an LSTM module according to the pedestrian orientation vector generated in (B); (D) calculating and storing a two-dimensional matrix representing attention distribution in a self-attention manner according to the attribute feature vectors generated in (A), and generating a relationship matrix of each orientation according to the pedestrian orientation vector generated in (B); (E) fully utilizing the relationship matrix in the form of graph convolution to perform attribute reasoning prediction according to the relationship matrix and the attribute feature vectors; and (F) inputting a test set into a pedestrian attribute recognition model for recognition, and evaluating experimental results.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of based on orientation perception Pedestrian attribute representation and learning method. BACKGROUND

[0002] In recent years, with the joint promotion of image processing, pattern recognition, computer vision technology, internet transmission technology and big data analysis technology, China's intelligent video monitoring industry has achieved rapid development, with the construction of smart city and safe city, monitoring system is widely used in public places, and giving monitoring system certain intelligent ability helps to reduce the burden of manual observation, improves efficiency. Pedestrian under the monitoring system is one of the most valuable analysis targets. Pedestrian attribute recognition refers to predicting a set of attributes from a predefined attribute list to describe the characteristics of a person image, and pedestrian structuring includes gender, age, height, hair accessories, clothing, item carrying and walking posture and other description information. Pedestrian attribute recognition has wide application prospect: pedestrian attribute information can help video investigation scene, so it is more in line with the practical needs of public security organs to crack down on crime; Pedestrian attribute information can be estimated in the automatic driving scene of unmanned vehicle field through monitoring video on target gender, age, clothing and other attributes, which can improve driving safety. After the advent of artificial intelligence technology, it was applied to video monitoring security industry in the 1990s, and has developed for decades. Pedestrian attribute recognition in monitoring scene has high value in practical application and academic research, and is widely concerned by people.

[0003] Pedestrian attribute recognition (PAR) aims to mine the attribute information of pedestrians from input images, which belongs to multi-task learning and multi-label learning. Pedestrian attribute recognition method uses a shared convolutional network to extract features, then uses a separate small network for classification and discrimination for each sub-task, or uses deep or traditional classifiers to classify and predict pedestrian attributes, to find out which attributes the person in the image has from an input image, so as to obtain the structured description of the image (image is a kind of unstructured data).

[0004] Pedestrian attribute recognition (PAR) aims to generate a structured description of pedestrians and plays an important role in surveillance. Current techniques that focus on 2D images can achieve good performance when the captured pedestrian direction does not change. Considering that attributes can be regarded as high-level semantics that are robust to viewpoint changes and viewing condition diversity. Objectively, most pedestrian images taken by cameras show pedestrians walking in different directions. Current PAR techniques focus on a single image without direction changes. These works ignore the direction of pedestrians, ignore the case where attributes can not be visible at different directions of pedestrians, and ignore the fact that the relationship of attributes perceived by mixing pictures of pedestrians from different directions can lead to confusion of the relationship of attributes. This ignorance brings difficulties to pedestrian attribute recognition. Therefore, in the landing scenario of diverse pedestrian direction changes, this ignorance limits the accuracy of PAR, the accuracy of these techniques cannot be maintained, and PAR remains a challenging task.

[0005] The direction of a pedestrian has a strong correlation with the pixel representation of the attributes of the pedestrian. On the one hand, recent attention techniques that have been well validated prove that adjusting the attention area of the model, providing attribute localization and body part, can improve the accuracy of attribute recognition, and a more accurate attention distribution can be achieved according to the direction guidance. On the other hand, the attribute relationship perceived by the model is affected by the direction, because different directions highlight different parts of the attribute relationship. For example, for a front-facing pedestrian image, the front-facing attribute relationship is highlighted, and the back-facing attribute relationship is hidden. Therefore, not distinguishing between directions will lead to two consequences. The first is that the attribute relationships of different directions are mixed together. The second is that the mixed attribute relationships lack contrast, resulting in weak signals for all attribute relationships. The state of a pedestrian contains the occlusion and orientation of the pedestrian, which can describe the effectiveness of the recognized attributes and the effectiveness of the perceived attribute relationships. By enhancing the attribute space attention area based on the direction of the pedestrian, the model can better focus on the attribute area, reduce the cognitive bias of the feature distribution, make the pixel semantics on which the model relies to judge attributes closer to humans, and improve the generalization ability of the model. By establishing direction-guided attribute relationship learning, the model extracts intrinsic and mutually distinguishable attribute relationships for each direction. There are currently three main problems with pedestrian attribute recognition technology:

[0006] 1) Current work does not consider the correlation between pedestrian direction and attributes. This correlation is important for improving the accuracy of PAR and is a logical prerequisite for the last two problems.

[0007] 2) Insufficient exploration of the correlation between pedestrian direction and attributes can ultimately reduce the generalization of the model when assigning attribute space attention. Equal treatment of different attributes leads to homogeneity of attribute space attention. Homogeneity leads to limited prediction accuracy.

[0008] 3) The correlation between the direction of pedestrians and the attribute relationship is not explored. Moreover, the attribute relationship extracted by the existing method is chaotic and weak. The attribute relationship needs to introduce direction information to guide the decoupling of the relationship in different directions, improve the saliency of the attribute relationship, and fully play the role of the attribute relationship in the PAR task. SUMMARY

[0009] The present application provides a pedestrian attribute representation and learning method based on orientation perception, which is compared with the mainstream method, has universality and scalability, can be compatible with different types of pedestrian data, has high recognition accuracy, and is suitable for pedestrian attribute recognition in actual indoor and outdoor monitoring scenes.

[0010] The pedestrian attribute representation and learning method based on orientation perception according to one embodiment of the present application comprises the following steps:

[0011] S1) Extract image features with Swin Transformer, constrain the network, associate feature maps with attributes, and convert feature maps into one-dimensional vectors, which have strong association with attributes, referred to as attribute feature vectors;

[0012] S2) According to the attribute feature vector generated in S1), generate a two-dimensional vector using the MEBOW module, which represents all possible orientation prediction results and the confidence of the prediction results of the detected image, and becomes the orientation vector of the pedestrian;

[0013] S3) According to the attribute feature vector generated in S1), generate the class activation map of the structured description of the pedestrian using the Grad-CAM method, and according to the orientation vector of the pedestrian generated in S2), store a batch of class activation map sequences of different attribute prediction positives, which become a pseudo sequence of spatial attention, and for each attribute, calculate and store the implicit intermediate quantity of the pseudo sequence using the LSTM module, which becomes the spatial attention modifier of each attribute;

[0014] S4) According to the attribute feature vector generated in S1), calculate and store a two-dimensional matrix representing attention distribution in a self-attention manner, which can be implemented by two fully connected layers, and according to the orientation vector of the pedestrian generated in S2), store two groups of different asymmetric heterogeneous graph edge weights for each orientation of the pedestrian, and the weights are dynamically generated by the pairing relationship between the input attribute feature vectors, the matrix represents the relationship between attributes, and becomes the relationship matrix of each orientation;

[0015] S5) The spatial attention modifier of each attribute generated according to S3) guides the spatial attention modification of the attribute feature, and the modification degree of the feature is measured in the form of Euclidean distance, becoming a spatial attention modification map of each attribute of the detected pedestrian, and the attribute feature vector and the spatial attention modification map of the pedestrian are fused through a residual connection structure, becoming a feature after spatial attention modification under the guidance of the orientation;

[0016] S6) The spatial attention modified feature generated in S5) and the relationship matrix of each orientation generated in S4) are fully utilized in the form of graph convolution to perform attribute reasoning prediction;

[0017] S7) A weighted cross-entropy loss function is selected to solve the problem of uneven positive and negative samples and balance the positive and negative samples;

[0018] S8) The test set is input into the pedestrian attribute recognition model for recognition, and the experimental results of the pedestrian attribute recognition model are evaluated.

[0019] According to a further embodiment of the application, the step S1) comprises:

[0020] S1.1) Selecting a Swin Transformer neural network as a feature extraction neural network;

[0021] S1.2) Outputting a feature map through the last layer of the feature extraction neural network, the shape of which is 2048x7x7, binding the feature map of multiple channels with the attribute of the feature, thereby binding the feature map with the attribute, and finally converting the corresponding feature map into a 1-dimensional vector to form an attribute feature vector.

[0022] According to a further embodiment of the application, the step S2) comprises:

[0023] S2.1) Selecting a MEBOW neural network as an orientation extraction neural network;

[0024] S2.2) Loading pre-trained model parameters for the orientation extraction neural network;

[0025] S2.3) Outputting a feature map through the last layer of the orientation extraction neural network, the shape of which is 24x3, and the orientation vector of the pedestrian.

[0026] According to a further embodiment of the application, the step S3) comprises:

[0027] S3.1) Keeping the shape of the generated spatial attention modifier of the attribute consistent with the shape of the class activation map of the generated pedestrian structured description;

[0028] S3.2) In the generation mode of the spatial attention modifier, the class activation map is converted into a pseudo sequence of spatial attention by the pedestrian orientation vector guidance, and is iteratively generated in the manner of LSTM.

[0029] According to a further embodiment of the application, the step S4) comprises:

[0030] S4.1) The number of generated attribute feature vectors is consistent with the number of last derived attributes;

[0031] S4.2) In the generation mode of the relationship matrix, the attribute feature vector is converted into a high-dimensional vector by a fully connected neural network, and the matrix multiplication is generated in the manner of self-attention.

[0032] According to a further embodiment of the application, the step S5) comprises:

[0033] S5.1) The number of generated spatial attention modified features is consistent with the number of last derived attributes;

[0034] S5.2) In the generation mode of the spatial attention modification map, the attribute feature vector is converted and generated by matrix multiplication of the attribute spatial attention modifier.

[0035] According to a further embodiment of the application, the step S6) comprises:

[0036] S6.1) According to the orientation vector of the pedestrian, the relationship matrix of the pedestrian in S4.2) is taken, and the final prediction result is generated by operating the attribute feature vector and the relationship matrix in the graph convolution operation, wherein the number of operation times of the relationship matrix is used as the transmission jump number of the attribute relationship, and the two sets of weights of the relationship matrix are used as the implementation of the asymmetry of the attribute relationship.

[0037] S6.2) To preserve the information lost in the operation process of the attribute feature vector information, a residual structure is added, wherein scores represent the probability of attribute prediction, and W_c represents the probability of c label occurrence.

[0038] According to a further embodiment of the application, the step S7) comprises the following specific steps:

[0039] S7.1) Iterative training is performed using the Adam optimizer with an initial learning rate of 0.0001, an exponentially decaying learning rate, and a data batch size of 32;

[0040] S7.2) The size of the input image is uniformly adjusted to 224 × 224, and then image enhancement steps such as random rotation, random size adjustment, and random horizontal flip are applied to construct a data batch;

[0041] S7.3) constantly iterate learning until the loss function value no longer decreases, the pedestrian attribute recognition model converges, the pedestrian appearance attribute recognition model parameters are saved, and the final pedestrian attribute recognition model is obtained.

[0042] The inventors of the present application have trained and tested the above-mentioned method of the present application on two published pedestrian attribute data sets RAP and PETA and PA100K, and the training and testing recognition accuracy is high, and the method has practical use in real life.

[0043] Compared with the prior art method, the relationship guided pedestrian attribute recognition method of the present application has the following advantages and beneficial effects:

[0044] (1) The present application improves the attribute description accuracy of a single picture;

[0045] (2) The present application obtains a more complete description of the attribute based on direction perception, focuses on the spatial area related to a specific attribute, reduces overfitting, improves generalization, and allocates more accurate spatial attention;

[0046] (3) The present application extracts the attribute relationship of each orientation of the pedestrian, enhances the effect of the attribute relationship, and improves the accuracy of the attribute prediction result;

[0047] (4) The present application first creates an orientation-based spatial attention correction sub and an attribute relationship mining framework, which can be applied to different databases and tasks, and has scalability. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a schematic diagram of the orientation perception-based pedestrian attribute representation and learning method employed in an embodiment of the present application.

[0049] Figure 2 is a network framework diagram of the orientation perception-based pedestrian attribute representation and learning method according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0051] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0052] As shown in Figure 1 , the present application decouples the attribute relationship matrix of the pedestrian direction guidance and distinguishes the weight of the attribute relationship of different orientations, taking direction guidance as the starting point, establishing the way of spatial attention accurate distribution and attribute relationship decoupling in pedestrian attribute recognition, and improving the accuracy of pedestrian attribute recognition. As shown in Figure 2 , a pedestrian attribute recognition method guided by orientation guidance according to the present application comprises the following steps:

[0053] A) using Swin Transformer to extract pedestrian image features, compressing features, and generating attribute feature vectors;

[0054] B) using Mebow to extract the orientation vector of the detected pedestrians in the picture, and establishing two branches according to the orientation, i.e. a non-directional perception branch and a directional perception branch;

[0055] C) generating the class activation map of the structured description of the pedestrians according to the attribute feature vectors generated in step A), and using the LSTM module to calculate and store the spatial attention correction sub of each attribute according to the orientation vector of the pedestrians generated in step B);

[0056] D) calculating and storing the graph structure of the attribute relationship in the form of self-attention according to the attribute feature vectors generated in step A), which can be implemented by two fully connected layers, and storing the graph edge weight for each orientation of the pedestrians according to the orientation vector of the pedestrians generated in step B), which represents the relationship between attributes and becomes the relationship matrix for each orientation;

[0057] E) guiding the spatial attention correction of the attribute features according to the spatial attention correction sub of each attribute generated in step C), and fusing into the spatial attention corrected features under the guidance of the orientation by a residual connection structure;

[0058] F) using the pedestrian orientation vector to generate different relationship matrices, and obtaining the feature after relationship propagation in the form of graph convolution; steps A-F establish the deep learning model of the method, i.e., the orientation-aware pedestrian attribute representation and learning model;

[0059] G) according to the feature after relationship propagation, performing attribute inference prediction through a classification network, and optimizing the parameters of the pedestrian attribute recognition model through training;

[0060] H) inputting a test set into the orientation-aware pedestrian attribute representation and learning model trained in G) to perform recognition, evaluating and iterating the measurement result of the pedestrian attribute recognition model, because the obtained model is end-to-end, the attribute of the pedestrian to be detected in the input picture in the identification is repeated in the steps A-G described in the training.

[0061] The contents of each step are specifically described as follows:

[0062] Step A includes:

[0063] Swin Transformer neural network is used to extract image features, the network is constrained, and the feature map is output through the last layer of the residual neural network, which has a shape of 2048x7x7. The feature map is bound with the attribute, and finally the corresponding feature map is converted into a 1-dimensional vector to form an attribute feature vector.

[0064] Step B includes:

[0065] The pre-trained model parameters are loaded into the orientation extraction neural network, the detected picture is input, and a two-dimensional vector is output to form a pedestrian orientation vector. The confidence threshold is taken as 0.3 to distinguish the non-directional perception branch and the directional perception branch.

[0066] Step C includes:

[0067] According to the attribute feature vector generated in step A), the Grad-CAM method is used to generate the class activation map of the pedestrian structured description, and according to the pedestrian orientation vector generated in step B), a batch of class activation map sequences of different attributes are classified and stored when the prediction is positive, which becomes a pseudo-sequence of spatial attention. The LSTM module is used to calculate and store the implicit intermediate quantity of the pseudo-sequence for each attribute, which becomes the spatial attention modifier of each attribute. According to the generated attribute feature vector and pedestrian direction vector, by comparing the features of the same attribute in different images under the same direction, learning the feature activation of the same attribute in different pictures based on the same state, realizing the comparison within the sequence, and specially extracting different features. These features have different spatial distributions. In this way, the isomorphism degree in the attribute feature extraction process is reduced, and the homogenization problem of attribute features is alleviated.

[0068] Step D comprises:

[0069] According to the generated attribute feature vector, the number of generated attribute feature vectors is consistent with the number of finally derived attributes, and the generation mode of the relationship matrix is to convert the attribute feature vector into a high-dimensional vector through a fully connected neural network, introduce a self-attention mechanism, and generate a two-dimensional matrix. The matrix represents the relationship between attributes, and the value of each point in the matrix represents the strength of the relationship between the corresponding two attributes. The decoupling of attribute relationships in each direction makes these relationships different from each other rather than mixed. In this way, the confusion of attribute relationships is reduced. In addition, the weight of the attribute relationship in each direction is improved, which alleviates the weak limitation of the previous method.

[0070] Step E comprises:

[0071] According to the spatial attention correction sub of each attribute generated in step C), the spatial attention correction of the attribute feature is guided, and the correction degree of the feature is measured in the form of Euclidean distance, becoming a spatial attention correction map of each attribute of the detected picture. The pedestrian attribute feature vector and the spatial attention correction map are fused through a residual connection structure to become a feature after spatial attention correction under the guidance of the direction, wherein the parameter of the residual connection structure is 0.35.

[0072] Step F comprises:

[0073] According to the relationship matrix and the attribute feature vector, the relationship matrix is fully utilized in the form of graph convolution to perform attribute reasoning prediction. The way of graph convolution is to operate the attribute feature vector and the relationship matrix to generate the result after the attribute relationship correction attribute feature, and the operation times of the relationship matrix represent the transmission jump number of the attribute relationship.

[0074] Step G comprises:

[0075] The attribute feature after relationship propagation is enhanced through a self-attention layer to enhance the weight of the singularity feature, and then through two fully connected layers to align the feature with the attribute classification result to obtain the final prediction result.

[0076] Step H comprises:

[0077] The test set is input into the orientation-aware pedestrian attribute representation and learning model trained in E) for recognition, and the experimental results of the pedestrian attribute recognition model are evaluated. Iterative training is performed using an Adam optimizer with an initial learning rate of 0.0001 and an exponentially decaying learning rate, and a batch size of 32. The input image size is uniformly adjusted to 224 × 224, and then image enhancement steps such as random rotation, random resizing, and random horizontal flipping are applied to construct a data batch. The model is iteratively trained over 30 rounds, optimizing towards the gradient direction with the smallest loss function value. The model parameters obtained in the training round with the smallest loss function value are saved as the model parameters used during testing. Once the pedestrian attribute recognition model converges, the pedestrian appearance attribute recognition model parameters are saved, resulting in the final pedestrian attribute recognition model.

[0078] Figure 1 This is an example of the implementation of the method of the present invention, wherein the images are two types of images from the PETA test set. The bottom shows the accurate recognition result obtained by fully utilizing the fusion of information from each frame.

[0079] According to one aspect of the present invention, a method for pedestrian attribute representation and learning based on orientation perception is provided, characterized by comprising the following steps:

[0080] S1) Extract image features using a deep learning model, compress the features, and generate attribute feature vectors; wherein the deep learning model includes one selected from Swin Transformer, ResNet-50, and ResNet-101.

[0081] S2) Based on the attribute feature vector generated in step S1), use the MEBOW module to generate a two-dimensional vector, called the pedestrian orientation vector, and establish two branches based on the pedestrian orientation vector, namely the non-direction perception branch and the direction perception branch.

[0082] S3) Based on the attribute feature vector generated in step S1), the Grad-CAM method is used to generate a class activation map of the structured description of the pedestrian. Based on the pedestrian orientation vector generated in step S2), a batch of class activation map sequences of different attributes are classified and stored, which become a pseudo sequence of spatial attention. The spatial attention modifier of each attribute is calculated and stored using the LSTM module. The spatial attention modifier of the attribute and the attribute feature vector are fused to obtain the spatial attention modified feature of the attribute.

[0083] S4) Based on the attribute feature vector generated in step S1), calculate and store a two-dimensional matrix representing the attention distribution as a graph structure of attribute relationships in a self-attention manner. Based on the pedestrian orientation vector generated in step S2), store two different sets of asymmetric heterogeneous weights for each orientation of the pedestrian. These weights are dynamically generated by the pairing relationship between the input attribute feature vectors. The two-dimensional matrix represents the relationship between attributes and becomes the relationship matrix for each orientation.

[0084] S5) Based on the spatial attention modifier of each attribute generated in step S3), the spatial attention of the attribute features is corrected. The attribute feature vector and spatial attention correction map of each attribute of the pedestrian are fused through the residual connection structure to become the spatial attention correction feature under orientation guidance.

[0085] S6) Based on the relation matrix and the spatial attention-corrected features, attribute reasoning and prediction are performed using the relation matrix in the form of graph convolution;

[0086] S7) The weighted cross-entropy loss function is used to solve the problem of uneven distribution of positive and negative samples and to balance the positive and negative samples.

[0087] S8) Input the test set into the pedestrian attribute recognition model for recognition, and evaluate the experimental results of the pedestrian attribute recognition model.

[0088] in:

[0089] Step S1) includes:

[0090] S1.1) Select one of Swing Transformer, ResNet-50, and ResNet-101 as the feature extraction neural network;

[0091] S1.2) The final layer of the feature extraction neural network outputs a feature map with a shape of 2048x7x7, forming an attribute feature vector.

[0092] Step S2) includes:

[0093] S2.1) Select the MEBOW neural network as the orientation extraction neural network;

[0094] S2.2) The final layer output feature map of the orientation extraction neural network is 24x3, which becomes the orientation vector of the pedestrian.

[0095] Step S3) includes:

[0096] S3.1) Ensure that the shape of the spatial attention modifier of the generated attributes is consistent with the shape of the class activation graph of the generated pedestrian structured description;

[0097] S3.2) The spatial attention modifier is generated by transforming the class activation map into a pseudo-sequence of spatial attention guided by the pedestrian's orientation vector, and then iteratively generating it using LSTM.

[0098] Step S4) includes:

[0099] S4.1) Ensure that the number of generated attribute feature vectors is consistent with the number of attributes obtained at the end;

[0100] S4.2) The relation matrix is ​​generated by transforming the attribute feature vectors into high-dimensional vectors through a fully connected neural network, and then generating them through matrix multiplication using self-attention.

[0101] Step S5) includes:

[0102] S5.1) Ensure that the number of generated spatial attention-corrected features is consistent with the number of attributes obtained at the end;

[0103] S5.2) The spatial attention correction map is generated by multiplying the attribute feature vectors by the spatial attention correction submatrix of the attributes.

[0104] Step S6) includes:

[0105] S6.1) Based on the pedestrian's orientation vector, take the relationship matrix and generate the final prediction result through convolution operation on the graph;

[0106] S6.2) To preserve the information lost during the operation of the attribute feature vector, a residual structure is added, where scores represent the probability of attribute prediction and W_c represents the probability of the c label appearing.

[0107] S6.3) Iterate learning until the loss function value no longer decreases, the pedestrian attribute recognition model converges, save the parameters of the pedestrian appearance attribute recognition model, and obtain the final pedestrian attribute recognition model.

[0108] According to a further aspect of the present invention, in the above-described pedestrian attribute recognition method for pedestrian direction guidance, step S6.1) employs an Adam optimizer with an initial learning rate of 0.0001, an exponentially decaying learning rate, and a data batch size of 32 for iterative training.

[0109] According to a further aspect of the present invention, in the above-described pedestrian attribute recognition method for pedestrian direction guidance, in step S6.2), the size of the input image is uniformly adjusted to 224 × 224, and then a data batch is constructed by applying steps such as random rotation, random resizing, and random horizontal flipping to enhance the image.

[0110] The above description only discloses specific embodiments of the present invention. Without departing from the scope of the claims of the present invention, those skilled in the art can make various corresponding changes and modifications based on the basic technical concept provided by the present invention.

Claims

1. A method for orientation-aware pedestrian attribute representation and learning, characterized in that The method comprises the following steps: S1) extracting image features with a deep learning model, performing feature compression, and generating an attribute feature vector; wherein the deep learning model comprises one selected from a Swin Transformer, a ResNet-50, and a ResNet-101; S2) generating a two-dimensional vector, referred to as a pedestrian orientation vector, using an MEBOW module according to the attribute feature vector generated in step S1), and establishing two branches, namely a non-directional perception branch and a directional perception branch, according to the pedestrian orientation vector; S3) generating a structured description class activation map of the pedestrian using a Grad-CAM method according to the attribute feature vector generated in step S1), and classifying and storing a batch of class activation map sequences of different attributes according to the pedestrian orientation vector generated in step S2) to form a pseudo sequence of spatial attention, calculating and storing a spatial attention modifier for each attribute using an LSTM module, fusing the spatial attention modifier of the attribute and the attribute feature vector to obtain an attribute spatial attention modified feature; S4) calculating and storing a two-dimensional matrix representing attention distribution as an attribute relationship graph in a self-attention manner according to the attribute feature vector generated in step S1), and storing two groups of different asymmetric heterogeneous weights for each orientation of the pedestrian according to the pedestrian orientation vector generated in step S2), wherein the weights are dynamically generated from the pairing relationship between the input attribute feature vectors, and the two-dimensional matrix represents the relationship between attributes and becomes a relationship matrix for each orientation; S5) guiding the spatial attention of the attribute feature to be modified according to the spatial attention modifier of each attribute generated in step S3), and fusing the attribute feature vector and the spatial attention modification graph of each attribute of the pedestrian through a residual connection structure to become a spatial attention modified feature under the guidance of the orientation; S6) performing attribute reasoning prediction in the form of graph convolution using the relationship matrix according to the relationship matrix and the spatial attention modified feature; S7) selecting a weighted cross-entropy loss function to solve the problem of uneven positive and negative samples and balance the positive and negative samples; S8) inputting a test set into the pedestrian attribute recognition model for recognition, and evaluating the experimental results of the pedestrian attribute recognition model, Wherein: The step S1) comprises: S1.1) selecting one from a Swin Transformer, a ResNet-50, and a ResNet-101 as a feature extraction neural network; S1.2) outputting a feature map through the last layer of the feature extraction neural network, which has a shape of 2048x7x7, forming an attribute feature vector, The step S2) comprises: S2.1) selecting an MEBOW neural network as an orientation extraction neural network; S2.2) outputting a feature map through the last layer of the orientation extraction neural network, which has a shape of 24x3, becoming a pedestrian orientation vector, The step S3) comprises: S3.1) making the shape of the generated attribute spatial attention modifier consistent with the shape of the generated pedestrian structured description class activation map; S3.2) In the generation mode of the spatial attention modifier, the class activation map is converted into a pseudo sequence of spatial attention by the pedestrian orientation vector guidance, and is generated iteratively in the form of LSTM, The step S4) includes: S4.1) The number of generated attribute feature vectors is consistent with the number of last derived attributes; S4.2) In the generation mode of the relationship matrix, the attribute feature vector is converted into a high-dimensional vector by a fully connected neural network, and the matrix multiplication is generated in the form of self-attention, The step S5) includes: S5.1) The number of generated spatial attention modified features is consistent with the number of last derived attributes; S5.2) In the generation mode of the spatial attention modification map, the attribute feature vector is converted into a spatial attention modification matrix by the attribute feature vector, The step S6) includes: S6.1) According to the orientation vector of the pedestrian, the relationship matrix is taken, and the last prediction result is generated by the operation of the graph convolution; S6.2) In order to retain the information lost in the operation process of the attribute feature vector, a residual structure is added, wherein scores represent the probability of attribute prediction, and W_c is the probability of c label occurrence; S6.3) Continuously iterate learning until the loss function value no longer decreases, the pedestrian attribute recognition model converges, the pedestrian appearance attribute recognition model parameters are saved, and the final pedestrian attribute recognition model is obtained.

2. The pedestrian attribute representation and learning method based on orientation perception according to claim 1, characterized in that: Step S6.1) uses the Adam optimizer with an initial learning rate of 0.0001, an exponentially decaying learning rate, and a data batch size of 32 for iterative training.

3. The pedestrian attribute representation and learning method based on orientation perception according to claim 1, characterized in that: In step S6.2), the size of the input image is uniformly adjusted to 224 × 224, and then image enhancement methods such as random rotation, random size adjustment and random horizontal flip are applied to construct a data batch.

4. A computer-readable storage medium storing a computer program, which can enable a processor to execute the pedestrian attribute representation and learning method based on orientation perception according to any one of claims 1-3.

Images