Training Method, System and Related Devices for Pedestrian Attribute Recognition Model

Abstract

The present invention is applicable to the field of computer vision applications and provides a training method, system, and related equipment for a pedestrian attribute recognition model. The method comprises: obtaining a dataset containing pedestrian images and filtering the dataset using the Apriori algorithm to obtain frequent item sets; performing binary attribute conversion on the frequent item sets to obtain a training dataset; constructing a pre-trained attribute recognition model including an improved ULSAM attention module, inputting the training dataset, and obtaining recognition accuracy results for pedestrian attributes; based on the recognition accuracy results, performing accuracy judgments on multiple pedestrian attributes with strong correlations according to preset association rules, and connecting the fully connected layers corresponding to the multiple pedestrian attributes that meet a preset accuracy threshold; and outputting a pedestrian attribute recognition model. By using the improved ULSAM attention module and the Apriori algorithm, the present invention optimizes the attribute recognition model structure and enhances recognition performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision applications, and in particular relates to a training method, system and related equipment for a pedestrian attribute recognition model. Background Art

[0002] Tasks in the field of computer vision include target detection, attribute recognition, image classification, etc., and pedestrian attribute recognition is an important task in the field of computer vision because it plays an important role in video surveillance. Its purpose is to identify the attributes of pedestrians in the picture, such as age, gender, top clothing, top color, etc.

[0003] Early pedestrian attribute recognition primarily relied on handcrafted low-level features. For example, the HOG algorithm proposed by N. Dalal et al. uses histograms of oriented gradients to capture local shapes. The SIFT scale-invariant feature transformation algorithm proposed by DGLowe et al. finds extreme points in spatial scale and extracts their position, scale, and rotation invariants to obtain local features. However, these algorithms, due to drawbacks such as uncontrollable human design, database dependency, long extraction times, and high workload, have not been ideal in practical applications.

[0004] In recent years, due to the advantage of deep learning in extracting richer features through multi-layer nonlinear transformations, a number of pedestrian attribute recognition algorithms based on deep learning have emerged. P. Sudowe et al. proposed a multi-branch classification layering algorithm (CAN) based on convolutional networks. The multi-branch classification layering only considers the global image and jointly learns all attributes for prediction. This results in a lack of consideration of fine-grained image recognition in the multi-branch classification layering algorithm. P. Liu et al. proposed a positioning guidance network (LGNet) that fuses local features with global features to locate areas corresponding to different attributes. However, the local and global features obtained by this type of algorithm may not be effective enough and cannot serve as an important basis for pedestrian attribute recognition.

[0005] In 2017, Wang et al. proposed the Non-local Net for video classification tasks in the field of computer vision. Its use of the attention mechanism as an information transfer structure resulted in higher performance than previous classification models, which promoted the widespread application of the attention mechanism in computer vision. The SE-Net proposed by Hu, J. et al. adaptively assigns weights to each channel, improving network performance, but only considers the channel direction. Based on the SE-Net, Saini, R. et al. proposed ULSAM (Ultra-Lightweight Subspace Attention), which has the ability to obtain a separate attention map from each feature subspace, thereby obtaining multi-scale and multi-frequency feature representations when extracting features. This is more ideal for fine-grained image classification. However, ULSAM suffers from the lack of interaction between sub-feature maps and ignores the relationship between multiple attributes in the feature. Summary of the Invention

[0006] The embodiments of the present invention provide a training method, system and related equipment for a pedestrian attribute recognition model, aiming to solve the problem that the original ULSAM attention module has insufficient ability to extract channel information of images, resulting in insufficient correlation between attributes.

[0007] In a first aspect, an embodiment of the present invention provides a method for training a pedestrian attribute recognition model, the method comprising:

[0008] Obtain a data set containing multiple pedestrian images, and filter the data set using an Apriori algorithm to obtain a frequent item set, wherein each pedestrian image in the frequent item set contains at least two pedestrian attributes;

[0009] Performing binary attribute conversion on the frequent item sets to obtain a training data set;

[0010] Constructing a pre-trained attribute recognition model including an improved ULSAM attention module, and inputting the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attributes, wherein the pre-trained attribute recognition model further includes a number of fully connected layers corresponding to the number of pedestrian attributes, and each fully connected layer corresponds to one of the pedestrian attributes;

[0011] Based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rules, and the fully connected layers corresponding to the pedestrian attributes that meet the preset accuracy threshold are connected;

[0012] The pre-trained attribute recognition model that has completed the connection of the fully connected layer is output to obtain a pedestrian attribute recognition model.

[0013] Furthermore, the step of obtaining a data set containing multiple pedestrian images and filtering the data set using the Apriori algorithm to obtain frequent itemsets is specifically as follows:

[0014] A preset support threshold and a preset confidence threshold of the Apriori algorithm are set according to the data set, and the pedestrian attributes in the data set are filtered and screened according to the preset support threshold and the preset confidence threshold to obtain the frequent item set, wherein at least some of the pedestrian attributes in the frequent item set have the strong correlation.

[0015] Furthermore, the step of performing binary attribute conversion on the frequent itemsets to obtain a training data set includes the following sub-steps:

[0016] Converting the multi-valued attributes in the frequent itemset into binary attributes;

[0017] The original binary attributes in the frequent item set are retained and integrated with the result of the multi-value attribute conversion to obtain the training data set.

[0018] Furthermore, the pre-trained attribute recognition model is based on the Resnet50 model, which includes an input layer, a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fully connected layer, and an output layer in a logical order. The first residual layer, the second residual layer, the third residual layer, and the fourth residual layer all have interaction channels between each other, wherein the first residual layer includes 3 residual modules, the second residual layer includes 4 residual modules, the third residual layer includes 6 residual modules, and the fourth residual layer includes 3 residual modules, and each of the residual modules is composed of 1*1 convolution, 3*3 convolution, and 1*1 convolution in a logical order.

[0019] Furthermore, the ULSAM attention module includes a spatial attention submodule and an SE-channel attention submodule, wherein the spatial attention submodule is located after the last 1*1 convolution of the first residual layer, and the SE-channel attention submodule is located after the output of the fully connected layer.

[0020] Furthermore, in the pre-trained attribute recognition model, a feature map output by the last 1*1 convolution of the first residual layer is defined as F, and F can be divided into g sub-feature maps corresponding to the number of the pedestrian attributes. Then, the sub-feature maps of the feature map F can be expressed as equation (1):

[0021] [F1,…F n ,…F g ] (1)

[0022] Where n and g are both positive integers;

[0023] Define the spatial attention submodule as F n The spatial attention feature map output after processing the input is F' n , then the spatial attention feature map F' n Satisfying the relationship (2):

[0024] F' n =sigmod(maxpool(F n ))*F n (2)

[0025] Define the channel connection and convolution operation of Resnet50 in the pre-trained attribute recognition model on the spatial attention feature map F' n The second feature map output after processing is F" n , then the second feature map F" n Satisfying the relationship (3):

[0026]

[0027] Define the SE-channel attention submodule to the second feature map F' n After the input is processed, the corrected feature map output by the output layer is Out, and the corrected feature map Out satisfies the relationship (4):

[0028] Out=FC2(Relu(FC1(maxpool(Concat[F”1,…F” n ,…F” g ])))) (4)

[0029] In the above equations (2), (3), and (4), maxpool represents the maximum pooling operation in the channel dimension, sigmoid and Relu represent activation functions, Concat represents the channel connection operation, Conv represents the convolution operation with a convolution kernel size of 1*1, and FC represents the fully connected operation of the fully connected layer.

[0030] Furthermore, based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rule, and the fully connected layers corresponding to the multiple pedestrian attributes that meet the preset accuracy threshold are connected, including the following sub-steps:

[0031] Obtaining the pedestrian attribute recognition accuracy corresponding to each of the pedestrian attributes in the pre-trained attribute recognition accuracy result;

[0032] Define any one of the pedestrian attributes as a consequent attribute, define the other pedestrian attributes that have the strong correlation with the consequent attribute as antecedent attributes, and determine whether the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold, wherein:

[0033] If the pedestrian attribute recognition accuracy rate corresponding to the antecedent attribute is higher than the preset accuracy threshold, connecting the fully connected layer corresponding to the antecedent attribute with the fully connected layer corresponding to the consequent attribute;

[0034] If the pedestrian attribute recognition accuracy rate corresponding to the antecedent attribute is less than or equal to the preset accuracy threshold, no processing is performed.

[0035] In a second aspect, an embodiment of the present invention further provides a training system for a pedestrian attribute recognition model, comprising:

[0036] An attribute association module is used to obtain a data set containing multiple pedestrian images and filter the data set using an Apriori algorithm to obtain a frequent item set, where each pedestrian image in the frequent item set contains at least two pedestrian attributes;

[0037] An attribute conversion module is used to convert the binary attributes of the frequent itemsets to obtain a training data set;

[0038] A pre-training module is used to construct a pre-trained attribute recognition model including an improved ULSAM attention module, and input the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attribute, wherein the pre-trained attribute recognition model further includes a number of fully connected layers corresponding to the number of pedestrian attributes, and each fully connected layer corresponds to one of the pedestrian attributes;

[0039] A fully connected association module is used to determine the accuracy of multiple pedestrian attributes with strong correlation according to the pre-trained attribute recognition accuracy results and a preset association rule, and to connect the fully connected layers corresponding to the pedestrian attributes that meet a preset accuracy threshold;

[0040] The model output module is used to output the pre-trained attribute recognition model that has completed the connection of the fully connected layer to obtain a pedestrian attribute recognition model.

[0041] In a third aspect, an embodiment of the present invention further provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the training method of the pedestrian attribute recognition model as described in any one of the above embodiments are implemented.

[0042] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the training method of the pedestrian attribute recognition model as described in any one of the above embodiments are implemented.

[0043] The beneficial effects achieved by the present invention are that the improved ULSAM attention module is added to the pedestrian attribute recognition model with channel interaction operation, and the pedestrian attributes are screened and the relationship is mined through the Apriori algorithm to change the structure and training method of the fully connected layer, so that the performance of the recognition model is enhanced and richer information can be recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of the steps of the training method of the pedestrian attribute recognition model provided by an embodiment of the present invention;

[0045] Figure 2 This is a sub-flow diagram of step S102 in the training method of the pedestrian attribute recognition model provided by an embodiment of the present invention;

[0046] Figure 3 2 is a schematic diagram of a multi-task learning hard-connected structure of a pre-trained attribute recognition model in an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the Resnet50 model structure used in the embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of the structure of the attention module provided by an embodiment of the present invention;

[0049] Figure 6 Schematic diagram of the overall structure of the pre-trained attribute recognition model provided by an embodiment of the present invention;

[0050] Figure 7 This is a sub-flow chart of step S104 in the training method of the pedestrian attribute recognition model provided by an embodiment of the present invention;

[0051] Figure 8 Schematic diagram of the fully connected structure of the pre-trained attribute recognition model provided by an embodiment of the present invention;

[0052] Figure 92 is a schematic diagram of the structure of a training system 200 for a pedestrian attribute recognition model provided by the present invention;

[0053] Figure 10 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] Please refer to Figure 1 , Figure 1 The following is a flowchart of a method for training a pedestrian attribute recognition model according to an embodiment of the present invention, comprising the following steps:

[0056] S101 , obtaining a data set including multiple pedestrian images, and filtering the data set using an Apriori algorithm to obtain a frequent item set, wherein each pedestrian image in the frequent item set includes at least two pedestrian attributes.

[0057] Specifically, the dataset contains multiple pedestrian images, each of which possesses various attributes, such as clothing type, clothing color, age, and gender. For each of these pedestrian images, the pedestrians can be categorized based on these attributes. The Apriori algorithm is a data mining algorithm used to calculate and identify frequently occurring data sets in data to facilitate relationship analysis between these sets. Frequently occurring data sets are referred to as frequent itemsets.

[0058] The Apriori algorithm mainly consists of connection and pruning. The connection method is mainly iterative class matrix calculation. Pruning is to remove unnecessary intermediate results of the class matrix calculation. The result obtained by pruning is the frequent item set. The frequent item set includes the concepts of standard support, confidence, and lift.

[0059] Standard support refers to the proportion of the number of occurrences in the associated data set to the total data set;

[0060] Confidence refers to the conditional probability of data. For example, for attributes x and y in the same data, the confidence of y relative to x can be expressed as:

[0061] Confidence(y->x)=P(x|y)=P(xy) / P(y)

[0062] Lift refers to the ratio of the probability of x occurring when y is present to the overall probability of x occurring. This can be expressed as:

[0063] Lift(y->x)=P(x|y)=Confidence(y->x) / P(x)

[0064] When the lift is greater than 1, it means that there is an effective strong correlation between y and x. When the lift is less than 1, it means that there is an invalid strong association rule between y and x. When the lift is equal to 1, it means that x and y are independent of each other.

[0065] In an embodiment of the present invention, by setting a preset support threshold and a preset confidence threshold of the Apriori algorithm, and then filtering the data set by the pedestrian attributes through the Apriori algorithm, the frequent item sets corresponding to the data set are obtained. Exemplarily, the embodiment of the present invention uses the public Market1501 data set as the data set for filtering by the Apriori algorithm. The Market1501 data set contains 32,668 pedestrian images and a variety of pedestrian attributes. For the Market1501 data set, the preset support threshold and the preset confidence threshold are set to 0.2 and 0.9, respectively. The frequent item sets obtained by filtering and calculating the Market1501 data set using the Apriori algorithm according to the above-mentioned preset support threshold and the preset confidence threshold are shown in Table 1.

[0066] Table 1 The frequent itemsets obtained by filtering and calculating the Market1501 dataset

[0067]

[0068] It should be noted that, depending on the data set, the preset support threshold, and the preset confidence threshold, the obtained results should also be correspondingly different. The data in Table 1 are only for illustrative purposes and are not intended to limit the present invention.

[0069] S102: Perform binary attribute conversion on the frequent itemsets to obtain a training data set.

[0070] For details, please refer to Figure 2 , Figure 2 This is a sub-flow diagram of step S102 in the training method of the pedestrian attribute recognition model provided by an embodiment of the present invention, which specifically includes the following sub-steps:

[0071] S1021: Convert the multi-valued attributes in the frequent itemset into binary attributes.

[0072] Any of the pedestrian attributes in the frequent item set may be a multi-valued attribute. The multi-valued attribute refers to an attribute that has multiple sets of values to describe an entity. In an embodiment of the present invention, taking the Market1501 dataset as an example, the pedestrian attributes contained in any pedestrian image are composed of 1 four-valued attribute, 9 two-valued attributes, 1 nine-valued attribute, and 1 two-valued attribute. In this step, the multi-valued attributes are all converted into the two-valued attributes.

[0073] S1022: retain the original binary attributes in the frequent item set, and integrate them with the result of the multi-value attribute conversion to obtain the training data set.

[0074] The converted multi-valued attributes are integrated with the original binary attributes. For example, after the Market1501 dataset is multi-valued converted, 32 binary attributes are obtained, and the shape of each image is converted from 64*128*3 to 224*224*3, thereby unifying the data size used for input into the pre-trained attribute recognition model.

[0075] S103. Construct a pre-trained attribute recognition model including an improved ULSAM attention module, and input the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attributes, wherein the pre-trained attribute recognition model further includes fully connected layers corresponding to the number of the pedestrian attributes, and each of the fully connected layers corresponds to one of the pedestrian attributes.

[0076] Please also refer to Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 , Figure 3 : is a schematic diagram of the multi-task learning hard-connected structure of the pre-trained attribute recognition model in an embodiment of the present invention, Figure 4 This is a schematic diagram of the Resnet50 model structure used in the embodiment of the present invention. Figure 5 is a schematic diagram of the structure of the attention module provided by an embodiment of the present invention, Figure 6 This is a schematic diagram of the overall structure of the pre-trained attribute recognition model provided by an embodiment of the present invention. In this embodiment of the present invention, the pre-trained attribute recognition model uses a hard-connected structure in multi-task learning. Multi-task joint learning can share data sets and improve model effects. Multi-task learning generally shares feature networks, but applies a separate network to each task. When used for pedestrian attribute recognition, multi-task learning is conducive to finding correlations in different subtasks.

[0077] The pre-trained attribute recognition model is based on the Resnet50 model, which includes an input layer, a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fully connected layer, and an output layer in a logical order. There are interaction channels between all residual layers, wherein the first residual layer includes 3 residual modules, the second residual layer includes 4 residual modules, the third residual layer includes 6 residual modules, and the fourth residual layer includes 3 residual modules. Each of the residual modules is composed of 1*1 convolution, 3*3 convolution, and 1*1 convolution in a logical order.

[0078] Furthermore, the improved ULSAM attention module includes a spatial attention submodule and an SE-channel attention submodule. The spatial attention submodule is located after the last 1*1 convolution of the first residual layer, and the SE-channel attention submodule is located after the output of the fully connected layer. The structure of the SE-channel attention submodule is related to the interaction channel in the Resnet50 model, which means that the SE-channel attention submodule has an information exchange relationship with the output of all residual layers, thereby improving the efficiency of information transmission in the network.

[0079] Furthermore, in the pre-trained attribute recognition model, a feature map output by the last 1*1 convolution of the first residual layer is defined as F, and F can be divided into g sub-feature maps corresponding to the number of the pedestrian attributes. Then, the sub-feature maps of the feature map F can be expressed as equation (1):

[0080] [F1,…F n ,…F g ](1)

[0081] Where n and g are both positive integers;

[0082] Define the spatial attention submodule as F n The spatial attention feature map output after processing the input is F' n , then the spatial attention feature map F' n Satisfying the relationship (2):

[0083] F' n =sigmod(maxpool(F n ))*F n (2)

[0084] Define the channel connection and convolution operation of Resnet50 in the pre-trained attribute recognition model on the spatial attention feature map F' n The second feature map output after processing is F" n , then the second feature map F" nSatisfying the relationship (3):

[0085]

[0086] Define the SE-channel attention submodule to the second feature map F' n After the input is processed, the corrected feature map output by the output layer is Out, and the corrected feature map Out satisfies the relationship (4):

[0087] Out=FC2(Relu(FC1(maxpool(Concat[F”1,…F” n ,…F” g ])))) (4)

[0088] In the above equations (2), (3), and (4), maxpool represents the maximum pooling operation in the channel dimension, sigmoid and Relu represent activation functions, Concat represents the channel connection operation, Conv represents the convolution operation with a convolution kernel size of 1*1, and FC represents the fully connected operation of the fully connected layer.

[0089] In an embodiment of the present invention, the training data set is input into the pre-trained attribute recognition model. After the pre-trained attribute recognition model extracts and recognizes the pedestrian attributes in the training data set, the pre-trained attribute recognition model outputs the recognized corrected feature map. By comparing the corrected feature map with the original pedestrian attributes, the pre-trained attribute recognition accuracy result of the pedestrian attributes can be obtained. Generally, the pre-trained attribute recognition accuracy result is a calculated value.

[0090] S104. Based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rules, and the fully connected layers corresponding to the pedestrian attributes that meet the preset accuracy threshold are connected.

[0091] Please refer to Figure 7 , Figure 7 This is a sub-flow diagram of step S104 in the training method of the pedestrian attribute recognition model provided by an embodiment of the present invention, including the following sub-steps:

[0092] S1041: Obtain the pedestrian attribute recognition accuracy corresponding to each pedestrian attribute in the pre-trained attribute recognition accuracy result.

[0093] Specifically, the pre-trained attribute recognition accuracy result in the embodiment of the present invention includes the recognition result of any of the pedestrian attributes, and the pedestrian attribute recognition accuracy corresponding to each of the pedestrian attributes can be calculated based on the feature correction map and the original pedestrian attributes, which is specifically a numerical value.

[0094] S1042. Define any one of the pedestrian attributes as a consequent attribute, define the other pedestrian attributes that have the strong correlation with the consequent attribute as antecedent attributes, and determine whether the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold.

[0095] For example, as shown in the data in Table 1, in the embodiment of the present invention, taking the pedestrian attributes down, cloth, and teenager as an example, when the Apriori algorithm is used to calculate the frequent item sets, the relevance of one attribute to the other attribute is calculated relative to one attribute, wherein down and cloth are the antecedent attributes (Antecedents), and teenager is the consequent attribute (Consequents). In order to improve the recognition accuracy of teenager as the consequent attribute, it is necessary to judge the recognition accuracy of down and cloth as the antecedent attributes. Here, a preset threshold, i.e., the preset accuracy threshold, needs to be defined, and the recognition accuracy of down and cloth is compared with the preset accuracy threshold for judgment, wherein:

[0096] S1043a: If the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold, connecting the fully connected layer corresponding to the antecedent attribute with the fully connected layer corresponding to the consequent attribute.

[0097] For details, please refer to Figure 8 , Figure 8 is a schematic diagram of the fully connected structure of the pre-trained attribute recognition model provided by an embodiment of the present invention. When the recognition accuracy of down and cloth exceeds the preset accuracy threshold, the layers corresponding to down and cloth in the fully connected layer (FC) are connected to the layer of teenager, that is:

[0098]

[0099] At the same time, taking the data in Table 1 as an example, when upsleeve is the consequent attribute, and the recognition accuracy rates of teenager, down, and cloth, which are the antecedent attributes of upsleeve, all exceed the preset accuracy threshold, the layers corresponding to teenager, down, and cloth in the fully connected layer are connected to the layer of upsleeve, that is:

[0100]

[0101] In this way, under the influence of attributes with strong correlation, the pedestrian attribute corresponding to a high recognition accuracy rate will also improve the recognition accuracy rate of its consequent attribute, thereby improving the recognition effect of the overall attribute recognition model.

[0102] S1043b: If the pedestrian attribute recognition accuracy rate corresponding to the antecedent attribute is less than or equal to the preset accuracy threshold, no processing is performed.

[0103] This situation refers to the situation where the recognition accuracy as the antecedent attribute is less than or equal to the accuracy threshold, that is, the current recognition accuracy is not sufficient to improve the accuracy of the consequent attribute. At this time, the corresponding level of the fully connected layer is not connected to the level of the consequent attribute. It should be noted that when there are multiple antecedent attributes, the hierarchical connection of multiple antecedent attributes is not canceled when the accuracy corresponding to one of the antecedent attributes is lower than the threshold, but only the hierarchical connection corresponding to the single antecedent attribute that does not meet the threshold is canceled.

[0104] S105 , outputting the pre-trained attribute recognition model that has completed the connection of the fully connected layer to obtain a pedestrian attribute recognition model.

[0105] For example, the comparison data of the ULSAM attention module used in the pedestrian attribute recognition model obtained by training and outputting the Market1501 dataset, the preset support threshold, the preset confidence threshold, the preset accuracy threshold and other parameters used in the embodiment of the present invention and other types of attention modules under the Accuary, Recall, Precision, and F1-Score indicators are shown in Table 2.

[0106] Table 2 Comparison data of ULSAM attention module and other types of attention modules

[0107] Model Accuary Recall Precision F1-Score Resnet 0.8898 0.7174 0.8153 0.7632 Resnet+SE 0.8964 0.7454 0.8197 0.7808 Resnet+ULSAM 0.9038 0.7745 0.8260 0.7994 Resnet+HS-ULSAM-B 0.9111 0.7932 0.8387 0.8153

[0108] Among them, Resnet+HS-ULSAM-B represents the pedestrian attribute recognition model in the embodiment of the present invention.

[0109] The ablation experiment comparison data of the pedestrian attribute recognition model and the model using other types of attention modules under the Accuary, Recall, Precision, and F1-Score indicators are shown in Table 3.

[0110] Table 3. Comparison of ablation experimental data between pedestrian attribute recognition model and models using other types of attention modules

[0111] Model Acc Recall Precision F1-Score Resnet 0.8898 0.7174 0.8153 0.7632 Resnet+ collaboration 0.8961 0.7328 0.8277 0.7773 Resnet+HS-ULSAM-A 0.9074 0.7987 0.8222 0.8103 Resnet+HS-ULSAM-B 0.9111 0.7932 0.8387 0.8153 Resnet+HS-ULSAM-B+ collaboration 0.9130 0.7971 0.8429 0.8194

[0112] Among them, Resnet+HS-ULSAM-B+synergy represents the pedestrian attribute recognition model that is collaboratively trained in combination with a hard-connected structure in an embodiment of the present invention.

[0113] For the pedestrian attributes teenager and upsleeve mentioned in the embodiment of the present invention, the comparative data of the recognition accuracy of the pedestrian attribute recognition model and the model using other types of attention modules on pedestrian attributes are shown in Table 4.

[0114] Table 4 Comparison of recognition accuracy of pedestrian attributes

[0115] Model teenager upSleeve Resnet 0.8266 0.9182 Resnet+ collaboration 0.8358 0.9240 Resnet+HS-ULSAM-B 0.8577 0.9280 Resnet+HS-ULSAM-B+ collaboration 0.8664 0.9343

[0116] The above data show that the pedestrian attribute recognition model provided by the embodiment of the present invention is superior to the neural network model with the traditional structure of the attention mechanism in terms of Accuary, Recall, Precision, and F1-Score indicators, and the recognition accuracy of pedestrian attributes is also improved.

[0117] The beneficial effects achieved by the present invention are that the improved ULSAM attention module is added to the pedestrian attribute recognition model with channel interaction operation, and the pedestrian attributes are screened and the relationship is mined through the Apriori algorithm to change the structure and training method of the fully connected layer, so that the performance of the recognition model is enhanced and richer information can be recognized.

[0118] The present invention also provides a training system for a pedestrian attribute recognition model. Figure 9 , Figure 9 2 is a schematic diagram of the structure of a training system 200 for a pedestrian attribute recognition model provided by the present invention, comprising:

[0119] The attribute association module 201 is configured to obtain a data set comprising a plurality of pedestrian images and filter the data set using an Apriori algorithm to obtain a frequent item set, wherein each pedestrian image in the frequent item set comprises at least two pedestrian attributes;

[0120] An attribute conversion module 202 is used to convert the frequent itemsets into binary attributes to obtain a training data set;

[0121] A pre-training module 203 is configured to construct a pre-trained attribute recognition model including an improved ULSAM attention module, and input the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attributes, wherein the pre-trained attribute recognition model further includes a number of fully connected layers corresponding to the number of pedestrian attributes, and each fully connected layer corresponds to one of the pedestrian attributes;

[0122] A fully connected association module 204 is configured to determine the accuracy of a plurality of pedestrian attributes having strong correlations according to the pre-trained attribute recognition accuracy results and a preset association rule, and to connect the fully connected layers corresponding to the pedestrian attributes that meet a preset accuracy threshold;

[0123] The model output module 205 is used to output the pre-trained attribute recognition model that has completed the connection of the fully connected layer to obtain a pedestrian attribute recognition model.

[0124] The training system 200 for the pedestrian attribute recognition model can implement the steps in the training method for the pedestrian attribute recognition model in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not repeat it here.

[0125] The embodiment of the present invention also provides a computer device, please refer to Figure 10 , Figure 10 3 is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and a computer program stored in the memory 302 and executable on the processor 301.

[0126] The processor 301 calls the computer program stored in the memory 302 to execute the steps of the park management method provided by the embodiment of the present invention. Figure 1 , specifically including:

[0127] S101 , obtaining a data set including multiple pedestrian images, and filtering the data set using an Apriori algorithm to obtain a frequent item set, wherein each pedestrian image in the frequent item set includes at least two pedestrian attributes.

[0128] Furthermore, the step of obtaining a data set containing multiple pedestrian images and filtering the data set using the Apriori algorithm to obtain frequent itemsets is specifically as follows:

[0129] A preset support threshold and a preset confidence threshold of the Apriori algorithm are set according to the data set, and the pedestrian attributes in the data set are filtered and screened according to the preset support threshold and the preset confidence threshold to obtain the frequent item set, wherein at least some of the pedestrian attributes in the frequent item set have the strong correlation.

[0130] S102: Perform binary attribute conversion on the frequent itemsets to obtain a training data set.

[0131] Furthermore, the step of performing binary attribute conversion on the frequent itemsets to obtain a training data set includes the following sub-steps:

[0132] Converting the multi-valued attributes in the frequent itemset into binary attributes;

[0133] The original binary attributes in the frequent item set are retained and integrated with the result of the multi-value attribute conversion to obtain the training data set.

[0134] S103. Construct a pre-trained attribute recognition model including an improved ULSAM attention module, and input the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attributes, wherein the pre-trained attribute recognition model further includes fully connected layers corresponding to the number of the pedestrian attributes, and each of the fully connected layers corresponds to one of the pedestrian attributes.

[0135] Furthermore, the pre-trained attribute recognition model is based on the Resnet50 model, which includes an input layer, a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fully connected layer, and an output layer in a logical order. The first residual layer, the second residual layer, the third residual layer, and the fourth residual layer have interaction channels in pairs, wherein the first residual layer includes 3 residual modules, the second residual layer includes 4 residual modules, the third residual layer includes 6 residual modules, and the fourth residual layer includes 3 residual modules, and each of the residual modules is composed of 1*1 convolution, 3*3 convolution, and 1*1 convolution in a logical order.

[0136] Furthermore, the improved ULSAM attention module includes a spatial attention submodule and an SE-channel attention submodule, wherein the spatial attention submodule is located after the last 1*1 convolution of the first residual layer, and the SE-channel attention submodule is located after the output of the fully connected layer.

[0137] Furthermore, in the pre-trained attribute recognition model, a feature map output by the last 1*1 convolution of the first residual layer is defined as F, and F can be divided into g sub-feature maps corresponding to the number of the pedestrian attributes. Then, the sub-feature maps of the feature map F can be expressed as equation (1):

[0138] [F1,…F n ,…F g ] (1)

[0139] Where n and g are both positive integers;

[0140] Define the spatial attention submodule as F nThe spatial attention feature map F' output after processing the input n , then the spatial attention feature map F' n Satisfying the relationship (2):

[0141] F' n =sigmod(maxpool(F n ))*F n (2)

[0142] Define the channel connection and convolution operation of Resnet50 in the pre-trained attribute recognition model on the spatial attention feature map F' n The second feature map output after processing is F" n , then the second feature map F" n Satisfying the relationship (3):

[0143]

[0144] Define the SE-channel attention submodule to the second feature map F' n After the input is processed, the corrected feature map output by the output layer is Out, and the corrected feature map Out satisfies the relationship (4):

[0145] Out=FC2(Relu(FC1(maxpool(Concat[F”1,…F” n ,…F” g ])))) (4)

[0146] In the above formula, maxpool represents the maximum pooling operation in the channel dimension, sigmoid and Relu represent activation functions, Concat represents the channel connection operation, Conv represents the convolution operation with a convolution kernel size of 1*1, and FC represents the fully connected operation of the fully connected layer.

[0147] S104. Based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rules, and the fully connected layers corresponding to the multiple pedestrian attributes that meet the preset accuracy threshold are connected.

[0148] Furthermore, based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rule, and the fully connected layers corresponding to the multiple pedestrian attributes that meet the preset accuracy threshold are connected, including the following sub-steps:

[0149] Obtaining the pedestrian attribute recognition accuracy corresponding to each of the pedestrian attributes in the pre-trained attribute recognition accuracy result;

[0150] Define any one of the pedestrian attributes as a consequent attribute, define the other pedestrian attributes that have the strong correlation with the consequent attribute as antecedent attributes, and determine whether the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold, wherein:

[0151] If the pedestrian attribute recognition accuracy rate corresponding to the antecedent attribute is higher than the preset accuracy threshold, connecting the fully connected layer corresponding to the antecedent attribute with the fully connected layer corresponding to the consequent attribute;

[0152] If the pedestrian attribute recognition accuracy rate corresponding to the antecedent attribute is less than or equal to the preset accuracy threshold, no processing is performed.

[0153] S105 , outputting the pre-trained attribute recognition model that has completed the connection of the fully connected layer to obtain a pedestrian attribute recognition model.

[0154] The computer device 300 provided in the embodiment of the present invention can implement the steps in the training method of the pedestrian attribute recognition model in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0155] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes and steps in the training method of the pedestrian attribute recognition model provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0156] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0157] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0158] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0159] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only a preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A training method for a pedestrian attribute recognition model, characterized in that: The method comprises: Obtain a data set containing multiple pedestrian images, and filter the data set using an Apriori algorithm to obtain a frequent item set, wherein each pedestrian image in the frequent item set contains at least two pedestrian attributes; Performing binary attribute conversion on the frequent item sets to obtain a training data set; Constructing a pre-trained attribute recognition model including an improved ULSAM attention module, and inputting the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attributes, wherein the pre-trained attribute recognition model further includes a number of fully connected layers corresponding to the number of pedestrian attributes, and each fully connected layer corresponds to one of the pedestrian attributes; Based on the pre-trained attribute recognition accuracy result, the accuracy of the multiple pedestrian attributes with strong correlation is judged according to the preset association rules, and the fully connected layers corresponding to the pedestrian attributes that meet the preset accuracy threshold are connected; Outputting the pre-trained attribute recognition model connected to the fully connected layer to obtain a pedestrian attribute recognition model; Wherein, the pre-trained attribute recognition model is based on the Resnet50 model, and the Resnet50 model includes an input layer, a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fully connected layer, and an output layer in a logical order. There are interaction channels between the first residual layer, the second residual layer, the third residual layer, and the fourth residual layer. The first residual layer includes 3 residual modules, the second residual layer includes 4 residual modules, the third residual layer includes 6 residual modules, and the fourth residual layer includes 3 residual modules. Each of the residual modules is composed of 1*1 convolution, 3*3 convolution, and 1*1 convolution in a logical order. The improved ULSAM attention module includes a spatial attention submodule and an SE-channel attention submodule, wherein the spatial attention submodule is located after the last 1*1 convolution of the first residual layer, and the SE-channel attention submodule is located after the output of the fully connected layer; In the pre-trained attribute recognition model, a feature map output by the last 1*1 convolution of the first residual layer is defined as F, and F can be divided into g sub-feature maps corresponding to the number of pedestrian attributes. Then, the sub-feature maps of the feature map F can be expressed as relation (1): [F1,…F n ,…F g ](1) Where n and g are both positive integers; Define the spatial attention submodule as F n The spatial attention feature map output after processing the input is F' n , then the spatial attention feature map F' n Satisfying the relationship (2): F’ n =sigmod(maxpool(F n ))*F n (2) Define the channel connection and convolution operation of Resnet50 in the pre-trained attribute recognition model on the spatial attention feature map F' n The second feature map output after processing is F" n , then the second feature map F" n Satisfying the relationship (3): Define the SE-channel attention submodule to the second feature map F' n After the input is processed, the corrected feature map output by the output layer is Out, and the corrected feature map Out satisfies the relationship (4): Out=FC2(Relu(FC1(maxpool(Concat[F”1,…F” n ,…F” g ])))) (4) In the above equations (2), (3), and (4), maxpool represents the maximum pooling operation in the channel dimension, sigmoid and Relu represent activation functions, Concat represents the channel connection operation, Conv represents the convolution operation with a convolution kernel size of 1*1, and FC represents the fully connected operation of the fully connected layer.

2. The method for training a pedestrian attribute recognition model according to claim 1, wherein: The step of obtaining a data set containing multiple pedestrian images and filtering the data set using the Apriori algorithm to obtain frequent item sets is specifically as follows: A preset support threshold and a preset confidence threshold of the Apriori algorithm are set according to the data set, and the pedestrian attributes in the data set are filtered and screened according to the preset support threshold and the preset confidence threshold to obtain the frequent item set, wherein at least some of the pedestrian attributes in the frequent item set have the strong correlation.

3. The method for training a pedestrian attribute recognition model according to claim 1, wherein: The step of performing binary attribute conversion on the frequent itemsets to obtain a training data set includes the following sub-steps: Converting the multi-valued attributes in the frequent itemset into binary attributes; The original binary attributes in the frequent item set are retained and integrated with the result of the multi-value attribute conversion to obtain the training data set.

4. The method for training a pedestrian attribute recognition model according to claim 2, wherein: The step of judging the accuracy of the multiple pedestrian attributes having strong correlation according to the pre-trained attribute recognition accuracy result and connecting the fully connected layers corresponding to the multiple pedestrian attributes that meet the preset accuracy threshold includes the following sub-steps: Obtaining the pedestrian attribute recognition accuracy corresponding to each of the pedestrian attributes in the pre-trained attribute recognition accuracy result; Define any one of the pedestrian attributes as a consequent attribute, define the other pedestrian attributes that have the strong correlation with the consequent attribute as antecedent attributes, and determine whether the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold, wherein: If the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is higher than the preset accuracy threshold, connecting the fully connected layer corresponding to the antecedent attribute with the fully connected layer corresponding to the consequent attribute; If the pedestrian attribute recognition accuracy corresponding to the antecedent attribute is less than or equal to the preset accuracy threshold, no processing is performed.

5. A training system for a pedestrian attribute recognition model, characterized in that: include: An attribute association module is used to obtain a data set containing multiple pedestrian images and filter the data set using an Apriori algorithm to obtain a frequent item set, where each pedestrian image in the frequent item set contains at least two pedestrian attributes; An attribute conversion module is used to convert the binary attributes of the frequent itemsets to obtain a training data set; A pre-training module is used to construct a pre-trained attribute recognition model including an improved ULSAM attention module, and input the training data set into the pre-trained attribute recognition model to obtain a pre-trained attribute recognition accuracy result for the pedestrian attribute, wherein the pre-trained attribute recognition model further includes a number of fully connected layers corresponding to the number of pedestrian attributes, and each fully connected layer corresponds to one of the pedestrian attributes; A fully connected association module is used to determine the accuracy of multiple pedestrian attributes with strong correlation according to the pre-trained attribute recognition accuracy results and a preset association rule, and to connect the fully connected layers corresponding to the pedestrian attributes that meet a preset accuracy threshold; A model output module, configured to output the pre-trained attribute recognition model after completing the connection of the fully connected layer to obtain a pedestrian attribute recognition model; Wherein, the pre-trained attribute recognition model is based on the Resnet50 model, and the Resnet50 model includes an input layer, a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fully connected layer, and an output layer in a logical order. There are interaction channels between the first residual layer, the second residual layer, the third residual layer, and the fourth residual layer. The first residual layer includes 3 residual modules, the second residual layer includes 4 residual modules, the third residual layer includes 6 residual modules, and the fourth residual layer includes 3 residual modules. Each of the residual modules is composed of 1*1 convolution, 3*3 convolution, and 1*1 convolution in a logical order. The improved ULSAM attention module includes a spatial attention submodule and an SE-channel attention submodule, wherein the spatial attention submodule is located after the last 1*1 convolution of the first residual layer, and the SE-channel attention submodule is located after the output of the fully connected layer; In the pre-trained attribute recognition model, a feature map output by the last 1*1 convolution of the first residual layer is defined as F, and F can be divided into g sub-feature maps corresponding to the number of pedestrian attributes. Then, the sub-feature maps of the feature map F can be expressed as relation (1): [F1,…F n ,…F g ](1) Where n and g are both positive integers; Define the spatial attention submodule as F n The spatial attention feature map output after processing the input is F' n , then the spatial attention feature map F' n Satisfying the relationship (2): F’ n =sigmod(maxpool(F n ))*F n (2) Define the channel connection and convolution operation of Resnet50 in the pre-trained attribute recognition model on the spatial attention feature map F' n The second feature map output after processing is F" n , then the second feature map F" n Satisfying the relationship (3): Define the SE-channel attention submodule to the second feature map F' n After the input is processed, the corrected feature map output by the output layer is Out, and the corrected feature map Out satisfies the relationship (4): Out=FC2(Relu(FC1(maxpool(Concat[F”1,…F” n ,…F” g ])))) (4) In the above equations (2), (3), and (4), maxpool represents the maximum pooling operation in the channel dimension, sigmoid and Relu represent activation functions, Concat represents the channel connection operation, Conv represents the convolution operation with a convolution kernel size of 1*1, and FC represents the fully connected operation of the fully connected layer.

6. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the method for training a pedestrian attribute recognition model as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for training a pedestrian attribute recognition model as described in any one of claims 1 to 4.

Images