Pedestrian attribute recognition system and method based on implicit dynamic feature allocation

Abstract

The present invention discloses a pedestrian attribute recognition system and method based on implicit dynamic feature allocation, which belongs to the field of computer vision technology and is characterized in that it includes the following steps: S1, preprocessing the input image; S2, extracting image features using an improved Transformer feature extractor to obtain a feature vector; S3, dynamically allocating the feature vector using a feature dynamic allocation module; S4, using a pedestrian attribute recognition network to perform attribute recognition on pedestrians in pedestrian images, and calculating error loss and accuracy, thereby helping to train the network. The present invention focuses on dynamically matching the high-dimensional features extracted by the Transformer-based feature extractor with pedestrian attributes, allowing each attribute to use its own features to perform a binary classification task, making the classification task of each attribute relatively independent, and better improving the recognition effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a pedestrian attribute recognition system and method based on implicit dynamic feature allocation. Background Art

[0002] Pedestrian attribute recognition refers to the use of computer vision and other technologies to classify the appearance attributes of people in pedestrian images, such as clothing, accessories, hair length and color. It is a branch of multi-label classification problems in practical applications. In the actual needs of suspect tracking and missing person notices in road monitoring scenarios, pedestrian attribute recognition plays a very important role and is widely used. Pedestrian attribute recognition can also be supplemented by technologies such as pedestrian re-identification and gait recognition, and play a role in all walks of life. In the field of pedestrian attribute recognition technology, most methods are based on deep learning convolutional neural networks. For example, convolutional neural networks are built using multiple layers of convolutional kernels of different sizes to extract features from input images, and finally the extracted high-level features are sent to the classifier through the fully connected layer for multi-classification recognition; there are also methods that change the multi-label multi-classification recognition of the last layer into multiple binary classification recognition, that is, convert the multi-classification problem into multiple binary classification problems, and classify each attribute separately and summarize it; there are also methods that divide human body parts into several torso blocks, such as head, upper body, and lower body, and classify and recognize each part separately to obtain more accurate results.

[0003] Most existing pedestrian attribute recognition technologies directly use the extracted high-dimensional features for attribute classification, or use the attention mechanism to make the model extract different features for different parts, but these methods cannot match the features with the attributes of pedestrians well, and do not assign more appropriate high-dimensional features to each attribute. In addition, most current pedestrian attribute recognition technologies use convolutional neural networks to extract features, but convolutional neural networks are prone to losing global information during the layer-by-layer sampling process and cannot perform well on some attributes. Summary of the invention

[0004] Technical Purpose

[0005] The present invention provides a pedestrian attribute recognition system and method based on implicit dynamic feature allocation; it focuses on dynamically matching the high-dimensional features extracted by a Transformer-based feature extractor with pedestrian attributes, allowing each attribute to use its own features to perform a binary classification task, making the classification task of each attribute relatively independent, and better improving the recognition effect.

[0006] Technical Solution

[0007] The first object of the present invention is to provide a pedestrian attribute recognition method based on implicit dynamic feature allocation, performing the following steps:

[0008] S1, preprocess the input image;

[0009] S2, using the improved Transformer feature extractor to extract image features and obtain feature vectors;

[0010] S3, dynamically allocating feature vectors using a feature dynamic allocation module;

[0011] S4. Use the pedestrian attribute recognition network to identify the attributes of pedestrians in pedestrian images, and calculate the error loss and accuracy to help train the network.

[0012] Preferably, the S1 is specifically:

[0013] S11, collect monitoring pictures, use the target detection algorithm to detect and crop pedestrians, obtain pictures containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3;

[0014] S12, screening and labeling the images, labeling each pedestrian image with 20-40 attribute values;

[0015] S13. Preprocess the pedestrian images, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise.

[0016] Preferably, S2 is specifically:

[0017] S21. Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain an image sequence I, which is expressed as:

[0018] {p1,p2,…,p 196}∈I

[0019] Afterwards, a 3*16*16 pixel classification token image block is added to the head of the image sequence I. The content of the classification token image block is random pixels, denoted as p0. The newly obtained image sequence P is expressed as:

[0020] {p0,p1,p2,…,p 196}∈P

[0021] After that, a position code representing the position information is added to each picture block of the picture sequence P. The size of the position code is the same as the size of the picture block. The initialization of the position code is expressed as follows:

[0022]

[0023]

[0024] Among them, pos represents the position of the image block p in the image sequence P, i represents the dimension of the position encoding, and the value range of i is: [0,...,d model / 2],d model is the hidden layer depth of the network, and its value is 512;

[0025] The position code is added to the elements of the picture sequence P to obtain a new picture sequence P′, which is expressed as:

[0026] P′=[p0,p1,…,p 196 ]+PE;

[0027] S22. Use the encoder Φ to extract features from the processed image sequence P′ to obtain a high-dimensional vector X of 197*384 dimensions, expressed as:

[0028] X = φ(P′);

[0029] The first dimension vector of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, called the classification feature vector (x cls ), expressed as:

[0030] x cls =X[0].

[0031] Preferably, S3 is specifically:

[0032] S31. Dynamic feature allocation is realized using k allocation blocks. Each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below:

[0033] x mcls =Linear(GELU(Linear(x cls )))

[0034] Among them: Linear is the fully connected layer, GELU is the activation function;

[0035] S32, k allocation blocks calculate k x_{mcls} respectively, concatenate them, and obtain the high-dimensional feature vector that finally expresses the characteristics of the image, which is expressed as follows:

[0036]

[0037] Preferably, S4 is specifically:

[0038] S41. Use multiple double-layer fully connected neural networks to classify the obtained k classification feature vectors, expressed as:

[0039] y = sigmoid(LeakyReLU(BN(FC(xmls ))));

[0040] Among them, sigmoid represents the activation function that finally realizes classification. Its role is to classify the feature vector into two categories and obtain the prediction result of yes or no. The role of the LeakyReLU activation function is to make the neural network nonlinear and enhance the expression ability of the linear network. The BN operation normalizes the distribution of the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence speed, controls the gradient explosion and gradient disappearance, and reduces overfitting. FC is a two-layer fully connected neural network.

[0041] S42, according to the predicted value y i and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows:

[0042]

[0043] Where: N represents the number of samples, M represents the number of attributes of the samples, and w l =exp(-p / θ 2 ) represents the weight of the lth attribute, p represents the proportion of positive samples in the training data set to all samples, and θ is the adjustment parameter.

[0044] A second object of the present invention is to provide a pedestrian attribute recognition system based on implicit dynamic feature allocation, comprising at least:

[0045] Image preprocessing module, which preprocesses the input image;

[0046] The feature extraction module uses the improved Transformer feature extractor to extract image features and obtain feature vectors;

[0047] A feature dynamic allocation module is used to dynamically allocate feature vectors;

[0048] The feature classification module uses the pedestrian attribute recognition network to identify the attributes of pedestrians in pedestrian images, and calculates the error loss and accuracy to help train the network.

[0049] Preferably, the preprocessing is specifically:

[0050] Collect surveillance images, use the target detection algorithm to detect and crop pedestrians, obtain images containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3;

[0051] Screen and label the images, with 20-40 attribute values ​​for each pedestrian image;

[0052] Pedestrian images are preprocessed, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise.

[0053] Preferably, feature extraction is specifically:

[0054] Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain the image sequence I, which is expressed as:

[0055] {p1,p2,…,p 196}∈I

[0056] Afterwards, a 3*16*16 pixel classification token image block is added to the head of the image sequence I. The content of the classification token image block is random pixels, denoted as p0. The newly obtained image sequence P is expressed as:

[0057] {p0,p1,p2,…,p 196}∈P

[0058] After that, a position code representing the position information is added to each picture block of the picture sequence P. The size of the position code is the same as the size of the picture block. The initialization of the position code is expressed as follows:

[0059]

[0060]

[0061] Among them, pos represents the position of the image block p in the image sequence P, i represents the dimension of the position encoding, and the value range of i is: [0,...,d model / 2],d model is the hidden layer depth of the network, and its value is 512;

[0062] The position code is added to the elements of the picture sequence P to obtain a new picture sequence P′, which is expressed as:

[0063] P′=[p0,p1,…,p 196 ]+PE;

[0064] Use the encoder Φ to extract features from the processed image sequence P′ to obtain a 197*384-dimensional high-dimensional vector X, expressed as:

[0065] X = φ(P′);

[0066] The first dimension vector of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, called the classification feature vector (x cls ), expressed as:

[0067] x cls =X[0].

[0068] Preferably, the dynamic allocation of features is specifically as follows:

[0069] Dynamic feature allocation is achieved using k allocation blocks. Each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below:

[0070] x mcls =Linear(GELU(Linear(x cls )))

[0071] Among them: Linear is the fully connected layer, GELU is the activation function;

[0072] The k allocation blocks calculate k x_{mcls} respectively, concatenate them, and finally obtain the high-dimensional feature vector that expresses the characteristics of the image, which is expressed as follows:

[0073]

[0074] The beneficial effect of this further solution is: classifying the obtained feature vectors into corresponding attribute features to obtain the final result.

[0075] Preferably, the feature classification is specifically:

[0076] Using multiple two-layer fully connected neural networks, the obtained k classification feature vectors are classified, expressed as:

[0077] y = sigmoid(LeakyReLU(BN(FC(x mls ))));

[0078] Among them, sigmoid represents the activation function that finally realizes classification. Its role is to classify the feature vector into two categories and obtain the prediction result of yes or no. The role of the LeakyReLU activation function is to make the neural network nonlinear and enhance the expression ability of the linear network. The BN operation normalizes the distribution of the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence speed, controls the gradient explosion and gradient disappearance, and reduces overfitting. FC is a two-layer fully connected neural network.

[0079] According to the predicted value y i and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows:

[0080]

[0081] Where: N represents the number of samples, M represents the number of attributes of the samples, and w l =exp(-p / θ 2 ) represents the weight of the lth attribute, p represents the proportion of positive samples in the training data set to all samples, and θ is the adjustment parameter.

[0082] The advantages and positive effects of the present invention are:

[0083] The present invention uses the features obtained by the feature extractor to perform dynamic feature allocation, so that a single feature discrimination information represents multiple discriminant features, and uses the attention mechanism to allocate weights between multiple discriminant features, establish a dependency relationship between the discriminant features, and significantly improve the recognition effect of pedestrian attributes.

[0084] The present invention focuses on dynamically matching the high-dimensional features extracted by the Transformer-based feature extractor with pedestrian attributes, allowing each attribute to use its own features to perform a binary classification task, making the classification task of each attribute relatively independent, and better improving the recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 is a flow chart of a preferred embodiment of the present invention;

[0086] Figure 2 A schematic diagram of a pedestrian attribute recognition network model structure in an embodiment of the present invention;

[0087] Figure 3 Schematic diagram of the implicit dynamic feature allocation module structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] In order to further understand the content, features and effects of the present invention, the following embodiments are given and described in detail as follows.

[0089] See also Figure 1 , a pedestrian attribute recognition method based on implicit dynamic feature allocation, including the following steps:

[0090] S1: Preprocess the input image; specifically includes the following steps:

[0091] S11: Use cameras and other equipment to collect road monitoring pictures, use target detection and other algorithms to detect and crop pedestrians, obtain pictures containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3;

[0092] S12: Use manual methods to screen and label the images, with 20-30 attribute values ​​annotated for each pedestrian image;

[0093] S13: Preprocess the pedestrian images, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise.

[0094] S2: Use the improved Transformer feature extractor to extract image features and obtain feature vectors; specifically, it includes the following steps:

[0095] S21: Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain an image sequence I, which is expressed as:

[0096] {p1,p2,…,p 196}∈I;

[0097] After that, a newly generated 3*16*16 pixel image block is added to the head of the image sequence I. The content of the image block is random pixels, called the classification token image block, denoted as p0. The newly obtained image sequence P is expressed as:

[0098] {p0,p1,p2,…,p 196}∈P;

[0099] After that, a code representing the position information is added to each picture block of the picture sequence P. The size of the position code is the same as the picture block, indicating the position of the picture block in the picture sequence, which is called the position code (PE). The initialization of the position code is as follows:

[0100]

[0101]

[0102] Among them, pos represents the position of the picture block p in the picture sequence P, i represents the dimension of PE, and the value range of i is: [0,...,d model / 2), d model is the hidden layer depth of the network, and its value is 512.

[0103] The position code is added to the elements of the original picture sequence P to obtain a new picture sequence P′, which is expressed as:

[0104] P′=[p0,p1,…,p 196 ]+PE;

[0105] S22: Use encoder Φ to extract features from the processed image sequence P′, and obtain a 197*384-dimensional high-dimensional vector X, which is expressed as:

[0106] X = φ(P′);

[0107] The first dimension vector of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, called the classification feature vector (x cls ), expressed as:

[0108] x cls =X[0].

[0109] S3: Use the feature dynamic allocation module to dynamically allocate feature vectors; specifically, it includes the following steps:

[0110] S31: Dynamic feature allocation is implemented using k allocation blocks, where each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below:

[0111] x mcls =Linear(GELU(Linear(x cls )))

[0112] Among them, Linear is the fully connected layer and GELU is the activation function.

[0113] S32: The k allocation blocks calculate k x_{mcls} respectively, concatenate them, and obtain the high-dimensional feature vector that finally expresses the characteristics of the image, which is expressed as follows:

[0114]

[0115] S4: Use the pedestrian attribute recognition network to identify the attributes of pedestrians in the pedestrian image and calculate indicators such as error loss and accuracy to help train the network; specifically, it includes the following steps:

[0116] S41: Use multiple two-layer fully connected neural networks to obtain k classification feature vectors for classification, expressed as:

[0117] y = sigmoid(LeakyReLU(BN(FC(x mls ))))

[0118] Among them, sigmoid represents the activation function that finally realizes classification, which is used to classify the feature vector into two categories and obtain the prediction result of yes or no. The LeakyReLU activation function is used to make the neural network nonlinear and enhance the expression ability of the linear network. The BN operation normalizes the distribution of the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence speed, controls the gradient explosion and gradient disappearance, and reduces overfitting. FC is a two-layer fully connected neural network.

[0119] S42: According to the predicted value y i and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows:

[0120]

[0121] Where N represents the number of samples, M represents the number of attributes of the samples, and w l =exp(-p / θ 2 ) represents the weight of the lth attribute, where p represents the proportion of positive samples in the sample to all samples, and θ is the adjustment parameter, which is set to 1 in the experiment.

[0122] A pedestrian attribute recognition system based on implicit dynamic feature allocation, which implements the above-mentioned pedestrian attribute recognition method based on implicit dynamic feature allocation, includes: an image preprocessing module, a feature extraction module, a feature dynamic allocation module, and a feature classification module; wherein:

[0123] The image preprocessing module: the input pedestrian image is uniformly cropped into a 224*224 pixel RGB image, and in order to enhance the robustness of the model, the image is enhanced using scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise methods, and the enhanced image replaces the original image input.

[0124] The feature extraction module uses the improved Transformer as a feature extractor to extract the feature vector of the original image layer by layer into a high-dimensional feature vector.

[0125] The feature dynamic classification module allocates the features extracted by the above module to different attributes according to the different feature focus parts, so that each attribute has several feature vectors.

[0126] The feature classification module performs multiple binary classifications on the features obtained by the above modules, and finally obtains the predicted values ​​of all attributes, and compares the predicted values ​​with the true values ​​to obtain the prediction accuracy;

[0127] Testing module: Use the trained pedestrian attribute recognition model to predict the attributes of the test images, compare the results with the true values, and obtain various test indicators.

[0128] As a preferred technical solution, the image preprocessing module includes: random cropping, random horizontal flipping, random rotation, random color deviation, random erasing, and random noise, and the parameters can be changed according to actual conditions.

[0129] As a preferred technical solution, the feature extraction module uses the Transformer model as the feature extractor and uses the last layer of the network as the output layer.

[0130] As a preferred technical solution, the feature classification module uses multiple two-layer fully connected classifiers, and the number of classifiers is consistent with the number of pedestrian attributes.

[0131] The technical solution of the present invention mainly includes the following steps S1 to S4:

[0132] S1: preprocess the acquired images;

[0133] In this example, step S1 specifically includes the following sub-steps:

[0134] S11: Use cameras and other equipment to collect road monitoring pictures, use target detection and other algorithms to detect and crop pedestrians, obtain pictures containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3;

[0135] S12: Use manual methods to screen and label the images, with 20-30 attribute values ​​annotated for each pedestrian image;

[0136] S13: Preprocess the pedestrian images, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise.

[0137] The beneficial effects of this further solution are: expanding the training data, avoiding overfitting caused by single data, and improving the generalization of the pedestrian attribute recognition network.

[0138] The step S2 uses an improved Transformer feature extractor to extract image features, specifically including the following steps:

[0139] S21: Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks of 3*16*16 pixels, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain an image sequence I, which is expressed as:

[0140] {p1,p2,…,p 196}∈I

[0141] After that, a newly generated 3*16*16 pixel image block is added to the head of the image sequence I. The content of the image block is random pixels, called the classification token image block, denoted as p0. The newly obtained image sequence P is expressed as:

[0142] {p0,p1,p2,…,p 196}∈P

[0143] After that, a code representing the position information is added to each picture block of the picture sequence P. The code size is the same as the picture block, indicating the position of the picture block in the picture sequence, which is called position code (PE). The initialization of the position code is as follows:

[0144]

[0145]

[0146] Among them, pos represents the position of the picture block p in the picture sequence P, i represents the dimension of PE, and the value range of i is: [0,…,d model / 2), d model is the hidden layer depth of the network, and its value is 512.

[0147] The position code is added to the elements of the original picture sequence P to obtain a new picture sequence P′, which is expressed as:

[0148] P′=[p0,p1,…,p 196 ]+PE

[0149] S22: Use encoder φ to extract features from the processed image sequence P′, and obtain a 197*384-dimensional high-dimensional vector X, which is expressed as:

[0150] X=φ(P′)

[0151] The first dimension vector of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, called the classification feature vector (x cls ), expressed as:

[0152] x cls =X[0]

[0153] Furthermore, the step S22 specifically includes the following sub-steps:

[0154] S221: Use a multi-head self-attention network structure to convert the input image sequence into a high-dimensional feature sequence with feature information. First, calculate the q value, k value, and v value of each vector in the sequence. The calculation method is as follows:

[0155] q i =Wq a i

[0156] k i =W k a i

[0157] v i =W v a i

[0158] Among them, i represents the position of the vector in the sequence, W q ,W k ,W v Represent three different conversion weights respectively.

[0159] S222: Use each q vector to perform attention calculation with each k vector to obtain the attention value α between the two vectors i and j ij , the attention value is calculated as follows:

[0160]

[0161] Where d is the dimension of q and k. Since the value of q·k increases with the increase of d, we use The results can be normalized. The softmax function is an activation function, which maps the numerical sequence to the interval (0,1) and the sum is 1.

[0162] S223: Calculate α {ij} and v j The product of , gets the output vector b, the calculation process is as follows:

[0163]

[0164] After multiple iterative calculations, the high-dimensional feature vector X representing the image sequence P can be obtained.

[0165] S224: Take out the first dimension vector of the high-dimensional vector as the high-dimensional feature representing the entire image, called the classification feature vector (x cls ), expressed as:

[0166] x cls =X[0]

[0167] The step S3 specifically includes the following sub-steps:

[0168] S31: Dynamic feature allocation is implemented using k allocation blocks, where each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below:

[0169] x mcls=Linear(GELU(Linear(x cls )))

[0170] Among them, Linear is the fully connected layer and GELU is the activation function.

[0171] S32: The k allocation blocks calculate k x_{mcls} respectively, concatenate them, and obtain the high-dimensional feature vector that finally expresses the characteristics of the image, which is expressed as follows:

[0172]

[0173] The beneficial effect of this further solution is: classifying the obtained feature vectors into corresponding attribute features to obtain the final result.

[0174] Furthermore, the step S4 specifically includes the following steps:

[0175] S41: Use multiple two-layer fully connected neural networks to obtain k classification feature vectors for classification, expressed as:

[0176] y = sigmoid(LeakyReLU(BN(FC(x mls ))))

[0177] Among them, sigmoid represents the activation function that finally realizes classification, which is used to classify the feature vector into two categories and obtain the prediction result of yes or no. The LeakyReLU activation function is used to make the neural network nonlinear and enhance the expression ability of the linear network. The BN operation normalizes the distribution of the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence speed, controls the gradient explosion and gradient disappearance, and reduces overfitting. FC is a two-layer fully connected neural network.

[0178] S42: According to the predicted value y i and truth value Calculate the loss function L of each sample in this single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows:

[0179]

[0180] Where N represents the number of samples, M represents the number of attributes of the samples, and w l =exp(-p / θ 2 ) represents the weight of the lth attribute, where p represents the proportion of positive samples in the sample to all samples, and θ is the adjustment parameter, which is set to 1 in the experiment.

[0181] In scale normalization, the pixels of all images are changed from original pixels to set pixels. In this method, the set pixels are 224*224. If the original image pixels are larger than the set pixels, the compression method is used to change the pixels; if the original pixels are smaller than the set pixels, the bilinear interpolation method is used to expand the elements to the set pixels.

[0182] In the random horizontal flip, the random value is set to 0.5, which means that half of the graphics are randomly flipped horizontally to replace the original image.

[0183] In random rotation, the random value is set to 0.5 and the rotation angle is set to ±20 degrees, that is, half of the images are randomly deviated from the center of the image at an angle of 20 degrees.

[0184] In the random color deviation, set the random value to 0.5, the hue value to ±0.5 degrees, and the saturation to ±0.5, which means that half of the images will be randomly raised or lowered by 0.5 degrees in hue and saturation.

[0185] In random erasing, the random value is set to 0.5 and the erasing value is set to 0.1, that is, 10% of the area of ​​half of the image is randomly erased and the pixel value of the target area is replaced with 0.

[0186] In the random noise, the random value is set to 0.5 and the noise range is set to 0.1, that is, the pixel values ​​in 10% of the area in half of the image are randomly replaced with random values ​​of 0-255.

[0187] S2: Use the improved Transformer feature extractor to extract image features and obtain feature vectors;

[0188] In this embodiment, the present invention first constructs a pedestrian attribute recognition model including an image embedding module, an encoder module and a feature dynamic allocation module in the Transformer. The model structure is as follows: Figure 2 shown.

[0189] The pedestrian attribute recognition model constructed by the present invention includes a picture embedding module (Patch Embedding), an encoder (Encoder) and a feature dynamic allocation module (LDT). They correspond to the following sub-steps respectively:

[0190] S21: The image embedding module takes the Transformer-base backbone network as an example. The backbone network consists of five parts, namely the image segmentation module, the linear mapping module, and the position information encoding module. The specific implementation of each module is as follows:

[0191] Given a pedestrian image x preprocessed by step S1, with a pixel size of 3*224*224, the image segmentation module divides the image into 196 16*16 pixel blocks, with no overlap between each block.

[0192] Subsequently, the linear mapping module uses a convolutional neural network to map the 196 3*16*16 image blocks into a 196*768-dimensional vector I. The convolution kernel size in the convolution layer is 3*16*16, the step size is 16, there is no zero padding, and the number is 768. The 196*768-dimensional vector I after the convolution operation replaces the original input image and can be expressed as:

[0193] {p1,p2,…,p 196}∈I,I∈R 196*768

[0194] In order to reduce the computational dimension of subsequent steps and enhance the feature expression capability, a newly generated 3*16*16 pixel image block is added to the head of the image sequence I. The content of the image block is random pixels, called the classification token image block, denoted as p_0. The newly obtained image sequence is expressed as:

[0195] {p0,p1,p2,…,p 196}∈P,P∈R 197*768

[0196] Afterwards, since the image block module in the previous step does not consider the order between the image blocks, that is, the order in the image blocks does not affect the vector representation result, the vector loses the information of the elements in the position image. Therefore, the position information encoding module is used to initialize a position vector PE∈R 1*196 , which can be expressed as:

[0197]

[0198]

[0199] Among them, pos represents the position of the picture block p in the picture sequence I, i represents the dimension of PE, and the value range of i is: [0,…,d model / 2), d model is the hidden layer depth of the network, which is 512. The vector P is added to the vector PE to obtain a new vector P′ of 196*768 dimensions, which is expressed as follows:

[0200] P′=[p0,p1,…,p 196 ]+PE

[0201] S22: Encoder module, taking the self-attention module as an example, the image sequence P∈R obtained in the above steps 197*768 Send it to the self-attention module, and after feature extraction through attention, a new feature vector X∈R is obtained 197*768 , and after 12 iterations, it is expressed as follows:

[0202] X=φ(P′)

[0203] Since an additional classification token image block is added in the above step S21, the classification token image block needs to be taken out in this step to represent the entire image vector, which is expressed as follows:

[0204] x cls =X[0],x cls ∈R 1*768

[0205] S3: Use the feature dynamic allocation module to dynamically allocate feature vectors, and use k allocation blocks to implement dynamic feature allocation, where each allocation block consists of two layers of fully connected neural networks, where the number of nodes in the input layer of the fully connected neural network is 768, the number of nodes in the first hidden layer is 384, and the number of output nodes in the second layer is 1. In this design implementation, k=64, which is expressed as follows:

[0206] x mcls =Linear(GELU(Linear(x cls )))x mcls ∈R 1*1

[0207] Linear is the fully connected layer and GELU is the activation function. Then, k allocation blocks calculate k x_{mcls} respectively, concatenate them, and finally obtain the high-dimensional feature vector x∈R that expresses the characteristics of the image. k , which is expressed as follows:

[0208]

[0209] S4: Use the pedestrian attribute recognition network to identify the attributes of pedestrians in pedestrian images and calculate indicators such as error loss and accuracy to help train the network.

[0210] After obtaining the final x that expresses the characteristics of the image, a fully connected layer is used to classify its attributes. The number of input nodes of the fully connected layer is k, and the number of output layer nodes is the number of attributes in the data set. For example, in the PEAT data set, the number of attributes is 35, that is, the number of output layer nodes is 35, which is expressed as:

[0211] y=sigmoid(LeakyReLU(BN(FC(x))))

[0212] Among them, sigmoid represents the activation function that finally realizes classification, which is used to classify the feature vector into two categories and obtain the prediction result of yes or no. The LeakyReLU activation function is used to make the neural network nonlinear and enhance the expression ability of the linear network. The BN operation normalizes the distribution of the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence speed, controls the gradient explosion and gradient disappearance, and reduces overfitting. FC is a two-layer fully connected neural network.

[0213] According to the predicted value y i and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows:

[0214]

[0215] Where N represents the number of samples, M represents the number of attributes of the samples, and w l =exp(-p / θ 2 ) represents the weight of the lth attribute, where p represents the proportion of positive samples in the sample to all samples, and θ is the adjustment parameter, which is set to 1 in the experiment.

[0216] The accuracy rate uses five indicators, namely mA, Accuracy, Precision, Recall and F1, which are calculated as follows:

[0217]

[0218]

[0219]

[0220]

[0221]

[0222] Among them, P, N, TP, TN, FP, and FN represent the total number of positive values, the total number of negative values, and the number of true positives, true negatives, false positives, and false negatives, respectively.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0224] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are within the scope of the technical solution of the present invention.

Claims

1. A pedestrian attribute recognition method based on implicit dynamic feature allocation, characterized in that: The steps include: S1. Preprocess the input image; specifically: S11, collect monitoring pictures, use the target detection algorithm to detect and crop pedestrians, obtain pictures containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3; S12, screening and labeling the images, labeling each pedestrian image with 20-40 attribute values; S13, preprocessing the pedestrian image, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise; S2. Use the improved Transformer feature extractor to extract image features and obtain feature vectors; specifically: S21. Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain an image sequence I, which is expressed as: Afterwards, a 3*16*16 pixel classification token image block is added to the head of the image sequence I. The content of the classification token image block is random pixels, denoted as p0. The newly obtained image sequence P is expressed as: After that, a position code representing the position information is added to each picture block of the picture sequence P. The size of the position code is the same as the size of the picture block. The initialization of the position code is expressed as follows: ； ； Among them, pos represents the position of the picture block p in the picture sequence P, i represents the dimension of the position encoding, and the value range of i is: , is the hidden layer depth of the network, and its value is 512; The position code is added to the elements of the image sequence P to obtain a new image sequence , expressed as: ； S22. Use encoder , for the processed image sequence Perform feature extraction to obtain a high-dimensional vector X of 197*384 dimensions, expressed as: ； The first dimension of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, which is called the classification feature vector ( ), expressed as: ； S3. Use the feature dynamic allocation module to dynamically allocate feature vectors; specifically: S31. Use The allocation blocks realize dynamic feature allocation. Each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below: in: is the fully connected layer, is the activation function; S32, k Each allocation block is calculated k indivual , concatenate them to get the high-dimensional feature vector that finally expresses the characteristics of the image, which is expressed as follows: ； S4. Use the pedestrian attribute recognition network to identify the attributes of pedestrians in pedestrian images, and calculate the error loss and accuracy to help train the network.

2. The pedestrian attribute recognition method based on implicit dynamic feature allocation according to claim 1 is characterized in that: S4 is specifically: S41, using multiple double-layer fully connected neural networks, the obtained k Classification feature vectors are classified, expressed as: ； in, sigmoid It represents the activation function that finally realizes the classification. Its function is to classify the feature vector into two categories and obtain the prediction result of yes or no. The role of the activation function is to make the neural network nonlinear and enhance the expressive power of the linear network. The operation normalizes the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence, controls gradient explosion and gradient disappearance, and reduces overfitting. It is a two-layer fully connected neural network; S42, according to the predicted value and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows: ； in: represents the number of samples, Indicates the number of attributes of the sample, Indicates The weight of the attribute, It indicates the proportion of positive samples in the training data set to all samples. To adjust the parameters.

3. A pedestrian attribute recognition system based on implicit dynamic feature allocation, characterized in that: At least: The image preprocessing module preprocesses the input image. The preprocessing is as follows: Collect surveillance images, use the target detection algorithm to detect and crop pedestrians, obtain images containing only pedestrians as a data set, and divide the data set into a training set and a test set in a ratio of 7:3; Screen and label the images, with 20-40 attribute values ​​for each pedestrian image; Preprocess pedestrian images, including scale normalization, random horizontal flipping, random rotation, random color deviation, random erasure, and random noise; The feature extraction module uses the improved Transformer feature extractor to extract image features and obtain feature vectors. The feature extraction is as follows: Using the image block embedding structure, the pedestrian images to be detected in the training set are divided into 196 image blocks, and the 196 image blocks are arranged in order from top to bottom and from left to right to obtain the image sequence I, which is expressed as: Afterwards, a 3*16*16 pixel classification token image block is added to the head of the image sequence I. The content of the classification token image block is random pixels, denoted as p0. The newly obtained image sequence P is expressed as: After that, a position code representing the position information is added to each picture block of the picture sequence P. The size of the position code is the same as the size of the picture block. The initialization of the position code is expressed as follows: ； ； Among them, pos represents the position of the picture block p in the picture sequence P, i represents the dimension of the position encoding, and the value range of i is: , is the hidden layer depth of the network, and its value is 512; The position code is added to the elements of the image sequence P to obtain a new image sequence , expressed as: ； Using the Encoder , for the processed image sequence Perform feature extraction to obtain a high-dimensional vector X of 197*384 dimensions, expressed as: ； The first dimension of the high-dimensional vector is taken out as the high-dimensional feature representing the entire image, which is called the classification feature vector ( ), expressed as: ； The feature dynamic allocation module is used to dynamically allocate feature vectors. The specific features of the dynamic allocation are as follows: use The allocation blocks realize dynamic feature allocation. Each allocation block consists of two layers of fully connected neural networks, where the number of hidden layer nodes is 384, as shown below: in: is the fully connected layer, is the activation function; k Each allocation block is calculated k indivual , concatenate them to get the high-dimensional feature vector that finally expresses the characteristics of the image, which is expressed as follows: ； The feature classification module uses the pedestrian attribute recognition network to identify the attributes of pedestrians in pedestrian images, and calculates the error loss and accuracy to help train the network.

4. The pedestrian attribute recognition system based on implicit dynamic feature allocation according to claim 3 is characterized in that: The feature classification is as follows: Using multiple two-layer fully connected neural networks, the obtained k Classification feature vectors are classified, expressed as: ； in, sigmoid It represents the activation function that finally realizes the classification. Its function is to classify the feature vector into two categories and obtain the prediction result of yes or no. The role of the activation function is to make the neural network nonlinear and enhance the expressive power of the linear network. The operation normalizes the input value of each layer of the neural network to a standard normal distribution with a mean of 0 and a variance of 1, which speeds up the network convergence, controls gradient explosion and gradient disappearance, and reduces overfitting. It is a two-layer fully connected neural network; According to the predicted value and truth value Calculate the loss function L of each sample in a single training, use the weighted binary cross entropy loss function, use the obtained loss function to calculate the gradient, train the network, and the loss function is calculated as follows: ； in: represents the number of samples, Indicates the number of attributes of the sample, Indicates The weight of the attribute, It indicates the proportion of positive samples in the training data set to all samples. To adjust the parameters.

Images