Pedestrian Attribute Recognition Method, Recognition Device, Electronic Device, and Storage Medium

Abstract

This application is applicable to the field of intelligent recognition technology, and provides a pedestrian attribute recognition method, a recognition device, an electronic device and a storage medium. The method includes obtaining an image to be recognized; using a Vision Transformer model to extract a semantic embedding vector of the image to be recognized; determining a preset dimension according to the number of attributes to be recognized, and determining attention values respectively corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; determining spatial embedding vectors respectively corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, where the spatial embedding vector represents the spatial position relationship of the attribute to be recognized in the image to be recognized; fusing each semantic embedding vector and each spatial embedding vector to obtain a target feature vector, and the target feature vector includes the correlation relationship between each attribute to be recognized; determining a pedestrian attribute recognition result according to the target feature vector, which improves the efficiency and accuracy of pedestrian attribute recognition.

Description

Technical Field

[0001] This application relates to the field of intelligent recognition technology, and in particular, to a pedestrian attribute recognition method, a recognition device, an electronic device, and a storage medium. Background Art

[0002] In recent years, pedestrian attribute recognition has been increasingly widely used in various fields. Pedestrian attribute recognition refers to using a computer to predict and analyze various attribute information related to pedestrians in an image. An accurate and efficient pedestrian attribute recognition method helps to carry out various analysis tasks based on pedestrian images.

[0003] Most current pedestrian attribute recognition technologies use a convolutional neural network to extract semantic information from the image to be recognized, and then directly perform attribute recognition based on the extracted semantic information. According to the current technical solution for pedestrian attribute recognition, the accuracy rate is relatively low.

[0004] Therefore, how to improve the accuracy of pedestrian attribute recognition has become an urgent problem to be solved. Summary of the Invention

[0005] Embodiments of this application provide a pedestrian attribute recognition method, a recognition device, an electronic device, and a storage medium, aiming to solve the problem of low accuracy of pedestrian attribute recognition.

[0006] In a first aspect, embodiments of this application provide a pedestrian attribute recognition method, the method including: obtaining an image to be recognized; using a Vision Transformer model to extract a semantic embedding vector of the image to be recognized; determining a preset dimension according to the number of attributes to be recognized, and determining attention values respectively corresponding to each of the attributes to be recognized according to the semantic embedding vector and the preset dimension; determining spatial embedding vectors respectively corresponding to each of the attributes to be recognized according to the semantic embedding vector and each of the attention values, the spatial embedding vector representing the spatial position relationship of the attribute to be recognized in the image to be recognized; for each semantic embedding vector, fusing the semantic embedding vector and each of the spatial embedding vectors to obtain a target feature vector, the target feature vector including the association relationship between each of the attributes to be recognized; determining a pedestrian attribute recognition result according to the target feature vector.

[0007] In a possible implementation, the extraction of the semantic embedding vector of the image to be recognized by the Vision Transformer model includes: Based on the Vision Transformer model, performing image segmentation on the image to be recognized to obtain a plurality of image patches; performing linear projection on each of the image patches to obtain first feature vectors respectively corresponding to each of the image patches; performing positional encoding on each of the first feature vectors to obtain second feature vectors respectively corresponding to each of the image patches; and extracting the semantic embedding vector of the image to be recognized according to each of the second feature vectors.

[0008] In a possible implementation, the extraction of the semantic embedding vector of the image to be recognized according to each of the second feature vectors includes: The Vision Transformer model uses the multi-head self-attention mechanism and each of the second feature vectors to determine third feature vectors respectively corresponding to each of the image patches, and the third feature vectors represent the association relationships between each of the image patches; The Vision Transformer model uses the feed-forward neural network and each of the third feature vectors to extract the semantic embedding vector of the image to be recognized.

[0009] In a possible implementation, the Vision Transformer model uses the multi-head self-attention mechanism and each of the second feature vectors to determine third feature vectors respectively corresponding to each of the image patches, including: The Vision Transformer model uses the multi-head self-attention mechanism to calculate the association degrees between each of the second feature vectors; Based on the association degrees between each of the second feature vectors, performing weighted summation on each of the second feature vectors to obtain third feature vectors respectively corresponding to each of the image patches.

[0010] In a possible implementation, the determination of the spatial embedding vectors respectively corresponding to each of the to-be-recognized attributes according to the semantic embedding vector and each of the attention values includes: Normalizing each of the attention values to obtain the normalized attention values; Based on the normalized attention values, performing weighted averaging on the semantic embedding vector to obtain the spatial embedding vectors respectively corresponding to each of the to-be-recognized attributes.

[0011] In a possible implementation, before determining the attention values respectively corresponding to each of the to-be-recognized attributes according to the semantic embedding vector and the preset dimension, the method further includes: Performing a class token removal operation on the semantic embedding vector.

[0012] In a possible implementation, the process of determining the to-be-recognized attribute includes: obtaining sample image data, where the sample image data includes a sample image and label information, and the label information characterizes the pedestrian attribute in the sample image; training the Vision Transformer model and a classifier based on the sample image data and determining the to-be-recognized attribute, where the classifier is used to determine the pedestrian attribute recognition result according to the target feature vector.

[0013] In a second aspect, an embodiment of the present application provides a pedestrian attribute recognition device, where the device includes: an acquisition module, configured to acquire an image to be recognized; a semantic embedding vector determination module, configured to extract a semantic embedding vector of the image to be recognized by using a Vision Transformer model; an attention value determination module, configured to determine a preset dimension according to the number of to-be-recognized attributes, and determine attention values respectively corresponding to each of the to-be-recognized attributes according to the semantic embedding vector and the preset dimension; a key embedding vector determination module, configured to determine spatial embedding vectors respectively corresponding to each of the to-be-recognized attributes according to the semantic embedding vector and each of the attention values, where the spatial embedding vector characterizes the spatial position relationship of the to-be-recognized attribute in the image to be recognized; a fusion module, configured to, for each of the semantic embedding vectors, fuse the semantic embedding vector and each of the spatial embedding vectors to obtain a target feature vector, where the target feature vector includes the association relationship between each of the to-be-recognized attributes; and an identification module, configured to determine a pedestrian attribute recognition result according to the target feature vector.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the method as described in the first aspect or any one of its implementation manners is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in the first aspect or any one of its implementation manners is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method as described in the first aspect or any one of its implementation manners are implemented.

[0017] The beneficial effects of the embodiments of this application compared with the prior art are as follows: Using the Vision Transformer model as a feature extractor, the semantic embedding vector of the image to be recognized is extracted. The preset dimension is determined according to the number of attributes to be recognized, and the attention values corresponding to each attribute to be recognized are determined based on the semantic embedding vector and the preset dimension. According to the semantic embedding vector and each attention value, the spatial embedding vectors corresponding to each attribute to be recognized are determined, that is, the spatial position relationship of each attribute to be recognized in the image to be recognized is determined. For each semantic embedding vector, the semantic embedding vector and each spatial embedding vector are fused to obtain the target feature vector, and the target feature vector contains the correlation relationships between the attributes to be recognized. When determining the pedestrian attribute recognition result based on the target feature vector, the attributes having a correlation relationship with the attribute to be recognized can be determined according to the target feature vector, and the influence of each attribute having a correlation relationship with the attribute to be recognized on the attribute to be recognized is considered, which improves the accuracy of pedestrian attribute recognition. At the same time, since the target feature vector is obtained by fusing the semantic embedding vector and the spatial embedding vectors corresponding to each attribute to be recognized, the target feature vector also contains the spatial position relationship of each attribute to be recognized in the image to be recognized. When performing pedestrian attribute recognition, the region related to the attribute to be recognized can also be effectively located according to the target feature vector, reducing the recognition range and improving the recognition efficiency. Therefore, the technical solution provided by this application can improve the efficiency and accuracy of pedestrian attribute recognition.

[0018] It can be understood that a pedestrian attribute recognition device, an electronic device, a computer-readable storage medium, and a computer program product provided by the embodiments of this application have the same beneficial effects as the above-mentioned pedestrian attribute recognition method, and will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a flowchart of a pedestrian attribute recognition method provided by an embodiment of this application;

[0021] Figure 2 It is a flowchart of using the Vision Transformer model to extract the semantic embedding vector of the image to be recognized provided by an embodiment of this application;

[0022] Figure 3 It is a structural block diagram of a pedestrian attribute recognition device provided by an embodiment of this application;

[0023] Figure 4 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0024] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from hindering the description of the present application.

[0025] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0026] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] As used in the specification of the present application and the appended claims, the term "if" can be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined" or "in response to determining" or "once detecting [the described condition or event]" or "in response to detecting [the described condition or event]" according to the context.

[0028] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0029] The reference to "an embodiment" or "some embodiments" or the like described in the specification of the present application means that a specific feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the statements "in an embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0030] Pedestrian attribute recognition refers to using a computer to predict and analyze various attribute information related to pedestrians in an image. In recent years, pedestrian attribute recognition has been more and more widely applied in various fields, such as video surveillance, smart cities, public safety, precise advertising placement, etc. An accurate and efficient pedestrian attribute recognition method helps the progress of various analysis tasks based on pedestrian images.

[0031] Most current pedestrian attribute recognition technologies use a convolutional neural network to extract semantic information from the image to be recognized, and then directly perform attribute recognition based on the extracted semantic information. According to this technical solution for pedestrian attribute recognition, the accuracy is relatively low.

[0032] To solve the above technical problems, this application proposes a pedestrian attribute recognition method, which includes: obtaining an image to be recognized; using a Vision Transformer model to extract a semantic embedding vector of the image to be recognized; determining a preset dimension according to the number of attributes to be recognized, and determining attention values corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; determining spatial embedding vectors corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, where the spatial embedding vector represents the spatial position relationship of the attribute to be recognized in the image to be recognized; for each semantic embedding vector, fusing the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, where the target feature vector contains the association relationship between each attribute to be recognized; and determining the pedestrian attribute recognition result according to the target feature vector, which improves the efficiency and accuracy of pedestrian attribute recognition.

[0033] For the convenience of understanding, the technical solution of this application will be introduced in detail below with reference to the accompanying drawings.

[0034] Figure 1 FIG. is a schematic flowchart of a pedestrian attribute recognition method provided by an embodiment of this application. For the convenience of description, only the parts related to this embodiment are shown. The method provided by this embodiment includes the following steps:

[0035] S110, obtaining an image to be recognized.

[0036] Optionally, the image to be recognized can be a single photo or a frame image in a video. Among them, the video can be a surveillance video captured by a security camera.

[0037] S120, using a Vision Transformer model to extract a semantic embedding vector of the image to be recognized.

[0038] Among them, the semantic embedding vector is used to represent the content such as objects and scenes in the image to be recognized. Specifically, the Vision Transformer model is a deep learning model that uses the Transformer architecture to process images in the field of computer vision. Using the Vision Transformer as the feature extractor for the image to be recognized in the embodiments of this application is actually applying the ability of the Transformer model to process sequence data to image data. By dividing the image to be recognized into multiple image patches and converting each image patch into a corresponding feature vector using linear projection and positional encoding, the Vision Transformer model can learn the complex relationships and high-level semantic features in the image to be recognized through the self-attention mechanism without relying on convolutional operations, and can accurately extract the semantic embedding vector of the image to be recognized.

[0039] S130. Determine a preset dimension according to the number of attributes to be recognized, and determine the attention value corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension.

[0040] Specifically, common attributes to be recognized include macroscopic attributes such as the gender, skin color, age, and body posture of pedestrians, as well as specific person attributes such as backpack type, clothing type and color, pants type and color, and current action. For example, the attributes to be recognized include male, female, glasses, white backpack, long hair, short hair, dress, black pants, etc.; the preset dimension is set in advance according to the number of attributes to be recognized.

[0041] As an example, set the preset dimension according to the number of attributes to be recognized and the dimension corresponding to the semantic embedding vector. It should be noted that the preset dimension can be set differently according to the actual situation, and this application does not limit this.

[0042] Specifically, the attention value represents the degree of attention to the attribute to be recognized. The higher the attention value, the higher the degree of attention to the attribute to be recognized. The attribute to be recognized with a higher attention value has a higher weight during the attribute recognition process, that is, it has a greater impact on attribute recognition. After determining the preset dimension, determine the attention value corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension through a fully connected layer.

[0043] In a specific implementation, perform an inner product operation on the extracted semantic embedding vector and the matrix corresponding to the preset dimension through a fully connected layer, convert the semantic embedding vector into a feature vector of the preset dimension, and the obtained feature vector is the attention value corresponding to each attribute to be recognized.

[0044] S140. Determine the spatial embedding vector corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value. The spatial embedding vector represents the spatial position relationship of the attribute to be recognized in the image to be recognized.

[0045] Specifically, a weighted average operation is performed on the semantic embedding vectors using the attention values corresponding to each to-be-recognized attribute, to obtain the spatial embedding vectors of each to-be-recognized attribute, that is, the spatial position relationship of each to-be-recognized attribute in the to-be-recognized image. When performing pedestrian attribute recognition on the to-be-recognized image, through each spatial embedding vector, it is possible to effectively locate the relevant regions of each to-be-recognized attribute in the to-be-recognized image, narrow the recognition range, and improve the efficiency of attribute recognition.

[0046] Exemplarily, when recognizing the attribute "glasses" in the to-be-recognized image, according to the spatial embedding vector of the "glasses" attribute, the relevant region of the "glasses" attribute in the to-be-recognized image is located as the head region. Then, when specifically recognizing the "glasses" attribute, the head region of the pedestrian is focused on.

[0047] S150. For each semantic embedding vector, fuse the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, where the target feature vector contains the correlation relationships between the to-be-recognized attributes.

[0048] Among them, the target feature vector is a feature vector containing the correlation relationships between the to-be-recognized attributes in all to-be-recognized attributes.

[0049] In a specific implementation, all semantic embedding vectors extracted from the to-be-recognized image and the spatial embedding vectors corresponding to each to-be-recognized attribute are input into a 4-layer Transformer encoder. Through the Transformer encoder, each semantic embedding vector and each spatial embedding vector are fused to capture the correlation relationships between the to-be-recognized attributes at the semantic feature level and the spatial feature level, and a target feature vector is obtained. Additionally, since the target feature vector is obtained by fusing the semantic embedding vectors and the spatial embedding vectors corresponding to each to-be-recognized attribute, the target feature vector also contains the spatial position relationship of each to-be-recognized attribute in the to-be-recognized image.

[0050] S160. Determine the pedestrian attribute recognition result according to the target feature vector.

[0051] In a possible implementation manner, the pedestrian attribute recognition result is the predicted probability value of the to-be-recognized attribute. Among them, the predicted probability value of the to-be-recognized attribute is the probability value that the to-be-recognized image has this to-be-recognized attribute.

[0052] As an example, the predicted probability value of the to-be-recognized attribute A is 75%, indicating that there is a 75% possibility that the to-be-recognized image has the to-be-recognized attribute A.

[0053] As another example, a preset threshold is set. When the predicted probability value of the attribute to be recognized is greater than the preset threshold, it indicates that the attribute to be recognized exists in the image to be recognized; when the predicted probability value of the attribute to be recognized is less than or equal to the preset threshold, it indicates that the attribute to be recognized does not exist in the image to be recognized.

[0054] Exemplarily, the preset threshold is set to 50%, and the predicted probability value of the attribute A to be recognized is 75%. Since the predicted probability value of the attribute A to be recognized is greater than the preset threshold, it indicates that the attribute A to be recognized exists in the image to be recognized.

[0055] In a specific implementation, the target feature vector is input into a classifier to obtain the recognition result of the pedestrian attribute for the image to be recognized. Since the target feature vector contains the correlation relationships between the attributes to be recognized, during the process of recognizing the pedestrian attributes through the target feature vector, the influences between the attributes to be recognized are fully considered, improving the accuracy of pedestrian attribute recognition.

[0056] Exemplarily, when recognizing the "female" attribute, according to the target feature vector, the attributes having a correlation relationship with the "female" attribute are determined, such as the attributes "skirt", "long hair", etc. Then, during the process of recognizing the "female" attribute, the attributes such as "skirt" and "long hair" are considered.

[0057] The technical solution provided in this embodiment uses the Vision Transformer model as a feature extractor to extract the semantic embedding vector of the image to be recognized, determines a preset dimension according to the number of attributes to be recognized, and determines the attention values corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; according to the semantic embedding vector and each attention value, determines the spatial embedding vectors corresponding to each attribute to be recognized, that is, determines the spatial position relationships of each attribute to be recognized in the image to be recognized; for each semantic embedding vector, fuses the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, and the target feature vector contains the correlation relationships between the attributes to be recognized; when determining the pedestrian attribute recognition result according to the target feature vector, it is possible to determine the attributes having a correlation relationship with the attribute to be recognized according to the target feature vector, consider the influences of the various attributes having a correlation relationship with the attribute to be recognized on the attribute to be recognized, improving the accuracy of pedestrian attribute recognition. At the same time, since the target feature vector is obtained by fusing the semantic embedding vector and the spatial embedding vectors corresponding to each attribute to be recognized, the target feature vector also contains the spatial position relationships of each attribute to be recognized in the image to be recognized. When performing pedestrian attribute recognition, it is also possible to effectively locate the region related to the attribute to be recognized according to the target feature vector, narrowing the recognition range and improving the recognition efficiency; therefore, the technical solution provided in this embodiment can improve the efficiency and accuracy of pedestrian attribute recognition.

[0058] Figure 2The figure is a schematic flowchart of a process for extracting semantic embedding vectors of an image to be recognized using a Vision Transformer model provided by an embodiment of the present application. In combination with Figure 2 As shown, based on the above embodiment, the technical solution in this embodiment is further described and optimized. Specifically, in this embodiment, using a Vision Transformer model to extract semantic embedding vectors of an image to be recognized includes:

[0059] S210, based on the Vision Transformer model, perform image segmentation on the image to be recognized to obtain multiple image patches.

[0060] S220, perform linear projection on each image patch to obtain first feature vectors corresponding to each image patch respectively.

[0061] S230, perform position encoding on each first feature vector to obtain second feature vectors corresponding to each image patch respectively.

[0062] S240, extract the semantic embedding vector of the image to be recognized according to each second feature vector.

[0063] In a specific implementation, based on the Vision Transformer model, the image to be recognized is segmented into multiple overlapping image patches of a fixed size (for example, 16x16 pixels), and the two-dimensional matrix data of the image to be recognized is converted into a series of sequence data that can be processed by the Transformer model. This segmentation method ensures a certain overlap between the image patches, which helps the subsequent Transformer model capture the local features and context information of the image to be recognized; then, a fully connected layer is used to perform linear projection on each image patch, mapping the pixel values in each image patch to a specified dimension to generate first feature vectors corresponding to each image patch respectively. The first feature vectors are feature vectors with a fixed length, which is convenient for subsequent processing. Among them, the pixel values of each image patch can be the red, green, and blue (RGB) values of each image patch; since the Transformer model itself does not have the ability to process the sequence order, position encoding needs to be added to each first feature vector to obtain second feature vectors corresponding to each image patch respectively, so as to retain the spatial position information of each image patch and ensure that the Transformer model can understand the relative position relationship between the image patches. The sequence of second feature vectors corresponding to each image patch obtained after linear projection and position encoding processing is input into the Transformer encoder, and the powerful self-attention mechanism of the Transformer is used to process the image features, further learning the relationship between the image patches to obtain the semantic embedding vector of the image to be recognized.

[0064] The technical solution provided in this embodiment uses a Vision Transformer model as a feature extractor to perform image segmentation, linear projection, and position encoding operations on the image to be recognized, so that the feature vectors of each image patch in the processed image to be recognized retain the spatial position information of each image patch. Then, the Transformer encoder is used to process the feature vectors of each image patch to learn the relationships between the image patches and extract the semantic embedding vector of the image to be recognized, improving the accuracy of extracting the semantic embedding vector in the image to be recognized. In addition, the feature vectors of each image patch in the processed image to be recognized have a fixed length, which is convenient for subsequent operations and improves the efficiency of extracting the semantic embedding vector in the image to be recognized.

[0065] Based on the above embodiment, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, extracting the semantic embedding vector of the image to be recognized according to each second feature vector includes:

[0066] The Vision Transformer model uses the multi-head self-attention mechanism and each second feature vector to determine the third feature vector corresponding to each image patch, and the third feature vector represents the association relationship between the image patches;

[0067] The Vision Transformer model uses the feed-forward neural network and each third feature vector to extract the semantic embedding vector of the image to be recognized.

[0068] In specific implementation, the Transformer encoder consists of multiple identical layers, and each layer includes a multi-head self-attention mechanism and a feed-forward neural network. Among them, the Vision Transformer model uses the multi-head self-attention mechanism to capture the dependence relationships between the image patches in the image to be recognized, learn the global features of the image to be recognized, and obtain the third feature vector representing the association relationship between the image patches based on the second feature vector corresponding to each image patch; the feed-forward neural network usually includes two linear transformations and a rectified linear unit (ReLU) activation function to process each third feature vector output by the multi-head self-attention mechanism, increasing the non-linear ability of the Vision Transformer model and further extracting the semantic embedding vector of the image to be recognized.

[0069] In the technical solution provided in this embodiment, the Vision Transformer model uses the multi-head self-attention mechanism to capture the dependency relationships between image patches in the image to be recognized, learn the global features of the image to be recognized, and then uses the feed-forward neural network to further process the output of the multi-head self-attention mechanism, improving the non-linear ability of the Vision Transformer model and the accuracy of extracting the semantic embedding vector of the image to be recognized.

[0070] Based on the above embodiment, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the Vision Transformer model uses the multi-head self-attention mechanism and each second feature vector to determine the third feature vector corresponding to each image patch, including:

[0071] The Vision Transformer model uses the multi-head self-attention mechanism to calculate the correlation degrees between each second feature vector;

[0072] Based on the correlation degrees between each second feature vector, perform weighted summation on each second feature vector to obtain the third feature vector corresponding to each image patch.

[0073] Specifically, in the multi-head self-attention mechanism, each second feature vector will interact with all other second feature vectors, calculate the correlation degrees between each pair, and perform weighted summation on each second feature vector according to the correlation degrees between each second feature vector to obtain the third feature vector corresponding to each image patch in the image to be recognized. The third feature vector reflects the correlation relationships between each image patch.

[0074] The technical solution provided in this embodiment learns the correlation relationships between each image patch in the image to be recognized through the multi-head self-attention mechanism, and further learns the semantic features of the image to be recognized based on the correlation relationships between each image patch, improving the accuracy of learning the semantic features of the image to be recognized.

[0075] Based on the above embodiment, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, according to the semantic embedding vector and each attention value, determine the spatial embedding vector corresponding to each to-be-recognized attribute, including:

[0076] Normalize each attention value to obtain the normalized attention value;

[0077] Based on the normalized attention value, perform weighted averaging on the semantic embedding vector to obtain the spatial embedding vector corresponding to each to-be-recognized attribute.

[0078] Specifically, in the field of machine learning, different evaluation metrics (i.e., different features in the feature vector) often have different dimensions and dimension units. Such a situation will affect the results of data analysis. In order to eliminate the influence of dimensions between metrics, data standardization processing needs to be carried out to solve the comparability between data metrics. After the original data is processed by data standardization, each metric is at the same order of magnitude, which is suitable for comprehensive comparative evaluation. Among them, the most typical one is the normalization processing of data. The purpose of normalization is to limit the preprocessed data (i.e., the attention values corresponding to each to-be-recognized attribute) within a certain range (such as [0, 1] or [-1, 1]), so as to eliminate the adverse effects caused by singular sample data. Singular sample data refers to sample vectors (i.e., feature vectors) that are particularly large or particularly small compared to other input samples.

[0079] In a specific implementation, the Softmax function is used to normalize each attention value, limit the attention values corresponding to each to-be-recognized attribute within the range of [0, 1], and then use the normalized attention values to perform weighted averaging on the semantic feature vector to obtain the spatial embedding vectors corresponding to each to-be-recognized attribute respectively.

[0080] The technical solution provided in this embodiment normalizes each attention value to obtain the normalized attention values, eliminates the influence of singular sample data, limits the attention values within a preset range, and then performs weighted averaging on the semantic embedding vector based on the normalized attention values to obtain each spatial embedding vector, improving the accuracy of determining the spatial embedding vectors corresponding to each to-be-recognized attribute.

[0081] On the basis of the above embodiment, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, before determining the attention values corresponding to each to-be-recognized attribute according to the semantic embedding vector and the preset dimension, the method further includes:

[0082] Performing an operation to remove class labels on the semantic embedding vector.

[0083] In a specific implementation, there may be label information in the to-be-recognized image. The label information represents the pedestrian attributes in the to-be-recognized image. Then, the semantic embedding vector extracted from the to-be-recognized image will also contain this label information, that is, the class label. Before determining the attention values corresponding to each to-be-recognized attribute according to the semantic embedding vector and the preset dimension, it is necessary to remove the class label in the semantic embedding vector to ensure that the semantic embedding vector does not carry a class label.

[0084] Before determining the attention values corresponding to each to-be-recognized attribute according to the semantic embedding vector and the preset dimension, the technical solution provided in this embodiment performs an operation of removing class labels on the semantic embedding vector, avoiding the influence of class labels on calculating the attention values, and improving the accuracy of calculating the attention values corresponding to each to-be-recognized attribute.

[0085] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the process of determining the to-be-recognized attribute includes:

[0086] Obtain sample image data, where the sample image data includes a sample image and label information, and the label information represents the pedestrian attributes in the sample image;

[0087] Train the Vision Transformer model and the classifier based on the sample image data and determine the to-be-recognized attribute. The classifier is used to determine the pedestrian attribute recognition result according to the target feature vector.

[0088] In a specific implementation, use the sample image and label information in the sample image data to train the parameters of the Vision Transformer model and the classifier, and determine the to-be-recognized attribute by learning the correspondence between the label information and the pedestrian attributes in the sample image data. Among them, the following formula is used as the loss function during the training process of the Vision Transformer model and the classifier:

[0089]

[0090] is the loss value of the Vision Transformer model, is the loss value of the classifier, λ is a hyperparameter, C is the number of to-be-recognized attributes, ρ n is the weight of the nth to-be-recognized attribute, σ(·) is the sigmoid activation function, y i represents the predicted value of the to-be-recognized attribute.

[0091] The technical solution provided in this embodiment trains the parameters of the Vision Transformer model and the classifier through the sample image data, and determines the to-be-recognized attribute by learning the correspondence between the label information and the pedestrian attributes in the sample image data, improving the effects of the Vision Transformer model and the classifier, and at the same time improving the accuracy of pedestrian attribute recognition.

[0092] Figure 3 This is a structural block diagram of a pedestrian attribute recognition device provided in an embodiment of the present application. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. Refer toFigure 3 The pedestrian attribute recognition device 300 may include an acquisition module 301, a semantic embedding vector determination module 302, an attention value determination module 303, a key embedding vector determination module 304, a fusion module 305, and an identification module 306.

[0093] The acquisition module 301 is configured to acquire an image to be recognized;

[0094] The semantic embedding vector determination module 302 is configured to extract a semantic embedding vector of the image to be recognized by using a Vision Transformer model;

[0095] The attention value determination module 303 is configured to determine a preset dimension according to the number of attributes to be recognized, and determine attention values corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension;

[0096] The key embedding vector determination module 304 is configured to determine spatial embedding vectors corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, where the spatial embedding vectors represent the spatial position relationship of the attributes to be recognized in the image to be recognized;

[0097] The fusion module 305 is configured to fuse the semantic embedding vector and each spatial embedding vector for each semantic embedding vector to obtain a target feature vector, where the target feature vector includes the association relationship between each attribute to be recognized;

[0098] The identification module 306 determines a pedestrian attribute recognition result according to the target feature vector.

[0099] The pedestrian attribute recognition device provided by the embodiment of the present application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0100] In one embodiment, the semantic embedding vector determination module 302 includes:

[0101] An image segmentation sub-module, configured to perform image segmentation on the image to be recognized based on a Vision Transformer model to obtain a plurality of image patches;

[0102] A linear projection sub-module, configured to perform linear projection on each image patch to obtain a first feature vector corresponding to each image patch;

[0103] A position encoding sub-module, configured to perform position encoding on each first feature vector to obtain a second feature vector corresponding to each image patch;

[0104] A feature extraction sub-module, configured to extract a semantic embedding vector of the image to be recognized according to each second feature vector.

[0105] In one embodiment, the feature extraction sub-module includes:

[0106] A feature vector determination unit, configured to use the multi-head self-attention mechanism and each second feature vector in the Vision Transformer model to determine a third feature vector corresponding to each image patch, where the third feature vector represents the correlation relationship between each image patch;

[0107] A feature extraction unit, configured to use the feed-forward neural network and each third feature vector in the Vision Transformer model to extract the semantic embedding vector of the image to be recognized.

[0108] In one embodiment, the feature vector determination unit includes:

[0109] An association degree calculation sub-unit, configured to use the multi-head self-attention mechanism in the Vision Transformer model to calculate the association degree between each second feature vector;

[0110] A feature vector calculation sub-unit, configured to perform weighted summation on each second feature vector based on the association degree between each second feature vector to obtain a third feature vector corresponding to each image patch.

[0111] In one embodiment, the key embedding vector determination module 304 includes:

[0112] An attention value processing sub-module, configured to normalize each attention value to obtain a normalized attention value;

[0113] A spatial embedding vector determination sub-module, configured to perform weighted averaging on the semantic embedding vector based on the normalized attention value to obtain a spatial embedding vector corresponding to each attribute to be recognized.

[0114] In one embodiment, the pedestrian attribute recognition device 300 further includes:

[0115] A class label removal module, configured to perform an operation of removing class labels on the semantic embedding vector.

[0116] In one embodiment, the pedestrian attribute recognition device 300 further includes:

[0117] A sample image data acquisition module, configured to acquire sample image data, where the sample image data includes a sample image and label information, and the label information represents the pedestrian attributes in the sample image;

[0118] An attribute to be recognized determination module, configured to train the Vision Transformer model and the classifier based on the sample image data and determine the attribute to be recognized, where the classifier is configured to determine the pedestrian attribute recognition result according to the target feature vector.

[0119] It should be noted that, for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiments of this application, for their specific functions and the technical effects brought, please refer to the method embodiment part for details, and will not be elaborated here.

[0120] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used for illustration. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit exists physically alone, or two or more units are integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0121] Figure 4 The following is a schematic structural diagram of an electronic device provided by an embodiment of this application. As Figure 4 shown, the electronic device 4 of this embodiment includes: at least one processor 40 ( Figure 4 only one is shown herein), a memory 41, and a computer program 42 stored in the memory 41 and operable on at least one processor 40. When the processor 40 executes the computer program 42, it implements the steps in the above Figure 1 or Figure 2 method embodiments, or implements the functions of each module / unit in the above Figure 3 device embodiments.

[0122] The electronic device 4 can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device 4 may include but is not limited to the processor 40 and the memory 41. Those skilled in the art can understand that Figure 4 merely examples of the electronic device 4 do not constitute a limitation to the electronic device 4, and it may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0123] The processor 40 can be a Central Processing Unit (CPU), and the processor 40 can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0124] In some embodiments, the memory 41 can be an internal storage unit of the electronic device 4, such as the hard disk or memory of the electronic device 4. In other embodiments, the memory 41 can also be an external storage device of the electronic device 4, such as a plug-in hard disk equipped on the electronic device 4, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 41 can also include both the internal storage unit and the external storage device of the electronic device 4. The memory 41 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of a computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0125] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0126] When an integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc.

[0127] A computer-readable storage medium provided by an embodiment of this application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0128] An embodiment of this application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments can be implemented.

[0129] A computer program product provided by an embodiment of this application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0130] In the above embodiments, the descriptions of each embodiment have their own emphases. For parts not detailed or recorded in a certain embodiment, the relevant descriptions of other embodiments can be referred to.

[0131] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by the combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0132] In the embodiments provided in the present application, it should be understood that the disclosed device / electronic device and method can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0133] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units. That is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A pedestrian attribute recognition method, characterized in that: The method comprises: Obtain the image to be recognized; Extracting a semantic embedding vector of the image to be identified using a Vision Transformer model; Determining a preset dimension according to the number of attributes to be identified, and determining an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; Determining, based on the semantic embedding vector and each of the attention values, a spatial embedding vector corresponding to each of the attributes to be identified, wherein the spatial embedding vector represents a spatial position relationship of the attribute to be identified in the image to be identified; For each of the semantic embedding vectors, the semantic embedding vector is fused with each of the spatial embedding vectors to obtain a target feature vector, where the target feature vector includes the association relationship between each of the attributes to be identified; Determining a pedestrian attribute recognition result according to the target feature vector; The extracting the semantic embedding vector of the image to be identified by using the Vision Transformer model includes: Based on the Vision Transformer model, segment the image to be recognized to obtain multiple image blocks; Performing linear projection on each of the image blocks to obtain first eigenvectors corresponding to each of the image blocks; Performing position encoding on each of the first eigenvectors to obtain a second eigenvector corresponding to each of the image blocks; Extracting a semantic embedding vector of the image to be identified based on each of the second feature vectors; The extracting the semantic embedding vector of the image to be identified according to each of the second feature vectors includes: The Vision Transformer model uses a multi-head self-attention mechanism and each second eigenvector to determine a third eigenvector corresponding to each image block, wherein the third eigenvector represents the association relationship between the image blocks; The Vision Transformer model uses a feedforward neural network and each of the third eigenvectors to extract a semantic embedding vector of the image to be identified; The Vision Transformer model uses a multi-head self-attention mechanism and each second eigenvector to determine a third eigenvector corresponding to each image block, including: The Vision Transformer model calculates the correlation between each second feature vector using the multi-head self-attention mechanism; Based on the correlation between the second eigenvectors, a weighted sum is performed on the second eigenvectors to obtain a third eigenvector corresponding to each of the image blocks.

2. The method according to claim 1, characterized in that Determining, based on the semantic embedding vector and each of the attention values, a spatial embedding vector corresponding to each of the attributes to be identified, includes: Normalizing each of the attention values to obtain a normalized attention value; The semantic embedding vectors are weighted averaged based on the normalized attention values to obtain spatial embedding vectors corresponding to each of the attributes to be identified.

3. The method according to claim 1, characterized in that Before determining the attention values corresponding to the attributes to be identified according to the semantic embedding vector and the preset dimension, the method further includes: A class label removal operation is performed on the semantic embedding vector.

4. The method according to any one of claims 1 to 3, characterized in that The process of determining the attribute to be identified includes: Acquire sample image data, where the sample image data includes a sample image and label information, where the label information represents attributes of a pedestrian in the sample image; The Vision Transformer model and the classifier are trained based on the sample image data and the attributes to be identified are determined. The classifier is used to determine the pedestrian attribute recognition result according to the target feature vector.

5. A pedestrian attribute recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be identified; A semantic embedding vector determination module, configured to extract the semantic embedding vector of the image to be identified using a Vision Transformer model; an attention value determination module, configured to determine a preset dimension according to the number of attributes to be identified, and determine an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; a key embedding vector determination module, configured to determine, based on the semantic embedding vector and each of the attention values, a spatial embedding vector corresponding to each of the attributes to be identified, wherein the spatial embedding vector represents a spatial positional relationship of the attribute to be identified in the image to be identified; a fusion module, configured to fuse the semantic embedding vector with each of the spatial embedding vectors for each of the semantic embedding vectors to obtain a target feature vector, wherein the target feature vector includes an association relationship between each of the attributes to be identified; The recognition module determines the pedestrian attribute recognition result according to the target feature vector. The semantic embedding vector determination module includes: An image segmentation submodule, configured to segment the image to be identified based on the Vision Transformer model to obtain a plurality of image blocks; a linear projection submodule, configured to perform linear projection on each of the image blocks to obtain a first eigenvector corresponding to each of the image blocks; a position encoding submodule, configured to perform position encoding on each of the first eigenvectors to obtain a second eigenvector corresponding to each of the image blocks; a feature extraction submodule, configured to extract a semantic embedding vector of the image to be identified based on each of the second feature vectors; The feature extraction submodule includes: a feature vector determining unit, configured to use the Vision Transformer model to determine a third feature vector corresponding to each of the image blocks using a multi-head self-attention mechanism and each of the second feature vectors, wherein the third feature vector represents an association relationship between the image blocks; a feature extraction unit, configured to extract a semantic embedding vector of the image to be identified by using a feedforward neural network and each of the third feature vectors in the Vision Transformer model; The feature vector determining unit includes: a correlation calculation subunit, configured to calculate the correlation between the second feature vectors using the multi-head self-attention mechanism in the Vision Transformer model; The feature vector calculation subunit is configured to perform weighted summation on each of the second feature vectors based on the correlation between the second feature vectors to obtain a third feature vector corresponding to each of the image blocks.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images