Pedestrian attribute recognition method and device, electronic equipment and storage medium

Abstract

The invention is suitable for the technical field of intelligent recognition, and provides a pedestrian attribute recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized image; a Vision Transform model is used to extract a semantic embedding vector of the image to be identified; determining a preset dimension according to the number of the to-be-identified attributes, and determining an attention value corresponding to each to-be-identified attribute according to the semantic embedding vector and the preset dimension; according to the semantic embedding vector and each attention value, determining a spatial embedding vector corresponding to each to-be-identified attribute, the spatial embedding vector representing a spatial position relationship of the to-be-identified attribute in the to-be-identified image; each semantic embedding vector and each spatial embedding vector are fused to obtain a target feature vector, and the target feature vector comprises association relationships among the to-be-identified attributes; and determining a pedestrian attribute recognition result according to the target feature vector. The efficiency and accuracy of pedestrian attribute recognition are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent recognition technology, and in particular to a pedestrian attribute recognition method, recognition device, electronic device and storage medium. Background Art

[0002] In recent years, pedestrian attribute recognition has been increasingly widely used in various fields. Pedestrian attribute recognition refers to the use of computers to predict and analyze various attribute information related to pedestrians in images. Accurate and efficient pedestrian attribute recognition methods are helpful for various analysis tasks based on pedestrian images.

[0003] Most of the current pedestrian attribute recognition technologies use convolutional neural networks to extract semantic information from the image to be recognized, and then directly perform attribute recognition based on the extracted semantic information. According to the current technical solution, the accuracy of pedestrian attribute recognition is low.

[0004] Therefore, how to improve the accuracy of pedestrian attribute recognition has become an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a pedestrian attribute recognition method, a recognition device, an electronic device and a storage medium, aiming to solve the problem of low pedestrian attribute recognition accuracy.

[0006] In a first aspect, an embodiment of the present application provides a method for pedestrian attribute recognition, the method comprising: obtaining an image to be recognized; extracting a semantic embedding vector of the image to be recognized using a Vision Transformer model; determining a preset dimension according to the number of attributes to be recognized, and determining an attention value corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; determining a spatial embedding vector corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, the spatial embedding vector representing the spatial position relationship of the attribute to be recognized in the image to be recognized; for each semantic embedding vector, fusing the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, the target feature vector containing the correlation relationship between each attribute to be recognized; and determining a pedestrian attribute recognition result according to the target feature vector.

[0007] In a possible implementation, the method of extracting the semantic embedding vector of the image to be identified by using the Vision Transformer model includes: performing image segmentation on the image to be identified based on the Vision Transformer model to obtain multiple image blocks; performing linear projection on each of the image blocks to obtain a first feature vector corresponding to each of the image blocks; performing position encoding on each of the first feature vectors to obtain a second feature vector corresponding to each of the image blocks; and extracting the semantic embedding vector of the image to be identified based on each of the second feature vectors.

[0008] In a possible implementation, the extracting the semantic embedding vector of the image to be identified based on each of the second feature vectors includes: the Vision Transformer model uses a multi-head self-attention mechanism and each of the second feature vectors to determine a third feature vector corresponding to each of the image blocks, wherein the third feature vector represents the association relationship between each of the image blocks; the Vision Transformer model uses a feedforward neural network and each of the third feature vectors to extract the semantic embedding vector of the image to be identified.

[0009] In one possible implementation, the Vision Transformer model uses a multi-head self-attention mechanism and each second eigenvector to determine a third eigenvector corresponding to each image block, including: the Vision Transformer model uses the multi-head self-attention mechanism to calculate the correlation between each second eigenvector; based on the correlation between each second eigenvector, weighted summation is performed on each second eigenvector to obtain a third eigenvector corresponding to each image block.

[0010] In one possible implementation, the spatial embedding vectors corresponding to each of the attributes to be identified are determined based on the semantic embedding vector and each of the attention values, including: normalizing each of the attention values ​​to obtain a normalized attention value; and weighted averaging the semantic embedding vector based on the normalized attention value to obtain a spatial embedding vector corresponding to each of the attributes to be identified.

[0011] In a possible implementation, before determining the attention values ​​corresponding to each of the attributes to be identified according to the semantic embedding vector and the preset dimension, the method further includes: performing a class label removal operation on the semantic embedding vector.

[0012] In one possible implementation, the process of determining the attributes to be identified includes: obtaining sample image data, the sample image data including a sample image and label information, the label information characterizing the attributes of pedestrians in the sample image; training the Vision Transformer model and classifier based on the sample image data and determining the attributes to be identified, the classifier being used to determine the pedestrian attribute recognition result based on the target feature vector.

[0013] In the second aspect, an embodiment of the present application provides a pedestrian attribute recognition device, which includes: an acquisition module, used to acquire an image to be recognized; a semantic embedding vector determination module, used to extract the semantic embedding vector of the image to be recognized by using a Vision Transformer model; an attention value determination module, used to determine a preset dimension according to the number of attributes to be recognized, and determine the attention value corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; a key embedding vector determination module, used to determine the spatial embedding vector corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, the spatial embedding vector representing the spatial position relationship of the attribute to be recognized in the image to be recognized; a fusion module, used to fuse the semantic embedding vector and each spatial embedding vector for each semantic embedding vector to obtain a target feature vector, wherein the target feature vector contains the association relationship between each attribute to be recognized; and a recognition module, which determines the pedestrian attribute recognition result according to the target feature vector.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any one of the implementation methods thereof.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects: using the Vision Transformer model as a feature extractor to extract the semantic embedding vector of the image to be identified, determining the preset dimension according to the number of attributes to be identified, and determining the attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; determining the spatial embedding vector corresponding to each attribute to be identified according to the semantic embedding vector and each attention value, that is, determining the spatial position relationship of each attribute to be identified in the image to be identified; for each semantic embedding vector, the semantic embedding vector and each spatial embedding vector are fused to obtain a target feature vector, which contains the correlation relationship between the attributes to be identified; and determining the pedestrian attribute recognition result according to the target feature vector When determining the attributes associated with the attributes to be identified based on the target feature vector, the influence of each attribute associated with the attributes to be identified on the attributes to be identified is considered, thereby improving the accuracy of pedestrian attribute recognition. At the same time, since the target feature vector is obtained by fusion of the semantic embedding vector and the spatial embedding vector corresponding to each attribute to be identified, the target feature vector also includes the spatial position relationship of each attribute to be identified in the image to be identified. When performing pedestrian attribute recognition, the area related to the attribute to be identified can be effectively located based on the target feature vector, thereby narrowing the recognition range and improving the recognition efficiency. Therefore, the technical solution provided in the present application can improve the efficiency and accuracy of pedestrian attribute recognition.

[0018] It can be understood that the pedestrian attribute recognition device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application have the same beneficial effects as the above-mentioned pedestrian attribute recognition method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flowchart of a pedestrian attribute recognition method provided in one embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process for extracting a semantic embedding vector of an image to be recognized using a Vision Transformer model provided in one embodiment of the present application;

[0022] Figure 3 A structural block diagram of a pedestrian attribute recognition device provided in one embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0025] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0026] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0028] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0030] Pedestrian attribute recognition refers to the use of computers to predict and analyze various attribute information related to pedestrians in images. In recent years, pedestrian attribute recognition has been increasingly widely used in various fields, such as video surveillance, smart cities, public safety, and targeted advertising. Accurate and efficient pedestrian attribute recognition methods are helpful for various analysis tasks based on pedestrian images.

[0031] Most of the current pedestrian attribute recognition technologies use convolutional neural networks to extract semantic information from the image to be recognized, and then directly perform attribute recognition based on the extracted semantic information. According to this technical solution, the accuracy of pedestrian attribute recognition is low.

[0032] In order to solve the above technical problems, the present application proposes a pedestrian attribute recognition method, which includes obtaining an image to be recognized; extracting a semantic embedding vector of the image to be recognized using a Vision Transformer model; determining a preset dimension according to the number of attributes to be recognized, and determining an attention value corresponding to each attribute to be recognized according to the semantic embedding vector and the preset dimension; determining a spatial embedding vector corresponding to each attribute to be recognized according to the semantic embedding vector and each attention value, the spatial embedding vector representing the spatial position relationship of the attribute to be recognized in the image to be recognized; for each semantic embedding vector, fusing the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, the target feature vector containing the correlation relationship between the attributes to be recognized; determining a pedestrian attribute recognition result according to the target feature vector, thereby improving the efficiency and accuracy of pedestrian attribute recognition.

[0033] For ease of understanding, the technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 This is a flow chart of a method for identifying pedestrian attributes provided in an embodiment of the present application. For the sake of convenience, only the part related to the present embodiment is shown. The method provided in the present embodiment includes the following steps:

[0035] S110, obtaining an image to be recognized.

[0036] Optionally, the image to be identified may be a single photo or a frame of an image in a video, wherein the video may be a surveillance video shot by a security camera.

[0037] S120, extracting a semantic embedding vector of the image to be recognized using the Vision Transformer model.

[0038] Among them, the semantic embedding vector is used to represent objects and scenes in the image to be identified. Specifically, the VisionTransformer model is a deep learning model that uses the Transformer architecture to process images in the field of computer vision. Using Vision Transformer as a feature extractor for the image to be identified in the embodiment of the present application is actually to apply the ability of the Transformer model to process sequence data to image data. By dividing the image to be identified into multiple image blocks, and using linear projection and position encoding to convert each image block into a corresponding feature vector, the VisionTransformer model can learn the complex relationships and high-level semantic features in the image to be identified through the self-attention mechanism without relying on convolution operations, and can accurately extract the semantic embedding vector of the image to be identified.

[0039] S130, determining a preset dimension according to the number of attributes to be identified, and determining an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension.

[0040] Specifically, common attributes to be identified include macro attributes such as gender, skin color, age, and body shape of pedestrians, as well as specific attributes such as backpack type, clothing type and color, pants type and color, current action, etc. For example, attributes to be identified include male, female, glasses, white backpack, long hair, short hair, dress, black pants, etc.; the preset dimension is set in advance by the number of attributes to be identified.

[0041] As an example, the preset dimension is set according to the number of attributes to be identified and the dimension corresponding to the semantic embedding vector. It should be noted that the preset dimension can be set differently according to actual conditions, and this application does not limit this.

[0042] Specifically, the attention value indicates the degree of attention paid to the attribute to be identified. The higher the attention value, the higher the attention paid to the attribute to be identified. The attribute to be identified with a higher attention value has a higher weight during the attribute identification process, that is, it has a greater impact on attribute identification. After determining the preset dimension, the attention value corresponding to each attribute to be identified is determined by the fully connected layer according to the semantic embedding vector and the preset dimension.

[0043] In a specific implementation, the extracted semantic embedding vector is inner-producted with the matrix corresponding to the preset dimension through a fully connected layer to convert the semantic embedding vector into a feature vector of the preset dimension. The obtained feature vector is the attention value corresponding to each attribute to be identified.

[0044] S140, determining a spatial embedding vector corresponding to each attribute to be identified according to the semantic embedding vector and each attention value, wherein the spatial embedding vector represents a spatial position relationship of the attribute to be identified in the image to be identified.

[0045] Specifically, the attention values ​​corresponding to the attributes to be identified are used to perform a weighted average operation on the semantic embedding vector to obtain the spatial embedding vector of each attribute to be identified, that is, the spatial position relationship of each attribute to be identified in the image to be identified. When performing pedestrian attribute recognition on the image to be identified, the spatial embedding vectors can effectively locate the relevant areas of each attribute to be identified in the image to be identified, narrowing the recognition range and improving the efficiency of attribute recognition.

[0046] For example, when identifying the attribute "glasses" in an image to be identified, based on the spatial embedding vector of the "glasses" attribute, the relevant area of ​​the "glasses" attribute in the image to be identified is located as the head area. Then, when specifically identifying the "glasses" attribute, the head area of ​​the pedestrian is focused on.

[0047] S150, for each semantic embedding vector, the semantic embedding vector is merged with each spatial embedding vector to obtain a target feature vector, where the target feature vector contains the association relationship between each attribute to be identified.

[0048] The target feature vector is a feature vector containing the association relationship between all the attributes to be identified.

[0049] In the specific implementation, all semantic embedding vectors extracted from the image to be identified and the spatial embedding vectors corresponding to each attribute to be identified are input into the 4-layer Transformer encoder, and each semantic embedding vector and each spatial embedding vector are fused through the Transformer encoder to capture the association between each attribute to be identified at the semantic feature level and the spatial feature level to obtain the target feature vector. In addition, since the target feature vector is obtained by fusion of the semantic embedding vector and the spatial embedding vector corresponding to each attribute to be identified, the target feature vector also contains the spatial position relationship of each attribute to be identified in the image to be identified.

[0050] S160, determining a pedestrian attribute recognition result according to the target feature vector.

[0051] In a possible implementation, the pedestrian attribute recognition result is a predicted probability value of the attribute to be recognized, wherein the predicted probability value of the attribute to be recognized is a probability value that the attribute to be recognized exists in the image to be recognized.

[0052] As an example, the predicted probability value of the attribute A to be identified is 75%, which means that there is a 75% possibility that the attribute A to be identified exists in the image to be identified.

[0053] As another example, a preset threshold is set. When the predicted probability value of the attribute to be identified is greater than the preset threshold, it indicates that the attribute to be identified exists in the image to be identified; when the predicted probability value of the attribute to be identified is less than or equal to the preset threshold, it indicates that the attribute to be identified does not exist in the image to be identified.

[0054] Exemplarily, the preset threshold is set to 50%, and the predicted probability value of the attribute A to be identified is 75%. Since the predicted probability value of the attribute A to be identified is greater than the preset threshold, it means that the attribute A to be identified exists in the image to be identified.

[0055] In the specific implementation, the target feature vector is input into the classifier to obtain the pedestrian attribute recognition result for the image to be recognized. Since the target feature vector contains the correlation between the attributes to be recognized, the influence between the attributes to be recognized is fully considered in the process of pedestrian attribute recognition through the target feature vector, which improves the accuracy of pedestrian attribute recognition.

[0056] For example, when identifying the attribute "female", based on the target feature vector, attributes associated with the attribute "female" are determined, such as the attributes "skirt", "long hair", etc. In the process of identifying the attribute "female", the attributes "skirt" and "long hair" are considered.

[0057] The technical solution provided in this embodiment uses the Vision Transformer model as a feature extractor to extract the semantic embedding vector of the image to be identified, determines the preset dimension according to the number of attributes to be identified, and determines the attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; determines the spatial embedding vector corresponding to each attribute to be identified according to the semantic embedding vector and each attention value, that is, determines the spatial position relationship of each attribute to be identified in the image to be identified; for each semantic embedding vector, fuses the semantic embedding vector and each spatial embedding vector to obtain a target feature vector, which contains the correlation relationship between the attributes to be identified; when determining the pedestrian attribute recognition result according to the target feature vector , it is possible to determine the attributes associated with the attributes to be identified based on the target feature vector, and consider the influence of each attribute associated with the attribute to be identified on the attribute to be identified, thereby improving the accuracy of pedestrian attribute recognition. At the same time, since the target feature vector is obtained by fusion of the semantic embedding vector and the spatial embedding vector corresponding to each attribute to be identified, the target feature vector also includes the spatial position relationship of each attribute to be identified in the image to be identified. When performing pedestrian attribute recognition, it is also possible to effectively locate the area related to the attribute to be identified based on the target feature vector, thereby narrowing the recognition range and improving the recognition efficiency. Therefore, the technical solution provided in this embodiment can improve the efficiency and accuracy of pedestrian attribute recognition.

[0058] Figure 2A flowchart of extracting a semantic embedding vector of an image to be recognized using a Vision Transformer model is provided in accordance with an embodiment of the present application. Figure 2 As shown, based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the semantic embedding vector of the image to be recognized is extracted using the Vision Transformer model, including:

[0059] S210, based on the Vision Transformer model, performing image segmentation on the image to be recognized to obtain multiple image blocks.

[0060] S220, performing linear projection on each image block to obtain a first eigenvector corresponding to each image block.

[0061] S230, position encoding is performed on each first eigenvector to obtain a second eigenvector corresponding to each image block.

[0062] S240: extracting a semantic embedding vector of the image to be identified according to each second feature vector.

[0063] In a specific implementation, based on the Vision Transformer model, the image to be recognized is divided into multiple overlapping image blocks of a fixed size (e.g., 16x16 pixels), and the two-dimensional matrix data of the image to be recognized is converted into a series of sequence data that can be processed by the Transformer model. This segmentation method ensures that there is a certain overlap between the image blocks, which helps the subsequent Transformer model capture the local features and contextual information of the image to be identified; then each image block is linearly projected through the fully connected layer, and the pixel values ​​in each image block are mapped to a specified dimension to generate a first feature vector corresponding to each image block. The first feature vector is a feature vector with a fixed length, which is convenient for subsequent processing. The pixel value of each image block can be the red, green, and blue (RGB) value of each image block; since the Transformer model itself does not have the ability to process sequence order, it is necessary to add position encoding to each first feature vector to obtain a second feature vector corresponding to each image block to retain the spatial position information of each image block and ensure that the Transformer model can understand the relative position relationship between image blocks. The second feature vector sequence corresponding to each image block obtained after linear projection and position encoding is input into the Transformer encoder, and the powerful self-attention mechanism of the Transformer is used to process image features, further learn the relationship between image blocks, and obtain the semantic embedding vector of the image to be identified.

[0064] The technical solution provided in this embodiment uses the Vision Transformer model as a feature extractor to perform image segmentation, linear projection and position encoding operations on the image to be identified, so that the feature vectors of each image block in the image to be identified obtained after processing retain the spatial position information of each image block, and then use the Transformer encoder to process the feature vectors of each image block, learn the relationship between each image block, and extract the semantic embedding vector of the image to be identified, thereby improving the accuracy of extracting the semantic embedding vector in the image to be identified; in addition, the feature vectors of each image block in the image to be identified obtained after processing have a fixed length, which is convenient for subsequent operations and improves the efficiency of extracting the semantic embedding vector in the image to be identified.

[0065] On the basis of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the semantic embedding vector of the image to be identified is extracted according to each second feature vector, including:

[0066] The Vision Transformer model uses a multi-head self-attention mechanism and each second eigenvector to determine the third eigenvector corresponding to each image block. The third eigenvector represents the association between the image blocks.

[0067] The Vision Transformer model uses a feedforward neural network and each third eigenvector to extract the semantic embedding vector of the image to be recognized.

[0068] In the specific implementation, the Transformer encoder is composed of multiple identical layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network. Among them, the Vision Transformer model uses a multi-head self-attention mechanism to capture the dependencies between the image blocks in the image to be identified, learn the global features of the image to be identified, and obtain the third feature vector representing the association between the image blocks based on the second feature vectors corresponding to each image block; the feedforward neural network usually contains two linear transformations and a rectified linear unit (Rectified Linear Unit, ReLU) activation function, which processes the third feature vectors output by the multi-head self-attention mechanism, increases the nonlinear ability of the Vision Transformer model, and further extracts the semantic embedding vector of the image to be identified.

[0069] The technical solution provided in this embodiment is that the Vision Transformer model uses a multi-head self-attention mechanism to capture the dependencies between image blocks in the image to be identified, learns the global features of the image to be identified, and then uses a feedforward neural network to further process the output of the multi-head self-attention mechanism, thereby improving the nonlinear ability of the Vision Transformer model and the accuracy of extracting the semantic embedding vector of the image to be identified.

[0070] On the basis of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the Vision Transformer model uses a multi-head self-attention mechanism and each second eigenvector to determine the third eigenvector corresponding to each image block, including:

[0071] The Vision Transformer model uses a multi-head self-attention mechanism to calculate the correlation between each second eigenvector;

[0072] Based on the correlation between the second eigenvectors, a weighted sum is performed on the second eigenvectors to obtain a third eigenvector corresponding to each image block.

[0073] Specifically, in the multi-head self-attention mechanism, each second eigenvector will interact with all other second eigenvectors, calculate the correlation between each pair, and perform weighted summation on each second eigenvector according to the correlation between each second eigenvector to obtain the third eigenvector corresponding to each image block in the image to be identified. The third eigenvector reflects the correlation relationship between each image block.

[0074] The technical solution provided in this embodiment learns the association relationship between each image block in the image to be identified through a multi-head self-attention mechanism, and further learns the semantic features of the image to be identified based on the association relationship between each image block, thereby improving the accuracy of learning the semantic features of the image to be identified.

[0075] On the basis of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, according to the semantic embedding vector and each attention value, the spatial embedding vector corresponding to each attribute to be identified is determined, including:

[0076] Normalize each attention value to obtain a normalized attention value;

[0077] Based on the normalized attention value, the semantic embedding vector is weighted averaged to obtain the spatial embedding vector corresponding to each attribute to be identified.

[0078] Specifically, in the field of machine learning, different evaluation indicators (i.e., different features in the feature vector) often have different dimensions and dimensional units, which will affect the results of data analysis. In order to eliminate the dimensional influence between indicators, data standardization is required to solve the comparability between data indicators. After the original data is processed by data standardization, each indicator is at the same order of magnitude, which is suitable for comprehensive comparative evaluation. Among them, the most typical is data normalization. The purpose of normalization is to limit the preprocessed data (i.e., the attention value corresponding to each attribute to be identified) to a certain range (such as [0,1] or [-1,1]), thereby eliminating the adverse effects caused by singular sample data. Singular sample data refers to sample vectors (i.e., feature vectors) that are particularly large or small relative to other input samples.

[0079] In the specific implementation, the Softmax function is used to normalize each attention value, and the attention value corresponding to each attribute to be identified is limited to the range of [0,1]. Then, the normalized attention value is used to perform weighted averaging on the semantic feature vector to obtain the spatial embedding vector corresponding to each attribute to be identified.

[0080] The technical solution provided in this embodiment normalizes each attention value to obtain a normalized attention value, eliminates the influence of singular sample data, limits the attention value within a preset range, and then performs weighted averaging on the semantic embedding vector based on the normalized attention value to obtain each spatial embedding vector, thereby improving the accuracy of determining the spatial embedding vector corresponding to each attribute to be identified.

[0081] On the basis of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, before determining the attention values ​​corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension, the method further includes:

[0082] Remove class labels from semantic embedding vectors.

[0083] In a specific implementation, there may be label information in the image to be identified, and the label information represents the attributes of the pedestrians in the image to be identified. Then the semantic embedding vector extracted from the image to be identified will also contain the label information, that is, the class label. Before determining the attention values ​​corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension, the class label in the semantic embedding vector needs to be removed to ensure that the semantic embedding vector does not contain the class label.

[0084] The technical solution provided in this embodiment removes the class label operation on the semantic embedding vector before determining the attention value corresponding to each attribute to be identified based on the semantic embedding vector and the preset dimension, thereby avoiding the influence of the class label on the calculation of the attention value and improving the accuracy of calculating the attention value corresponding to each attribute to be identified.

[0085] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the process of determining the attribute to be identified includes:

[0086] Acquire sample image data, where the sample image data includes a sample image and label information, where the label information represents attributes of a pedestrian in the sample image;

[0087] The Vision Transformer model and classifier are trained based on sample image data and the attributes to be identified are determined. The classifier is used to determine the pedestrian attribute recognition result based on the target feature vector.

[0088] In the specific implementation, the sample images and label information in the sample image data are used to train the parameters of the Vision Transformer model and the classifier, and the attributes to be identified are determined by learning the correspondence between the label information in the sample image data and the attributes of the pedestrian. In the training process of the Vision Transformer model and the classifier, the following formula is used as the loss function:

[0089]

[0090] is the loss value of the Vision Transformer model, is the loss value of the classifier, λ is a hyperparameter, C is the number of attributes to be identified, ρ n is the weight of the nth attribute to be identified, σ(·) is the sigmoid activation function, y i Represents the predicted value of the attribute to be identified.

[0091] The technical solution provided in this embodiment trains the parameters of the Vision Transformer model and the classifier through sample image data, and determines the attributes to be identified by learning the correspondence between the label information in the sample image data and the pedestrian attributes, thereby improving the effects of the Vision Transformer model and the classifier, and at the same time improving the accuracy of pedestrian attribute recognition.

[0092] Figure 3 This is a structural block diagram of a pedestrian attribute recognition device provided by an embodiment of the present application. For the sake of convenience, only the parts related to the embodiment of the present application are shown. Figure 3 The pedestrian attribute recognition device 300 may include an acquisition module 301, a semantic embedding vector determination module 302, an attention value determination module 303, a key embedding vector determination module 304, a fusion module 305 and a recognition module 306.

[0093] An acquisition module 301 is used to acquire an image to be recognized;

[0094] A semantic embedding vector determination module 302, used to extract a semantic embedding vector of the image to be identified using a Vision Transformer model;

[0095] An attention value determination module 303 is used to determine a preset dimension according to the number of attributes to be identified, and to determine an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension;

[0096] A key embedding vector determination module 304 is used to determine a spatial embedding vector corresponding to each attribute to be identified according to the semantic embedding vector and each attention value, wherein the spatial embedding vector represents the spatial position relationship of the attribute to be identified in the image to be identified;

[0097] A fusion module 305 is used to fuse the semantic embedding vector with each spatial embedding vector for each semantic embedding vector to obtain a target feature vector, wherein the target feature vector includes the association relationship between each attribute to be identified;

[0098] The recognition module 306 determines a pedestrian attribute recognition result according to the target feature vector.

[0099] A pedestrian attribute recognition device provided in an embodiment of the present application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0100] In one embodiment, the semantic embedding vector determination module 302 includes:

[0101] The image segmentation submodule is used to segment the image to be recognized based on the Vision Transformer model to obtain multiple image blocks;

[0102] A linear projection submodule, used to perform linear projection on each image block to obtain a first eigenvector corresponding to each image block;

[0103] A position encoding submodule, used for performing position encoding on each first feature vector to obtain a second feature vector corresponding to each image block;

[0104] The feature extraction submodule is used to extract the semantic embedding vector of the image to be identified according to each second feature vector.

[0105] In one embodiment, the feature extraction submodule includes:

[0106] A feature vector determination unit, used for the Vision Transformer model to determine third feature vectors corresponding to each image block by using a multi-head self-attention mechanism and each second feature vector, wherein the third feature vector represents an association relationship between each image block;

[0107] The feature extraction unit is used for the Vision Transformer model to extract the semantic embedding vector of the image to be identified by using the feedforward neural network and each third feature vector.

[0108] In one embodiment, the feature vector determination unit includes:

[0109] The correlation calculation subunit is used in the Vision Transformer model to calculate the correlation between the second eigenvectors using the multi-head self-attention mechanism;

[0110] The feature vector calculation subunit is used to perform weighted summation on each second feature vector based on the correlation between each second feature vector to obtain a third feature vector corresponding to each image block.

[0111] In one embodiment, the key embedding vector determination module 304 includes:

[0112] The attention value processing submodule is used to normalize each attention value to obtain a normalized attention value;

[0113] The spatial embedding vector determination submodule is used to perform weighted averaging on the semantic embedding vector based on the normalized attention value to obtain the spatial embedding vector corresponding to each attribute to be identified.

[0114] In one embodiment, the pedestrian attribute recognition device 300 further includes:

[0115] The class tag removal module is used to remove the class tag operation on the semantic embedding vector.

[0116] In one embodiment, the pedestrian attribute recognition device 300 further includes:

[0117] A sample image data acquisition module is used to acquire sample image data, where the sample image data includes a sample image and label information, where the label information represents attributes of pedestrians in the sample image;

[0118] The module for determining attributes to be identified is used to train the Vision Transformer model and the classifier based on sample image data and determine the attributes to be identified. The classifier is used to determine the pedestrian attribute recognition result according to the target feature vector.

[0119] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0120] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0121] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: at least one processor 40 ( Figure 4 Only one is shown in the figure), a memory 41, and a computer program 42 stored in the memory 41 and executable on at least one processor 40, the processor 40 executes the computer program 42 to implement the above Figure 1 or Figure 2 The steps in the method embodiment, or the implementation of the above Figure 3 Functions of each module / unit in the device embodiment.

[0122] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 4 may include but is not limited to a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation on the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0123] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0124] In some embodiments, the memory 41 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. In other embodiments, the memory 41 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 41 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0125] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0127] A computer-readable storage medium provided in an embodiment of the present application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0128] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0129] A computer program product provided in an embodiment of the present application has the same beneficial effects as the above-mentioned pedestrian attribute recognition method.

[0130] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0132] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A pedestrian attribute recognition method, characterized in that: The method comprises: Obtain an image to be recognized; Extracting a semantic embedding vector of the image to be identified using a Vision Transformer model; Determine a preset dimension according to the number of attributes to be identified, and determine an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; Determine, according to the semantic embedding vector and each of the attention values, a spatial embedding vector corresponding to each of the attributes to be identified, wherein the spatial embedding vector represents a spatial position relationship of the attribute to be identified in the image to be identified; For each of the semantic embedding vectors, the semantic embedding vector is fused with each of the spatial embedding vectors to obtain a target feature vector, wherein the target feature vector includes an association relationship between each of the attributes to be identified; A pedestrian attribute recognition result is determined according to the target feature vector.

2. The method according to claim 1, characterized in that The extracting the semantic embedding vector of the image to be identified by using the Vision Transformer model includes: Based on the Vision Transformer model, segment the image to be recognized to obtain multiple image blocks; Performing linear projection on each of the image blocks to obtain first eigenvectors corresponding to each of the image blocks; Performing position encoding on each of the first feature vectors to obtain a second feature vector corresponding to each of the image blocks; A semantic embedding vector of the image to be identified is extracted according to each of the second feature vectors.

3. The method according to claim 2, characterized in that The step of extracting the semantic embedding vector of the image to be identified according to each of the second feature vectors includes: The Vision Transformer model uses a multi-head self-attention mechanism and each of the second eigenvectors to determine a third eigenvector corresponding to each of the image blocks, wherein the third eigenvector represents an association relationship between the image blocks; The Vision Transformer model uses a feedforward neural network and each of the third feature vectors to extract a semantic embedding vector of the image to be identified.

4. The method according to claim 3, characterized in that The Vision Transformer model uses a multi-head self-attention mechanism and each of the second feature vectors to determine a third feature vector corresponding to each of the image blocks, including: The Vision Transformer model calculates the correlation between the second feature vectors using the multi-head self-attention mechanism; Based on the correlation between the second eigenvectors, weighted summation is performed on the second eigenvectors to obtain third eigenvectors corresponding to the image blocks.

5. The method according to claim 1, characterized in that The determining, according to the semantic embedding vector and each of the attention values, a spatial embedding vector corresponding to each of the attributes to be identified, comprises: Normalizing each of the attention values ​​to obtain a normalized attention value; The semantic embedding vectors are weighted averaged based on the normalized attention values ​​to obtain spatial embedding vectors corresponding to each of the attributes to be identified.

6. The method according to claim 1, characterized in that Before determining the attention values ​​respectively corresponding to the attributes to be identified according to the semantic embedding vector and the preset dimension, the method further includes: A class label removal operation is performed on the semantic embedding vector.

7. The method according to any one of claims 1 to 6, characterized in that: The process of determining the attribute to be identified includes: Acquire sample image data, the sample image data including a sample image and label information, the label information representing attributes of a pedestrian in the sample image; The Vision Transformer model and the classifier are trained based on the sample image data and the attributes to be identified are determined. The classifier is used to determine the pedestrian attribute recognition result according to the target feature vector.

8. A pedestrian attribute recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be recognized; A semantic embedding vector determination module, used to extract the semantic embedding vector of the image to be identified using a Vision Transformer model; An attention value determination module, used to determine a preset dimension according to the number of attributes to be identified, and determine an attention value corresponding to each attribute to be identified according to the semantic embedding vector and the preset dimension; A key embedding vector determination module, used to determine a spatial embedding vector corresponding to each of the attributes to be identified according to the semantic embedding vector and each of the attention values, wherein the spatial embedding vector represents a spatial position relationship of the attribute to be identified in the image to be identified; A fusion module, for fusing the semantic embedding vector with each of the spatial embedding vectors for each of the semantic embedding vectors to obtain a target feature vector, wherein the target feature vector includes an association relationship between each of the attributes to be identified; The recognition module determines a pedestrian attribute recognition result according to the target feature vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images