Pedestrian attribute identification method and system based on feature suppression and orthogonal constraint
By using Vision Transformer and selective feature activation method in pedestrian attribute recognition technology, combined with orthogonal feature activation loss, the problem of feature representation imbalance and overfitting is solved, and the recognition performance of pedestrian attributes is improved.
Patent Information
- Application Number
- CN202411063968.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-26
AI Technical Summary
Existing pedestrian attribute recognition technology is difficult to effectively capture remote contextual relationships and secondary attributes in pedestrian images, resulting in imbalance in feature representation, major attributes are overemphasized while minor attributes are ignored, resulting in overfitting and performance degradation.
Vision Transformer (ViT) is used as the backbone network, combining the selective feature activation method (SFAM) and orthogonal feature activation loss (LOFA) to extract more robust features and reduce feature correlations, ensuring that all attribute-specific regions are concerned , and the LOFA loss enhances the robustness and diversity of feature representations are introduced through the SFAM module without parameter suppression model attention to the main attribute area.
Effectively capture remote contextual relationships between specific areas of different attributes, reduce overfitting, improve attention to secondary attributes, enhance the robustness and diversity of feature representations, and improve the overall performance of pedestrian attribute recognition.
Smart Images

Figure CN119027979A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of pedestrian attribute recognition, and in particular relates to a pedestrian attribute recognition method and system based on feature suppression and orthogonal constraints. Background Art
[0002] Pedestrian attribute recognition (PAR) is a multi-label classification problem that aims to predict a set of semantic attributes such as age, gender, hairstyle, etc. Due to its wide applications in surveillance and public safety, many efforts have been made to improve its performance in real-world scenarios. Current research in this field shows that PAR remains challenging. In different pedestrian images, the same attribute captured has significant intra-attribute differences due to changes in background, illumination, viewpoint, and pose. Specifically, even the same pedestrian attribute has different representations in different pedestrian images. The intra-attribute variations can lead to redundancy, irrelevance, and noise in attribute-related features.
[0003] Existing PAR works use CNNs, such as ResNet50 or InceptionNet, to obtain attribute-related features from attribute regions of pedestrians. Although PAR has achieved good performance due to the success of CNN architecture, the task is far from being solved due to the challenges brought by the poor quality of pedestrian images. In addition, PAR is different from general single-label image classification because it involves predicting the presence of multiple attributes in a single image, making the contextual relationships between different attributes crucial. For example, when the model determines that there is a high probability that a person is wearing a skirt, the contextual relationship may indicate a higher probability of attributes such as being female or having long hair, which are usually related to wearing a skirt. However, exploiting the contextual relationship between attributes remains challenging for CNN. This limitation mainly stems from the fact that the convolution and downsampling operators process one neighboring region at a time, which hinders the learning of global context and captures long-term dependencies between neighboring regions, even though they may contain attributes with contextual relationships.
[0004] Recently, ViT has become the first choice for various computer vision tasks due to its impressive performance and versatility. Unlike CNNs that rely on convolution and downsampling operators, ViT utilizes a multi-head self-attention module to process different regions simultaneously. This allows ViT to capture long-range dependencies and preserve fine-grained details. Despite ViT's ability to establish contextual relationships, it is still necessary to ensure that the extracted features accurately represent all attributes. This is mainly due to the problem of imbalanced attribute distribution, which has an adverse impact on the performance of PAR. Specifically, the PAR model may tend to prioritize capturing more frequently annotated attributes, such as hairstyle, clothing type, while less frequently annotated attributes, such as glasses, backpacks, etc., may receive less attention or be ignored. In other words, the attributes of the primary annotations may be overemphasized, resulting in overfitting, while the attributes of the secondary annotations may be ignored. Therefore, another key issue in PAR is how to further mine more comprehensive attribute representations based on ViT.
[0005] In view of this, it is very meaningful to propose a pedestrian attribute recognition method and system based on feature suppression and orthogonal constraints. Summary of the invention
[0006] In order to solve the problem that the existing PAR methods usually rely on convolutional neural networks (CNNs) as the backbone network to extract pedestrian features, but CNN only processes one adjacent area at a time, resulting in the loss of long-range mutual relations between specific areas of different attributes, the present invention provides a pedestrian attribute recognition method and system based on feature suppression and orthogonal constraints, which utilizes Vision Transformer (ViT) instead of CNN as the backbone of PAR, aiming to establish a long-range relationship model and extract more robust features. However, PAR has an inherent attribute imbalance problem, which causes ViT to naturally pay more attention to attributes with higher frequency in the training set, while ignoring some pedestrian attributes with lower frequency. The features extracted by ViT cannot tolerate the problem of unbalanced attribute distribution, and the present invention is intended to solve this problem.
[0007] In a first aspect, the present invention proposes a pedestrian attribute recognition method based on feature suppression and orthogonal constraints, the method comprising the following steps:
[0008] Take any image in the acquired dataset D as the input of the backbone network ViT to identify multiple pedestrian attributes;
[0009] Use multiple Transformer encoding layers to extract pedestrian attribute features from the input image and output features f;
[0010] The output feature f is then used as the input of the selective feature activation method SFAM, and the output feature F S ;
[0011] Finally, the two output features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result.
[0012] Preferably, the taking any image in the acquired data set D as the input of the backbone network ViT further includes further processing the image:
[0013] Let the image be x i , whose corresponding label is y i ∈{0,1} M , M represents the number of categories of pedestrian attributes, and the corresponding label y i Use 1 to mark the negative, and 0 to mark the negative.
[0014] The image is divided into Z image blocks of size P*P*C, Z = (H*W) / (P*P), where P represents the size of the image block, C represents the number of image channels, H represents the height of the image, and W represents the width of the image. A learnable marker is added to form the input of the backbone network ViT. Represents Z in The dimension size is (Z+1)×C.
[0015] Further preferably, the output feature f is used as the input of the selective feature activation method SFAM, and the output feature F S include:
[0016] The output features of the backbone network ViT First, the output features Converted into a four-dimensional feature tensor Where B represents the image batch size during training, C, H, and W represent the number of channels, height, and width of the feature, respectively;
[0017] Calculate the activation F of each feature in the feature vector A , expressed as:
[0018]
[0019] Among them, F i =F(i,:,:),F i Indicates that each feature size is a slice of H*W, and p is set to 2;
[0020] Calculate the average of the activation values of the jth row to represent each stripe r j The correlation is expressed as:
[0021]
[0022] Among them, F A represents feature activation, and W represents width;
[0023] All the calculated r j Values are ranked from smallest to largest.
[0024] Further preferably, it also includes:
[0025] The selective feature activation method SFAM adopts a suppression factor τ to select the first (τ×H) features for a certain degree of suppression, that is, among the first (τ×H) feature values, its feature value is scaled by a scaling factor α, and if it is not the first (τ×H) feature value, its feature value remains unchanged.
[0026] Further preferably, the two features f and F are outputted S They are sent to the linear classification layer for attribute probability prediction, including:
[0027] Using the orthogonal feature activation method, the loss L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S Mutual enhancement, the orthogonal feature activation method loses L OFA It is expressed as:
[0028]
[0029] Among them, B represents the image batch size during training, j represents a channel, and the range is 1<=j<=C. <f ij ·F sij > represents two features.
[0030] Further preferably, the method further includes: regarding the entire pedestrian attribute recognition problem as a multi-label classification task, and using a binary cross entropy BCELoss as a loss function. The binary cross entropy loss function BCELoss is expressed as follows:
[0031]
[0032]
[0033] Among them, N represents the number of samples, M represents the number of attributes, Represents image x i Enter the algorithm framework, and perform i The predicted probability value of the jth attribute in , y i,j represents the attribute label, σ is the preset activation function, ω j represents the imbalance suppression factor, ω j As shown below:
[0034]
[0035] Among them, r j Represents the proportion of positive samples of the j-th attribute in the training set.
[0036] Preferably, adding the two probability predictions to obtain a final prediction result includes: directly adding the two obtained prediction probability values to obtain a final prediction result.
[0037] In a second aspect, an embodiment of the present invention provides a pedestrian attribute recognition system based on feature suppression and orthogonal constraints, comprising:
[0038] A backbone network ViT module is configured to use any image in the acquired data set D as an input of the backbone network ViT to identify multiple pedestrian attributes, extract pedestrian attribute features from the input image using multiple Transformer encoding layers, and output features f;
[0039] A selective feature activation method SFAM module is configured to use the output feature f as an input of the selective feature activation method SFAM, and output the feature F S ;
[0040] The probability prediction module is configured to output two features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result;
[0041] Orthogonal feature activation method loss L OFA module, configured to use the orthogonal feature activation method loss L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S mutual reinforcement.
[0042] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.
[0044] Compared with the prior art, the beneficial results of the present invention are:
[0045] Different from the existing PAR method, the method of the present invention uses ViT to capture the long-range contextual relationship between specific regions of different attributes; a parameter-free SFAM module is proposed, which first calculates the model's attention to each region and suppresses the attribute regions with high attention to a certain extent, so that the model can pay attention to more attributes that are easy to ignore; in addition, L OFA Loss is used to enhance the robustness and diversity of feature representations from different feature spaces. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.
[0047] Figure 1 is an overall framework diagram of the pedestrian attribute recognition method based on feature suppression and orthogonal constraints of the embodiment of the present invention;
[0048] Figure 2 A schematic flow chart of a pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to an embodiment of the present invention;
[0049] Figure 3 A schematic diagram of the architecture of a pedestrian attribute recognition system based on feature suppression and orthogonal constraints according to an embodiment of the present invention;
[0050] Figure 4 It is a schematic diagram of the structure of a computer device suitable for implementing an electronic device of an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0053] The present invention proposes a pedestrian attribute recognition method and system based on feature suppression and orthogonal constraints. The overall framework diagram is as follows: Figure 1As shown in the figure, the whole method mainly consists of three main modules: backbone network (ViT), selective feature activation method (SFAM in the figure), orthogonal feature activation method loss (L OFA ).for
[0054] First, Figure 2 It is shown that an embodiment of the present invention discloses a pedestrian attribute recognition method based on feature suppression and orthogonal constraints, such as Figure 2 As shown, the method comprises the following steps:
[0055] S1. Take any image in the acquired dataset D as the input of the backbone network ViT to identify multiple pedestrian attributes;
[0056] Specifically, taking any image in the acquired data set D as the input of the backbone network ViT also includes further processing the image:
[0057] S11, the image is recorded as x i , whose corresponding label is y i ∈{0,1} M , M represents the number of categories of pedestrian attributes, and the corresponding label y i Use 1 to mark the negative, and 0 to mark the negative.
[0058] S12, the image is divided into Z image blocks of size P*P*C, Z = (H*W) / (P*P), where P represents the size of the image block, C represents the number of image channels, H represents the height of the image, and W represents the width of the image, plus a learnable marker, which is combined into the input of the backbone network ViT Represents Z in The dimension size is (Z+1)×C.
[0059] Above Figure 1 Receive any image x in the dataset D i As input, it aims to recognize multiple pedestrian attributes.
[0060] S2, using multiple Transformer encoding layers to extract pedestrian attribute features from the input image and output features f;
[0061] S3, then use the output feature f as the input of the selective feature activation method SFAM, and output feature F S ;
[0062] Specifically, the step includes:
[0063] S31, output characteristics of the backbone network ViT First, the output features Converted into a four-dimensional feature tensor Where B represents the image batch size during training, C, H, and W represent the number of channels, height, and width of the feature, respectively;
[0064] S32, calculate the activation F of each feature in the feature vector A , expressed as:
[0065]
[0066] Among them, F i =F(i,:,:),F i Indicates that each feature size is a slice of H*W, and p is set to 2;
[0067] Calculate the average of the activation values of the jth row to represent each stripe r j The correlation is expressed as:
[0068]
[0069] Among them, F A represents the activation of the feature, and W represents the width;
[0070] S33, all the calculated r j Values are ranked from smallest to largest.
[0071] In a specific embodiment, all the calculated r j The values are sorted from smallest to largest, and then the first few values are suppressed.
[0072] Furthermore, it also includes:
[0073] The selective feature activation method SFAM adopts a suppression factor τ to select the first (τ×H) features for a certain degree of suppression, that is, among the first (τ×H) feature values, its feature value is scaled by a scaling factor α, and if it is not the first (τ×H) feature value, its feature value remains unchanged.
[0074] S4, finally output the two features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result.
[0075] Specifically, the orthogonal feature activation method is used to lose L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S Mutual enhancement, the orthogonal feature activation method loses L OFA It is expressed as:
[0076]
[0077] Among them, B represents, j represents, <f ij ·F sij > indicates.
[0078] Furthermore, in this embodiment, the entire pedestrian attribute recognition problem is regarded as a multi-label classification task, and the binary cross entropy BCELoss is used as the loss function. The binary cross entropy loss function BCELoss is expressed as follows:
[0079]
[0080]
[0081] Among them, N represents the number of samples, M represents the number of attributes, Represents image x i Enter the algorithm framework, and perform i The predicted probability value of the jth attribute in , y i,j represents the attribute label, σ is the preset activation function, ω j represents the imbalance suppression factor, ω j As shown below:
[0082]
[0083] Among them, r j Represents the proportion of positive samples of the j-th attribute in the training set.
[0084] Preferably, adding the two probability predictions to obtain a final prediction result includes: directly adding the two obtained prediction probability values to obtain a final prediction result.
[0085] It should be noted that in other embodiments, the addition of two probability predictions may also be performed in a weighted manner.
[0086] Different from the existing PAR method, the method disclosed in the embodiment of the present invention uses ViT to capture the long-range contextual relationship between specific regions of different attributes. The embodiment of the present invention proposes a parameter-free SFAM module, which first calculates the model's attention to each region and suppresses the attribute regions with high attention to a certain extent, so that the model can pay attention to more attributes that are easy to ignore. In addition, we introduce L OFA Loss is used to enhance the robustness and diversity of feature representations from different feature spaces.
[0087] In a specific embodiment, Figure 1 As shown, in order to realize the proposed pedestrian attribute recognition method based on feature suppression and orthogonal constraints, the present invention mainly includes the following steps:
[0088] Step S10: Figure 1 Receive any image x in the dataset D i As input, its purpose is to identify multiple pedestrian attributes. The image x i The corresponding label is y i ∈{0,1} M , M represents the number of categories of pedestrian attributes, and the corresponding y i The image is divided into Z blocks of size P*P*C, Z = (H*W) / (P*P), and a learnable marker is added to form the input of the backbone network ViT.
[0089] Step S20: The entire pedestrian attribute recognition problem is regarded as a multi-label classification task, and the binary cross entropy (BCELoss) is used as the loss function. The expression of the loss function is as follows:
[0090]
[0091]
[0092] Among them, N represents the number of samples, M represents the number of attributes, Represents image x i Enter the algorithm framework, and perform i The predicted probability value of the jth attribute in , y i,j represents the attribute label, σ is the preset activation function, ω j represents the imbalance suppression factor, ω j As shown below:
[0093]
[0094] Among them, r j Represents the proportion of positive samples of the j-th attribute in the training set.
[0095] Step S30: The Selective Feature Activation Method (SFAM) proposed in the present invention aims to learn a more robust feature representation and ensure that all attribute-specific regions can be learned. SFAM receives the output features of ViT As input, first Converted into a four-dimensional feature tensor C, H, and W represent the number of channels, height, and width of the feature, respectively, and B represents the image batch size during training. Next, we calculate the activation F of each feature in the feature vector A , which can be expressed as:
[0096]
[0097] Among them, F i =F(i,:,:),F i Indicates that each feature size is a slice of H*W, and p is set to 2;
[0098] Calculate the average of the activation values of the jth row to represent each stripe r j The correlation is expressed as:
[0099]
[0100] Among them, F A represents the activation of the feature, and W represents the width;
[0101] All the calculated r j The values are ranked from small to large. SFAM uses a suppression factor τ to select the first (τ×H) features for a certain degree of suppression. That is, among the first (τ×H) feature values, its feature value is scaled by a scaling factor α. If it is not the first (τ×H) feature value, its feature value remains unchanged. Through SFAM, we get feature F S .
[0102] Step S40: Our method generates two features, one (f) from the backbone network ViT and the other (F S ) comes from the output of SFAM. These two features are sent to a linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result. We expect these two attribute predictions to complement each other, which means that f and F S can encode attributes from a more diverse feature space. Since f and F S By sharing the final classification layer, the PAR model may encourage the same distribution of the two features to quickly minimize the final loss. We believe that two highly correlated features cannot complement each other. The orthogonal feature activation method loss (L in the figure) OFA ) is used to reduce the correlation of features and make the feature space more comprehensive. This can promote the mutual enhancement of the two characteristics, further increase their complementarity, and ultimately improve the overall effectiveness and adaptability of the PAR method. OFA It can be expressed as:
[0103]
[0104] Among them, B represents the image batch size during training, j represents a channel, and the range is 1<=j<=C. <f ij ·F sij > represents two features.
[0105] In a second aspect, the embodiment of the present invention further discloses a pedestrian attribute recognition system based on feature suppression and orthogonal constraints, such as Figure 3 As shown, it includes: a backbone network ViT module 31, a selective feature activation method SFAM module 32, a probability prediction module 33 and an orthogonal feature activation method loss L OFA Module 34.
[0106] In a specific embodiment, the backbone network ViT module 31 is configured to use any image in the acquired data set D as the input of the backbone network ViT to identify multiple pedestrian attributes, use multiple Transformer encoding layers to extract pedestrian attribute features from the input image and output feature f; the selective feature activation method SFAM module 32 is configured to use the output feature f as the input of the selective feature activation method SFAM, and output feature F S ; Probability prediction module 33, configured to output the two features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added to get the final prediction result; the orthogonal feature activation method loss L OFA Module 34 is configured to use an orthogonal feature activation method loss L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S mutual reinforcement.
[0107] Reference below Figure 4 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer device 600 (a server or terminal device as shown). Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0108] like Figure 4 As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 603 or a program loaded from a storage part 609 to a random access memory (RAM) 604. In the RAM 604, various programs and data required for the operation of the device 600 are also stored. The CPU 601, GPU 602, ROM 603 and RAM 604 are connected to each other through a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.
[0109] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, etc.; an output section 608 including a display such as a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 609 including a hard disk, etc.; and a communication section 610 including a network interface card such as a LAN card, a modem, etc. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 611 as needed so that a computer program read therefrom is installed into the storage section 609 as needed.
[0110] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 610, and / or installed from the removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processing unit (GPU) 602, the above-mentioned functions defined in the method of the present invention are executed.
[0111] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus or device, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution device, apparatus or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution device, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0112] Computer program code for performing the operations of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0113] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0114] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and the modules described may also be arranged in a processor.
[0115] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: uses any image in the acquired data set D as the input of the backbone network ViT to identify multiple pedestrian attributes; uses multiple Transformer encoding layers to extract pedestrian attribute features from the input image and outputs feature f; and then uses the output feature f as the input of the selective feature activation method SFAM to output feature F S ; Finally, the two features f and F are output S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result.
[0116] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.
Claims
1. A pedestrian attribute recognition method based on feature suppression and orthogonal constraints, characterized in that: The method comprises the following steps: Take any image in the acquired dataset D as the input of the backbone network ViT to identify multiple pedestrian attributes; Use multiple Transformer encoding layers to extract pedestrian attribute features from the input image and output features f; The output feature f is then used as the input of the selective feature activation method SFAM, and the output feature F S ; Finally, the two output features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result.
2. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 1 is characterized in that: The method of using any image in the acquired data set D as the input of the backbone network ViT also includes further processing the image: Let the image be x i , whose corresponding label is y i ∈{0,1} M , M represents the number of categories of pedestrian attributes, and the corresponding label y i Use 1 to mark the negative, and 0 to mark the negative. The image is divided into Z image blocks of size P*P*C, Z = (H*W) / (P*P), where P represents the size of the image block, C represents the number of image channels, H represents the height of the image, and W represents the width of the image. A learnable marker is added to form the input of the backbone network ViT. Represents Z in The dimension size is (Z+1)×C.
3. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 2 is characterized in that: The output feature f is then used as the input of the selective feature activation method SFAM, and the output feature F S include: The output features of the backbone network ViT First, the output features Converted into a four-dimensional feature tensor Where B represents the image batch size during training, C, H, and W represent the number of channels, height, and width of the feature, respectively; Calculate the activation F of each feature in the feature vector A , expressed as: Among them, F i =F(i,:,:),F i Indicates that each feature size is a slice of H*W, and p is set to 2; Calculate the average of the activation values of the jth row to represent each stripe r j The correlation is expressed as: Among them, F A represents the activation of the feature, and W represents the width; All the calculated r j Values are ranked from smallest to largest.
4. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 3 is characterized in that: Also includes: The selective feature activation method SFAM adopts a suppression factor τ to select the first (τ×H) features for a certain degree of suppression, that is, among the first (τ×H) feature values, its feature value is scaled by a scaling factor α. If it is not the first (τ×H) feature value, Its characteristic value remains unchanged.
5. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 4 is characterized in that: Finally, the two output features f and F S They are sent to the linear classification layer for attribute probability prediction, including: Using the orthogonal feature activation method, the loss L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S Mutual enhancement, the orthogonal feature activation method loses L OFA It is expressed as: Among them, B represents the image batch size during training, j represents a channel, and the range is 1<=j<=C. <f ij ·F sij > represents two features.
6. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 2 is characterized in that: Also includes: The entire pedestrian attribute recognition problem is regarded as a multi-label classification task, and the binary cross entropy BCELoss is used as the loss function. The binary cross entropy loss function BCELoss is expressed as follows: Among them, N represents the number of samples, M represents the number of attributes, Represents image x i Enter the algorithm framework, and perform i The predicted probability value of the jth attribute in , y i,j represents the attribute label, σ is the preset activation function, ω j represents the imbalance suppression factor, ω j As shown below: Among them, r j Represents the proportion of positive samples of the j-th attribute in the training set.
7. The pedestrian attribute recognition method based on feature suppression and orthogonal constraints according to claim 1 is characterized in that: Then add the two probability predictions together to get the final prediction results including: The two predicted probability values are directly added together to obtain the final prediction result.
8. A pedestrian attribute recognition system based on feature suppression and orthogonal constraints, characterized in that: include: A backbone network ViT module is configured to use any image in the acquired data set D as an input of the backbone network ViT to identify multiple pedestrian attributes, extract pedestrian attribute features from the input image using multiple Transformer encoding layers, and output features f; A selective feature activation method SFAM module is configured to use the output feature f as an input of the selective feature activation method SFAM, and output the feature F S ; The probability prediction module is configured to output two features f and F S They are sent to the linear classification layer for attribute probability prediction, and then the two probability predictions are added together to get the final prediction result; Orthogonal feature activation method loss L OFA module, configured to use the orthogonal feature activation method loss L OFA To reduce feature correlation, make the feature space more comprehensive, and promote the two characteristics f and F S mutual reinforcement.
9. An electronic device, comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
