Pedestrian attribute identification method based on image-attribute and frequency domain-space domain dual collaborative learning

Abstract

The invention discloses a pedestrian attribute identification method based on image-attribute and frequency domain-space domain dual collaborative learning, and relates to the technical field of computer vision. An image-attribute collaborative learning framework is provided, and visual information and attribute tags are integrated to perceive finer semantic understanding of pedestrian attributes; an attribute vector representation is obtained through learnable attribute cues, a pedestrian image is input into an image encoder to obtain a visual vector representation, the attribute vector representation and the visual vector representation are mapped into a joint image-attribute collaboration space, and semantic association between the image and the attribute is learned through contrastive learning loss. A frequency domain-spatial domain collaborative learning module is provided, interactive guide learning is carried out by using a cross attention mechanism involving a frequency domain amplitude spectrum component and a phase spectrum component, and collaborative learning with spatial information is carried out. And frequency-space double-domain information in the pedestrian image is comprehensively explored and utilized to obtain more robust visual features. The method has significant advantages in the aspect of improving pedestrian attribute recognition task performance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pedestrian attribute recognition method based on image-attribute and frequency-space domain dual collaborative learning. Background Art

[0002] Given a person image, the goal of pedestrian attribute recognition task is to identify a set of semantic attributes (such as age, gender and clothing) to describe the characteristics of the pedestrian. With the continuous development of deep learning technology, the performance of pedestrian attribute recognition task has been significantly improved. However, in practical applications, pedestrian attribute recognition task still faces some challenges, such as complex background, changes in illumination and changes in posture, which makes pedestrian attribute recognition task more challenging. Therefore, the key to pedestrian attribute recognition task is to accurately mine robust feature representation of pedestrian images. Recently, multimodal learning models based on vision-language pre-training have attracted more and more attention, which aims to combine vision with its attribute information to provide richer and more comprehensive feature representation. Through the multimodal learning model with vision-language pre-training, semantic information can be effectively extracted from images, thereby greatly improving the accuracy and robustness of downstream tasks. In the pedestrian attribute recognition task, the attributes of pedestrians usually contain multiple words, which contain rich semantic information that needs to be explored. However, traditional pedestrian attribute recognition methods often use pedestrian attribute information as labels to optimize the output features of pedestrian attribute recognition network. Although these methods have achieved good results, they rarely consider combining attribute information with images for collaborative learning.

[0003] In addition, frequency domain information can focus on the distribution and intensity of different frequency components in the image signal. By analyzing the frequency domain information, features related to pedestrian attributes, such as texture, edges, etc., can be extracted. These features tend to show more significant and stable performance in the frequency domain, which helps to more accurately identify pedestrian attributes. Similarly, spatial domain information can focus on the spatial distribution characteristics of pixels in the image. By analyzing the spatial domain information, attribute information such as the shape, posture, and clothing of pedestrians can be obtained. Therefore, frequency domain and spatial domain information can complement each other in the task of pedestrian attribute recognition and jointly improve the accuracy and stability of recognition. By combining the feature extraction methods in the frequency domain and spatial domain, the pedestrian attribute information can be more comprehensively described, thereby improving the recognition performance. However, frequency domain and spatial collaborative learning are rarely involved in pedestrian attribute recognition tasks. Summary of the invention

[0004] The purpose of the present invention is to provide a pedestrian attribute recognition method based on image-attribute and frequency-space domain dual collaborative learning to address the above problems in the prior art. The method integrates visual information and attribute labels, combines the complementary advantages of frequency domain and space domain information, and realizes fine recognition and accurate classification of pedestrian attributes.

[0005] The pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency domain-spatial domain of the present invention comprises the following steps:

[0006] 1) An image-attribute co-learning framework is proposed, which integrates visual information and attribute labels to perceive a more refined semantic understanding of pedestrian attributes. The image-attribute co-learning framework obtains attribute vector representations through a series of learnable attribute cues, inputs pedestrian images into an image encoder to obtain visual vector representations, and maps both attribute vector representations and visual vector representations into a joint image-attribute co-space, and learns the semantic association between images and attributes through contrastive learning loss.

[0007] 2) A frequency-domain-spatial-domain collaborative learning module is proposed to comprehensively explore and utilize the diversified information of pedestrian images in both frequency and spatial domains to achieve robust feature extraction and attribute recognition and obtain more robust visual features; a cross-attention mechanism involving frequency-domain amplitude spectrum components and phase spectrum components is used to interactively guide learning and collaboratively learn with spatial information.

[0008] In step 1), an image-attribute collaborative learning framework is proposed, comprising the following steps:

[0009] (1) Learnable attribute hints: First, the attribute labels are converted into learnable attribute hints, which are constructed as follows: a person has [xxx] attributes. The learnable attribute hints are then input into the text encoder of the contrastive language-image pre-training model (CLIP) to generate the corresponding text vector representation of the pedestrian attributes.

[0010] (2) Visual vector representation extraction: The pedestrian image is divided into a series of fixed-size, non-overlapping blocks, with a total of blocks, where P represents the block size; these block sequences are converted into 1D vectors through trainable linear projection, and these 1D vectors are input into the CLIP image encoder to obtain visual vector representation;

[0011] (3) Using a two-stage training strategy training framework:

[0012] Phase 1: Freeze the parameters of the text encoder and image encoder, fine-tune the learnable attribute cues of the text in the dataset to better capture the semantic information of the attributes; optimize the framework through multi-classification contrastive learning loss, including image-to-attribute loss and attribute-to-image loss;

[0013] The second stage: fix the learnable attribute hints and text encoders, optimize the parameters of the image encoder, and improve the image encoder's ability to capture attribute information; use binary cross entropy loss and image-attribute similarity score to jointly optimize the classifier output, improve recognition accuracy, better capture and understand the attribute information in pedestrian images, and enable the extracted visual features to more accurately match the attribute features;

[0014] Furthermore, the first stage focuses on training learnable attribute cues; in this stage, the parameters of the text encoder and the image encoder are frozen and do not participate in training, and the text cues in the dataset are fine-tuned; by fine-tuning the text cues, the proposed image-attribute co-learning framework can better understand the semantic information of the attributes; specifically, each learnable attribute cue corresponds to an attribute label, and each pedestrian image corresponds to multiple attribute cues; a multi-classification contrastive learning loss is designed to optimize the proposed image-attribute co-learning framework, which can be expressed as follows:

[0015] (1)

[0016] in, Representing images To properties Multi-class contrastive learning loss, property To Image Multi-class contrastive learning loss; A label representing a given pedestrian attribute; represents the index of images and attributes in a mini-batch during training, Indicates the first Attributes; Indicates the similarity between two vector representations; Represents the first Feature representation of an image; Indicates the first Attributes represent;

[0017] Since the proposed image-attribute and frequency-space dual collaborative learning method adopts a frequency-space dual-branch learning strategy, the feature representation of the frequency domain is assumed as , the feature of the spatial domain is expressed as , then the overall loss function of the first stage can be expressed as follows:

[0018] (2)

[0019] Furthermore, the second stage training process will focus on optimizing the image encoder; in this stage, the learnable attribute hints and the parameters of the text encoder are fixed and no longer updated; the present invention uses the attribute vector representation learned in the first stage as a stable attribute reference and focuses on improving the performance of the image encoder; in the second stage, by optimizing the parameters of the image encoder to better capture and understand the attribute information in the pedestrian image, the extracted visual features can more accurately match the attribute features; the image encoder better understands the image content and establishes a more accurate correspondence with the attributes;

[0020] In the second stage, the binary cross entropy loss is used to optimize the attribute categories output by the classifier layer of the proposed method. , its mathematical expression is as follows:

[0021] (3)

[0022] in, Represents the attribute label of the pedestrian image, Represents the sigmoid function, which is used to convert the attribute category output by the classifier layer Convert to probability value;

[0023] In the second stage, in order to fully explore the semantic correspondence between pedestrian images and pedestrian attributes, the image vector representation is calculated To attribute vector representation The similarity score between , and used it to optimize the predicted value of the proposed method, and its mathematical expression is as follows:

[0024] (4)

[0025] The total loss in the second stage can be expressed as:

[0026] (5)

[0027] in, is a weight hyperparameter used to balance the loss function and The optimization weight of .

[0028] In step 2), the frequency domain-spatial domain collaborative learning module is as follows:

[0029] Features of the CLIP image encoder output , first transform it from the spatial domain to the frequency domain using the Fast Fourier Transform (FFT) to obtain the amplitude spectrum components and phase spectrum components , can be expressed mathematically as follows:

[0030] (6)

[0031] in, and They correspond to the fast Fourier transform FFT ( )’s real and imaginary parts;

[0032] In the frequency domain, the amplitude spectrum components Related to certain attributes of pedestrians, such as clothing texture, facial features, etc.; Phase spectrum component It can help to more accurately identify the pedestrian's profile, posture and other attributes; considering the amplitude spectrum component and phase spectrum components Contains different and diverse attribute information, through the amplitude spectrum component and phase spectrum components Interactive learning is performed to fully promote mutual learning between the two components; wherein the interactive operation is achieved by using cross-multi-head attention (MCA); specifically, the amplitude spectrum component is set to and , phase spectrum component is set to and , then the mathematical expression of the interaction operation can be expressed as:

[0033] (7)

[0034] in, and Indicates two convolutional layer; next, the features improved by the above interactions are transformed back to the spatial domain through inverse fast Fourier transform and added to the matrix with the spatial domain features to complete the collaborative learning of the spatial domain and frequency domain;

[0035] Then, the amplitude spectrum components obtained through the above interaction operation are transformed by inverse fast Fourier transform (IFFT) and phase spectrum components Transformed into the spatial domain, the improved features are obtained , and spatial domain characteristics They are used together for collaborative learning in the frequency-space domain to fully utilize the frequency-space domain information of attributes to improve performance.

[0036] Compared with the prior art, the present invention has the following outstanding technical effects and advantages:

[0037] 1. Improve recognition accuracy: Through the image-attribute collaborative learning framework, the framework integrates visual information and attribute labels to perceive a more refined semantic understanding of pedestrian attributes, which can more accurately capture the semantic association between images and attributes and improve the accuracy of attribute recognition.

[0038] 2. Enhanced robustness: This paper proposes a frequency-space domain collaborative learning module, which aims to fully explore and utilize the frequency-space dual-domain information in pedestrian images to obtain more robust visual features. This module uses a cross-attention mechanism involving frequency domain amplitude spectrum components and phase spectrum components to interactively guide learning, and then conducts collaborative learning with spatial information. It makes full use of the diverse information of the image in the frequency domain and spatial domain to improve the robustness and generalization ability of feature extraction.

[0039] 3. Two-stage training strategy: Through phased training, first optimize the semantic information of attribute prompts, then optimize the performance of the image encoder, achieve gradual optimization from coarse to fine, and further improve the recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The invention discloses a pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency-space domain.

[0041] Figure 2 It is a flow chart of the two-stage training strategy of the present invention.

[0042] Figure 3 This is a flow chart of the frequency domain-spatial domain collaborative learning module of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the following embodiments will further illustrate the present invention in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] The overall process of implementing the present invention comprises the following steps:

[0045] The present invention mainly includes two parts: an image-attribute collaborative learning module and a frequency-space domain collaborative learning module. First, an image-attribute collaborative learning framework is proposed, which integrates visual information and attribute labels to perceive a more refined semantic understanding of pedestrian attributes. Specifically, the image-attribute collaborative learning framework obtains attribute vector representations through a series of learnable attribute prompts, and then inputs the pedestrian image into the image encoder to obtain a visual vector representation, and maps both the attribute vector representation and the visual vector representation into a joint image-attribute collaborative space, and learns the semantic association between the image and the attribute through contrastive learning loss. In addition, the present invention proposes a frequency-space domain collaborative learning module, which uses a cross-attention mechanism involving frequency domain amplitude spectrum components and phase spectrum components for interactive guided learning, and then performs collaborative learning with spatial information. The proposed frequency-space domain collaborative learning module aims to fully explore and utilize the frequency-space dual-domain information in pedestrian images to obtain more robust visual features.

[0046] 1. The image-attribute collaborative learning framework, such as Figure 1 As shown, the following steps are included:

[0047] 1) Learnable attribute prompts

[0048] Due to the lack of text description in the pedestrian attribute recognition task, the accuracy and efficiency of recognition will be greatly affected. In view of this, the present invention proposes an innovative method. First, the attribute labels are processed and converted into learnable attribute prompts. The attribute prompts here have a specific construction form, namely: "a person has [xxx] attributes". This construction form aims to concretize the abstract attribute labels to make them more consistent with the logic of model processing. Among them, [x xx] represents specific attribute content, which can be information related to various pedestrian attributes such as age, gender, clothing style, etc. In this way, the attribute labels are given richer semantic connotations.

[0049] These carefully constructed learnable attribute cues are then input into the text encoder of the contrastive language-image pre-training model (CLIP). The CLIP model has excellent ability in processing the relationship between natural language and images, and its text encoder can deeply encode the input text to mine the semantic information contained therein. In this process, through the processing of the text encoder, the learnable attribute cues are converted into corresponding text vector representations of pedestrian attributes. These text vector representations can accurately reflect the information carried by the attribute cues, providing an important foundation for the subsequent pedestrian attribute recognition task.

[0050] 2) Visual vector representation extraction part

[0051] For pedestrian images, we first divide them into a series of fixed-size, non-overlapping blocks, with a total of blocks, where P represents the block size. These 2D image data, which originally existed in the form of blocks, are then converted into 1D vectors through trainable linear projection. These 1D vectors are then input into the CLIP image encoder to obtain visual vector representations. The CLIP image encoder is a highly professional module that has been trained with a large amount of data. It can perform complex calculations and analyses on the input vectors to extract the visual-related feature information and finally obtain visual vector representations. These visual vector representations can accurately characterize the visual features of pedestrian images, providing a key basis for the subsequent fusion with text vector representations and pedestrian attribute recognition tasks.

[0052] 3) Two-stage training strategy

[0053] As mentioned above, due to the lack of text description in the pedestrian attribute recognition task, the present invention innovatively proposes to convert attribute labels into learnable attribute prompts. In this way, the image and attributes can be organically combined to perceive higher-level and richer semantic information. However, it should be noted that the unoptimized learnable attribute prompts in the initial state have obvious limitations. It cannot effectively capture semantic information, which leads to challenges for the model in understanding and processing pedestrian attribute-related content. Therefore, in order to overcome this difficulty, the present invention carefully designs and adopts a two-stage training strategy to train the proposed framework. The specific situation is as follows Figure 2 shown.

[0054] The first stage focuses on optimizing the learnable attribute cues. In this stage, in order to ensure the pertinence and stability of the training, the parameters of the text encoder and the image encoder are frozen so that they do not participate in the training process. At the same time, the focus is on optimizing the learnable attribute cues. Through this targeted fine-tuning, the proposed image-attribute collaborative learning framework can gradually establish a deeper and more accurate understanding of the attribute semantic information. Specifically, in the design of the present invention, each learnable attribute cue corresponds to a specific attribute label, and each pedestrian image corresponds to multiple attribute cues. Based on this structural feature, the present invention further designs a multi-classification contrastive learning loss to optimize the proposed image-attribute collaborative learning framework, and its specific expression can be expressed as:

[0055] (1)

[0056] in, Representing images To properties Multi-class contrastive learning loss, property To Image Multi-class contrastive learning loss. A label representing a given pedestrian attribute. represents the index of images and attributes in a mini-batch during training, Indicates the first attributes. Indicates the similarity between two vector representations; Represents the first Feature representation of an image. Indicates the first An attribute representation.

[0057] Since the proposed image-attribute and frequency-space dual collaborative learning method adopts a frequency-space dual-branch learning strategy, the feature representation of the frequency domain is assumed as , the feature of the spatial domain is expressed as , then the overall loss function of the first stage can be expressed as follows:

[0058] (2)

[0059] The second phase of the training process will focus on optimizing the image encoder. In this phase, the learnable attribute cues and the parameters of the text encoder are fixed and no longer updated. The present invention uses the attribute vector representation learned in the first phase as a stable attribute reference and focuses on improving the performance of the image encoder. In the second phase, the parameters of the image encoder are optimized to better capture and understand the attribute information in the pedestrian image, so that the extracted visual features can more accurately match the attribute features.

[0060] In this way, the image encoder can better understand the image content and establish more accurate correspondence with the attributes. In the second stage, the binary cross entropy loss is used to optimize the attribute categories output by the classifier layer of the proposed method. , its mathematical expression is as follows:

[0061] (3)

[0062] in, Represents the attribute label of the pedestrian image, Represents the sigmoid function, which is used to convert the attribute category output by the classifier layer Convert to probability value.

[0063] In addition, in the second stage, in order to fully explore the semantic correspondence between pedestrian images and pedestrian attributes, the present invention calculates the image vector representation To attribute vector representation The similarity score between , and used it to optimize the predicted value of the proposed method, and its mathematical expression is as follows:

[0064] (4)

[0065] Therefore, the total loss of the second stage can be expressed as:

[0066] (5)

[0067] The frequency-spatial collaborative learning module (FSCL) aims to fully exploit the diverse information of pedestrian images in the frequency and spatial domains to achieve robust feature extraction and attribute recognition. In the pedestrian attribute recognition (PAR) task, the effective use of image information is crucial for accurately identifying pedestrian attributes, while frequency domain information is often neglected in previous studies. The design of the FSCL module is to make up for this deficiency by deeply exploring the information in the frequency and spatial domains to provide the model with a more comprehensive feature representation. Specifically, Figure 3 As shown, the features of the CLIP image encoder output are , first transform it from the spatial domain to the frequency domain using the Fast Fourier Transform (FFT) to obtain the amplitude spectrum components and phase spectrum components The amplitude spectrum component reflects the intensity information of different frequency components, while the phase spectrum component contains the location information of the image structure. Both components are important for understanding the intrinsic characteristics of the image and the potential connection with pedestrian attributes. The transformation from the spatial domain to the frequency domain can be expressed mathematically as follows:

[0068] (6)

[0069] in and They correspond to the fast Fourier transform FFT ( )'s real and imaginary parts.

[0070] In the frequency domain, the amplitude spectrum components Related to certain attributes of pedestrians, such as clothing texture, facial features, etc. Phase spectrum component It can help to more accurately identify the pedestrian's profile, posture and other attributes. Considering the amplitude spectrum component and phase spectrum components Contains different and diverse attribute information. The present invention uses the amplitude spectrum component and phase spectrum components Interactive learning is performed to fully promote mutual learning between the two components. The interactive operation is achieved by using cross-multi-head attention (MCA). Specifically, the amplitude spectrum component is set to and , phase spectrum component is set to and , then the mathematical expression of the interaction operation can be expressed as:

[0071] (7)

[0072] in, and Indicates two Next, the features improved by the above interactions are transformed back to the spatial domain through inverse fast Fourier transform and added to the matrix with the spatial domain features to complete the collaborative learning of the spatial domain and frequency domain.

[0073] Then, the amplitude spectrum components obtained through the above interaction operation are transformed by inverse fast Fourier transform (IFFT) and phase spectrum components Transformed into the spatial domain, the improved features are obtained , and spatial domain characteristics Together they are used for collaborative learning in the frequency domain and the space domain. In this way, the present invention can make full use of the frequency-space domain information of the attribute to improve the performance.

[0074] In order to verify the experimental effect of the present invention, the method is compared with the existing methods on four commonly used pedestrian attribute recognition datasets (PETA, PA100k, RAPV1 and RAPV2). The experimental results are shown in Table 1.

[0075] Table 1

[0076]

[0077] As can be seen from Table 1, the proposed method FSIA performs well in multiple indicators on various data sets, showing high accuracy and robustness. Specifically, on the PETA data set, FSIA has achieved leading performance in indicators such as mA, Accu, Prec, Recall and F1. For example, mA reached 90.08%, significantly higher than other comparison methods. This shows that FSIA can accurately identify pedestrian attributes on the PETA data set, and has good performance in accuracy, precision, recall, etc. Compared with traditional methods such as IAA-Caps and AttExpIB-Net, FSIA has significantly improved in various indicators. Taking IAA-Caps as an example, in the mA indicator, FSIA is nearly 5 percentage points higher than it, indicating that FSIA can extract features and identify attributes more effectively. Compared with some recent methods such as VTB*, PromptPAR*, FSIA also has obvious advantages. It has surpassed these methods in multiple indicators, showing its innovation and effectiveness in handling pedestrian attribute recognition tasks. In addition, it can be seen that on the PA100k dataset, FSIA also performs well on this dataset, leading in indicators such as mA, Recall, and F1. For example, Recall reaches 92.20%, indicating that FSIA can more comprehensively detect the attributes of actual pedestrians and reduce omissions. Compared with other methods, FSIA has a better balance in both accuracy and recall, and can provide more reliable data support for practical applications. On the RAP_v1 dataset, FSIA achieved the best results in mA, Accu, Prec, and Recall indicators, and the F1 indicator is second only to EALC. On the RAP_v2 dataset, FSIA performed best in all indicators. This shows that FSIA has strong adaptability and excellent performance on different RAP datasets. Compared with earlier and recent methods, FSIA can achieve leading or very competitive results on both datasets. For example, compared with DAFL, on the RAP_v1 dataset, FSIA has a higher Recall indicator, indicating that it is more comprehensive in detecting pedestrian attributes.

[0078] The outstanding performance of the FSIA method on multiple data sets proves that it has strong generalization ability and can adapt to the characteristics of different data sets and the requirements of pedestrian attribute recognition tasks. The higher mA index means that FSIA performs well in the accuracy of overall attribute recognition and can correctly identify more pedestrian attributes. The high Recall index shows that FSIA can more comprehensively detect the actual attributes of pedestrians and reduce omissions. In practical applications, such as accurate recognition and comprehensive detection of pedestrian attributes in monitoring scenarios, it is of great significance for subsequent behavior analysis, target tracking and other tasks. Through comparative experimental results with many methods, the effectiveness and innovation of the FSIA method in the field of pedestrian attribute recognition are fully demonstrated, providing a new and better performance solution for research in this field, which will help promote the further development of pedestrian attribute recognition technology.

[0079] A large number of experiments show that the image-attribute and frequency-space domain dual collaborative learning method proposed in this invention has significant advantages in improving the performance of pedestrian attribute recognition tasks.

[0080] The above embodiments are only preferred embodiments of the present invention and cannot be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency-space domain, characterized by The following steps are involved: 1) An image-attribute co-learning framework is proposed, which integrates visual information and attribute labels to perceive a more refined semantic understanding of pedestrian attributes. The image-attribute co-learning framework obtains attribute vector representations through a series of learnable attribute cues, inputs pedestrian images into an image encoder to obtain visual vector representations, and maps both attribute vector representations and visual vector representations into a joint image-attribute co-space, and learns the semantic association between images and attributes through contrastive learning loss. 2) A frequency-space collaborative learning module is proposed to fully explore and utilize the diversified information of pedestrian images in both frequency and space domains to achieve robust feature extraction and attribute recognition and obtain more robust visual features; The cross-attention mechanism involving the frequency-domain amplitude spectrum component and the phase spectrum component is used for interactive guided learning, which is collaboratively learned with spatial information.

2. The pedestrian attribute recognition method based on image-attribute and frequency-space domain dual collaborative learning as claimed in claim 1, characterized in that In step 1), an image-attribute collaborative learning framework is proposed, comprising the following steps: (1) Learnable attribute hints: First, the attribute labels are converted into learnable attribute hints, which are constructed as follows: a person has [xxx] attributes; and the learnable attribute hints are input into the text encoder of the contrastive language-image pre-training model CLIP to generate the corresponding text vector representation of the pedestrian attributes; (2) Visual vector representation extraction: The pedestrian image is divided into a series of fixed-size, non-overlapping blocks, with a total of blocks, where P represents the block size; these block sequences are converted into 1D vectors through trainable linear projection, and these 1D vectors are input into the CLIP image encoder to obtain visual vector representation; (3) Using a two-stage training strategy training framework: Phase 1: Freeze the parameters of the text encoder and image encoder, fine-tune the learnable attribute cues of the text in the dataset to better capture the semantic information of the attributes; optimize the framework through multi-classification contrastive learning loss, including image-to-attribute loss and attribute-to-image loss; The second stage: fix the learnable attribute prompts and text encoders, optimize the parameters of the image encoder, and improve the image encoder's ability to capture attribute information; use binary cross entropy loss and image-attribute similarity score to jointly optimize the classifier output, improve recognition accuracy, better capture and understand the attribute information in pedestrian images, and enable the extracted visual features to more accurately match the attribute features.

3. The pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency-space domain as claimed in claim 2, characterized in that In step (3), the first stage focuses on training learnable attribute cues. In this stage, the parameters of the text encoder and the image encoder are frozen and do not participate in training, and the text cues in the dataset are fine-tuned. By fine-tuning the text cues, the proposed image-attribute co-learning framework understands the semantic information of the attributes. Specifically, each learnable attribute cue corresponds to an attribute label, and each pedestrian image corresponds to multiple attribute cues. A multi-classification contrastive learning loss is designed to optimize the proposed image-attribute co-learning framework, which is expressed as follows: (1) in, and Respectively represent images To properties Multi-class contrastive learning loss and attributes To Image Multi-class contrastive learning loss; A label representing a given pedestrian attribute; Represents the index of images and attributes in a mini-batch during training; Indicates the similarity between two vector representations; Represents the first Feature representation of an image; Indicates the first Attributes represent; Since the proposed image-attribute and frequency-space dual collaborative learning method adopts a frequency-space dual-branch learning strategy, the feature representation of the frequency domain is assumed as , the feature of the spatial domain is expressed as , then the overall loss function of the first stage is expressed as follows: (2) in, and Respectively represent images To properties Multi-class contrastive learning loss and attributes To Image Multi-class contrastive learning loss; A label representing a given pedestrian attribute; Represents the index of images and attributes in a mini-batch during training.

4. The pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency-space domain as claimed in claim 2, characterized in that In step (3), the second stage training process will focus on optimizing the image encoder; in this stage, the learnable attribute cues and the parameters of the text encoder are fixed and no longer updated; the attribute vector representation learned in the first stage is used as a stable attribute reference to focus on improving the performance of the image encoder; in the second stage, the parameters of the image encoder are optimized to better capture and understand the attribute information in the pedestrian image, so that the extracted visual features can more accurately match the attribute features; The image encoder better understands the image content and establishes more accurate correspondence with attributes; In the second stage, the binary cross entropy loss is used to optimize the attribute categories output by the classifier layer of the proposed method. , its mathematical expression is as follows: (3) in, Represents the attribute label of the pedestrian image, Represents the sigmoid function, which is used to convert the attribute category output by the classifier layer Convert to predicted probability value; In the second stage, in order to fully explore the semantic correspondence between pedestrian images and pedestrian attributes, the image vector representation is calculated To attribute vector representation The similarity score between , and used it to optimize the predicted value of the proposed method, and its mathematical expression is as follows: (4) The total loss in the second stage is expressed as: (5) in, is a weight hyperparameter used to balance the loss function and The optimization weight of .

5. The pedestrian attribute recognition method based on dual collaborative learning of image-attribute and frequency-space domain as claimed in claim 1, characterized in that In step 2), the frequency domain-spatial domain collaborative learning module is as follows: Features of the CLIP image encoder output First, use the fast Fourier transform FFT to transform it from the spatial domain to the frequency domain to obtain the amplitude spectrum component and phase spectrum components , expressed mathematically as follows: (6) in, and They correspond to the fast Fourier transform FFT ( )’s real and imaginary parts; In the frequency domain, the amplitude spectrum components Related to certain attributes of pedestrians, including clothing texture, facial features; phase spectrum component Helps to more accurately identify pedestrian attributes, including outline and posture; taking into account the amplitude spectrum components and phase spectrum components Contains different and diverse attribute information, through the amplitude spectrum component and phase spectrum components Interactive learning is performed to fully promote mutual learning between the two components; wherein the interactive operation achieves mutual learning by using cross multi-head attention MCA; specifically, the amplitude spectrum component is set to and , phase spectrum component is set to and , then the mathematical expression of the interaction operation is expressed as: (7) in, and Indicates two The convolutional layer transforms the features improved by the above interactions back to the spatial domain through inverse fast Fourier transform and adds them to the matrix with the spatial domain features to complete the collaborative learning of the spatial domain and frequency domain. The amplitude spectrum components obtained by the above interaction operation are transformed by inverse fast Fourier transform (IFFT). and phase spectrum components Transformed into the spatial domain, the improved features are obtained , and spatial domain characteristics They are used together for collaborative learning in the frequency-space domain to fully utilize the frequency-space domain information of attributes to improve performance.

Images