A cross-modal pedestrian re-identification method based on character attribute assistance

Abstract

The application discloses a kind of based on character attribute auxiliary cross-modal pedestrian re-identification method, comprising: obtaining pedestrian image data, character attribute annotation is carried out to cross-modal dataset;ResNet50 in the form of double-flow network is used as backbone network to extract features;Then the features output by double-flow network are subjected to average pooling operation, and then a multi-level fusion scheme is used for feature fusion, and the optimal result is selected as the fusion layer of the network;The attribute is divided into global attribute and local attribute, and the network learns global features and local features assisted by attributes, and the model is trained and tested by combining the designed loss function, to obtain the final recognition accuracy of cross-modal pedestrian re-identification.The application considers that the auxiliary research of character attribute and the research of global-local features are combined, which can effectively improve the accuracy of recognition.

Description

Technical Field

[0001] This invention relates to a cross-modal pedestrian re-identification method based on person attributes, specifically a visible light-infrared light cross-modal pedestrian re-identification method based on person attributes, belonging to the field of computer vision. Background Technology

[0002] In recent years, with the rapid development of computer vision, pedestrian re-identification has gradually become a research hotspot, aiming to retrieve and identify pedestrian images captured by different cameras in different scenes. Due to the complex variations in viewpoint, pose, occlusion, and lighting, pedestrian re-identification is extremely challenging in real-world scenarios. Although current pedestrian re-identification methods are constantly improving, most research focuses on visible light images, neglecting the fact that in low-light environments such as at night, visible light cameras struggle to capture effective human information, significantly reducing the performance of these methods in specific scenarios. Therefore, infrared cameras, capable of capturing effective pedestrian images in dim light, have been widely adopted.

[0003] However, due to the significant differences between the two modalities, infrared images lack some strong distinguishing cues compared to visible light images, such as color. This makes it a major challenge to explore the common elements in both modalities, reduce the feature differences of the same pedestrian in the cross-modal context, and establish the connection between the two. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a cross-modal pedestrian re-identification method based on person attribute assistance. This method explores the commonalities between heterogeneous images and introduces attribute labels as auxiliary information to further increase the similarity of the same pedestrian in cross-modal scenarios, thereby improving the accuracy of recognition.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0006] In a first aspect, the present invention provides a cross-modal person re-identification method based on person attribute assistance, comprising:

[0007] Obtain images of the pedestrians to be identified;

[0008] The corresponding modal features are extracted from the pedestrian image to be identified using the corresponding ResNet50-based backbone network.

[0009] The modal features are then subjected to average pooling to obtain one-dimensional features;

[0010] The one-dimensional feature is input into the trained cross-modal pedestrian re-identification model;

[0011] The pedestrian re-identification result is determined based on the output of the cross-modal pedestrian re-identification model;

[0012] The training method for the cross-modal person re-identification model includes:

[0013] S1 acquires a cross-modal pedestrian image dataset, performs person attribute annotation on the cross-modal pedestrian image dataset, and obtains a cross-modal pedestrian image dataset with person attribute annotation; wherein the cross-modal pedestrian image dataset includes a visible light pedestrian image dataset and an infrared light pedestrian image dataset;

[0014] S2 uses ResNet50, presented as a two-stream network, as the backbone network to extract features from the visible light pedestrian image dataset and infrared pedestrian image dataset labeled with human attributes, and obtains multimodal shared features.

[0015] S3 performs average pooling on the multimodal shared features and then uses a multi-level fusion method to fuse the features to obtain a fused one-dimensional feature dataset.

[0016] S4 uses the fused one-dimensional feature dataset to train the ResNet50-based cross-modal pedestrian re-identification model, thus obtaining the trained cross-modal pedestrian re-identification model.

[0017] Obtain a cross-modal pedestrian image dataset and perform person attribute annotation on the dataset, including:

[0018] The SYSU-MM01 dataset was used to construct a cross-modal pedestrian image dataset. By comparing and analyzing images in both visible and infrared light modalities, human attributes were selected and labeled using binary labels. These human attributes included gender, hair length, wearing glasses, wearing a hat, wearing a coat, sleeve length of a top, length of bottoms, pants, skirt, backpack, handbag, and shoulder bag.

[0019] In some embodiments, ResNet50, presented as a two-stream network, is used as the backbone network for feature extraction, including:

[0020] The first and second stages of ResNet50 were used as feature extractors for specific modalities to be trained separately on visible light and infrared light images, respectively, and specific features for the two modalities were extracted.

[0021] The last three stages of ResNet50 are used for shared network feature embedding, which extracts multimodal shared features by mapping two modality-specific features to a unified feature space.

[0022] In some embodiments, after performing average pooling on the multimodal shared features, a multi-level fusion method is used for feature fusion, including:

[0023] The output features of the third stage of ResNet50 after average pooling are used as the mid-layer features Mid2_fea, and the output features of the fifth stage of ResNet50 after average pooling are used as the high-layer features High_fea. The mid-layer features and the high-layer features are then fused to obtain the fused feature Fea1.

[0024] The fused feature Fea1 is passed through a 1x1 convolutional layer, a batch normalization network layer, and an activation function layer to obtain the fused one-dimensional feature.

[0025] In some embodiments, training a ResNet50-based cross-modal person re-identification model using the fused one-dimensional feature dataset includes:

[0026] The cross-modal pedestrian re-identification model includes a global feature learning module and a local feature learning module. Attributes are divided into global attributes and local attributes. Global attributes focus on the overall picture, using pedestrian identity information and gender as classification labels for the global feature learning module; local attributes focus on details, using other person attributes besides gender as classification labels for the local feature learning module.

[0027] The fused one-dimensional features are input into the cross-modal pedestrian re-identification model for training, with the goal of minimizing the total loss, to obtain the trained cross-modal pedestrian re-identification model.

[0028] In some embodiments, the total loss of the cross-modal person re-identification model is:

[0029] Loss = Loss1 + βLoss2

[0030] Where Loss1 is the loss of the global feature learning module, Loss2 is the loss of the local feature learning module, and β is the weight parameter that balances the global feature loss and the local feature loss.

[0031] The loss Loss1 of the global feature learning module is:

[0032] Loss1 = Loss1 id +αLoss1 ge

[0033]

[0034]

[0035] Where Loss1 is the loss of the global feature learning module, and α represents the weight parameter balancing the pedestrian identity classification loss and the gender attribute classification loss; Loss1 id L represents the pedestrian identification classification loss.t Indicates the loss of the triplet. The Softmax loss is used for pedestrian identity classification. Let α1 be the heterogeneous center loss corresponding to pedestrian identity classification. Weighting coefficients; Loss1 ge This represents the loss for gender attribute classification. The Softmax loss is used for classifying gender attributes. The heterogeneous center loss corresponding to gender attribute classification, α2 is Weighting coefficients;

[0036] The loss (Loss2) of the local feature learning module is:

[0037]

[0038] Among them, Loss2 parti (i = 1, 2, 3, 4) represents the loss of the local features corresponding to each class of local attributes.

[0039] Furthermore, the triplet loss L t The formula is as follows:

[0040] L t =max(d(a,p)-d(a,n)+margin,0)

[0041] Where a is the anchor sample, p is the positive sample, n is the negative sample, d(a,p) represents the feature distance between the anchor sample and the positive sample, and d(a,n) represents the feature distance between the anchor sample and the negative sample. Using margin as a threshold, the distance between positive sample pairs is shortened, and the distance between negative sample pairs is widened, so that pedestrian images with the same identity form a cluster in the feature space.

[0042] The Softmax loss formula is as follows:

[0043]

[0044] Among them, z y Let z be the value of the y-th class in the true category. i K represents the predicted value for each class in a multi-class classification task, and K represents the total number of classes in the multi-class classification task.

[0045] The formula for the loss of heterogeneous centers is as follows:

[0046]

[0047] in, These are the feature distribution centers of the i-th pedestrian category in the visible light mode and the infrared light mode, respectively, x i,1,jLet x represent the j-th visible light image feature of the i-th pedestrian. i,2,j Let U represent the j-th infrared image feature of the i-th pedestrian, U represent the total number of pedestrian categories, and M and N represent the number of visible light and infrared images of the i-th type of person, respectively.

[0048] In a second aspect, the present invention provides a cross-modal pedestrian re-identification device based on person attribute assistance, including a processor and a storage medium;

[0049] The storage medium is used to store instructions;

[0050] The processor is configured to operate according to the instructions to perform the steps of the method according to the first aspect.

[0051] Thirdly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0052] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0053] This invention proposes a cross-modal person re-identification method based on person attribute assistance. Utilizing a dual-stream network with shared parameters, it effectively acquires specific and shared features from different modalities. Furthermore, by fusing shared features from different levels, it can more effectively explore mid-to-high-level semantic information. Simultaneously, by leveraging attributes shared by both modalities, it can better enhance the model's generalization ability and establish a "communication bridge" between different modalities. Experiments have verified its superior effectiveness in improving recognition accuracy, demonstrating its potential for wider application. Attached Figure Description

[0054] Figure 1 This is an example of person attribute annotation in a cross-modal person re-identification method based on person attribute assistance provided in an embodiment of the present invention;

[0055] Figure 2 This is a model structure diagram of a cross-modal person re-identification method based on person attribute assistance provided in an embodiment of the present invention;

[0056] Figure 3 This is a flowchart of a cross-modal pedestrian re-identification method based on person attribute assistance provided by an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0058] Example 1

[0059] A cross-modal person re-identification method based on person attribute assistance includes:

[0060] Obtain images of the pedestrians to be identified;

[0061] The corresponding modal features are extracted from the pedestrian image to be identified using the corresponding ResNet50-based backbone network.

[0062] The modal features are then subjected to average pooling to obtain one-dimensional features;

[0063] The one-dimensional feature is input into the trained cross-modal pedestrian re-identification model;

[0064] The pedestrian re-identification result is determined based on the output of the cross-modal pedestrian re-identification model;

[0065] The training method for the cross-modal person re-identification model includes:

[0066] S1 acquires a cross-modal pedestrian image dataset, performs person attribute annotation on the cross-modal pedestrian image dataset, and obtains a cross-modal pedestrian image dataset with person attribute annotation; wherein the cross-modal pedestrian image dataset includes a visible light pedestrian image dataset and an infrared light pedestrian image dataset;

[0067] S2 uses ResNet50, presented as a two-stream network, as the backbone network to extract features from the visible light pedestrian image dataset and infrared pedestrian image dataset labeled with human attributes, and obtains multimodal shared features.

[0068] S3 performs average pooling on the multimodal shared features and then uses a multi-level fusion method to fuse the features to obtain a fused one-dimensional feature dataset.

[0069] S4 uses the fused one-dimensional feature dataset to train the ResNet50-based cross-modal pedestrian re-identification model, thus obtaining the trained cross-modal pedestrian re-identification model.

[0070] In some embodiments, a visible-infrared cross-modal pedestrian re-identification method based on person attributes includes the following steps:

[0071] Step 1: Construct data samples by obtaining the publicly available SYSU-MM01 dataset online. Divide the data samples into training and test sets in a 7:3 ratio and pre-label the attributes. The SYSU-MM01 dataset is a representative dataset in the field of cross-modal person re-identification, containing 30,071 visible light images and 15,792 infrared light images.

[0072] (1) The pedestrian images obtained in the first step are preprocessed; and 12 personal attributes (gender, hair length, wearing glasses, wearing a hat, wearing a coat, sleeve length of the top, length of the bottom, pants, skirt, backpack, handbag, shoulder bag) are selected and labeled with binary labels (0 or 1).

[0073] Step 2: Input the preprocessed pedestrian images of the two modalities obtained in Step 1 into a two-stream network for feature extraction. The specific steps are as follows:

[0074] (1) Obtain images of two modalities with the same number of corresponding identities from the training set using a random batch sampling strategy. That is, randomly sample P people, and then randomly sample K visible light images and K infrared light images from these P people respectively. The size of the training batch is 2*P*K.

[0075] (2) The sampled visible light image sample set and infrared light image sample set The inputs are fed into two branches of the ResNet50 feature extraction module, respectively, to extract specific features for two different modalities. Let i be the feature map of the i-th visible light image in the visible light image sample set. Let R be the feature map of the i-th infrared image in the infrared image sample set, where R represents the set of real numbers, C is the channel of the image, H is the height of the image, and W is the width of the image.

[0076] (3) Then map the specific features of the two modes into a unified feature space to obtain the features shared by the two modes.

[0077] Step 3: After performing average pooling on the shared features obtained in Step 2 through the modality sharing branch, feature fusion is performed using a multi-level fusion method. The specific steps are as follows:

[0078] (1) Name the output features of the last three stages of the ResNet50 backbone network as Mid2, Mid3 and High respectively, and perform average pooling operation on them respectively;

[0079] Specifically, the output features of the third and fourth stages of ResNet50 after average pooling are used as mid-layer features and named Mid2_fea and Mid3_fea, respectively. The output features of the fifth stage of ResNet50 after average pooling are used as high-layer features and named High_fea.

[0080] (2) The features after average pooling were used as the input of the fusion module, and experiments were conducted according to the proposed multi-level fusion scheme.

[0081] The proposed multi-level integration scheme is as follows:

[0082] The three fusion features obtained through the three fusion schemes are named Fea1, Fea2, and Fea3 respectively.

[0083] Fea1 √ √ Fea2 √ √ Fea3 √ √ √

[0084] (3) The fused features are dimensionality reduced by passing through a 1x1 convolutional layer, a batch normalization network layer, and an activation function layer to obtain a one-dimensional feature representation, which serves as the input to the feature learning module, thereby improving the discriminative power of global and local features.

[0085] Step 4: Input the one-dimensional features obtained in Step 3 into the multi-task network, divide the attributes into global attributes and local attributes, supervise the network to learn attribute-aided global and local features, and combine them with the designed loss function to obtain the final recognition accuracy of cross-modal person re-identification. The specific steps are as follows:

[0086] (1) In the global feature learning module, pedestrian identity information and gender attribute are used as classification labels; the loss function described in this module is jointly determined by triplet loss, softmax loss, and heterogeneous center loss, and can be expressed as:

[0087]

[0088]

[0089] Loss1 = Loss1 ia +aLoss1 ge (1.3)

[0090] Among them, Loss1 id L represents the total loss corresponding to the pedestrian identity classification. t Indicates the loss of the triplet. The Softmax loss is used for pedestrian identity classification. Let α1 be the heterogeneous center loss corresponding to pedestrian identity classification. The weighting coefficients. Loss1 ge This represents the total loss corresponding to the gender attribute classification. The Softmax loss is used for classifying gender attributes. The heterogeneous center loss corresponding to gender attribute classification, α2 is The weight coefficients are: Loss1 is the loss corresponding to the global feature learning module, and α represents the weight parameter balancing the pedestrian identity classification loss and the attribute classification loss.

[0091] (2) In the local feature learning module, the remaining 11 attributes other than gender are used as classification labels; and the remaining attributes are divided into 4 local attributes according to the proportion of the human body from top to bottom: local attribute 1 (hair length, wearing glasses, wearing a hat), local attribute 2 (upper sleeve length, backpack), local attribute 3 (wearing a coat, carrying a bag, shoulder bag), and local attribute 4 (skirt, pants, lower garment length), with proportions of (0, 1 / 5), (1 / 5, 1 / 2), (1 / 5, 3 / 4), and (1 / 2, 1) respectively; the loss function described in this module consists of Softmax loss and heterogeneous center loss, which can be expressed as:

[0092]

[0093] Here, Loss2 refers to the loss corresponding to the local feature learning module. parti (i = 1, 2, 3, 4) represents the loss of the local features corresponding to each type of local attribute;

[0094] (3) Finally, the total loss of the network is the sum of the global feature loss and the local feature loss:

[0095] Loss = Loss1 + βLoss2

[0096] Where Loss is the total loss of the network, Loss1 is the loss corresponding to the global feature learning module, Loss2 is the loss corresponding to the local feature learning module, and β is the weight parameter to balance the global feature loss and the local feature loss.

[0097] (4) Based on the Rank-k, mAP, and mINP results of cross-modal person re-identification obtained from the test, select

[0098] Select the optimal solution for multi-level feature fusion.

[0099] The evaluation metrics involved here are: Rank-k represents the accuracy rate that the top k images sorted by similarity contain images of the same person as the person in the query image; mAP represents the mean precision, which reflects the degree to which the retriever ranks all correct images in the dataset at the top of the sort list; and mINP represents the average negative sample penalty rate for all query samples, which is the proportion of correct samples found up to the last correct result.

[0100] (5) The loss function mentioned above is calculated as follows:

[0101] (5.1) The formula for triplet loss is as follows:

[0102] L t =max(d(a,p)-d(a,n)+margin,0)

[0103] Where a is the anchor sample, p is the positive sample, n is the negative sample, d(a,p) represents the feature distance between the anchor sample and the positive sample, and d(a,n) represents the feature distance between the anchor sample and the negative sample. Using margin as a threshold, the distance between positive sample pairs is shortened, and the distance between negative sample pairs is widened, so that pedestrian images with the same identity form clusters in the feature space.

[0104] (5.2) The Softmax loss formula is as follows:

[0105]

[0106] Among them, z y Let z be the value of the y-th class in the true category. i Let K be the predicted value for each class in a multi-class classification task, and K represent the total number of classes in the multi-class classification task. For multi-class classification tasks, the goal is to improve the properties of sample features in the metric space: intra-class compactness and inter-class diversity.

[0107] (5.3) The formula for the loss of heterogeneous centers is as follows:

[0108]

[0109] in, These are the feature distribution centers of the i-th pedestrian category in the visible light mode and the infrared light mode, respectively, x i,1,j Let x represent the j-th visible light image feature of the i-th pedestrian. i,2,j Let represent the j-th infrared image feature of the i-th pedestrian, U represent the total number of pedestrian categories, and M and N represent the number of visible light and infrared images of the i-th pedestrian category, respectively. Cross-modal similarity within each category is improved by constraining the center distance of each category.

[0110] Example 2

[0111] Secondly, this embodiment provides a cross-modal pedestrian re-identification device based on person attribute assistance, including a processor and a storage medium;

[0112] The storage medium is used to store instructions;

[0113] The processor is configured to operate according to the instructions to perform the steps of the method according to Embodiment 1.

[0114] Example 3

[0115] Thirdly, this embodiment provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0116] Example 4

[0117] A cross-modal person re-identification system based on person attribute assistance includes:

[0118] Feature extraction module: Acquire pedestrian images of two different modalities respectively, and after preprocessing, input them into the dual-stream backbone network. The feature extractor of the specific modality extracts specific features of the two modalities, and the feature embedder of the shared network extracts shared features.

[0119] Feature fusion module: Used to fuse shared features from different levels to explore the impact of multi-level feature fusion on the results;

[0120] Global Feature Learning Module: This module is used to learn global features based on global attributes after the fused one-dimensional features are input into the multi-task network.

[0121] Local Feature Learning Module: This module is used to learn local features based on local attributes after the fused one-dimensional features are input into the multi-task network.

[0122] This invention improves the similarity between pedestrian images with the same identity but different modalities by performing attribute annotation on cross-modal datasets and using person attribute assistance, effectively enhancing the model's generalization ability. Furthermore, by combining global-local feature research with attribute research, it can better enhance the feature representation ability and further improve the recognition accuracy.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] It should be noted that the above description of the present invention is merely illustrative, and the invention is not limited thereto. Therefore, the specific embodiments of the present invention may differ. While the main theory of the present invention remains unchanged, all other embodiments derived by those skilled in the art through modifications or substitutions that are understandable under the guidance of the present invention are considered to be within the protection scope of the present invention.

Claims

1. A cross-modal person re-identification method based on person attribute assistance, characterized in that, include: Obtain images of the pedestrians to be identified; The corresponding modal features are extracted from the pedestrian image to be identified using the corresponding ResNet50-based backbone network. The modal features are then subjected to average pooling to obtain one-dimensional features; The one-dimensional feature is input into the trained cross-modal pedestrian re-identification model; The pedestrian re-identification result is determined based on the output of the cross-modal pedestrian re-identification model; The training method for the cross-modal person re-identification model includes: S1. Obtain a cross-modal pedestrian image dataset, and perform person attribute annotation on the cross-modal pedestrian image dataset to obtain a cross-modal pedestrian image dataset with person attribute annotation; wherein the cross-modal pedestrian image dataset includes a visible light pedestrian image dataset and an infrared light pedestrian image dataset; wherein the person attributes include gender, hair length, wearing glasses, wearing a hat, wearing a coat, upper garment sleeve length, lower garment length, pants, skirt, backpack, handbag, and shoulder bag; S2. The visible light pedestrian image dataset and infrared pedestrian image dataset with person attribute annotations are used to extract features using a ResNet50 network presented in a two-stream network format as the backbone network to obtain multimodal shared features, including: The first and second stages of ResNet50 were used as feature extractors for specific modalities to be trained separately on visible light and infrared light images, respectively, and specific features for the two modalities were extracted. The last three stages of ResNet50 are used for shared network feature embedding, which extracts multimodal shared features by mapping two modality-specific features to a unified feature space. S3. After performing average pooling on the multimodal shared features, a multi-level fusion method is used to fuse the features to obtain a fused one-dimensional feature dataset, including: taking the output features of the third stage of ResNet50 after average pooling as the mid-layer feature Mid2_fea, taking the output features of the fifth stage of ResNet50 after average pooling as the high-layer feature High_fea, fusing the mid-layer features and the high-layer features to obtain the fused feature Fea1, and passing the fused feature Fea1 through a 1x1 convolutional layer, a batch normalization network layer, and an activation function layer to obtain the fused one-dimensional feature; S4. Using the fused one-dimensional feature dataset, train the ResNet50-based cross-modal person re-identification model to obtain the trained cross-modal person re-identification model, including: The cross-modal pedestrian re-identification model includes a global feature learning module and a local feature learning module. Attributes are divided into global attributes and local attributes. Global attributes focus on the overall picture, using pedestrian identity information and gender as classification labels for the global feature learning module; local attributes focus on details, using other person attributes besides gender as classification labels for the local feature learning module. The fused one-dimensional features are input into the cross-modal pedestrian re-identification model for training, with the goal of minimizing the total loss, to obtain the trained cross-modal pedestrian re-identification model.

2. The cross-modal person re-identification method based on person attribute assistance according to claim 1, characterized in that, Obtain a cross-modal pedestrian image dataset and perform person attribute annotation on the dataset, including: We used the SYSU-MM01 dataset to construct a cross-modal pedestrian image dataset. By comparing and analyzing images in both visible and infrared light modes, we selected human attributes and labeled them with binary tags.

3. The cross-modal person re-identification method based on person attribute assistance according to claim 1, characterized in that, Total loss of cross-modal person re-identification model for: ； in, The loss of the global feature learning module, The loss of the local feature learning module, These are the weight parameters that balance the global feature loss and the local feature loss; Loss of the global feature learning module for: ； ； ； in, It is the loss of the global feature learning module. This represents the weight parameters that balance the pedestrian identity classification loss and the gender attribute classification loss; This indicates loss related to pedestrian identification classification. Indicates the loss of the triplet. The Softmax loss is used for pedestrian identity classification. The heterogeneous center loss corresponding to pedestrian identity classification. for Weighting coefficients; This represents the loss for gender attribute classification. The Softmax loss is used for classifying gender attributes. The heterogeneity center loss is used for gender attribute classification. for Weighting coefficients; Loss of the local feature learning module for: ； in, Let i be the loss of the local features corresponding to each type of local attribute, i=1,2,3,4.

4. The cross-modal person re-identification method based on person attribute assistance according to claim 3, characterized in that, Triple loss The formula is as follows: )； Where a is the anchor sample, p is the positive sample, and n is the negative sample. Indicates anchor point sample and positive Feature distance of the sample This represents the feature distance between anchor samples and negative samples; using margin as a threshold, the distance between positive sample pairs is shortened, and the distance between negative sample pairs is widened, ultimately enabling pedestrian images with the same identity to form clusters in the feature space. The Softmax loss formula is as follows: ； in, Let y be the value of the y-th class in the true category. K represents the predicted value for each class in a multi-class classification task, and K represents the total number of classes in the multi-class classification task. The formula for the loss of heterogeneous centers is as follows: ； in, , These are the feature distribution centers of the i-th pedestrian category in the visible light mode and the infrared light mode, respectively. This represents the j-th visible light image feature of the i-th pedestrian. Let U represent the j-th infrared image feature of the i-th pedestrian, U represent the total number of pedestrian categories, and M and N represent the number of visible light and infrared images of the i-th type of person, respectively.

5. A cross-modal pedestrian re-identification device based on person attribute assistance, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 4.

6. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images