Method of Visual-Language Infrared Person Re-Identification Network Model Based on Binary Mixture

Abstract

The present invention proposes a method for a visual-language infrared pedestrian re-identification network model based on binary mixing. The method includes: inputting visible-light images and infrared images into a binary separator for binary operation to generate binary mixed images; encoding the visible-light images and infrared images using an image encoder to generate image features; encoding the visible-light images, infrared images, and binary mixed images through a text generation module and a text encoder to generate text features. The binary mixing auxiliary method proposed by the present invention can deeply analyze the differences between visible-light and infrared images, and effectively compensate for the text prediction differences between visible and infrared through a specific mixing strategy, so as to better identify pedestrians in complex infrared image scenarios and improve the accuracy of infrared pedestrian re-identification.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a method for a visual - language infrared pedestrian re - identification network model based on bipartite mixing. Background Art

[0002] Visible - infrared person re - identification (VI - ReID) is a cross - modal pedestrian retrieval problem aiming to match person identities under different imaging modalities. However, the unique modality of infrared images generates significant domain differences and more complex environments, such as the loss of common color, texture, and detail information in images. Therefore, previous ReID methods based solely on visible images are not suitable for the VI - ReID task. In recent years, driven by the rapid development of deep - learning technology, significant progress has been made in this field, and how to improve the accuracy of model search has become an important issue in infrared pedestrian re - identification.

[0003] Currently, infrared pedestrian re - identification networks are mainly divided into two types: feature - level methods and image - level methods. Feature - level methods mainly focus on feature learning, aiming to map images of different modalities into a shared embedding space to reduce differences. Although methods based on original images can achieve good results in some cases, they still face significant difference problems when processing different - modality data in the feature space. On the other hand, image - level methods reduce the modality differences between VIS and IR images in the image space by using generative adversarial networks (GANs) to convert images from one modality to another or by creating intermediate - modality images for training. However, cross - modal images usually contain noise, which affects the output quality and poses a major challenge to the stability of the training process.

[0004] To further improve the performance of the model, many researchers have conducted experiments using multi - modal pre - training, such as adding artificial text descriptions or introducing CLIP as a driver. However, these methods require additional artificial annotation information or ignore the complementarity of multi - modalities, thus failing to obtain rich and pure semantics, which hinders the development and effectiveness of multi - modal pre - training models. Summary of the Invention

[0005] In view of the above situation, the main purpose of the present invention is to propose a method for a visual - language infrared pedestrian re - identification network model based on bipartite mixing to solve the above - mentioned technical problems.

[0006] The present invention proposes a method for a visual - language infrared pedestrian re - identification network model based on bipartite mixing, and the method includes the following steps:

[0007] Step 1: Construct a bipartite separator and a mixer based on the image bipartite mechanism. The bipartite separator and the mixer constitute the bipartite mixing module. Construct an image encoder based on the diverse embedding expansion network. Construct a text-image collaborative mixing module based on the text-image fusion mechanism. The image encoder, the bipartite mixing module, the text generation module, the text encoder of the ViLT model, and the text-image collaborative mixing module constitute the person re-identification network model;

[0008] Step 2: Input the visible light image and the infrared image into the bipartite separator for bipartite operation to obtain image cropping blocks;

[0009] Input the image cropping blocks into the mixer for cropping block exchange to generate a bipartite mixed image;

[0010] Step 3: Input the visible light image and the infrared image into the image encoder for image encoding to generate image features;

[0011] Step 4: Encode the visible light image, the infrared image, and the bipartite mixed image through the text generation module and the text encoder to generate text features;

[0012] Step 5: Input the image features and the text features into the text-image collaborative mixing module for hybrid centralization operation, and perform feature set fusion through the attention weight controller to generate weighted integrated features;

[0013] Step 6: Construct cross-entropy loss, triplet loss, center-guided pair mining loss, and orthogonal loss based on the image features;

[0014] Construct bidirectional weighted distribution loss based on the weighted integrated features;

[0015] Optimize the recognition network model based on the cross-entropy loss, triplet loss, center-guided pair mining loss, orthogonal loss, and bidirectional weighted distribution loss to obtain an optimized person re-identification network model;

[0016] Use the optimized person re-identification network model to obtain the recognition result.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] 1. The present invention introduces the ViLT multi-modal pre-training model, which reduces the dependence on manually labeled text by automatically predicting text descriptions; by automatically predicting text through the ViLT model, it can dynamically generate corresponding text descriptions based on real-time data, greatly improving the prediction accuracy of the model; at the same time, because the multi-modal pre-training model has the ability to learn multiple data patterns, it can show good adaptability in different scenarios, thus significantly improving the versatility of the model;

[0019] 2. The binary hybrid assistance method proposed by the present invention can deeply analyze the differences between visible light and infrared images, and through a specific hybrid strategy, effectively make up for the text prediction differences between visible and infrared, so as to better identify pedestrians in complex infrared image scenes and improve the accuracy of infrared pedestrian re-identification.

[0020] 3. The proposed graphic-text collaborative hybrid module aims to overcome the problem of biased weight distribution in the interaction process between image and text information; this module can intelligently adjust the weights of image and text information in the interaction process, enabling the model to accurately focus on the information useful to itself, avoiding information deviation caused by uneven weight distribution, and thus effectively improving the model performance to achieve more accurate identification in the pedestrian re-identification task.

[0021] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be understood through the embodiments of the present invention. Brief Description of the Drawings

[0022] Figure 1 It is a flowchart of the steps of the method for the visual language infrared pedestrian re-identification network model based on binary hybrid proposed by the present invention.

[0023] Figure 2 It is an overall framework diagram of the network model of the method for the visual language infrared pedestrian re-identification network model based on binary hybrid proposed by the present invention.

[0024] Figure 3 It is a schematic diagram of the structure of the binary hybrid module of the method for the visual language infrared pedestrian re-identification network model based on binary hybrid proposed by the present invention.

[0025] Figure 4 It is an architecture diagram of the graphic-text collaborative hybrid module of the method for the visual language infrared pedestrian re-identification network model based on binary hybrid proposed by the present invention. Detailed Embodiments

[0026] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals are the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0027] Referring to the following description and drawings, these and other aspects of the embodiments of the present invention will become clear. In these descriptions and drawings, some specific embodiments of the embodiments of the present invention are specifically disclosed as some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0028] Please refer to Figure 1 , an embodiment of the present invention proposes a method for a visual - language infrared pedestrian re - identification network model based on binary mixing. The method includes the following steps:

[0029] Step 1: Construct a binary separator and a mixer based on an image binary mechanism. The binary separator and the mixer constitute a binary mixing module. Construct an image encoder based on a diverse embedding expansion network, and construct a text - image collaborative mixing module based on a text - image fusion mechanism. The image encoder, the binary mixing module, the text generation module, the text encoder of the ViLT model, and the text - image collaborative mixing module constitute a pedestrian re - identification network model.

[0030] Step 2: Input the visible - light image and the infrared image into the binary separator for binary operation to obtain image cropping blocks;

[0031] Input the image cropping blocks into the mixer for cropping - block exchange to generate a binary - mixed image.

[0032] Please refer to Figure 3 , in Step 2, when the visible - light image and the infrared image are input into the binary separator for binary operation to obtain image cropping blocks, the following relational expressions exist for the corresponding process:

[0033] ;

[0034] Wherein, represents the output left - hand part of the image, represents the output right - hand part of the image, represents the output upper - hand part of the image, represents the output lower - hand part of the image, represents the width of the image, represents the height of the image, represents left - hand cropping, represents right - hand cropping, represents upper - hand cropping, represents lower - hand cropping;

[0035] In the step of inputting the image cropping blocks into the mixer for cropping - block exchange to generate a binary - mixed image, the following relational expressions exist for the corresponding process:

[0036] ;

[0037] Wherein, represents the binary - mixed image, and represent two different cropping blocks, represents the exchange operation, represents the visible - light image, represents the infrared image.

[0038] Furthermore, for different datasets, we can flexibly choose the left - right or up - down scheme according to the distribution of pedestrians in the image. Through such a binary operation, each input image can be divided into two different cropped blocks, providing a basis for the subsequent mixer to exchange cropped blocks between two images, and then generating a mixed image containing visible and infrared pedestrian information to eliminate the text feature differences between different modalities.

[0039] Step 3: Input the visible - light image and the infrared image into an image encoder for image encoding to generate image features;

[0040] Construct cross - entropy loss, triplet loss, center - guided pair mining loss, and orthogonal loss based on the image features, and optimize the recognition model based on the cross - entropy loss, triplet loss, center - guided pair mining loss, and orthogonal loss.

[0041] In step 3, inputting the visible - light image and the infrared image into an image encoder for image encoding to generate image features specifically includes the following steps:

[0042] Input the visible - light image and the infrared image into the two - stream residual network in the image encoder for feature extraction to obtain the extracted features in the first stage;

[0043] Perform feature aggregation on the extracted features in the first stage to obtain the current - stage features;

[0044] Perform feature aggregation on the current - stage features to obtain the next - stage features;

[0045] Input the extracted features in the last stage into a multi - branch convolution structure for embedding processing to generate image features.

[0046] Furthermore, the visible - light image and the infrared image are input into a two - stream residual network (two - stream ResNet - 50 network), which consists of four stages for feature extraction. In each stage, the encoder aggregates the features of the previous stage to explore richer representations. After the last stage, the encoder uses a multi - branch convolution structure to generate diverse embedded features, thereby generating image features.

[0047] Step 4: Encode the visible - light image, the infrared image, and the binary - mixed image through a text generation module and a text encoder to generate text features.

[0048] Please refer to Figure 2 In step 4, encoding the visible - light image, the infrared image, and the binary - mixed image through a text generation module and a text encoder to generate text features specifically includes the following steps;

[0049] Use the text generation module to perform template presetting to obtain a preset text template;

[0050] Use the ViLT model to perform image visual processing on visible light images, infrared images, and dichotomous mixed images to generate semantic information of visible light images, semantic information of infrared images, and semantic information of dichotomous mixed images;

[0051] Use the ViLT model to synchronously receive the preset text template, and perform semantic information filling processing on the semantic information of visible light images, semantic information of infrared images, and semantic information of dichotomous mixed images respectively to generate a template after semantic filling;

[0052] Extract semantic information from the template after semantic filling through a text encoder to generate text features.

[0053] It should be noted that the text templates of visible light images, infrared images, and dichotomous mixed images take "a [MASK] is wearing [MASK] pants and [MASK] sports shoes, facing the [MASK] direction, and dressed in [MASK] clothing" as an example. Specific semantic information is inferred from the image to replace the [MASK] marks in the text template of the dichotomous mixed image, thereby generating a template after semantic filling.

[0054] Step 5: Input the image features and text features into the text-image collaborative hybrid module for hybrid centralized operation, and perform feature set fusion through an attention weight controller to generate weighted integrated features;

[0055] Please refer to Figure 4 , in the said Step 5, input the image features and text features into the text-image collaborative hybrid module for hybrid centralized operation, and perform feature set fusion through an attention weight controller to generate weighted integrated features, which specifically includes the following steps:

[0056] Input the text features into the fully connected layer for dimension matching processing to obtain text features after dimension matching;

[0057] Input the image features and the text features after dimension matching into the text-image collaborative hybrid module for hybrid set operation to obtain a feature set of visible light image features and visible light text features, a feature set of infrared image features and infrared text features, a feature set of visible light image features and dichotomous mixed text features, and a feature set of infrared image features and dichotomous mixed text features;

[0058] For the feature sets of visible light image features and visible light text features, infrared image features and infrared text features, visible light image features and binary mixed text features, and infrared image features and binary mixed text features, the attention weight controller is used to perform weight calculation processing to obtain the weighted integrated features of visible light images and text features, infrared images and text features, visible light images and binary mixed text features, and infrared image features and binary mixed text features.

[0059] It should be noted that the text features are input into the hybrid collaborative module after passing through the fully connected layer for dimension matching with the image features.

[0060] In the step of using the attention weight controller to perform weight calculation processing on the feature sets of visible light image features and visible light text features, infrared image features and infrared text features, visible light image features and binary mixed text features, and infrared image features and binary mixed text features to obtain the weighted integrated features of visible light images and text features, infrared images and text features, visible light images and binary mixed text features, and infrared image features and binary mixed text features, the following relational expressions exist for the corresponding process:

[0061] ;

[0062] ;

[0063] Among them, represents the weighted integrated feature of visible light image features and visible light text features, represents the weighted integrated feature of infrared image features and infrared text features, represents the weighted integrated feature of visible light image features and binary mixed text features, represents the weighted integrated feature of infrared image features and binary mixed text features, represents the visible light text features, represents the infrared text features, represents the binary mixed text features, represents the weight calculation result from image to text, represents the weight calculation result from text to image, represents being processed by the normalization function, represents the dimension of the image features, represents the control factor for image-to-text attention learning, represents the dimension of the text features, Represents the control factor for text-to-image attention learning, Represents the obtained feature weights, Represents the text features Of the transpose, Represents the image features Of the transpose.

[0064] Step 6, construct cross-entropy loss, triplet loss, center-guided pair mining loss, and orthogonal loss based on the image features;

[0065] Construct bidirectional weighted distribution loss based on the weight-integrated features;

[0066] Optimize the recognition network model based on cross-entropy loss, triplet loss, center-guided pair mining loss, orthogonal loss, and bidirectional weighted distribution loss to obtain the optimized person re-identification network model;

[0067] Use the optimized person re-identification network model to obtain the recognition result.

[0068] Constructing cross-entropy loss, triplet loss, center-guided pair mining loss, and orthogonal loss based on the image features further includes the following sub-steps:

[0069] Construct cross-entropy loss based on the image features;

[0070] Among them, for constructing the cross-entropy loss, the relational expressions in the corresponding process are as follows:

[0071]

[0072] Among them, Represents the cross-entropy loss, Represents the number of samples, Represents the sample index, Represents the number of categories, Represents the category index, Represents the true label, Represents the logarithmic function, Represents the probability predicted by the person re-identification model;

[0073] Construct triplet loss based on the image features;

[0074] Among them, for constructing the triplet loss, the relational expressions in the corresponding process are as follows:

[0075] ;

[0076] Among them, Represents the triplet loss, Represents the distance function, Represents after feature extraction processing, And Represent two different positive samples of the same category, Representing Negative samples of different categories, Representing the boundary parameter;

[0077] Construct a center-guided pair mining loss based on image features;

[0078] Among them, for the construction of the center-guided pair mining loss, the relationship in the corresponding process is as follows:

[0079] ;

[0080] Among them, Represents the center-guided pair mining loss, Represents the Euclidean distance between two embeddings, Represents the visible image features, Represents the infrared image features, Represents the embedding generated from the visible image features, Represents the embedding generated from the infrared image features, Represents the hyperparameter.

[0081] Construct an orthogonal loss based on image features;

[0082] Among them, for the construction of the orthogonal loss, the relationship in the corresponding process is as follows:

[0083] ;

[0084] Among them, Represents the orthogonal loss, and Represent the embedding vectors of different branches generated by the module, Represents the total number of branches generated by the module, and Respectively represent two embedding vectors generated from different branches.

[0085] It should be noted that the cross-entropy loss, triplet loss, center-guided pair mining loss, and orthogonal loss are used to optimize the image features.

[0086] Construct a bidirectional weighted distribution loss based on the weighted integrated features, specifically including the following steps:

[0087] Recombine the image-to-text weighted integrated features and the text-to-image weighted integrated features into pairs, and assign indicators for cosine similarity calculation to obtain the cosine similarity score of the attention weights between the image-to-text weighted integrated features and the text-to-image weighted integrated features;

[0088] Calculate the relative entropy of the cosine similarity score of the attention weights between the image-to-text weighted integrated feature and the text-to-image weighted integrated feature to obtain the first relative entropy calculation result, and construct the image-to-text weighted distribution loss based on the first relative entropy calculation result;

[0089] Exchange the input positions of the image-to-text weighted integrated feature and the text-to-image weighted integrated feature, and calculate the relative entropy to obtain the second relative entropy calculation result, and construct the text-to-image weighted distribution loss based on the second relative entropy calculation result;

[0090] Add the image-to-text weighted distribution loss and the text-to-image weighted distribution loss to construct the bidirectional weighted distribution loss.

[0091] In the step of recombining the image-to-text weighted integrated feature and the text-to-image weighted integrated feature into pairs and assigning indicators for cosine similarity calculation to obtain the cosine similarity score of the attention weights between the image-to-text weighted integrated feature and the text-to-image weighted integrated feature, the relational expressions existing in the corresponding process are as follows:

[0092] ;

[0093] Among them, represents the cosine similarity score of the attention weights of image-to-text and text-to-image, represents the exponential function, represents the similarity function, represents the attention weight vector of image-to-text, represents the variable in the summation, represents the number of pairs of image and text attention weights, and respectively represent two text-to-image attention weight vectors, represents the temperature hyperparameter that controls the peak of the probability distribution, represents the index of the image, represents the index of the text;

[0094] In the step of calculating the relative entropy of the cosine similarity score of the attention weights between the image-to-text weighted integrated feature and the text-to-image weighted integrated feature to obtain the first relative entropy calculation result, and constructing the image-to-text weighted distribution loss based on the first relative entropy calculation result, the relational expressions existing in the corresponding process are as follows:

[0095] ;

[0096] Among them, represents the image-to-text weighted distribution loss, Indicates that after relative entropy calculation, Indicates the parallel relationship between two probability distributions, Indicates the logarithmic function, Indicates the true matching probability, Indicates an extremely small number;

[0097] In the step of swapping the input positions of the image-to-text weighted integrated feature and the text-to-image weighted integrated feature, performing relative entropy calculation, obtaining the second relative entropy calculation result, and constructing the text-to-image weighted distribution loss based on the second relative entropy calculation result, the relational expressions existing in the corresponding process are as follows:

[0098] ;

[0099] Among them, Indicates the text-to-image weighted distribution loss;

[0100] In the step of adding the image-to-text weighted distribution loss and the text-to-image weighted distribution loss to construct the bidirectional weighted distribution loss, the relational expressions existing in the corresponding process are as follows:

[0101] ;

[0102] Among them, Indicates the bidirectional weighted distribution loss.

[0103] It should be noted that the bidirectional weighted distribution loss ensures the symmetry of the learning process, enabling the model to learn complementary information from both directions. This helps the model to more comprehensively understand the relationship between images and texts and enhances its generalization ability.

[0104] Furthermore, the total loss is the cross-entropy loss, triplet loss, center-guided pair mining loss, orthogonal loss, and bidirectional weighted distribution loss. The relational expressions existing in the corresponding process are as follows:

[0105] ;

[0106] Among them, Indicates the total loss.

[0107] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0108] In the description of this specification, the description referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0109] The above-described embodiments merely represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims

1. A method for infrared person re-identification network model based on binary mixture of visual language, characterized in that: The method comprises the following steps: Step 1: Based on the image binary division mechanism, a binary separator and a mixer are constructed. The binary separator and the mixer constitute a binary hybrid module. An image encoder is constructed based on a diverse embedding extension network. A picture-text collaborative hybrid module is constructed based on the picture-text fusion mechanism. The image encoder, the binary hybrid module, the text generation module, the text encoder of the ViLT model, and the picture-text collaborative hybrid module constitute a person re-identification network model. Step 2: Input the visible light image and the infrared image into a binary separator for binary operation to obtain an image clipping block; Input the image clipping blocks into the mixer to exchange the clipping blocks and generate a binary mixed image; Step 3: Input the visible light image and the infrared image into an image encoder for image encoding to generate image features; Step 4: Encode the visible light image, infrared image and binary mixed image through a text generation module and a text encoder to generate text features; Step 5: Input the image features and text features into the image-text collaborative hybrid module for hybrid weighting operation, and perform feature set fusion through the attention weight controller to generate weighted integrated features; Step 6: Construct cross entropy loss, triplet loss, center-guided pair mining loss and orthogonal loss based on image features; Construct a bidirectional weighted distribution loss based on weighted integration features; Based on the cross entropy loss, triplet loss, center-guided pair mining loss, orthogonal loss and bidirectional weighted distribution loss, the recognition network model is optimized to obtain the optimized pedestrian re-identification network model; The recognition results are obtained using the optimized pedestrian re-identification network model.

2. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 1 is characterized in that: In step 2, the visible light image and the infrared image are input into the binary separator for binary operation to obtain image clipping blocks. The corresponding process has the following relationship: ； in, Indicates outputting the left half of the image. Indicates outputting the right half of the image. Indicates outputting the upper part of the image. Indicates outputting the lower half of the image. Indicates the width of the image. Indicates the height of the image. Indicates clipping to the left. Indicates trimming to the right. Indicates upward trimming. It means cutting downwards; In the step of inputting the image clipping block into the mixer to exchange the clipping block and generate a binary mixed image, the corresponding process has the following relationship: ； in, represents a binary mixed image, and Represents two different clipping blocks, represents a swap operation, represents a visible light image, Represents an infrared image.

3. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 2 is characterized in that: In step 3, the visible light image and the infrared image are input into an image encoder for image encoding to generate image features, which specifically includes the following steps: The visible light image and the infrared image are input into the two-stream residual network in the image encoder for feature extraction to obtain the extracted features of the first stage; Perform feature aggregation on the extracted features of the first stage to obtain the features of the current stage; Aggregate the features of the current stage to obtain the features of the next stage; The extracted features in the last stage are input into the multi-branch convolution structure for embedding to generate image features.

4. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 3 is characterized in that: In the step 4, the visible light image, the infrared image and the binary mixed image are encoded by a text generation module and a text encoder to generate text features, which specifically includes the following steps: Using the text generation module to preset the template, a preset text template is obtained; The ViLT model is used to perform image visual processing on visible light images, infrared images and binary mixed images to generate semantic information of visible light images, infrared images and binary mixed images; The ViLT model is used to synchronously receive the preset text template, and the semantic information of the visible light image, the semantic information of the infrared image, and the semantic information of the binary mixed image are respectively filled with semantic information to generate a semantically filled template; The semantically filled template is subjected to a text encoder to extract semantic information and generate text features.

5. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 4 is characterized in that: In step 5, the image features and text features are input into the image-text collaborative hybrid module for hybrid weighting operation, and feature set fusion is performed through the attention weight controller to generate weighted integrated features, which specifically includes the following steps: Input the text features into the fully connected layer for dimension matching to obtain the text features after dimension matching; Input the image features and the dimensionally matched text features into the image-text collaborative mixing module for mixed set operation, and obtain the feature set of visible light image features and visible light text features, the feature set of infrared image features and infrared text features, the feature set of visible light image features and binary mixed text features, and the feature set of infrared image features and binary mixed text features; The feature sets of visible light image features and visible light text features, the feature sets of infrared image features and infrared text features, the feature sets of visible light image features and binary mixed text features, and the feature sets of infrared image features and binary mixed text features are weighted by using an attention weight controller to obtain weighted integrated features of visible light image and text features, weighted integrated features of infrared image and text features, weighted integrated features of visible light image and binary mixed text features, and weighted integrated features of infrared image features and binary mixed text features.

6. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 5 is characterized in that: In the steps of calculating weights of the feature sets of visible light image features and visible light text features, the feature sets of infrared image features and infrared text features, the feature sets of visible light image features and binary mixed text features, and the feature sets of infrared image features and binary mixed text features using an attention weight controller to obtain weighted integrated features of visible light image and text features, weighted integrated features of infrared image and text features, weighted integrated features of visible light image and binary mixed text features, and weighted integrated features of infrared image features and binary mixed text features, the corresponding processes have the following relationship: ； ； in, Represents the weighted integrated features of visible light image features and visible light text features, Represents the weighted integrated features of infrared image features and infrared text features, Represents the weighted integrated features of visible light image features and binary mixed text features, Represents the weighted integrated features of infrared image features and binary mixed text features, Represents the visible light text feature, Indicates infrared text features, represents binary mixed text features, Represents the weight calculation result from image to text, Represents the weight calculation result from text to image, It means that after normalization function processing, Represents the dimension of image features, represents the control factor for image-to-text attention learning, Represents the dimension of text features, represents the control factor for text-to-image attention learning, represents the obtained feature weight, Representing text features The transpose of Representing image features The transpose of .

7. The method of the binary mixed visual language infrared pedestrian re-identification network model according to claim 6 is characterized in that: Based on the image features, the cross entropy loss, triplet loss, center-guided pair mining loss and orthogonal loss are constructed, which also includes the following sub-steps: Construct cross entropy loss based on image features; Among them, the cross entropy loss is constructed, and the relationship between the corresponding process is as follows: ； in, represents the cross entropy loss, represents the number of samples, represents the sample index, represents the number of categories, Represents the category index, represents the true label, represents the logarithmic function, Represents the probability predicted by the pedestrian re-identification model; Construct triplet loss based on image features; Among them, the relationship between the construction of triple loss and the corresponding process is as follows: ； in, represents the triplet loss, represents the distance function, It means that after feature extraction, and represents two different positive samples of the same category, Representation and Negative samples of different categories, represents the boundary parameter; Constructing center-guided pair mining loss based on image features; Among them, the relationship between the construction center guidance and mining loss and the corresponding process is as follows: ； in, represents the center-guided mining loss, represents the Euclidean distance between two embeddings, represents the visible light image features, Indicates the infrared image features, represents the embedding generated from visible light image features, represents the embedding generated from infrared image features, represents a hyperparameter; Construct orthogonal loss based on image features; Among them, the orthogonal loss is constructed, and the relationship between the corresponding process is as follows: ； in, represents the orthogonal loss, and Represents the embedding vectors of different branches generated by the module, Indicates the total number of branches generated by the module. and Represent two embedding vectors generated from different branches respectively.

8. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 7 is characterized in that: The bidirectional weighted distribution loss is constructed based on the weight integration feature, which includes the following steps: The weighted integrated features from image to text and the weighted integrated features from text to image are recombined into pairs, and indicators are assigned to perform cosine similarity calculations to obtain the cosine similarity scores of the attention weights between the weighted integrated features from image to text and the weighted integrated features from text to image; Perform relative entropy calculation on the cosine similarity scores of the attention weights between the weighted integrated features from image to text and the weighted integrated features from text to image to obtain a first relative entropy calculation result, and construct a weighted distribution loss from image to text based on the first relative entropy calculation result; The input positions of the weighted integrated features from image to text and the weighted integrated features from text to image are exchanged, and relative entropy calculation is performed to obtain a second relative entropy calculation result, and a weighted distribution loss from text to image is constructed based on the second relative entropy calculation result; The weighted distribution loss from image to text and the weighted distribution loss from text to image are added to construct a bidirectional weighted distribution loss.

9. The method of the visual language infrared pedestrian re-identification network model based on binary mixture according to claim 8 is characterized in that: In the step of recombining the image-to-text weighted integrated features and the text-to-image weighted integrated features into pairs and assigning indicators for cosine similarity calculation to obtain the cosine similarity score of the attention weight between the image-to-text weighted integrated features and the text-to-image weighted integrated features, the corresponding process has the following relationship: ； in, represents the cosine similarity score between the image-to-text attention weight and the text-to-image attention weight, represents the exponential function, represents the similarity function, represents the attention weight vector from image to text, and Represent two text-to-image attention weight vectors, represents the variable in the summation, represents the number of image and text attention weight pairs, represents the attention weight vector from text to image, represents the temperature hyperparameter that controls the peak of the probability distribution, Represents the index of the image, Represents the index of the text; In the step of constructing the weighted distribution loss from image to text based on the first relative entropy calculation result, the relative entropy calculation result is obtained by performing a relative entropy calculation on the cosine similarity score of the attention weight between the weighted integrated features from image to text and the weighted integrated features from text to image. The corresponding process has the following relationship: ； in, represents the weighted distribution loss from image to text, It means that after relative entropy calculation, represents the parallel relationship between two probability distributions, represents the logarithmic function, represents the true matching probability, Indicates a very small number; In the step of exchanging the input positions of the weighted integrated features from image to text and the weighted integrated features from text to image, and performing relative entropy calculation to obtain a second relative entropy calculation result, and constructing a weighted distribution loss from text to image based on the second relative entropy calculation result, the corresponding process has the following relationship: ； in, represents the weighted distribution loss from text to image; In the step of adding the weighted distribution loss from image to text and the weighted distribution loss from text to image to construct the bidirectional weighted distribution loss, the relationship between the corresponding processes is as follows: ； in, represents the bidirectional weighted distribution loss.

Images