Infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion

Abstract

The application provides an infrared pedestrian re-identification method based on frequency and visual-language adaptive fusion, which comprises the following steps: inputting a visible image and an infrared image into an image encoder, and realizing the extraction of features from low-level textures to high-level semantics through multi-stage residual processing; inputting the visible image and the infrared image into a text feature processing module, and generating a plurality of candidate text descriptions associated with the pedestrian image; and inputting the global image features and the optimal text features into a picture-text frequency mixing module, and obtaining image-text mixed features through frequency domain conversion, cross-modal interaction and calculation of the attention mechanism. The application realizes the deep alignment of cross-modal features through frequency domain conversion and the attention mechanism, balances the complementary information of images and texts in the frequency dimension, avoids the problem of single modal dominant feature representation, and thus improves the comprehensive utilization of cross-modal information by the model, so that more accurate recognition is realized in the pedestrian re-identification task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to an infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion. BACKGROUND

[0002] Visible-infrared pedestrian re-identification (VI-ReID) aims to realize the identity matching of pedestrians under continuous monitoring in day and night, which is of great significance to improve the real combat capability of security systems in all-weather scenes. However, in practical applications such as night campus security, existing technologies face severe challenges. In such scenarios, the images of pedestrians captured by night infrared cameras need to be quickly compared with the targets recorded by daytime visible light cameras to realize the positioning of specific personnel. However, this process has the following outstanding problems:

[0003] (1) Night infrared images are limited by imaging mechanisms and can only provide gray contours, lacking key visual features such as color and texture. At the same time, visible light images in low light are prone to motion blur and noise interference, further weakening the quality of available information. The modal difference between these two types of images is extremely magnified, making it difficult for traditional methods based on single modal or simple multi-modal fusion to achieve effective feature alignment and discrimination.

[0004] (2) Due to the lack of clear structural information such as edges and directions of pedestrians in infrared images, conventional feature extraction methods cannot fully capture their directional structural features, resulting in a decrease in feature discrimination, especially in complex situations such as target occlusion and posture variation.

[0005] (3) Existing methods often use fixed text templates to generate auxiliary descriptions, which cannot adaptively select discriminative texts based on image content, leading to inconsistencies between texts and image semantics. In addition, in the feature fusion process, there is a lack of mechanism to balance and interact image and text information in the frequency dimension, which can easily lead to one modality (such as image) dominating the final feature representation, weakening the complementary advantage of cross-modal, and causing misidentification.

[0006] The above problems collectively result in high misidentification rate and large retrieval delay of existing systems in night cross-period pedestrian matching tasks, severely limiting the real-time response capability and recognition accuracy of security systems. SUMMARY

[0007] In view of the above situation, the main purpose of the present application is to propose a beneficiation equipment alarm method and system based on semantic analysis to solve the above technical problems.

[0008] The present application proposes an infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion, which comprises the following steps:

[0009] Step 1, a pedestrian re-identification network model is formed based on an image encoder, a text feature processing module and a graph-text frequency mixing module;

[0010] Step 2, the visible image and the infrared image are input into the image encoder, and through multi-stage residual processing, extraction of low-level texture to high-level semantic features is realized to obtain global image features;

[0011] Step 3, the visible image and the infrared image are input into the text feature processing module, a plurality of candidate text descriptions associated with the pedestrian image are generated, and the optimal text features are adaptively selected from the plurality of candidate text descriptions associated with the pedestrian image;

[0012] Step 4, the global image features and the optimal text features are input into the graph-text frequency mixing module, and through frequency domain conversion, cross-modal interaction and calculation of the attention mechanism, image-text mixed features are obtained; identity classification loss and triplet loss are constructed based on the classification output of the global image features; relative entropy loss is constructed based on the image-text mixed features and the global image features; the pedestrian re-identification network model is optimized by using the identity classification loss, the triplet loss and the relative entropy loss, and an optimized pedestrian re-identification network model is obtained;

[0013] The visible image and the infrared image are input into the optimized pedestrian re-identification network model to obtain the final global image features, and a classification regression operation is performed on the final global image features to obtain the recognition result.

[0014] Compared with the prior art, the present application has the following advantages:

[0015] 1. The present application adaptively selects the optimal description from the diversified candidate texts through a semantic similarity calculation and evaluation mechanism. It can flexibly adjust the text selection strategy according to the real-time features of the input image, effectively improve the semantic consistency of text and image modalities, and enhance the adaptability of the model to complex scenes, significantly improving the feature matching accuracy in cross-modal tasks;

[0016] 2. The present application strengthens the directional structure information in the image through horizontal / vertical phase / amplitude extraction and Euler expansion mechanism. It can accurately capture key directional features such as horizontal edges and vertical textures, making up for the deficiency of traditional methods in representing directional structure, so that the model can still maintain sensitive perception of target structure in complex background, thereby improving the discriminability of feature description.

[0017] 3. The present application realizes deep alignment of cross-modal features through frequency domain transformation and attention mechanism, balances the complementary information of image and text in frequency dimension, avoids the problem of single modal dominant feature representation, thereby improving the comprehensive utilization ability of the model to cross-modal information, and realizing more accurate recognition in the pedestrian re-identification task.

[0018] Additional aspects and advantages of the present application will be set forth in part in the description which follows, and in part will be obvious from the description or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The step flow chart of the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion proposed by the present application.

[0020] Figure 2 The network model architecture diagram of the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion proposed by the present application.

[0021] Figure 3 The direction perception feature enhancement module architecture diagram of the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion proposed by the present application.

[0022] Figure 4 The adaptive text feature selection module architecture diagram of the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion proposed by the present application.

[0023] Figure 5 The image-text frequency mixing module architecture diagram of the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion proposed by the present application. DETAILED DESCRIPTION

[0024] Embodiments of the present application are described in detail below with reference to the attached drawings, which show by way of example, embodiments in which the same or similar elements or elements having the same or similar functions are denoted by the same reference numerals throughout the drawings. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, and should not be understood as limiting the present application.

[0025] These and other aspects of embodiments of the present application will be more apparent from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of embodiments of the present application are specifically disclosed to provide some ways of implementing principles of embodiments of the present application, but it should be understood that the scope of embodiments of the present application is not limited thereto.

[0026] Referring to Figure 1 , the present application proposes an infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion, which comprises the following steps:

[0027] Step 1, a pedestrian re-identification network model is constructed based on an image encoder, a text feature processing module and an image-text frequency mixing module.

[0028] In step 1, a pedestrian re-identification network model is constructed based on an image encoder, a text feature processing module and a graph-text frequency hybrid module; wherein the image encoder comprises an initial convolution block, a plurality of residual stages composed of bottleneck blocks and a direction-aware feature enhancement module; the plurality of residual stages composed of bottleneck blocks comprises a first residual stage, a second residual stage, a third residual stage and a fourth residual stage; the direction-aware feature enhancement module comprises a horizontal / vertical phase extraction component, a horizontal / vertical amplitude extraction component, a channel processing component and an Euler expansion module.

[0029] The text feature processing module comprises a text prediction module, a text encoder and an adaptive text feature selection module.

[0030] In step 2, the visible image and the infrared image are input into the image encoder, and multi-stage residual processing is performed to extract low-level texture to high-level semantic features to obtain global image features.

[0031] Referring to Figure 2 and Figure 3 In step 2, the visible image and the infrared image are input into the image encoder, and multi-stage residual processing is performed to extract low-level texture to high-level semantic features to obtain global image features, comprising the following steps:

[0032] A pre-trained ResNet-50 model is used as the image encoder, the visible image and the infrared image are input into the image encoder, an initial convolution operation is performed by the initial convolution block, and maximum pooling processing is performed to obtain processed features;

[0033] The processed features are processed by the first residual stage, the second residual stage and the third residual stage to obtain a feature map output by the third residual stage;

[0034] The feature map output by the third residual stage is input into the direction-aware feature enhancement module, the horizontal / vertical phase extraction component and the horizontal / vertical amplitude extraction component are used to extract horizontal phase information, vertical phase information, horizontal amplitude information and vertical amplitude information respectively, and the Euler expansion module is used to fuse the horizontal phase information and the vertical phase information, the horizontal amplitude information and the vertical amplitude information by Euler formula to generate direction phase features and direction amplitude features;

[0035] The channel dimension features are extracted from the feature map output by the third residual stage by the channel processing component;

[0036] The channel dimension features, the direction phase features and the direction amplitude features are spliced and fused along the channel dimension to generate the output of the direction-aware feature enhancement module;

[0037] The output of the direction-aware feature enhancement module is input to a fourth residual stage for processing and global average pooling to generate global image features.

[0038] The feature map output after the third residual stage processing is input to the direction-aware feature enhancement module, and horizontal / vertical phase extraction components and horizontal / vertical amplitude extraction components are used to extract horizontal phase information, vertical phase information, horizontal amplitude information and vertical amplitude information, respectively. The relationship of the corresponding process is as follows:

[0039] ;

[0040] Among them, represents the horizontal phase information, represents the linear activation function, represents the horizontal branch batch normalization, represents the vertical phase convolution, represents the feature map output after the third residual stage processing, represents the vertical phase information, represents the vertical branch batch normalization, represents the vertical amplitude convolution, represents the horizontal amplitude information, represents the horizontal amplitude convolution, represents the vertical amplitude information, represents the vertical amplitude convolution;

[0041] In the step of fusing the horizontal phase information and the vertical phase information, the horizontal amplitude information and the vertical amplitude information by Euler formula through the Euler expansion module to generate direction phase features and direction amplitude features, the relationship of the corresponding process is as follows:

[0042] ;

[0043] Among them, represents the direction phase feature, represents the direction amplitude feature, represents the 1x7 convolution, represents the 7x1 convolution, represents the dot product operation, represents the cosine function, represents the sine function, represents the concatenation operation in the channel dimension;

[0044] In the step of concatenating and fusing the channel dimension features, the direction phase features and the direction amplitude features along the channel dimension to generate the output of the direction-aware feature enhancement module, the relationship of the corresponding process is as follows:

[0045] ;

[0046] wherein, represents the output of the direction-aware feature enhancement module, represents a 1x1 convolution, represents feature concatenation in the channel dimension, represents channel dimension features.

[0047] Further, the image encoder adopts a pre-trained ResNet-50 model to extract discriminative visual features from the input image. The architecture contains an initial convolutional block, four consecutive residual stages, and an integrated direction-aware feature enhancement module, aiming to capture hierarchical visual representations. The horizontal and vertical structural features extracted by the direction-aware feature enhancement module can accurately cover the key structural information of pedestrians, and are used to strengthen the representation ability of image features for pedestrian details.

[0048] Step 3, input the visible image and the infrared image into the text feature processing module to generate a plurality of candidate text descriptions associated with the pedestrian image, and adaptively screen the optimal text feature from the plurality of candidate text descriptions associated with the pedestrian image.

[0049] Referring to Figure 4 , in step 3, the visible image and the infrared image are input into the text feature processing module to generate a plurality of candidate text descriptions associated with the pedestrian image, and the optimal text feature is adaptively screened from the plurality of candidate text descriptions associated with the pedestrian image, comprising the following steps:

[0050] The ViLT model is selected as the text prediction module, a text template is constructed, the visible image and the infrared image are input into the text prediction module, and the visual clues of the visible image and the infrared image are used to fill the placeholders of the text template in semantics, to generate a complete text description;

[0051] The BERT model is used as the text encoder, the complete text description is input into the text encoder, the complete text description is converted into a token ID sequence through the tokenizer shared by the text prediction module, and the text feature is generated by using the last hidden state of the BERT model on the token ID sequence;

[0052] The global image feature and the text feature are input into the adaptive text feature selection module for normalization processing, to obtain the normalized image feature and the normalized text feature;

[0053] The normalized image feature and the normalized text feature are subjected to cosine similarity calculation, to obtain a similarity matrix of the image and the text;

[0054] The matrix elements of the image and text similarity matrix are aggregated by average pooling, and the optimal text features are screened according to the index of the maximum value.

[0055] The global image features and text features are input into the adaptive text feature selection module for normalization to obtain normalized image features and normalized text features, and the relationship formula existing in the corresponding process is as follows:

[0056] ;

[0057] Among them, denotes the normalized image features, denotes the global image features, denotes the L2 norm, denotes a constant, denotes the normalized text features, denotes the text features;

[0058] In the step of calculating the cosine similarity of the normalized image features and the normalized text features to obtain the image and text similarity matrix, the relationship formula existing in the corresponding process is as follows:

[0059] ;

[0060] Among them, denotes the image and text similarity matrix, denotes the initial value related coefficient, denotes the transpose of the normalized text features , denotes the exponential function, denotes the logarithmic function.

[0061] Further, the text prediction module selects the ViLT model, which is a lightweight visual language converter that integrates visual and text information for cross-modal understanding. Using the Masked Language Modeling (MLM) mechanism, ViLT iteratively predicts and fills the mask tokens in the text template. At the same time, the BERT model is used as the text encoder, which is a pre-trained bidirectional converter model optimized for deep context understanding of text.

[0062] It should be noted that the text template takes "the age of

MASK

MASK

MASK

MASK

MASK

MASK

MASK

[0063] Step 4, input the global image feature and the optimal text feature into the image-text frequency mixing module, obtain the image-text mixed feature through frequency domain conversion, cross-modal interaction and calculation of attention mechanism; construct identity classification loss and triplet loss based on the classification output of the global image feature, construct relative entropy loss based on the image-text mixed feature and the global image feature, optimize the pedestrian re-identification network model by using the identity classification loss, the triplet loss and the relative entropy loss, and obtain the optimized pedestrian re-identification network model;

[0064] Input the visible image and the infrared image into the optimized pedestrian re-identification network model to obtain the final global image feature, and perform classification regression operation on the final global image feature to obtain the recognition result.

[0065] Please refer to Figure 5 In step 4, the global image feature and the optimal text feature are input into the image-text frequency mixing module, and the image-text mixed feature is obtained through frequency domain conversion, cross-modal interaction and calculation of attention mechanism, which includes the following steps:

[0066] The global image feature and the optimal text feature are input into the image-text frequency mixing module, and are processed by convolution layers respectively, and are reshaped into two-dimensional matrices to obtain the reshaped image feature tensor and the reshaped text feature tensor;

[0067] The reshaped image feature tensor and the reshaped text feature tensor are converted to frequency domain by discrete cosine transform respectively to obtain the converted image feature tensor and the converted text feature tensor;

[0068] The converted image feature tensor and the converted text feature tensor are interacted in the frequency domain by attention mechanism to obtain the frequency domain interaction feature;

[0069] The frequency domain interaction feature is restored to the spatial domain by inverse discrete cosine transform to generate the image-text fusion feature;

[0070] The image-text fusion feature is subjected to parallel Hadamard product operation with the global image feature and the optimal text feature respectively to obtain the product result of the image branch and the product result of the text branch;

[0071] The product result of the text branch is calculated to obtain the adjusted text branch product result, and then the adjusted text branch product result is subjected to matrix difference operation with the optimal text feature to obtain the adjusted text branch feature;

[0072] The product result of the image branch is added to the adjusted text branch feature to obtain the image-text mixed feature.

[0073] The global image features and the optimal text features are input into a graph-text frequency mixing module, processed by convolution layers respectively, and reshaped into two-dimensional matrices. The relationship formula corresponding to the process is as follows:

[0074] ;

[0075] wherein, represents the reshaped image feature tensor, represents a dimension reshaping operator, represents a batch size, represents a channel dimension, represents a block size, represents a target dimension;

[0076] In the step of feature interaction in the frequency domain by attention mechanism on the image feature tensor converted to frequency and the text feature tensor converted to frequency, the relationship formula corresponding to the process is as follows:

[0077] ;

[0078] wherein, represents an attention matrix, represents the image feature tensor converted to frequency, represents the text feature tensor converted to frequency, represents a normalization function, represents the transpose of the text feature tensor converted to frequency , represents a scaling factor.

[0079] The product result of the text branch is adjusted to calculate an adjustment parameter, and the product result of the text branch and the adjustment parameter are subjected to matrix difference operation. The relationship formula corresponding to the process is as follows:

[0080] ;

[0081] wherein, represents an adjustment parameter, represents a summation operation along the target dimension , and represent two learnable query direction parameters, and represent two learnable key direction parameters, represents an initial bias parameter.

[0082] It should be noted that in Figure 2 , represents an identity classification loss, represents a triplet loss, representing image-text fusion features.

[0083] It should be understood that each of the elements of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or combinations thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application-specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field-programmable gate arrays (FPGA), and the like.

[0084] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0085] The above-described embodiments only express several implementation manners of the present application, which are described in a more specific and detailed manner, but cannot be understood as limitations on the patent scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. An infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion, characterized in that, The method comprises the following steps: Step 1: constructing a pedestrian re-identification network model based on an image encoder, a text feature processing module and a graph-text frequency mixing module; Step 2: inputting the visible image and the infrared image into the image encoder, and realizing the extraction of the global image features from low-level texture to high-level semantic features through multi-stage residual processing to obtain the global image features; Step 3: inputting the visible image and the infrared image into the text feature processing module to generate a plurality of candidate text descriptions associated with the pedestrian image, and adaptively screening the optimal text features from the plurality of candidate text descriptions associated with the pedestrian image; Step 4: inputting the global image features and the optimal text features into the graph-text frequency mixing module, and obtaining the image-text mixed features through frequency domain conversion, cross-modal interaction and calculation of the attention mechanism; The identity classification loss and the triplet loss are constructed based on the classification output of the global image features, the relative entropy loss is constructed based on the image-text mixed features and the global image features, and the pedestrian re-identification network model is optimized by using the identity classification loss, the triplet loss and the relative entropy loss to obtain the optimized pedestrian re-identification network model; The visible image and the infrared image are input into the optimized pedestrian re-identification network model to obtain the final global image features, and the classification regression operation is performed on the final global image features to obtain the recognition result; In the step 3, the visible image and the infrared image are input into the text feature processing module to generate a plurality of candidate text descriptions associated with the pedestrian image, and the optimal text features are adaptively screened from the plurality of candidate text descriptions associated with the pedestrian image, which specifically comprises the following steps: The ViLT model is selected as the text prediction module, a text template is constructed, the visible image and the infrared image are input into the text prediction module, the visual clues of the visible image and the infrared image are used to fill the placeholders of the text template in the semantic aspect to generate a complete text description; The BERT model is used as the text encoder, the complete text description is input into the text encoder, the complete text description is converted into a token ID sequence through the tokenizer shared by the text prediction module, and the text features are generated by using the last hidden state of the BERT model; The global image features and the text features are input into the adaptive text feature selection module for normalization to obtain the normalized image features and the normalized text features; The cosine similarity of the normalized image features and the normalized text features is calculated to obtain the similarity matrix of the image and the text; The matrix elements of the similarity matrix of the image and the text are aggregated through average pooling, and the optimal text features are screened according to the index of the maximum value.

2. The method of claim 1, wherein the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion is characterized in that, In the step 1, the pedestrian re-identification network model is constructed based on the image encoder, the text feature processing module and the graph-text frequency mixing module; The image encoder comprises an initial convolution block, a plurality of residual stages composed of bottleneck blocks and a direction perception feature enhancement module; the plurality of residual stages composed of bottleneck blocks comprises a first residual stage, a second residual stage, a third residual stage and a fourth residual stage. The direction perception feature enhancement module comprises a horizontal / vertical phase extraction component, a horizontal / vertical amplitude extraction component, a channel processing component, and an Euler expansion module. The text feature processing module comprises a text prediction module, a text encoder, and an adaptive text feature selection module.

3. The method of claim 2, wherein the infrared pedestrian re-identification method is based on frequency and visual-linguistic adaptive fusion. In the step 2, the visible image and the infrared image are input into the image encoder, and the extraction of low-level texture to high-level semantic feature is realized through multi-stage residual processing to obtain the global image feature, specifically comprising the following steps: The pre-trained ResNet-50 model is used as the image encoder, the visible image and the infrared image are input into the image encoder, the initial convolution operation is performed through the initial convolution block, and the maximum pooling processing is performed to obtain the processed feature; The processed feature is processed through the first residual stage, the second residual stage and the third residual stage to obtain the feature map output after the third residual stage processing; The feature map output after the third residual stage processing is input into the direction perception feature enhancement module, the horizontal phase information, the vertical phase information, the horizontal amplitude information and the vertical amplitude information are extracted through the horizontal / vertical phase extraction component and the horizontal / vertical amplitude extraction component, and the Euler formula is used to fuse the horizontal phase information and the vertical phase information, the horizontal amplitude information and the vertical amplitude information through the Euler expansion module to generate the direction phase feature and the direction amplitude feature; The channel dimension feature is extracted from the feature map output after the third residual stage processing using the channel processing component to obtain the channel dimension feature; The channel dimension feature, the direction phase feature and the direction amplitude feature are spliced and fused along the channel dimension to generate the output of the direction perception feature enhancement module; The output of the direction perception feature enhancement module is input into the fourth residual stage for processing, and the global average pooling processing is performed to generate the global image feature.

4. The method of claim 3, wherein the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion is characterized in that, The feature map output after the third residual stage processing is input into the direction perception feature enhancement module, the horizontal phase information, the vertical phase information, the horizontal amplitude information and the vertical amplitude information are extracted through the horizontal / vertical phase extraction component and the horizontal / vertical amplitude extraction component, and the corresponding relationship of the process is as follows: ； wherein, represents horizontal phase information, represents a linear activation function, represents horizontal branch batch normalization, represents horizontal phase convolution, represents a feature map output after a third residual stage processing, represents vertical phase information, represents vertical branch batch normalization, represents vertical phase convolution, represents horizontal amplitude information, represents horizontal amplitude convolution, represents vertical amplitude information, represents vertical amplitude convolution; In the step of using the Euler formula to fuse the horizontal phase information and the vertical phase information, the horizontal amplitude information and the vertical amplitude information through the Euler expansion module to generate the direction phase feature and the direction amplitude feature, the corresponding relationship of the process is as follows: ； wherein, represents a directional phase feature, represents a directional amplitude feature, represents a 1 x 7 convolution, represents a 7 x 1 convolution, represents a dot product operation, represents a cosine function, represents a sine function, represents a concatenation operation over the channel dimension; In the step of splicing and fusing the channel dimension feature, the direction phase feature and the direction amplitude feature along the channel dimension to generate the output of the direction perception feature enhancement module, the corresponding relationship of the process is as follows: ； wherein, represents the output of the direction perception feature enhancement module, represents a 1x1 convolution, represents feature concatenation in the channel dimension, represents a channel dimension feature.

5. The method of claim 4, wherein the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion is characterized in that, The global image feature and the text feature are input into the adaptive text feature selection module for normalization to obtain the normalized image feature and the normalized text feature, and the corresponding relationship of the process is as follows: ； wherein, denotes a normalized image feature, denotes a global image feature, denotes an L2 norm, denotes a constant, denotes a normalized text feature, denotes a text feature; In the step of calculating the cosine similarity of the normalized image feature and the normalized text feature to obtain the similarity matrix of the image and the text, the corresponding relationship of the process is as follows: ； wherein, represents a similarity matrix of images and texts, represents an initial value a coefficient related to, represents a transpose of normalized text features , represents an exponential function, represents a logarithmic function.

6. The method of claim 5, wherein the infrared pedestrian re-identification method based on frequency and visual-linguistic adaptive fusion is characterized in that, In the step 4, the global image feature and the optimal text feature are input into the image-text frequency mixing module, and through the cross-modal interaction and calculation of the frequency domain conversion and the attention mechanism, the image-text mixed feature is obtained, which specifically includes the following steps: The global image feature and the optimal text feature are input into the image-text frequency mixing module, and are processed by convolution layers respectively, and are reshaped into two-dimensional matrices to obtain the reshaped image feature tensor and the reshaped text feature tensor; The reshaped image feature tensor and the reshaped text feature tensor are converted to the frequency domain through discrete cosine transformation respectively to obtain the converted image feature tensor and the converted text feature tensor; The converted image feature tensor and the converted text feature tensor are interacted through the attention mechanism in the frequency domain to obtain the frequency domain interaction feature; The frequency domain interaction feature is restored to the spatial domain through inverse discrete cosine transformation to generate the image-text fusion feature; The image-text fusion feature is subjected to parallel Hadamard product operation with the global image feature and the optimal text feature respectively to obtain the product result of the image branch and the product result of the text branch respectively; The product result of the text branch is calculated to obtain the adjusted parameter, and the product result of the text branch after the parameter calculation is subjected to matrix difference operation with the optimal text feature to obtain the adjusted text branch feature; The product result of the image branch is added to the adjusted text branch feature to obtain the image-text mixed feature.

7. The method of claim 6, wherein the infrared pedestrian re-identification method is based on frequency and visual-linguistic adaptive fusion. The global image feature and the optimal text feature are input into the image-text frequency mixing module, and are processed by convolution layers respectively, and are reshaped into two-dimensional matrices, and the relationship formula corresponding to the process exists as follows: ； wherein, represents the reshaped image feature tensor, represents the dimension reshaping operator, represents the batch size, represents the channel dimension, represents the block size, represents the target dimension; In the step of interacting the converted image feature tensor and the converted text feature tensor through the attention mechanism in the frequency domain, the relationship formula corresponding to the process exists as follows: ； wherein, denotes the attention matrix, denotes the image feature tensor converted to frequency, denotes the text feature tensor converted to frequency, denotes the normalization function, denotes the transpose of the text feature tensor converted to frequency , denotes the scaling factor.

8. The method of claim 7, wherein the infrared pedestrian re-identification method is based on frequency and visual-linguistic adaptive fusion. The product result of the text branch is adjusted to calculate the adjustment parameter, and the product result of the text branch is subjected to matrix difference operation with the adjustment parameter, and the relationship formula corresponding to the process exists as follows: ； wherein, denotes an adjustment parameter, denotes a summation operation along the target dimension denotes a summation operation along the target dimension and denote two learnable query direction parameters, and denote two learnable key direction parameters, denotes an initial bias parameter.

Images