A pedestrian re-identification method and device based on Transformer network

Abstract

The present invention provides a person re-identification method and device based on a Transformer network, which is achieved by dividing the original image into two branches; linearly mapping the two branches to obtain a first sequence and a second sequence; adding new parameters to the first sequence and the second sequence to generate a third sequence and a fourth sequence; placing the third sequence and the fourth sequence into corresponding different layers of the Transformer network for feature extraction, thereby obtaining a first global feature and a first local feature; fusing the first global feature and the first local feature to obtain a second global feature and a second local feature; processing the second local feature, placing the processed second local feature and the second global feature into specific layers of the Transformer network for feature extraction, and calculating the overall loss of the extracted features according to the corresponding loss function. This method can effectively improve the accuracy and robustness of the person re-identification task. The device also has the above-mentioned beneficial effects.

Description

Technical Field

[0001] The present invention relates to the technical field of network image processing, and in particular to a pedestrian re-identification method and device based on a Transformer network. Background Art

[0002] With economic development and technological advancements, person re-identification (PIR) has become a research hotspot in intelligent video analysis and has garnered widespread attention in academia. PIR utilizes computer vision technology to determine whether a target person exists in images or videos captured by different cameras or surveillance cameras. However, due to varying camera resolutions, camera angles, lighting variations, and other objective factors, obtaining high-quality images is often difficult, presenting significant challenges for PIR technology.

[0003] Currently, with the development of deep learning, convolutional neural networks (CNNs) are widely used to extract representational features for person re-identification. However, while typical CNNs focus only on extracting local features, global features are particularly important for person re-identification. Transformers, derived from natural language processing (NLP), take an entire sequence as input, placing greater emphasis on extracting global features than CNNs. Transformer networks are also a common choice for extracting features from images or videos. However, in the field of person re-identification, the information extracted from both global and local features is crucial for the accuracy and robustness of the task, and the existing Transformer network feature extraction methods are not suitable for improving the accuracy and robustness of person re-identification.

[0004] Therefore, it is an urgent problem to be solved by those skilled in the art to provide a pedestrian re-identification method and device based on a Transformer network, which can effectively improve the accuracy and robustness of pedestrian re-identification tasks compared with CNN and the existing Transformer network. Summary of the Invention

[0005] The present invention aims to provide a person re-identification method and device based on a Transformer network. This method features clear logic, streamlined steps, and simple operation. This method addresses the shortcomings of CNN convolutional neural networks and existing Transformer networks in person re-identification tasks, effectively improving the accuracy and robustness of such tasks. This device also achieves the aforementioned beneficial effects.

[0006] Based on the above objectives, the technical solutions provided by the present invention are as follows:

[0007] A person re-identification method based on a Transformer network includes the following steps:

[0008] Divide the original image to obtain a first branch and a second branch;

[0009] linearly mapping the first branch and the second branch to obtain a first sequence and a second sequence;

[0010] Add parameters to the first sequence and the second sequence respectively to obtain a third sequence and a fourth sequence;

[0011] Put the third sequence and the fourth sequence into the corresponding layers of the Transformer network for feature extraction to obtain the first global feature and the first local feature;

[0012] feature fusing the first global feature and the first local feature to obtain a second global feature and a second local feature;

[0013] The processed second local features and the second global features are respectively placed into specific layers of the Transformer network for feature extraction, and the overall loss is calculated according to the corresponding loss function;

[0014] The third sequence is the first sequence after adding parameters, and the fourth sequence is the second sequence after adding parameters.

[0015] Preferably, before dividing the original image to obtain the first branch and the second branch, the method further includes the following steps:

[0016] Preset first plaque and preset second plaque;

[0017] inputting the original image;

[0018] Wherein, the first plaque size is larger than the second plaque size.

[0019] Preferably, the parameters specifically include: auxiliary information and position coding information.

[0020] Preferably, the step of placing the third sequence and the fourth sequence into corresponding layers of a Transformer network to perform feature extraction to obtain the first global feature and the first local feature specifically includes the following steps:

[0021] Obtain the first layer number corresponding to the third sequence according to the ablation experiment;

[0022] Obtain the second layer number corresponding to the fourth sequence according to the ablation experiment;

[0023] Putting the third sequence into the Transformer network with the first number of layers, and putting the fourth sequence into the Transformer network with the second number of layers, respectively performing feature extraction to obtain the first global feature and the first local feature of the third sequence and the first global feature and the first local feature of the fourth sequence;

[0024] Among them, feature extraction is based on feature interaction between the encoder and decoder in the Transformer network.

[0025] Preferably, the feature fusion of the first global feature and the first local feature to obtain a second global feature and a second local feature specifically includes the following steps:

[0026] The first global feature of the third sequence and the first local feature of the fourth sequence, as well as the first local feature of the third sequence and the first global feature of the fourth sequence are respectively put into the cross attention network for feature fusion to obtain the second global feature and the second local feature.

[0027] Preferably, before the feature fusing the first global feature and the first local feature to obtain the second global feature and the second local feature, the following steps are further included:

[0028] In the cross-attention network, the fully functional nodes are replaced by mapping and inverse mapping relationships.

[0029] Preferably, before respectively placing the processed second local features and the second global features into a specific layer of the Transformer network for feature extraction and respectively calculating the overall loss according to the corresponding loss function, the following steps are further included:

[0030] performing a shuffling operation on the second local features;

[0031] The shuffling operation specifically includes: disrupting the auxiliary information and the position encoding information in the second local feature.

[0032] Preferably, the specific layer is specifically the last layer in the Transformer network.

[0033] Preferably, the processing of the second local features and the second global features is respectively put into a specific layer of the Transformer network for feature extraction, and the overall loss is obtained by calculating the corresponding loss function, which specifically includes the following steps:

[0034] The second local feature and the second global feature after the shuffle operation are respectively put into the last layer of the Transformer network for feature extraction to obtain a third global feature and a third local feature;

[0035] Obtain a first loss according to the third global feature and a preset loss function ID Loss;

[0036] Obtaining a second loss according to the third local feature and a preset loss function Triple Loss;

[0037] Obtaining an overall loss based on the first loss and the second loss;

[0038] The overall loss is specifically an average value of the first loss and the second loss.

[0039] A person re-identification device based on a Transformer network, comprising:

[0040] A division module, configured to divide the original image into a first branch and a second branch;

[0041] A mapping module, configured to linearly map the first branch and the second branch to obtain a first sequence and a second sequence;

[0042] a parameter adding module, configured to add parameters to the first sequence and the second sequence to convert them into a third sequence and a fourth sequence;

[0043] a feature extraction module, configured to extract a first global feature and a first local feature of the third sequence and the fourth sequence respectively;

[0044] A feature fusion module, configured to fuse the first global feature and the first local feature to obtain a second global feature and a second local feature;

[0045] a processing module, configured to process the second local feature;

[0046] A calculation module is used to calculate and obtain the overall loss based on the processed second local features and the second global features and the corresponding loss function.

[0047] The present invention provides a pedestrian re-identification method based on a Transformer network, which is achieved by dividing an original image into two branches; linearly mapping the two branches respectively to obtain a first sequence and a second sequence; adding new parameters to the first sequence and the second sequence to generate a third sequence and a fourth sequence; placing the third sequence and the fourth sequence into corresponding different layers of the Transformer network for feature extraction, thereby obtaining a first global feature and a first local feature; fusing the first global feature and the first local feature to obtain a second global feature and a second local feature; processing the second local feature and not processing the second global feature, placing the processed second local feature and the second global feature into specific layers of the Transformer network for feature extraction, and calculating the overall loss of the extracted features according to corresponding loss functions.

[0048] In fact, the applicant discovered during their research on related issues in the field of person re-identification that, while previous CNN methods focused more on local, fine-grained feature representation, person re-identification requires more than just fine-grained features. Due to the influence of a series of objective factors, such as lighting, environmental background changes, and camera monitoring angle changes, accurate judgment cannot be made solely based on local features. Recent research has shown that CNN's adaptability to massive amounts of data is not as good as expected. On the contrary, the performance of the Transformer model improves as the amount of data increases, making the Transformer's advantage over CNN on large datasets in person re-identification more pronounced. Furthermore, CNN focuses on extracting local features and is prone to local optimization. Furthermore, the pooling in general CNN networks can cause information loss. Existing technologies use Transformer networks to extract global features, which are equally important for fine-grained features in the task of person re-identification.

[0049] On this basis, this case divides the original image into two branches, one branch extracts coarse-grained features and the other extracts fine-grained features. The obtained two-dimensional branches representing coarse-grained and fine-grained features are then converted into one-dimensional sequences, resulting in the first and second sequences. This allows for simultaneous feature extraction of both coarse and fine grains, greatly improving the accuracy of person re-identification tasks. To avoid a series of adverse factors caused by model recognition bias due to objective factors such as lighting, angle impressions, or different background environments in the images captured by the camera, parameters are added to the first and second sequences to obtain third and fourth sequences. The third and fourth sequences are then placed in the corresponding layers of the Transformer network for feature extraction. After extracting features from the two different branches, feature fusion is performed. As the coarse and fine grain information between the two branches interact, features selected at different scales can be fused together. After feature fusion, local features are pre-processed to achieve the effect of data image enhancement, making the output process more robust. Finally, the processed local features and global features are respectively calculated using corresponding loss functions to obtain the overall loss.

[0050] The present invention also provides a pedestrian re-identification device based on a Transformer network, which includes a module for implementing the above method. Since the device adopts the corresponding module, the device should have the same beneficial effects as the above method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 A flowchart of a person re-identification method based on a Transformer network provided by an embodiment of the present invention;

[0053] Figure 2 Flowchart before step S1 provided for an embodiment of the present invention;

[0054] Figure 3 A flowchart of step S4 provided in an embodiment of the present invention;

[0055] Figure 4 A flowchart of step S6 provided in an embodiment of the present invention;

[0056] Figure 5A schematic diagram of the structure of a pedestrian re-identification device based on a Transformer network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0058] The embodiments of the present invention are written in a progressive manner.

[0059] The present invention provides a person re-identification method based on a Transformer network. This method addresses the technical problem that, in existing technologies, CNNs focus only on extracting local features, while existing Transformer networks tend to extract global features. Both have certain drawbacks, hindering the accuracy and robustness of person re-identification tasks.

[0060] A person re-identification method based on a Transformer network includes the following steps:

[0061] S1. Divide the original image to obtain a first branch and a second branch;

[0062] S2. linearly mapping the first branch and the second branch to obtain a first sequence and a second sequence;

[0063] S3. Add parameters to the first and second sequences respectively to obtain the third and fourth sequences;

[0064] S4. Place the third sequence and the fourth sequence into the corresponding layers of the Transformer network to perform feature extraction to obtain the first global feature and the first local feature;

[0065] S5. Feature fusion of the first global feature and the first local feature to obtain a second global feature and a second local feature;

[0066] S6. Place the processed second local features and the second global features into specific layers of the Transformer network for feature extraction, and calculate the overall loss according to the corresponding loss functions.

[0067] The third sequence is the first sequence after adding parameters, and the fourth sequence is the second sequence after adding parameters.

[0068] In step S1, the existing Transformer network is mainly used to obtain features of global sequences. In the task of pedestrian re-identification, obtaining fine-grained features is also particularly important. Therefore, the first step is to extract features through two different patches (in the CNN learning and training process, instead of processing the entire image at once, the image is first divided into multiple small blocks. The kernel, filter, or feature detector only looks at one block of the image at a time. This small block is called a patch, and then the filter moves to another patch of the image) to obtain the first branch B-Branch and the second branch S-Branch.

[0069] In step S2, the two branches simultaneously obtain one-dimensional tokens (sequences, the basic unit of processing in the Transformer network) through a layer of linear mapping. Each branch obtains a one-dimensional token, namely the first sequence and the second sequence;

[0070] In step S3, in order to avoid a series of adverse factors caused by model recognition bias due to objective factors such as the influence of lighting factors, angle impressions, or different background environments on the images captured by the camera, corresponding parameters are added to the first sequence and the second sequence respectively, to obtain the first sequence after the parameters are added and the second sequence after the parameters are added, namely the third sequence and the fourth sequence;

[0071] In step S4, the two branches of different sizes are designed according to the number of layers of the Transformer to which they are most adapted. Therefore, the two branches are placed in Transformers of different layers for feature extraction, thereby obtaining the first global feature cls-token and the first local feature patch-tokens.

[0072] In step S5, when fusion is performed, the first global feature cls-token of one branch B-Branch and the first local feature patch-tokens of another branch S-Branch are fused and interacted; the same is true for the other branch S-Branch. After the fusion is completed, the second global feature and the second local feature are obtained.

[0073] In step S6, the processed second local feature patch-tokens′ and the second global feature cls-token′ are respectively placed in a specific layer of the Transformer network for feature extraction, and the overall loss is calculated according to the loss function in the corresponding Transformer network.

[0074] Preferably, before step S1, the following steps are further included:

[0075] A1. Preset first plaque and preset second plaque;

[0076] A2. Input the original image;

[0077] The first plaque size is larger than the second plaque size.

[0078] In step A1, two patches of different sizes are set to extract features, resulting in two branches of different sizes. Specifically, the two branches are: a large branch (B-Branch), which divides the image with a larger first patch size and obtains coarse-grained features; and a small branch (S-Branch), which divides the image with a smaller second patch size and obtains fine-grained features.

[0079] In step A2, an original image obtained in various scenarios such as image retrieval, security environment, or criminal investigation is input.

[0080] Preferably, the parameters in step S3 are specifically: auxiliary information and position coding information.

[0081] In actual application, the parameters include auxiliary information and position encoding information. Auxiliary information (such as viewpoint information in the image or camera information such as camera ID) is used to avoid model inaccuracies caused by objective problems such as camera pixels or lighting angles. Position encoding information refers to the fact that Transformer contains a key technology - position encoding, which can improve the model's perception of position information and make up for the lack of position information in the Self Attention mechanism. By adding auxiliary information and position encoding information, the Transformer network can be assisted to better perform feature extraction and feature fusion.

[0082] Preferably, step S4 specifically includes the following steps:

[0083] B1. Obtain the first layer number corresponding to the third sequence based on the ablation experiment;

[0084] B2. Obtain the second layer number corresponding to the fourth sequence based on the ablation experiment;

[0085] B3. Place the third sequence into a Transformer network with the first number of layers, and place the fourth sequence into a Transformer network with the second number of layers, and perform feature extraction on each to obtain the first global feature and first local feature of the third sequence, and the first global feature and first local feature of the fourth sequence;

[0086] Among them, feature extraction is based on feature interaction between the encoder and decoder in the Transformer network.

[0087] In steps B1 and B2, the ablation experiment means that when the author proposes a new solution, this solution changes multiple conditions or parameters at the same time. Then in the ablation experiment, the experimenter will control one condition or parameter at a time to see the result, to see which condition or parameter has a greater impact on the result. In short, it is the control variable method. The number of layers corresponding to the third sequence and the fourth sequence is obtained by designing an ablation experiment. Generally, the number of Transformer layers that are maximally adaptable to the third sequence and the fourth sequence is 3-7 layers. The specific number of the first layer and the second layer can be selected according to actual needs. It only needs to meet the requirement that the first layer and the second layer are not equal. In this embodiment, the first layer is 4 and the second layer is 5;

[0088] In step B3, the third sequence is placed in the Transformer network of the fourth layer, and the second sequence is placed in the Transformer network of the fifth layer. Feature extraction is performed on the third sequence and the fourth sequence respectively, so as to obtain the first global feature cls-token and the first local feature patch-tokens of the third sequence and the first global feature cls-token and the first local feature patch-tokens of the fourth sequence; the specific operation method of feature extraction is to directly use the encoder-decoder (encoding-decoding) in the Transformer network for feature interaction; it should be noted that the Encoder-Decoder (encoding-decoding) is not a specific model, but a general framework. The Encoder and Decoder parts can be any text, speech, image, video data, and the model can adopt CNN, RNN, BiRNN, LSTM, GRU, etc. In this embodiment, it is a general framework in the Transformer network.

[0089] Preferably, step S5 specifically includes the following steps:

[0090] The first global feature of the third sequence and the first local feature of the fourth sequence, as well as the first local feature of the third sequence and the first global feature of the fourth sequence are respectively put into the cross attention network for feature fusion to obtain the second global feature and the second local feature.

[0091] In actual application, it should be noted that the function of feature fusion is to accept two sets of features as input and output two sets of updated features; the second global feature clls-token′ is the updated first global feature cls-token, and the second local feature patch-tokens′ is the updated first global feature patch-tokens;

[0092] The Cross Attention Network (CAN) primarily consists of an embedding operation and a Cross Attention Module. The embedding operation is primarily used for branch extraction, while the Cross Attention Module is used. The CAN is ultimately composed of a local classifier and a global classifier. The local classifier calculates the similarity between the support set features and the query set features using the cosine distance between the two, thereby obtaining a probability value for the query set features. The global classifier performs classification directly through a fully connected layer and a softmax algorithm.

[0093] Preferably, before step S5, the following steps are further included:

[0094] In the crisscross attention network, the full-function nodes are replaced by mapping and anti-mapping relationships.

[0095] In actual application, in order to reduce the impact of time complexity and space complexity brought by Transformer, the full-function node (FFN) in the cross-attention is replaced with mapping and inverse mapping, which greatly improves the rate of the cross-attention network.

[0096] Preferably, before step S6, the following steps are further included:

[0097] Performing a shuffling operation on the second local feature;

[0098] The shuffling operation specifically includes: disrupting the auxiliary information and position encoding information in the second local feature.

[0099] In practice, in this embodiment, the shuffling operation is specifically to shuffle the position information and auxiliary information of different sequence tokens in the second local feature patch-tokens′ when a token contains position information (position embedding) and patch embedding information (patch embedding). This enhances robustness, but the patch embedding remains unchanged and remains in the original position.

[0100] Preferably, the specific layer is specifically the last layer in the Transformer network.

[0101] In actual application, the second local feature patch-tokens′ and the second global feature cls-token′ are respectively placed in the last layer of the Transformer network.

[0102] Preferably, step S6 specifically includes the following steps:

[0103] C1. The second local feature and the second global feature after the shuffle operation are respectively put into the last layer of the Transformer network for feature extraction to obtain the third global feature and the third local feature;

[0104] C2. Obtain the first loss based on the third global feature and the preset loss function ID Loss;

[0105] C3. Obtain the second loss based on the third local feature and the preset loss function Triple Loss;

[0106] C4. Determine the overall loss based on the first loss and the second loss;

[0107] The overall loss is specifically the average of the first loss and the second loss.

[0108] In step C1, after the shuffling operation, the second local feature patch-tokens′ is passed to the last layer of the Transformer network for feature extraction to obtain the third local feature patch-tokens″; at the same time, the features of the second global feature cls-tokens′ are not shuffled, but directly passed to the last layer of the Transformer network for feature extraction to obtain the third global feature cls-tokens″;

[0109] In steps C2 and C3, ID Loss and Triple Loss are commonly used loss functions for person re-identification tasks. The third global feature, cls-tokens″, is input into the ID Loss loss function to calculate the first loss, and the third local feature, patch-tokens, is input into the Triple Loss loss function to calculate the second loss.

[0110] In step C4, the overall loss is calculated based on the first loss and the second loss, that is, the average value of the sum of the first loss and the second loss.

[0111] A person re-identification device based on a Transformer network, comprising:

[0112] A division module, configured to divide the original image into a first branch and a second branch;

[0113] A mapping module, configured to linearly map the first branch and the second branch to obtain a first sequence and a second sequence;

[0114] A parameter adding module is used to add parameters to the first sequence and the second sequence to convert them into the third sequence and the fourth sequence;

[0115] a feature extraction module, configured to extract first global features and first local features of the third sequence and the fourth sequence respectively;

[0116] A feature fusion module, configured to fuse the first global feature and the first local feature to obtain a second global feature and a second local feature;

[0117] A processing module, configured to process the second local feature;

[0118] The calculation module is used to calculate the overall loss based on the processed second local features and the second global features and the corresponding loss function.

[0119] During actual application, the division module divides the original image input into the first branch and the second branch, and then inputs the first branch and the second branch into the mapping module; the mapping module linearly maps the first branch and the second branch respectively to obtain the first sequence and the second sequence, and transmits the first sequence and the second sequence to the parameter adding module; the parameter adding module adds parameters to the first sequence and the second sequence respectively to form the third sequence and the fourth sequence, and transmits the third sequence and the fourth sequence to the feature extraction module; the feature extraction module extracts the first global feature and the first local feature from the third sequence and the fourth sequence respectively, and transmits the first global feature and the first local feature to the feature fusion module; the feature fusion module fuses the first global feature and the first local feature to obtain the second global feature and the second local feature, and transmits the second local feature to the processing module, and transmits the second global feature to the calculation module; the processing module processes the second local feature, and transmits the processed second local feature to the calculation module; the calculation module calculates the overall loss based on the processed second local feature and the second global feature and the corresponding loss function.

[0120] In the embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0121] In addition, all functional modules in the embodiments of the present invention may be integrated into one processor, or each module may be a separate device, or two or more modules may be integrated into one device; the functional modules in the embodiments of the present invention may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0122] Those skilled in the art will understand that all or part of the steps of the above-mentioned method embodiment can be completed by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above-mentioned method embodiment are executed; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0123] It should be understood that the use of "system," "device," "unit," and / or "module" in this application is merely a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0124] As used in this application and the claims, unless the context clearly indicates an exception, the terms "a," "an," "an," and / or "the" are not intended to refer to the singular and may include the plural, unless the context clearly indicates otherwise. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements. The phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, product, or apparatus that includes the elements.

[0125] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0126] If a flow chart is used in this application, the flow chart is used to illustrate the operations performed by the system according to the embodiments of the application. It should be understood that the previous or subsequent operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. At the same time, other operations can be added to these processes, or one or more operations can be removed from these processes.

[0127] The above is a detailed introduction to the pedestrian re-identification method and device based on the Transformer network provided by the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field. The general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest scope consistent with the principles and novel features disclosed in this article.

Claims

1. A person re-identification method based on Transformer network, characterized in that: The steps include: Divide the original image to obtain a first branch and a second branch; linearly mapping the first branch and the second branch to obtain a first sequence and a second sequence; Add parameters to the first sequence and the second sequence respectively to obtain a third sequence and a fourth sequence; Put the third sequence and the fourth sequence into the corresponding layers of the Transformer network for feature extraction to obtain the first global feature and the first local feature; feature fusing the first global feature and the first local feature to obtain a second global feature and a second local feature; The processed second local features and the second global features are respectively placed into specific layers of the Transformer network for feature extraction, and the overall loss is calculated according to the corresponding loss function; The third sequence is the first sequence after adding parameters, and the fourth sequence is the second sequence after adding parameters; Before dividing the original image to obtain the first branch and the second branch, the following steps are also included: Preset first plaque and preset second plaque; inputting the original image; wherein the first plaque size is larger than the second plaque size; The parameters specifically include: auxiliary information and position coding information; The step of placing the third sequence and the fourth sequence into the corresponding layers of the Transformer network to perform feature extraction to obtain the first global feature and the first local feature specifically includes the following steps: Obtain the first layer number corresponding to the third sequence according to the ablation experiment; Obtain the second layer number corresponding to the fourth sequence according to the ablation experiment; Putting the third sequence into the Transformer network with the first number of layers, and putting the fourth sequence into the Transformer network with the second number of layers, respectively performing feature extraction to obtain the first global feature and the first local feature of the third sequence and the first global feature and the first local feature of the fourth sequence; Among them, feature extraction is based on the feature interaction between the encoder and decoder in the Transformer network; The feature fusing the first global feature and the first local feature to obtain a second global feature and a second local feature specifically includes the following steps: The first global feature of the third sequence and the first local feature of the fourth sequence, as well as the first local feature of the third sequence and the first global feature of the fourth sequence are respectively put into the cross attention network for feature fusion to obtain the second global feature and the second local feature.

2. The person re-identification method based on the Transformer network according to claim 1, characterized in that: Before the feature fusing the first global feature and the first local feature to obtain a second global feature and a second local feature, the following steps are further included: In the cross-attention network, the fully functional nodes are replaced by mapping and inverse mapping relationships.

3. The person re-identification method based on the Transformer network according to claim 2, characterized in that: Before respectively placing the processed second local features and the second global features into a specific layer of the Transformer network for feature extraction and respectively calculating the overall loss according to the corresponding loss function, the following steps are also included: performing a shuffling operation on the second local features; The shuffling operation specifically includes: disrupting the auxiliary information and the position encoding information in the second local feature.

4. The person re-identification method based on the Transformer network according to claim 3, characterized in that: The specific layer is specifically the last layer in the Transformer network.

5. The person re-identification method based on Transformer network according to claim 4, characterized in that: The second local feature and the second global feature are respectively placed into a specific layer of the Transformer network for feature extraction, and the overall loss is obtained by calculating the corresponding loss function, which specifically includes the following steps: The second local feature and the second global feature after the shuffle operation are respectively put into the last layer of the Transformer network for feature extraction to obtain a third global feature and a third local feature; Obtain a first loss according to the third global feature and a preset loss function ID Loss; Obtaining a second loss according to the third local feature and a preset loss function Triple Loss; Obtaining an overall loss based on the first loss and the second loss; The overall loss is specifically an average value of the first loss and the second loss.

6. A person re-identification device based on a Transformer network, characterized in that: include: A division module, configured to divide the original image into a first branch and a second branch; A mapping module, configured to linearly map the first branch and the second branch to obtain a first sequence and a second sequence; a parameter adding module, configured to add parameters to the first sequence and the second sequence to convert them into a third sequence and a fourth sequence; a feature extraction module, configured to extract a first global feature and a first local feature of the third sequence and the fourth sequence respectively; A feature fusion module, configured to fuse the first global feature and the first local feature to obtain a second global feature and a second local feature; a processing module, configured to process the second local feature; A calculation module, configured to calculate an overall loss based on the processed second local feature and the second global feature and a corresponding loss function; An initial module, used for presetting a first patch and a second patch; inputting the original image; wherein the first plaque size is larger than the second plaque size; The parameters added by the parameter adding module are specifically: auxiliary information and position coding information; A feature extraction module, specifically used to obtain the first layer number corresponding to the third sequence according to an ablation experiment; Obtain the second layer number corresponding to the fourth sequence according to the ablation experiment; Putting the third sequence into the Transformer network with the first number of layers, and putting the fourth sequence into the Transformer network with the second number of layers, respectively performing feature extraction to obtain the first global feature and the first local feature of the third sequence and the first global feature and the first local feature of the fourth sequence; Among them, feature extraction is based on the feature interaction between the encoder and decoder in the Transformer network; The feature fusion module is specifically used to place the first global feature of the third sequence and the first local feature of the fourth sequence, and the first local feature of the third sequence and the first global feature of the fourth sequence into a cross attention network for feature fusion to obtain a second global feature and a second local feature.

Images