A pedestrian re-identification method and system of double-branch key feature recombination and a medium

Abstract

The application discloses a kind of dual-branch key feature reorganization pedestrian re-identification method, system and medium, wherein method includes: obtaining image data;Image data is input into trained pedestrian re-identification model and is handled, and outputs pedestrian re-identification result;The pedestrian re-identification model includes ViT module and a local branch;Wherein, ViT module is used as main network, for extracting the global information of pedestrian;Local branch includes key feature reorganization module and encoder module, the key feature reorganization module is used to filter out the key information that the most contribution is to discriminative feature from global information, and the encoder module is used to extract feature according to key information, and obtain the local information of pedestrian with high discriminability.The application is combined in the manner of using global information and local information, to exclude the interference of background and shielding, effectively improve the performance of pedestrian re-identification network, and can be widely applied in image recognition technical field.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a pedestrian re-identification method, system and medium based on dual-branch key feature reconstruction. Background Technology

[0002] Pedestrian re-identification is a cross-camera retrieval task in multi-camera, multi-pedestrian scenarios. It is generally considered an important research topic in the field of computer vision and is of great significance in promoting social security and realizing smart cities.

[0003] Pedestrian re-identification is a challenging task. The shooting areas of multiple cameras do not overlap, and pedestrians have different perspectives and postures under different cameras, which makes even the same pedestrian appear very different under different cameras.

[0004] Currently, deep learning-based methods dominate the field of person re-identification. Training a deep learning model for person re-identification can be divided into the following steps: First, a set of pedestrian images in the training set are processed and input into a deep neural network. After a large amount of computation, a set of feature vectors representing pedestrians are obtained. Then, the vectors are substituted into the designed loss function to calculate the loss value. Then, the model parameters are updated through gradient backpropagation and an optimizer. Thus, it can be seen that designing a network model that effectively extracts pedestrian features is crucial. Person re-identification methods can be divided into the following categories: (1) Person re-identification methods based on global images. After extracting global features of pedestrians in the image through layers of convolutional neural networks, this approach achieved good results at the time. However, due to the inherent characteristics of convolutional neural networks, the resolution will gradually decrease irreversibly during the inevitable downsampling process, making it easy for the network to ignore local discriminative features. Furthermore, directly extracting features from the entire image is easily affected by occlusion, background, and human pose, thus affecting the recognition rate in these situations. (2) The pedestrian re-identification method based on linear block division divides the pedestrian image into linear blocks and then inputs them into the network, prompting the network to pay more attention to local areas. The network can utilize local features. Information matching and classification have achieved good results, and even effectively improved the recognition rate in environments with partial occlusion. However, under severe occlusion, even local information contains a lot of interference noise, and the block strategy is powerless to address the misalignment of people; (3) Pedestrian re-identification methods based on image segmentation models or pose estimation models. Both models can extract pedestrian information from background information and then hand it over to the subsequent pedestrian re-identification model. This two-stage strategy significantly improves the accuracy of recognition. However, due to the introduction of additional models, it inevitably leads to a large consumption of computing power and a significant decrease in the final recognition rate. The above depends on the cleanliness of the pedestrian information extracted in the first stage. Since the image segmentation model or pose estimation model will inevitably face cross-domain incompatibility when processing the pedestrian re-identification database, it will treat the discriminative fine-grained features on the pedestrian as interference and classify them as background information, resulting in the loss of key pedestrian information; (4) Pedestrian re-identification methods based on attention mechanism. There are many attention mechanisms based on convolution operation that can mine the most discriminative regions in pedestrian images. However, due to the limitation of the receptive field and the resolution loss caused by downsampling, the convolution kernel can usually only focus on a small region and cannot solve the problem of long-term dependence. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a pedestrian re-identification method, system and medium for dual-branch key feature reconstruction.

[0006] The technical solution adopted in this invention is:

[0007] A pedestrian re-identification method based on dual-branch key feature reconstruction includes the following steps:

[0008] Acquire image data;

[0009] The image data is input into the trained pedestrian re-identification model for processing, and the pedestrian re-identification result is output.

[0010] The pedestrian re-identification model includes a ViT module and a local branch;

[0011] The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians.

[0012] Furthermore, the ViT module performs the following processing on the input image:

[0013] For an input image I∈R H×W×C Where H represents height, W represents width, and C represents the number of channels, a convolutional kernel with length and width of P and stride of S is used to linearly map the input image I, resulting in N sequences of dimension D. D is determined by the number of convolution kernels, i∈1,2,...,N, where:

[0014]

[0015] Generate a sequence x P Learnable cls tags of equal length, and the cls tags are associated with the sequence x. P The spatial information is combined and integrated into the combination through learnable positional encoding, resulting in the input sequence z0 represented as follows:

[0016]

[0017] In the formula, x cls ∈R 1×D Indicates the cls marker, E pos ∈R (N+1)×D Represented as positional encoding;

[0018] After the linear mapping is completed, the input sequence z0 is fed into the encoder part, which contains multiple transformers. The encoder part includes a multi-head self-attention module and a multilayer perceptron module. The formula for calculating the output sequence is as follows:

[0019] z′ l=MSA(LN(z) l-1 ))+z l-1

[0020] z l =MLP(LN(z′) l ))+z′ l

[0021] In the formula, LN(·) represents the inter-layer normalization operation, MSA(·) represents the multi-head attention operation, and MLP(·) represents the multi-layer perception operation.

[0022] Furthermore, after the input sequence z0 passes through the encoder part of the transformer, the cls label is extracted separately and put into the classifier for classification to obtain the classification result. During the training of the pedestrian re-identification model, the overall loss is calculated based on the classification result using identity loss and triplet loss, and the parameters of the entire model are optimized through backpropagation.

[0023] Furthermore, the key feature reconstructing module obtains key information in the following ways:

[0024] Set a learnable parameter θ∈R for the inter-layer feature map weights of length L. L×1 The feature maps of each layer in the ViT module are summed to obtain the final global high-attention self-attention feature map:

[0025]

[0026] In the formula, attn_map i The feature map representing the cls-tagged portion.

[0027] Furthermore, the encoder module obtains local information in the following way:

[0028] The local information z is composed of the top ρ% of key information with the largest values ​​in the global high-attention self-attention feature map. k :

[0029]

[0030] In the formula, N represents the number of sequences with dimension D.

[0031] Furthermore, the pedestrian re-identification model is trained using a similarity comparison loss, the expression of which is:

[0032] Loss con =Sim(x) global ,x prm )

[0033] Where Sim(·) represents the Hadamah product, x global The x represents the cls flag, which is the last output in the global branch. prm This indicates the cls flag of the last output in a local branch.

[0034] Furthermore, the pedestrian re-identification model is trained in the following manner:

[0035] Obtain pedestrian image data, perform augmentation processing on the pedestrian image data, and then obtain the training set;

[0036] Model construction: The ViT module is used as the backbone network, and a local branch is added, which includes a key feature reorganization module and an encoder module;

[0037] The model is trained using a training set and a preset loss function to obtain a trained pedestrian re-identification model.

[0038] Another technical solution adopted in this invention is:

[0039] A pedestrian re-identification system based on dual-branch key feature reconstruction includes:

[0040] The data acquisition module is used to acquire image data;

[0041] The recognition and processing module is used to input image data into the trained pedestrian re-identification model for processing and output pedestrian re-identification results.

[0042] The pedestrian re-identification model includes a ViT module and a local branch;

[0043] The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians.

[0044] Another technical solution adopted in this invention is:

[0045] A pedestrian re-identification device based on dual-branch key feature reconstruction includes:

[0046] At least one processor;

[0047] At least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0049] Another technical solution adopted in this invention is:

[0050] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0051] The beneficial effects of this invention are: by using a combination of global and local information, this invention eliminates interference from background and occlusion, effectively improving the performance of the pedestrian re-identification network. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a structural diagram of the person re-identification model based on transformer in an embodiment of the present invention;

[0054] Figure 2 This is a flowchart of a pedestrian re-identification method based on the reconstruction of key features in a dual-branch manner, as described in an embodiment of the present invention. Detailed Implementation

[0055] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0057] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0058] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0059] Multi-head attention mechanisms based on transformers, because they do not involve downsampling, can capture long-term dependencies and thus preserve more effective information. Originally proposed in the field of natural language processing for processing one-dimensional sequence information, transformers, due to their excellent performance, were later applied to the image domain and termed Vision Transformer (ViT). This involves cropping an image into small, uniformly sized patches, and through convolutional mapping and flattening operations, transforming each patch into one-dimensional sequence information. A self-trainable cls (colours) tag is then generated and combined with the generated one-dimensional sequence to form a complete input sequence. Since images are two-dimensional signals with rich positional information, this information is lost when flattened to one dimension. Therefore, during training, a trainable parameter with a length equal to the total number of patches plus one is set as the image's positional encoding information and superimposed on the input sequence. The input sequence then passes through multiple transformer encoder modules. The cls tag learns discriminative features representing the input image during the self-attention process, and finally, pedestrian classification is performed based on the cls tag. It is evident that although the transformer has been modified for image tasks, it is not fully adapted to image tasks. Apart from the aforementioned position encoding information, there are no additional improvements for image tasks.

[0060] In summary, using only the global image cannot enable the model to focus on local details. Linear block segmentation methods struggle to avoid background interference and are affected by human body misalignment. While additional image segmentation or pose estimation models can extract effective human body parts, they introduce additional parameters and computational overhead. The final result heavily depends on the extraction capabilities of the image segmentation or pose estimation models and suffers from domain incompatibility. While ViT alone achieves good results, it lacks the ability to extract local information and improves image processing capabilities. To address these issues, considering the multi-head self-attention mechanism in ViT's encoder, which focuses on itself and generates an attention map, high-value regions represent key areas of high attention, while low-value regions represent low-attention background or occluded areas.

[0061] Therefore, this invention proposes (1) a key feature reorganization module, which fully considers the attention maps generated by the multi-head self-attention mechanism in all encoder parts, selects the key regions of the high attention part with the highest ρ% maximum value to form local features, and inputs them into the local branch. This solves the problem that traditional ViT only uses the global image and cannot pay attention to local regions. Moreover, the extracted local features are all pedestrian information with high discriminative power, and the background or occlusion information that causes interference is filtered out. It achieves the effect of image segmentation or pose estimation models, but avoids the increase in the number of parameters and computation caused by the introduction of additional models. The problem is that, considering that the global features contain all the local feature information and that the local features are all highly concerned, the global features will also focus on the extracted local feature positions when passing through the encoder part of the subsequent ViT, which is something we do not want to see. Therefore, this invention also proposes (2) a similarity comparison loss, which is used to widen the similarity between global features and local features, forcing local branches to use local features to find discriminative features in high-concern areas, while the main branch avoids focusing on the same areas as the local branches in the focus of global features, and instead forces the main branch to focus on low-concern areas but areas with discriminative information.

[0062] like Figure 2 As shown, this embodiment provides a pedestrian re-identification method based on dual-branch key feature reconstruction, including the following steps:

[0063] S1. Acquire image data;

[0064] S2. Input the image data into the trained pedestrian re-identification model for processing, and output the pedestrian re-identification result.

[0065] The pedestrian re-identification model includes a ViT module and a local branch;

[0066] The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians.

[0067] The pedestrian re-identification model is trained through the following steps:

[0068] A1. Obtain pedestrian image data, perform augmentation processing on the pedestrian image data, and obtain the training set;

[0069] A2. Model Construction: The ViT module is used as the backbone network, and a local branch is added. The local branch includes a key feature reorganization module and an encoder module.

[0070] A3. The model is trained using a training set and a preset loss function to obtain the trained pedestrian re-identification model.

[0071] In this embodiment, the dataset enhancement includes methods commonly used in pedestrian re-identification tasks, such as randomly cropping, randomly flipping, and randomly erasing the input image, which can effectively enhance the robustness of the network.

[0072] During model training, the aim is to fully utilize image information, extract discriminative information from images, map images to feature space, map images with the same label to nearby locations, and differentiate the positions of images with different labels in feature space. During model testing, the query set and retrieval set from the pedestrian re-identification dataset are mapped to feature space according to the trained model. Based on the images in the query set, the retrieval set finds the 10 closest images to it, and the retrieval results are returned in order of distance.

[0073] See Figure 1 , Figure 1This is a structural diagram of the transformer-based pedestrian re-identification model. This embodiment uses ViT (i.e., the ViT module) as the backbone network to extract global pedestrian information. To enhance the transformer's ability to capture local information, a local branch is included. A novel key feature reorganization module is used to filter out the key information that contributes most to discriminative features from the global information. Then, the encoder module is used for feature extraction to obtain highly discriminative local information about the pedestrian. Considering that the global branch already contains all the information from the local branch, and that the two branches will inevitably focus on the same area, especially the area selected by the feature reorganization module, wasting valuable computational resources, this embodiment also proposes a similarity contrast loss to suppress the two branches from focusing on the same direction, allowing them to focus on different points to find more discriminative features in areas of interest, thus strengthening the entire network's ability to represent pedestrian features.

[0074] The following provides a detailed explanation of each module of the pedestrian re-identification model.

[0075] For ViT, given an input image I∈R H×W×C Where H represents height, W represents width, and C represents the number of channels, an image I is linearly mapped using a convolutional kernel with length and width of P and stride of S, resulting in N sequences of dimension D. D is determined by the number of convolution kernels, i∈1,2,...,N, where:

[0076]

[0077] Then generate a sequence x. P Learnable cls tags of equal length, and the cls tags are compared with x. P By combining spatial information through learnable positional encoding, the input sequence of the transformer can be represented as:

[0078]

[0079] Where x cls ∈R 1×D Indicates the cls marker, E pos ∈R (N+1)×D This is represented as a positional code.

[0080] After the linear mapping is completed, z0 will be input into the encoder part of a transformer with L transformers, including a multi-head self-attention module and a multilayer perception module. The output is calculated by equations (3) and (4):

[0081] z′ l=MSA(LN(z) l-1 ))+z l-1 (3)

[0082] z l =MLP(LN(z′) l ))+z′ l (4)

[0083] Where LN(·) represents inter-layer normalization operation, MSA(·) represents multi-head attention operation, and MLP(·) represents multi-layer perception operation.

[0084] In the multi-head self-attention module, multi-head is simply a linear segmentation operation that helps the module focus on more discriminative regions without changing the underlying operation. Therefore, we will only introduce the self-attention part below. The self-attention module will process the input z... l-1 With three self-trained weight matrices W q W k W v ∈R D×D Performing matrix operations is a self-extraction of information, resulting in three pairs of input z. l-1 The representation of q,k,v∈R (N+1)×D Then, matrix multiplication is performed on the transposes of matrices q and k, which is equivalent to performing operations on different representations of the same information. This is the origin of self-attention. Based on this operation, the attention matrix attn of the entire input can be extracted. l =qk T ,attn l ∈R (N+1)×(N+1) As can be seen from the operation, the positions with high values ​​in the attention matrix are the regions of high interest from the multi-head self-attention module. Then, a matrix multiplication operation is performed between the attention matrix attn and v to calculate the pedestrian's feature values ​​based on the regions of high interest. Since the multilayer sensing operation is not modified or needed in this method, it will not be quantitatively analyzed. Simply put, the multilayer sensing operation is a linear mapping of the output of the multi-head self-attention module.

[0085] After passing through the encoder layers of transformers, the z0 representing pedestrian image information extracts the cls label separately and feeds it into a classifier for classification. In pedestrian re-identification tasks, the overall loss is typically calculated using identity loss and triplet loss based on the results, and then the entire model parameters are optimized through backpropagation. Using only ViT for pedestrian re-identification, and validating it on the Market1501 dataset, we obtained 88.2% map coverage and 95.2% Rank-1 coverage.

[0086] As mentioned above, combining global and local information in pedestrian re-identification tasks can extract more robust pedestrian features. To extract more crucial local information, this method designs a key feature reorganization module. Considering that the most important part of the transformer, the multi-head self-attention module, itself focuses on highly discriminative pedestrian features, each multi-head self-attention module can provide an attention map of the pedestrian patch. Since we only represent pedestrian features based on the cls label, we only need the feature map attn_map of the cls label part. l =attn l [:,0,1:], considering the need to fully utilize the encoder part of each transformer, a learnable parameter θ∈R for the inter-layer feature map weights of length L is set. L×1 Then, the feature maps from each layer are summed to obtain the final global high-attention self-attention feature map:

[0087]

[0088] The local information z is composed of the top ρ% of key information with the largest values ​​in the global high-attention self-attention feature map. k :

[0089]

[0090] Where ψ(·) represents the filtering function, used to select key local features. i∈1,2,...,N represents the i-th patch in the l-th layer. Then, a copy of the cls label is combined with the filtered local information and input into the encoder part of a brand new transformer for feature extraction.

[0091] The key feature reconstructing module proposed in this embodiment only uses the attention map information of the transformer itself, without using other attention map calculation modules, thus avoiding unnecessary computation. This module can extract pedestrian patch-level granular information while using only image-level labels, effectively selecting discriminative local information, removing background and occlusion interference, and reducing additional labeling work. After adding the key feature reconstructing module to the backbone network, the map performance improved by 0.5%, and the Rank-1 performance improved by 0.1%, which is a very good improvement on the Market1501 dataset, which is almost pushed to its limits.

[0092] This embodiment also considers that the global branch itself contains all the information in the local branch, and the local branch contains only high-interest regions. Therefore, it is difficult to avoid two branches focusing on the same discriminative region. So this method proposes a similarity comparison loss, the formula of which is as follows:

[0093] Loss con =Sim(x) global ,x prm (7)

[0094] Where Sim(·) represents the Hadamah product, x global The x represents the cls flag, which is the last output in the global branch. prm This represents the cls label of the last output in the local branch. Using this loss effectively focuses the two branches on different regions, learning more features that can characterize pedestrians, thus enhancing the robustness of the entire network. After training with the similarity contrast loss added to the network, the map performance improved by 0.3%, and the Rank-1 performance improved by 0.2%.

[0095] In summary, this embodiment uses ViT as the backbone network and proposes a key feature reorganization module. High-interest patches are selected as inputs to the branch networks to fully extract local fine-grained features. Considering that the backbone and branches contain the same information, they inevitably focus on the same areas. This embodiment proposes a similarity comparison loss to guide the two branches to explore other low-interest areas but with discriminative features, thus enhancing the network's ability to represent pedestrian features. This method has also undergone extensive validation experiments on the standard pedestrian re-identification dataset Market1501, achieving state-of-the-art performance with 89% map accuracy and 95.3% Rank-1, as shown in Table 1 below.

[0096] Table 1

[0097] methods mAP(%) Rank-1 (ms) ViT 88.2 95.0 ViT+prm 88.7 95.1 <![CDATA[ViT+prm+Loss con ]]> 89.0 95.3

[0098] This embodiment also provides a pedestrian re-identification system based on dual-branch key feature reconstruction, including:

[0099] The data acquisition module is used to acquire image data;

[0100] The recognition and processing module is used to input image data into the trained pedestrian re-identification model for processing and output pedestrian re-identification results.

[0101] The pedestrian re-identification model includes a ViT module and a local branch;

[0102] The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians.

[0103] This embodiment of the pedestrian re-identification system based on dual-branch key feature reconstruction can execute the pedestrian re-identification method based on dual-branch key feature reconstruction provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0104] This embodiment also provides a pedestrian re-identification device for dual-branch key feature reconstruction, including:

[0105] At least one processor;

[0106] At least one memory for storing at least one program;

[0107] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 2 The method shown.

[0108] This embodiment of the pedestrian re-identification device based on dual-branch key feature reconstruction can execute the pedestrian re-identification method based on dual-branch key feature reconstruction provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0109] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 2 The method shown.

[0110] This embodiment also provides a storage medium storing instructions or programs that can execute the pedestrian re-identification method based on dual-branch key feature reconstruction provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0111] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0112] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0113] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0115] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0116] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0117] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0118] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0119] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A pedestrian re-identification method based on dual-branch key feature reconstruction, characterized in that, Includes the following steps: Acquire image data; The image data is input into the trained pedestrian re-identification model for processing, and the pedestrian re-identification result is output. The pedestrian re-identification model includes a ViT module and a local branch; The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians. The key feature reconstruction module obtains key information through the following methods: Set a length of Inter-layer feature map weights can be learned parameters The feature maps of each layer in the ViT module are summed to obtain the final global high-attention self-attention feature map: In the formula, Feature map representing the cls-tagged portion; The encoder module obtains local information in the following ways: Select the top value with the largest value from the global high-attention self-attention feature map. Key information constitutes local information. : In the formula, This represents the number of sequences of dimension D; Indicates the cls tag; Indicates the filtering function; , Indicates the first Layer i One patch; The pedestrian re-identification model is trained using a similarity comparison loss, the expression of which is: in, Represented as Hadamaji, This indicates the cls flag, representing the last output in the global branch. This indicates the cls flag of the last output in a local branch.

2. The pedestrian re-identification method based on dual-branch key feature reconstruction according to claim 1, characterized in that, The ViT module performs the following processing on the input image: For an input image ,in Represents height, Represents width, Representing the number of channels, using a variable with a length and width of [value missing]. And the step size is The convolution kernel for the input image Perform a linear mapping to obtain A sequence of dimension D D is determined by the number of convolution kernels. ,in: Generate a sequence Learnable cls tags of equal length, and the cls tags are associated with the sequence. The input sequence is obtained by combining spatial information through learnable positional encoding. It is expressed as follows: In the formula, Indicates the cls tag, Represented as positional encoding; After the linear mapping is completed, the input sequence will be... The input is fed into the encoder, which contains multiple transformers. The encoder includes a multi-head self-attention module and a multilayer perceptron module. The formula for calculating the output sequence is as follows: In the formula, This indicates the inter-layer normalization operation. This indicates a bullish attention-based operation. This represents a multi-layer sensing operation.

3. The pedestrian re-identification method based on dual-branch key feature reconstruction according to claim 2, characterized in that, Input sequence After passing through the encoder part of the transformer, the cls tag is extracted separately and put into the classifier for classification to obtain the classification result. During the training of the pedestrian re-identification model, the overall loss is calculated based on the classification result using identity loss and triplet loss, and the parameters of the entire model are optimized through backpropagation.

4. The pedestrian re-identification method based on dual-branch key feature reconstruction according to claim 1, characterized in that, The pedestrian re-identification model was trained in the following manner: Obtain pedestrian image data, perform augmentation processing on the pedestrian image data, and then obtain the training set; Model construction: The ViT module is used as the backbone network, and a local branch is added, which includes a key feature reorganization module and an encoder module; The model is trained using a training set and a preset loss function to obtain a trained pedestrian re-identification model.

5. A pedestrian re-identification system based on dual-branch key feature reconstruction, used to implement the method described in any one of claims 1-4, characterized in that, include: The data acquisition module is used to acquire image data; The recognition and processing module is used to input image data into the trained pedestrian re-identification model for processing and output pedestrian re-identification results. The pedestrian re-identification model includes a ViT module and a local branch; The ViT module is used as the backbone network to extract global information of pedestrians. The local branches include a key feature reorganization module and an encoder module. The key feature reorganization module is used to filter out the key information that contributes the most to the discriminative features from the global information. The encoder module is used to extract features based on the key information to obtain highly discriminative local information of pedestrians.

6. A pedestrian re-identification system based on dual-branch key feature reconstruction, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-4.

7. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-4.

Images