Pedestrian re-identification method and device, electronic equipment and storage medium

Abstract

The invention provides a pedestrian re-identification method and device, electronic equipment and a storage medium, and relates to the technical field of pedestrian re-identification. The method comprises the steps of obtaining re-identification model input data corresponding to a to-be-identified original pedestrian image; the input data of the re-identification model at least comprises an original pedestrian image; inputting the re-recognition model input data into a pedestrian re-recognition model trained to be converged, and generating a corresponding first re-recognition feature; the pedestrian re-identification model at least comprises a first visual Transform model or a second visual Transform model; the first visual Transform model comprises an overlapped slice semantic feature extraction module, a foreground feature enhancement module in a global branch and a visual angle perception module in a local branch; the second visual Transform model comprises a slice enhancement module, an auxiliary information embedding module and a multi-scale fusion module; and performing re-identification matching processing according to the first re-identification feature to generate a corresponding re-identification result. According to the method, the pedestrian re-identification precision can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pedestrian re-identification, and particularly relates to a pedestrian re-identification method and device, an electronic device and a storage medium. BACKGROUND

[0002] Pedestrian re-identification is a task of searching and matching pedestrians across non-overlapping monitoring cameras in different monitoring scenes. Pedestrian re-identification mainly recognizes pedestrians by comparing the appearance, posture, body shape and gait of pedestrians. Compared with a face recognition algorithm that needs to obtain a clear face image to identify pedestrians, a pedestrian re-identification algorithm can identify pedestrians using a low-resolution image, thereby avoiding the need for high-definition accuracy of a monitoring camera.

[0003] In related technologies, image features are extracted layer by layer through a convolutional neural network, and then a classifier and a regressor are combined to realize target positioning and pedestrian re-identification. However, the convolutional neural network can only focus on a few local regions, and meanwhile, a down-sampling operation will cause loss of detailed information. Therefore, in the field of pedestrian re-identification, a Transformer network is gradually adopted to make up for the loss of detailed information and global feature recognition by using its multi-head attention mechanism. However, the Transformer network currently applied to pedestrian re-identification has the problem of low recognition accuracy.

[0004] Therefore, the recognition accuracy of the Transformer network currently applied to pedestrian re-identification in related technologies is low, which leads to the need for further optimization of the accuracy of pedestrian re-identification. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a pedestrian re-identification method, device, electronic device and storage medium to improve the accuracy of pedestrian re-identification.

[0006] In a first aspect, an embodiment of the present application provides a pedestrian re-identification method, comprising:

[0007] obtaining re-identification model input data corresponding to an original pedestrian image to be identified; the re-identification model input data at least includes the original pedestrian image;

[0008] inputting the re-identification model input data into a trained pedestrian re-identification model to generate corresponding first re-identification features; the pedestrian re-identification model at least includes a first visual Transformer model or a second visual Transformer model; the first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in a global branch and a perspective perception module in a local branch; the second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module and a multi-scale fusion module.

[0009] performing a re-identification matching process according to the first re-identification feature to generate a corresponding re-identification result.

[0010] In some embodiments of the first aspect, the pedestrian re-identification model is a first visual Transformer model;

[0011] obtaining re-identification model input data corresponding to the original pedestrian image to be identified, comprising:

[0012] performing semantic segmentation processing on the original pedestrian image by using a pre-trained U-Net network to generate a corresponding semantic segmentation image;

[0013] determining the original pedestrian image and the semantic segmentation image as the re-identification model input data.

[0014] In some embodiments of the first aspect, inputting the re-identification model input data into the trained pedestrian re-identification model to generate a corresponding first re-identification feature, comprising:

[0015] performing feature extraction on the semantic segmentation image and the original pedestrian image by using an overlapping slice semantic feature extraction module to generate first semantic overlapping slice features corresponding to the semantic segmentation image and first original slice features corresponding to the original pedestrian image;

[0016] performing fusion of the first semantic overlapping slice features and the first original slice features by using a foreground feature enhancement module to generate a corresponding first global feature;

[0017] performing view feature extraction on the first semantic overlapping slice features by using a view perception module to generate a corresponding first semantic view feature;

[0018] performing feature fusion of the first global feature and the first semantic view feature to generate a corresponding first re-identification feature.

[0019] In some embodiments of the first aspect, before inputting the re-identification model input data into the trained pedestrian re-identification model to generate a corresponding first re-identification feature, further comprising:

[0020] performing semantic segmentation processing on the labeled pedestrian image training set by using a U-Net network to generate a corresponding semantic segmentation training set;

[0021] training a preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set;

[0022] judging whether the preset first visual Transformer model is converged by using a triplet loss function;

[0023] If it is determined that the preset first visual Transformer model converges, the preset first visual Transformer model is determined as the pedestrian re-identification model trained to convergence.

[0024] In some embodiments of the first aspect, the preset first visual Transformer model is trained based on the semantic segmentation training set and the pedestrian image training set, including:

[0025] The semantic segmentation training image in the semantic segmentation training set and the pedestrian training image in the pedestrian image training set are input into the preset first visual Transformer model for iterative training; the pedestrian training image corresponds to the semantic segmentation training image;

[0026] For each semantic segmentation training image and corresponding pedestrian training image, the preset first visual Transformer model performs the following processing:

[0027] The overlapping slice semantic feature extraction module in the preset first visual Transformer model is used to extract features of the semantic segmentation training image and the pedestrian training image, to generate second semantic overlapping slice features corresponding to the semantic segmentation training image and second original slice features corresponding to the pedestrian training image;

[0028] The foreground feature enhancement module in the preset first visual Transformer model is used to fuse the second semantic overlapping slice features and the second original slice features to generate corresponding second global features;

[0029] The view angle perception module in the preset first visual Transformer model is used to extract view angle features from the second semantic overlapping slice features to generate corresponding second semantic view angle features;

[0030] The second global features and the second semantic view angle features are fused to generate corresponding second re-identification features.

[0031] In some embodiments of the first aspect, the pedestrian re-identification model is a second visual Transformer model;

[0032] Before the first re-identification features are generated by inputting the re-identification model input data into the pedestrian re-identification model trained to convergence, the method further includes:

[0033] The preset pedestrian image data set is preprocessed by random gray block replacement and random clipping to generate a corresponding image enhanced training set;

[0034] The preset second visual Transformer model is trained based on the image enhanced training set;

[0035] The preset second visual Transformer model is jointly optimized by using a triplet loss function and a cross-entropy loss function until the preset second visual Transformer model converges.

[0036] The preset second visual Transformer model is determined as the trained-to-converge pedestrian re-identification model.

[0037] In some embodiments of the first aspect, the preset second visual Transformer model is trained based on the image enhancement training set, including:

[0038] The enhanced training images in the image enhancement training set are input into the preset second visual Transformer model for iterative training.

[0039] For each enhanced training image, the preset second visual Transformer model performs the following processing:

[0040] The slice enhancement module in the preset second visual Transformer model is used to perform slice feature extraction and feature fusion of different scales on the training images in the image enhancement training set, to generate corresponding local semantic features.

[0041] The auxiliary information embedding module in the preset second visual Transformer model is used to load pre-trained camera information and direction information.

[0042] The local semantic features, the camera information, and the direction information are fused to generate corresponding viewpoint-invariant features.

[0043] The multi-scale fusion module is used to perform shift and slice shuffling operations and feature scale division operations on the viewpoint-invariant features, to generate third re-identification features of multiple scales.

[0044] Based on the same inventive concept, in a second aspect, the embodiments of the present application further provide a pedestrian re-identification device, including:

[0045] An acquisition module is configured to acquire re-identification model input data corresponding to an original pedestrian image to be identified, wherein the re-identification model input data at least includes the original pedestrian image.

[0046] The first generation module is configured to input the re-identification model input data into the trained pedestrian re-identification model to generate corresponding first re-identification features. The pedestrian re-identification model at least includes a first visual Transformer model or a second visual Transformer model. The first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in a global branch, and a view perception module in a local branch. The second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module, and a multi-scale fusion module.

[0047] The second generation module is configured to perform re-identification matching processing based on the first re-identification features to generate corresponding re-identification results.

[0048] In some embodiments of the second aspect, the pedestrian re-identification model is the first visual Transformer model. The acquisition module is specifically configured to:

[0049] The pre-trained U-Net network is used to perform semantic segmentation processing on the original pedestrian image to generate a corresponding semantic segmentation image. The original pedestrian image and the semantic segmentation image are determined as the re-identification model input data.

[0050] In some embodiments of the second aspect, the first generation module is specifically configured to:

[0051] The overlapping slice semantic feature extraction module is used to perform feature extraction on the semantic segmentation image and the original pedestrian image to generate first semantic overlapping slice features corresponding to the semantic segmentation image and first original slice features corresponding to the original pedestrian image. The foreground feature enhancement module is used to fuse the first semantic overlapping slice features and the first original slice features to generate corresponding first global features. The view perception module is used to perform view feature extraction on the first semantic overlapping slice features to generate corresponding first semantic view features. The first global features and the first semantic view features are fused to generate corresponding first re-identification features.

[0052] In some embodiments of the second aspect, the device further includes:

[0053] The first training module is configured to perform semantic segmentation processing on the labeled pedestrian image training set using the U-Net network to generate a corresponding semantic segmentation training set. The pre-set first visual Transformer model is trained based on the semantic segmentation training set and the pedestrian image training set. The pre-set first visual Transformer model is determined to be converged by using a triplet loss function. If it is determined that the pre-set first visual Transformer model is converged, the pre-set first visual Transformer model is determined as the trained pedestrian re-identification model.

[0054] In some embodiments of the second aspect, when training the preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set, the first training module is specifically configured to:

[0055] input the semantic segmentation training image in the semantic segmentation training set and the pedestrian training image in the pedestrian image training set into the preset first visual Transformer model for iterative training; the pedestrian training image corresponds to the semantic segmentation training image;

[0056] for each semantic segmentation training image and corresponding pedestrian training image, the preset first visual Transformer model performs the following processing:

[0057] extract features of the semantic segmentation training image and the pedestrian training image by using an overlapping slice semantic feature extraction module in the preset first visual Transformer model to generate second semantic overlapping slice features corresponding to the semantic segmentation training image and second original slice features corresponding to the pedestrian training image; fuse the second semantic overlapping slice features and the second original slice features by using a foreground feature enhancement module in the preset first visual Transformer model to generate corresponding second global features; extract perspective features of the second semantic overlapping slice features by using a perspective perception module in the preset first visual Transformer model to generate corresponding second semantic perspective features; and fuse the second global features and the second semantic perspective features to generate corresponding second re-identification features.

[0058] In some embodiments of the second aspect, the pedestrian re-identification model is a second visual Transformer model; and the device further comprises:

[0059] a second training module configured to: pre-process the preset pedestrian image dataset by using random gray block replacement and random clipping mixing to generate a corresponding image enhanced training set; train the preset second visual Transformer model based on the image enhanced training set; jointly optimize the preset second visual Transformer model by using a triplet loss function and a cross-entropy loss function until the preset second visual Transformer model converges; and determine the preset second visual Transformer model as the trained-to-converge pedestrian re-identification model.

[0060] In some embodiments of the second aspect, when training the preset second visual Transformer model based on the image enhanced training set, the second training module is specifically configured to:

[0061] input an enhanced training image in the image enhanced training set into the preset second visual Transformer model for iterative training;

[0062] For each enhanced training image, the preset second visual Transformer model is processed as follows:

[0063] The slice enhancement module in the preset second visual Transformer model is used to perform slice feature extraction and feature fusion of different scales on the training images in the image enhancement training set, to generate corresponding local semantic features; the auxiliary information embedding module in the preset second visual Transformer model is used to load the pre-trained camera information and direction information; the local semantic features, the camera information and the direction information are fused to generate corresponding viewpoint-invariant features; the multi-scale fusion module is used to perform shift and slice shuffle operations and feature scale division operations on the viewpoint-invariant features, to generate multi-scale third re-identification features.

[0064] Based on the same inventive concept, in a third aspect, the embodiments of the present application also provide an electronic device, which comprises:

[0065] a memory and a processor;

[0066] the memory stores computer execution instructions;

[0067] the processor executes the computer execution instructions stored in the memory, to implement the pedestrian re-identification method according to any one of the first aspect.

[0068] Based on the same inventive concept, in a fourth aspect, the embodiments of the present application also provide a computer readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the pedestrian re-identification method according to any one of the first aspect.

[0069] According to the pedestrian re-identification method, device, electronic device and storage medium provided by the embodiments of the present application, the re-identification model input data corresponding to the original pedestrian image to be identified is obtained, the re-identification model input data is input to the converged pedestrian re-identification model for training, to generate corresponding first re-identification features, and meanwhile, the re-identification matching processing is performed according to the first re-identification features, to generate corresponding re-identification results. Since the pedestrian re-identification model at least comprises a first visual Transformer model or a second visual Transformer model. The first visual Transformer model comprises an overlapping slice semantic feature extraction module, a foreground feature enhancement module in a global branch and a viewpoint perception module in a local branch, which can enhance the foreground feature expression, reduce the background interference, and thus improve the pedestrian re-identification accuracy. The second visual Transformer model comprises a slice enhancement module, an auxiliary information embedding module and a multi-scale fusion module, which can enhance the features and local details, and thus improve the pedestrian re-identification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 Fig. 1 shows a flow diagram of a pedestrian re-identification method according to an embodiment of the present application;

[0071] Figure 2 Fig. 2 shows another flow diagram of a pedestrian re-identification method according to an embodiment of the present application;

[0072] Figure 3 Fig. 3 shows a model training framework according to an embodiment of the present application;

[0073] Figure 4 Fig. 4 shows a single Transformer structure according to an embodiment of the present application;

[0074] Figure 5 Fig. 5 shows an overlapping slice according to an embodiment of the present application;

[0075] Figure 6 Fig. 6 shows a Transformer layer projection according to an embodiment of the present application;

[0076] Figure 7 Fig. 7 shows another model training framework according to an embodiment of the present application;

[0077] Figure 8 Fig. 8 shows a slice enhancement architecture according to an embodiment of the present application;

[0078] Figure 9 Fig. 9 shows an attention mechanism architecture according to an embodiment of the present application;

[0079] Figure 10 Fig. 10 shows a structure diagram of a pedestrian re-identification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0080] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below in combination with the drawings and embodiments.

[0081] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. The present application can be implemented without some of these specific details by those skilled in the art. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0082] It is to be understood that the terminology used herein such as first and second, and the like, is merely for distinguishing one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0083] Embodiment 1

[0084] The pedestrian re-identification method provided by the embodiments of the present application can be executed by a pedestrian re-identification device and an electronic device, and the like. The following will be described by taking the pedestrian re-identification method executed by an electronic device as an example.

[0085] As shown in Figure 1 The method provided by the embodiments of the present application can include steps S101 to S103.

[0086] S101, obtaining re-identification model input data corresponding to an original pedestrian image to be identified. The re-identification model input data at least includes the original pedestrian image.

[0087] Illustratively, the original pedestrian image refers to a pedestrian image to be identified. The re-identification model input data at least includes the original pedestrian image, and can also include semantic segmentation images, camera information, direction information and the like. The re-identification model input data is related to a pedestrian re-identification model.

[0088] S102, inputting the re-identification model input data into a trained pedestrian re-identification model to generate corresponding first re-identification features. The pedestrian re-identification model at least includes a first visual Transformer model or a second visual Transformer model. The first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in a global branch and a perspective perception module in a local branch. The second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module and a multi-scale fusion module.

[0089] Illustratively, the first visual Transformer model and the second visual Transformer model both further include respective Transformer layers of the visual Transformer. The Transformer is a deep learning network architecture based on a self-attention mechanism.

[0090] Exemplarily, the pedestrian re-identification model can be in multiple manners, for example, the pedestrian re-identification model is a first visual Transformer model, the pedestrian re-identification model is a second visual Transformer model, or the pedestrian re-identification model is a combination of the first visual Transformer model and the second visual Transformer model. If the pedestrian re-identification model is a combination of the first visual Transformer model and the second visual Transformer model, the re-identification model input data are input into the first visual Transformer model and the second visual Transformer model respectively, corresponding first features and second features are generated, and weighted fusion is performed to generate a final first re-identification feature. During training, joint training can also be performed.

[0091] S103, performing re-identification matching processing according to the first re-identification feature to generate a corresponding re-identification result.

[0092] Exemplarily, the manner of generating the re-identification result can adopt a general manner, for example, it can be matching with the features of the images in the database, and the matching manner can adopt Euclidean distance matching.

[0093] According to the pedestrian re-identification method provided in the embodiments of the present application, the re-identification model input data corresponding to the original pedestrian image to be identified are obtained, the re-identification model input data are input into the trained pedestrian re-identification model for training to generate corresponding first re-identification features, and at the same time, re-identification matching processing is performed according to the first re-identification features to generate corresponding re-identification results. Since the pedestrian re-identification model at least includes a first visual Transformer model or a second visual Transformer model. The first visual Transformer model can enhance the foreground feature expression and reduce the background interference, thereby improving the pedestrian re-identification accuracy. The second visual Transformer model can enhance the features and local details, thereby improving the pedestrian re-identification accuracy.

[0094] Embodiment 2

[0095] As shown in Figure 2 The pedestrian re-identification method provided in the embodiments of the present application is based on the pedestrian re-identification method provided in Embodiment 1 of the present application, and further description can include steps S201 to S204. The pedestrian re-identification model is a first visual Transformer model.

[0096] S201, performing semantic segmentation processing on the original pedestrian image by using a pre-trained U-Net network to generate a corresponding semantic segmentation image.

[0097] Exemplarily, the U-Net network is a classical symmetrical encoder-decoder structure convolutional neural network, which can be used for image segmentation.

[0098] S202, determining the original pedestrian image and the semantic segmentation image as the input data of the re-identification model.

[0099] S203, inputting the input data of the re-identification model into the pedestrian re-identification model trained to convergence to generate the corresponding first re-identification feature.

[0100] In some embodiments, S203 can be specifically as follows:

[0101] The semantic segmentation image and the original pedestrian image are subjected to feature extraction by using the overlapping slice semantic feature extraction module to generate the first semantic overlapping slice feature corresponding to the semantic segmentation image and the first original slice feature corresponding to the original pedestrian image.

[0102] The first semantic overlapping slice feature and the first original slice feature are fused by using the foreground feature enhancement module to generate the corresponding first global feature.

[0103] The first semantic overlapping slice feature is subjected to view angle feature extraction by using the view angle perception module to generate the corresponding first semantic view angle feature.

[0104] The first global feature and the first semantic view angle feature are subjected to feature fusion to generate the corresponding first re-identification feature.

[0105] Exemplarily, the overlapping slice semantic feature extraction module extracts features in an overlapping manner, which can better preserve the edge information of the semantic image. The foreground feature enhancement module extracts global features irrelevant to the background by fusing the first semantic overlapping slice feature and the first original slice feature to weaken the background interference. The view angle perception module extracts semantic view angle features suitable for the visual Transformer structure to assist in learning the view angle invariant features.

[0106] In some embodiments, S203 further includes a training process of the first visual Transformer model, which is specifically as follows:

[0107] The labeled pedestrian image training set is subjected to semantic segmentation processing by using the U-Net network to generate the corresponding semantic segmentation training set.

[0108] The preset first visual Transformer model is trained based on the semantic segmentation training set and the pedestrian image training set.

[0109] Whether the preset first visual Transformer model is converged is judged by using a triplet loss function.

[0110] If it is determined that the preset first visual Transformer model converges, the preset first visual Transformer model is determined as the pedestrian re-identification model trained to convergence.

[0111] For example, the pedestrian image training set can be manually annotated. The triplet loss function is a loss function for metric learning, which optimizes the distances of anchor samples, positive samples and negative samples in a high-dimensional embedding space, so that samples of the same class are close to each other and samples of different classes are far away from each other, thereby improving the optimization effect of model training.

[0112] In some embodiments, the process of training the preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set can be specifically as follows:

[0113] The semantic segmentation training image in the semantic segmentation training set and the pedestrian training image in the pedestrian image training set are input into the preset first visual Transformer model for iterative training. The pedestrian training image corresponds to the semantic segmentation training image.

[0114] For each semantic segmentation training image and corresponding pedestrian training image, the preset first visual Transformer model performs the following processing:

[0115] The overlapping slice semantic feature extraction module in the preset first visual Transformer model is used to extract features from the semantic segmentation training image and the pedestrian training image, to generate second semantic overlapping slice features corresponding to the semantic segmentation training image and second original slice features corresponding to the pedestrian training image.

[0116] The foreground feature enhancement module in the preset first visual Transformer model is used to fuse the second semantic overlapping slice features and the second original slice features to generate corresponding second global features.

[0117] The perspective feature extraction module in the preset first visual Transformer model is used to extract perspective features from the second semantic overlapping slice features to generate corresponding second semantic perspective features.

[0118] The second global features and the second semantic perspective features are fused to generate corresponding second re-identification features.

[0119] For example, the process of the preset first visual Transformer model during training is similar to the process of actual application, and details can be referred to the aforementioned application process, which will not be repeated here.

[0120] S204, re-identification matching processing is performed according to the first re-identification features to generate corresponding re-identification results.

[0121] The model of the embodiment combines semantic perspective features with visual Transformer organically. In the global branch, the method of fusing semantic slice features with original features is used to extract foreground features irrelevant to the background of the pedestrian image and enhance the expression of the foreground features. In the local branch, the perspective feature of the pedestrian is extracted by using the pixel statistics method to reduce the background interference and enhance the perspective feature learning.

[0122] To better understand the training process of the pedestrian re-identification method provided by the embodiments of the present application, further description is made as follows. The pedestrian re-identification method of the embodiments takes an electronic device such as a communication device as the execution subject, and the process is specifically as follows:

[0123] As shown in Figure 3 The first visual Transformer model mainly consists of three parts: an overlapping slice semantic feature extraction module, a foreground feature enhancement module in the global branch, and a perspective perception module in the local branch. In the figure, L ce denotes cross-entropy loss, L ri denotes re-identification loss, and the superscript numbers 1-4 denote local regions (front view, rear view, overhead view, and side view). The superscript g denotes global features. V1 to V4 denote local features, and the subscript numbers 1-4 also denote the four-view local regions.

[0124] First, video frame pictures are intercepted from a real monitoring camera in a certain area, and the interception range includes: sunny night (2:00-3:00 in the morning), sunny day, rainy day, snowy day, and three days (11:00-12:00 at noon). One picture is intercepted per minute from each monitoring video, a total of 7200 pictures are intercepted for manual labeling, and a pedestrian target image segmentation visual dataset with accurate labeling is constructed.

[0125] Then, according to the labeled pedestrian target segmentation set, a U-Net-based pedestrian image perspective segmentation network is trained to infer the original dataset (for example, a pedestrian image training set) and the re-identification dataset in the re-identification task, and respectively obtain a new perspective segmentation dataset (for example, a semantic segmentation training set) and a re-identification dataset.

[0126] When the first visual Transformer model is trained: the semantic segmentation image is sliced in an overlapping manner, which meets the input of the visual Transformer network while retaining more edge information. In the global branch, the semantic slice features are fused with the original slice features to enhance the expression of the foreground features. In the local branch, the slice pixel statistics method is used to extract perspective features (front view, rear view, side view, and overhead view) with significant differences to enhance the robustness of the model to perspective differences. The features obtained by training are optimized by using a triplet loss to accelerate the convergence of the loss.

[0127] A single Transformer structure such as Figure 4 As shown, Z i-1 is the layer input, Z i is the layer output. i-1 After layer normalization, multi-head attention module, layer normalization, and multi-layer perceptron, the output is Z i .

[0128] Overlapping Slice Semantic Feature Extraction: The semantically segmented image is sliced ​​and fed into the foreground feature enhancement module and the view perception module to extract semantic information. A U-Net network is used as the pedestrian image semantic segmentation framework. Labeled pedestrian images are divided into training and test sets. Each labeled image contains five labels: background, front view, rear view, top view, and side view. SEResNetXt50 is used as the backbone network, and training is supervised by cross-entropy loss. The loss function is as follows:

[0129]

[0130] Where N represents the number of samples, y gt represents the true label, y pr Represents the predicted label.

[0131] Secondly, the previously labeled dataset is inferred based on the trained U-Net network to obtain a new segmentation dataset (such as a semantic segmentation training set). The segmentation network is used to predict the semantic image of the pedestrian target, and five different segmentation results are obtained: background, front view, rear view, top view, and side view of the image. The pixel values ​​of each result are defined as (0, 1, 2, 3, 4) respectively.

[0132] Finally, the new segmentation dataset is matched with the original dataset (same as the visual Transformer image block method) to semantically segment the image Use overlapping slices to divide it into N slices to retain more edge information and enhance the regional perspective feature representation, which is expressed as:

[0133] And i∈[1,N], the calculation formula for sorting is as follows:

[0134]

[0135] Among them, H and W represent the height and width of the image respectively, S represents the stride of the sliding window, P represents the size of the sliding window, and P d Represents the size of the padding.

[0136] like Figure 5 As shown, Figure 5In (a), image block features are extracted in a non-overlapping slice manner. The white box region in the pedestrian target semantic slice image contains two non-overlapping image blocks, which are respectively denoted as class slice V c In the first class slice and the fourth class slice The first class slice does not participate in the extraction of the lateral feature V4, so that the first class slice loses the edge information of the red dashed line region. Figure 5 In (b), image block features are extracted in an overlapping slice manner. In the process of extracting the lateral feature V4, the overlapping slice can better preserve the edge information of the semantic image.

[0137] Foreground feature enhancement: in the visual Transformer, the image I C×H×W is divided into N image blocks, denoted as and is flattened and mapped to a D-dimensional feature space through linear projection to meet the input dimension of the network, denoted as:

[0138]

[0139] wherein, represents the obtained original slice feature, F(·) represents the flattening and linear projection operation, and a learnable position embedding ( Figure 3 position) and a learnable class token ( Figure 3 category) are inserted to add position information and learn global features.

[0140] By fusing the semantic overlapping slice feature and the original slice feature of the visual Transformer network, a background-independent global feature is extracted to weaken the background interference. The specific steps include:

[0141] For a semantic image, the semantic slice feature is obtained through the flattening and linear projection operation in the visual Transformer:

[0142]

[0143] wherein, represents the obtained semantic feature slice, and F(·) represents the flattening and linear projection operation.

[0144] As shown in Figure 6 , in the first layer of the visual Transformer, for the original image, the original slice feature is extracted​ And integrate semantic slice features and position embedding As input, it is represented as:

[0145]

[0146] Among them, f c ∈R D represents a learnable category token for classification, Represents a learnable position embedding to preserve the flattened features and spatial information. The slice features representing the original image contain a large amount of information related to the original image identity to extract unique pedestrian target identity representation. It represents the corresponding semantic image slice features, carries the semantic information of the target pedestrian, filters the background information and encodes the foreground features of the pedestrian target to enhance the foreground feature expression.

[0147] Finally, the Transformer layer is stacked to learn the global feature representation of the target. The kth visual Transformer layer The calculation formula is:

[0148]

[0149]

[0150] in, represents the input of the k-th layer visual Transformer,

[0151] R (N+1)×D represents the output of the k-th layer visual Transformer, NORM(·) represents regularization, MLP(·) represents multi-layer perceptron, and MSA(·) represents multi-head self-attention. The expression is:

[0152]

[0153] Among them, Q, K, V represent query, key and value respectively, and the dimensions are D q ,D k ,D v , is The multi-head self-attention divides Q, K, V into h times and performs h times of self-attention calculation, so that each head can obtain the feature representation of the fusion key information. Finally, the output of each head is linearly projected and concatenated to form the final output. It is forwarded through 13 Transformer layers, including the TL2 to TL 1 and TL g (g represents the global feature). Global feature vector V g (g represents the global feature) is composed as follows:

[0154]

[0155] Among them, TL g (·) represents the global visual Transformer layer, Represents the output of the 12th layer visual Transformer, represent The introduction of semantic slice features makes the global feature vector V g The interference of the image background is reduced. In order to further distinguish objects with similar appearance, perspective features are further introduced to increase the network's ability to express perspective-invariant features.

[0156] Perspective perception: In the local branch of the Transformer, semantic perspective features suitable for the visual Transformer structure are extracted to assist in learning perspective-invariant features.

[0157] Count the pixel values ​​of each semantic slice and perform classification based on the pixel values:

[0158] When semantic slicing When the pixel values ​​in are consistent, the pixel value is directly assigned as the category Calculate when pixel values ​​are inconsistent The frequency of the values ​​that appear in the statistics, the value with the highest frequency is assigned as the category

[0159] Assign a direction category to each semantic slice through pixel value statistics of the semantic slice. They represent the pedestrian target background, front view, rear view, top view, and side view directions respectively.

[0160] Then, according to the category of each semantic slice, the semantic perspective features are extracted. p Will Divided into 5 categories of token feature groups, namely X c ,c∈{0,1,2,3,4}X,ce{0,1,2,3,4}, can be expressed as:

[0161]

[0162] in, Represents the category token, C pRepresents the category of semantic slice, j∈[1,N] represents the index, represents the features under the c-th perspective, N c Table 2. The number of slices of the cth semantic category, N c The pixel statistics module calculates the number of pixels and D represents the dimension of the category token feature. To avoid the interference of background feature X0, X0 is discarded and only four view features X are retained. c , c∈{1,2,3,4}.

[0163] X c Feed into the local visual Transformer layer TL l Learn four view feature vectors V c , expressed as:

[0164]

[0165] Among them, TL l (·) represents the local visual Transformer layer, is the output of the local visual Transformer layer, is the category token feature. During testing, the four view feature vectors V c ,c∈{1,2,3,4} are concatenated to form the view feature representation V l .

[0166] V l =Concat(V1,V2,V3,V4)

[0167] Among them, Concat(·) represents the feature concatenation operation, V l The four view feature vectors V1, V2, V3, and V4 represent the four view direction features of the target. The introduction of semantic view features enables the view perception module to extract target features from different viewpoints, improving the object re-identification network's ability to express view-invariant features.

[0168] Example 3

[0169] The person re-identification method provided in this embodiment of the present application is further described based on the person re-identification method provided in Example 1 of the present application, and may include steps S301 to S303. The person re-identification model is a second visual Transformer model.

[0170] S301: Obtain re-identification model input data corresponding to an original pedestrian image to be identified. The re-identification model input data at least includes the original pedestrian image.

[0171] Exemplarily, the re-identification model input data can include an original pedestrian image, camera information, and direction information.

[0172] S302, input the re-identification model input data into the trained pedestrian re-identification model to generate corresponding first re-identification features.

[0173] Exemplarily, the second visual Transformer model generates the first re-identification features based on the original pedestrian image, the camera information, and the direction information. Alternatively, the second visual Transformer model can only be based on the original pedestrian image, and the pre-trained camera information and direction information are called by the auxiliary information embedding module.

[0174] In some embodiments, the pedestrian re-identification model is a second visual Transformer model. Before S302, a training process is further included, which is specifically as follows:

[0175] Random gray block replacement and random cutout mixing are adopted to pre-process a preset pedestrian image dataset to generate a corresponding image enhanced training set.

[0176] The preset second visual Transformer model is trained based on the image enhanced training set.

[0177] The preset second visual Transformer model is jointly optimized by using a triplet loss function and a cross-entropy loss function until the preset second visual Transformer model converges.

[0178] The preset second visual Transformer model is determined as the trained pedestrian re-identification model.

[0179] Exemplarily, random gray block replacement refers to randomly selecting a local area (block) in an image, converting it into a gray block, simulating color distortion or illumination change of a part of the area, thereby enhancing color robustness. Random cutout mixing refers to randomly cutting a local area from another image to mix into the current image with transparency, simulating occlusion or complex background, thereby improving the anti-occlusion ability.

[0180] The second visual Transformer model removes the view segmentation network, simplifies the training process, and improves the ability to extract fine-grained local subtle difference features of the pedestrian image. The image gray block replacement, random cutout mixing method, and auxiliary information embedding module are introduced, which increases the diversity of the data representation and improves the robustness of the model.

[0181] In some embodiments, the process of training the preset second visual Transformer model based on the image enhanced training set can be specifically as follows:

[0182] Input the enhanced training image in the image enhancement training set into the preset second visual Transformer model for iterative training.

[0183] For each enhanced training image, the preset second visual Transformer model performs the following processing:

[0184] The slice enhancement module in the preset second visual Transformer model is used to perform slice feature extraction and feature fusion of different scales on the training images in the image enhancement training set, to generate corresponding local semantic features.

[0185] The auxiliary information embedding module in the preset second visual Transformer model is used to load pre-trained camera information and direction information.

[0186] The local semantic features, the camera information, and the direction information are fused to generate corresponding viewpoint-invariant features.

[0187] The multi-scale fusion module is used to perform shift and slice shuffling operations and feature scale division operations on the viewpoint-invariant features, to generate multi-scale third re-identification features.

[0188] For example, the slice pyramid network and the slice attention enhancement method are used in the slice enhancement module, so as to perform slice feature extraction and feature fusion of different scales, to improve the spatial correlation between image slices in the visual Transformer.

[0189] The local semantic features, the camera information, and the direction information are fused to generate corresponding viewpoint-invariant features. Joint input of camera / viewpoint information and original features, using the global modeling capability of the Transformer, the finally output feature will tend to have viewpoint invariance, so this output feature is called viewpoint-invariant feature.

[0190] The multi-scale fusion module re-groups the multi-scale features, divides different scales, and can also calculate the loss of the multi-scale third re-identification features respectively, enhancing the local feature extraction capability of the model.

[0191] S303, re-identification matching processing is performed according to the first re-identification feature, to generate corresponding re-identification results.

[0192] In order to better understand the training process of the pedestrian re-identification method provided in the embodiments of the present application, the following will be further described in detail. The pedestrian re-identification method of the present embodiment takes an electronic device such as a communication device as the execution subject, and the process is as follows:

[0193] Slice augmentation and multi-scale feature based re-identification: Visual Transformer needs to perform slicing operation when processing images, which makes the different slices of the pedestrian target forcedly segmented (such as the torso, accessories, etc.), damages the complete information of the human target, and reduces the relevance between the slices. In addition, the pedestrian target pictures collected in the real scene are easily affected by the light intensity, resolution, etc., which makes the model insufficient in extracting fine-grained local subtle difference features. In the embodiment, the second visual Transformer model is an end-to-end slice augmentation and multi-scale feature based re-identification network model.

[0194] As shown in Figure 7 , the second visual Transformer model mainly consists of three parts, which are a slice augmentation module, an auxiliary information embedding module, and a multi-scale fusion module.

[0195] Random gray block replacement and random cutting mixed: During training, local random gray block replacement (gray replacement in Figure 7 ) and local random cutting mixed (cutting mixed in Figure 7 ) are used as data augmentation strategies in the data preprocessing stage. By using the random gray block replacement method, a rectangular region of the image is randomly selected, and the pixel points in the corresponding rectangular region of the gray image are replaced, and the calculation formula is as follows.

[0196]

[0197] rect=RandPositon(I C×H×W )

[0198]

[0199] Where I C×H×W represents the input image, t(·) represents the gray transform function, I gray represents the obtained gray image, RandPositon(·) represents a rectangular box randomly generated in I C×H×W , and LGT(·) represents assigning the pixel value of the corresponding gray image in the rectangular box to I C×H×W , represents the sample after local gray block transformation, and y represents the label of the transformed image.

[0200] To increase the diversity of data, the random cutting mixed method is used to combine two training samples and and their labels y1 and y2 to generate a new training sample, and the generated new sample and its label are represented as:

[0201]

[0202] y new = λy1 + (1 - λ)y2

[0203] where M ∈ {0, 1} represents a gray-scale binary region in which one image replaces another image, the operator represents an element-wise multiplication operation, λ represents the combination ratio between two images, and λ is randomly sampled according to a Beta distribution in (0, 1).

[0204] Secondly, in the slice enhancement module, four different sizes of convolution kernels are used to divide the image into four different sizes of slices from coarse to fine, to construct a slice pyramid network, to enhance the relevance between slices, and to embed a slice attention module to mine finer-grained image information.

[0205] The slice enhancement module uses a slice pyramid network constructed in a coarse-to-fine slice manner, and combines a slice attention mechanism to improve the mining ability of fine-grained features. As shown in FIG. 3, the slice enhancement module is composed of a slice pyramid network and a slice attention module, and the specific process is as follows: Figure 8

[0206] First, given an image I C×H×W , according to the structure of the Transformer network, N fixed-size slices are divided from the image I C×H×W . Different sizes of convolution kernels (2x2, 4x4, 8x8, 16x16) are used for convolution to obtain four different sizes of slices from coarse to fine. The height H and the width W of each size of slice are in the following proportion:

[0207] H3 = 2H2 = 8H1 = 16H0

[0208] W3 = 2W2 = 8W1 = 16W0

[0209] For each size of slice , a slice pyramid network is constructed, and feature fusion is performed from the largest slice at the top of the pyramid network to the smallest target slice 2D at the bottom of the pyramid network. The above two operations are repeatedly performed until the smallest target slice at the bottom of the pyramid is reached.

[0210] In each downsampling operation F 2D ​Previously, a slice attention enhancement module is added to improve the mining ability of slice for image fine-grained features. The slice attention enhancement module formula is as follows,

[0211]

[0212] Wherein, PA(·) represents the slice attention enhancement operation, F 2D represents the down-sampling operation with the convolution kernel size of 3x3, P3", P2", P2" represent the enhanced features with different sizes respectively, represents the final output slice feature.

[0213] As Figure 9 shown, the slice attention enhancement module mainly consists of two parts of channel attention module and spatial attention module. The input slice feature is respectively subjected to feature extraction operation in channel and spatial dimensions, and the channel and spatial attention features are fused to obtain the final fine-grained output feature P i ".

[0214] In the channel dimension, the channel attention feature is extracted from the input slice . Firstly, it is subjected to maximum pooling F Max and average pooling F Avg to generate two feature maps and to retain the channel dimension features. Then, by squeezing and replacing operation F sq is performed on and to obtain and Next, to realize the cross-channel mutual perception between features, one-dimensional convolution F 1D operation is performed on and by using a convolution kernel with a size of 3x3 to obtain and After that, the sum of and is obtained to obtain the fused feature After Sigmoid activation function F sig , the is obtained. Finally, the channel dimension feature weight is obtained by decompression operation F unsq .

[0215]

[0216] The spatial attention feature of slice

[0217] In the spatial dimension, the spatial attention feature is extracted from the input slice extract spatial attention features.

[0218] First, pass it through parallel max-pooling F max and average-pooling F Avg operations to generate two feature maps and to preserve spatial dimensional features. Then, concatenate and to obtain and pass it through a two-dimensional convolution operation F 2D with a 3x3 convolution kernel to change the number of channels in to 1 to obtain Finally, apply a Sigmoid activation function F sig to to obtain spatial dimensional feature weights

[0219] Thus, the calculation method of spatial feature weights is shown in the following formula:

[0220]

[0221] The spatial attention features of the slice are as follows.

[0222]

[0223] wherein, represents a feature concatenation operation in the channel dimension, represents an element-wise multiplication operation.

[0224] Finally, combine the channel attention features and the spatial attention features to output the slice attention features P i ″:

[0225]

[0226] wherein, ω1 and ω2 are weights that change with the gradient, and ω1+ω2=1.

[0227] An auxiliary information embedding module is introduced to introduce a learnable one-dimensional embedding to preserve camera and direction information. First, auxiliary information such as viewing angle (e.g., background, side view, front view, top view, rear view, etc.), camera, etc. is introduced through a learnable one-dimensional embedding to preserve the viewing angle, together with a learnable class token, a learnable position embedding, as input of the Transformer network. Let the number of camera roles be N C , initialize a learnable camera embedding S C ∈R NC×D, where D is the dimension size, given an image I C×H×W The camera identity is r, and its camera embedding information is S C [r]. Camera embedding for a certain image I C ×H×W All slices of are the same. C×H×W All slices of are encoded into a learnable perspective embedding S via the perspective label q V [q],

[0228] N V The number of perspective IDs.

[0229] Since the camera is embedded in S C [r] and perspective embedding S V [q] is different types of information, and the camera information and the viewing angle information are encoded together as

[0230] Transformer transforms an image I C×H×W Divide into N fixed-size slices Introducing an additional category token f in Transformer c ∈R D As a global feature representation, add a learnable position embedding Used to fuse spatial information and send it to the first Transformer layer to get an original input sequence embedding Indicates that Original slice features obtained by flattening and linear projection.

[0231] Finally, in order to introduce auxiliary information such as camera and viewpoint, the sequence with camera identity r and viewpoint identity q is inserted into the Transformer layer together with the original input sequence to learn the viewpoint invariant features, as shown in the formula:

[0232]

[0233] in, represents the original input sequence, and λ represents the hyperparameter for balancing the embedding weights of the auxiliary information.

[0234] like Figure 7 As shown in Figure 2, the Transformer layer includes global branches and local branches. In the multi-scale fusion module, the model robustness is improved by using shift and slice shuffling operations, and the shuffled features are divided into different scales, and the loss optimization is performed on the features at each scale (for example, Figure 7In order to improve the generalization ability of visual Transformer and reduce the overfitting problem, a multi-scale feature representation learning method is proposed to learn local fine-grained features and global context features. For the features obtained above, a hard mining triplet loss and a cross-entropy loss are combined to optimize the performance of the re-identification network. In the evaluation stage, the cosine distance is used to measure the similarity between the query set and the gallery set, and the corresponding evaluation results are given.

[0235] The input of the last layer of the Transformer is divided into k groups, and these groups are connected to the shared class token The input of the last layer of the Transformer is divided into k groups, and these groups are connected to the shared class token into a shared Transformer layer to learn k local features {f l j |j=1,2,…,k},f l j represent the output features of the jth group. To solve the problem that each group of local features only considers a part of the continuous slice embedding and cannot fully utilize the global dependence of the Transformer, a shift mixing operation is performed on the slice embedding:

[0236] The first step is to move the first m slices of the input of the last layer of the Transformer except the class token to the end, such as moving m steps to

[0237] The second step is to further shuffle the order of the moved slices through a slice mixing operation of k groups. To further solve the problem that the k groups of local slice features only consider a part of the slice embedding when calculating the loss and cannot fully utilize the global feature dependence, a multi-scale fusion module is set up to divide the shifted and mixed slice features into three levels of scales:

[0238] The local features of the first level of scale contain 4 groups of local slice features, and each feature group has N / 4 slices. In the second level of scale, two consecutive local slice feature groups of the first level of scale are fused to form two local slice feature groups each feature group has N / 2 slices. In the third level of scale, the local slice feature groups of the second level of scale are fused to obtain a global slice feature group each feature group has N slices. Each feature group in each level of scale calculates a loss, and the three levels of scales fully utilize the multi-level scale features of the slices and can fully utilize the global dependence.

[0239] Embodiment 4

[0240] As Figure 10As shown, the pedestrian re-identification device 400 provided by the embodiments of the present application can include:

[0241] The acquisition module 401 is configured to acquire re-identification model input data corresponding to the original pedestrian image to be identified. The re-identification model input data at least includes the original pedestrian image.

[0242] The first generation module 402 is configured to input the re-identification model input data into the trained pedestrian re-identification model to generate corresponding first re-identification features. The pedestrian re-identification model at least includes a first visual Transformer model or a second visual Transformer model. The first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in a global branch, and a perspective perception module in a local branch. The second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module, and a multi-scale fusion module.

[0243] The second generation module 403 is configured to perform re-identification matching processing according to the first re-identification features to generate corresponding re-identification results.

[0244] In some embodiments, the pedestrian re-identification model is the first visual Transformer model. The acquisition module 401 is specifically configured to:

[0245] The pre-trained U-Net network is used to perform semantic segmentation processing on the original pedestrian image to generate a corresponding semantic segmentation image. The original pedestrian image and the semantic segmentation image are determined as the re-identification model input data.

[0246] In some embodiments, the first generation module 402 is specifically configured to:

[0247] The overlapping slice semantic feature extraction module is used to extract features of the semantic segmentation image and the original pedestrian image to generate first semantic overlapping slice features corresponding to the semantic segmentation image and first original slice features corresponding to the original pedestrian image. The foreground feature enhancement module is used to fuse the first semantic overlapping slice features and the first original slice features to generate corresponding first global features. The perspective perception module is used to extract perspective features of the first semantic overlapping slice features to generate corresponding first semantic perspective features. The first global features and the first semantic perspective features are fused to generate corresponding first re-identification features.

[0248] In some embodiments, the pedestrian re-identification device 400 further includes:

[0249] The first training module is configured to perform semantic segmentation processing on the labeled pedestrian image training set by using a U-Net network to generate a corresponding semantic segmentation training set. The preset first visual Transformer model is trained based on the semantic segmentation training set and the pedestrian image training set. Whether the preset first visual Transformer model converges is determined by using a triplet loss function. If it is determined that the preset first visual Transformer model converges, the preset first visual Transformer model is determined as the pedestrian re-identification model trained to converge.

[0250] In some embodiments, when the first training module trains the preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set, the first training module is specifically configured to:

[0251] The semantic segmentation training image in the semantic segmentation training set and the pedestrian training image in the pedestrian image training set are input into the preset first visual Transformer model for iterative training. The pedestrian training image corresponds to the semantic segmentation training image. For each semantic segmentation training image and corresponding pedestrian training image, the preset first visual Transformer model performs the following processing: the overlapping slice semantic feature extraction module in the preset first visual Transformer model is used to perform feature extraction on the semantic segmentation training image and the pedestrian training image to generate second semantic overlapping slice features corresponding to the semantic segmentation training image and second original slice features corresponding to the pedestrian training image. The foreground feature enhancement module in the preset first visual Transformer model is used to fuse the second semantic overlapping slice features and the second original slice features to generate corresponding second global features. The perspective perception module in the preset first visual Transformer model is used to perform perspective feature extraction on the second semantic overlapping slice features to generate corresponding second semantic perspective features. The second global features and the second semantic perspective features are fused to generate corresponding second re-identification features.

[0252] In some embodiments, the pedestrian re-identification model is a second visual Transformer model. The pedestrian re-identification device 400 further includes:

[0253] The second training module is configured to perform preprocessing on the preset pedestrian image dataset by using random gray block replacement and random clipping mixing to generate a corresponding image enhancement training set. The preset second visual Transformer model is trained based on the image enhancement training set. The preset second visual Transformer model is jointly optimized by using a triplet loss function and a cross-entropy loss function until the preset second visual Transformer model converges. The preset second visual Transformer model is determined as the pedestrian re-identification model trained to converge.

[0254] In some embodiments, the second training module is specifically configured to:

[0255] inputting the augmented training images in the image augmented training set into the preset second visual Transformer model for iterative training. For each augmented training image, the preset second visual Transformer model performs the following processing: adopting a slice augmentation module in the preset second visual Transformer model to perform slice feature extraction and feature fusion of different scales on the training images in the image augmented training set, to generate corresponding local semantic features. Adopting an auxiliary information embedding module in the preset second visual Transformer model to load pre-trained camera information and direction information. Fusing the local semantic features, the camera information and the direction information to generate corresponding viewpoint-invariant features. Adopting a multi-scale fusion module to perform shift and slice shuffling operations and feature scale division operations on the viewpoint-invariant features, to generate multi-scale third re-identification features.

[0256] The pedestrian re-identification device provided by the embodiments of the present application has the beneficial effects and implementation manners of the pedestrian re-identification methods provided by embodiments 1 to 3 of the present application, and specific descriptions can be referred to the specific descriptions of the pedestrian re-identification methods in embodiments 1 to 3 above. The present embodiment will not be repeated here.

[0257] Embodiment 4

[0258] The embodiments of the present application further provide an electronic device, which comprises a memory and a processor. The memory stores computer execution instructions. The processor executes the computer execution instructions stored in the memory to implement the pedestrian re-identification methods in embodiments 1 to 3.

[0259] The electronic device provided by the embodiments of the present application has the beneficial effects and implementation manners of the pedestrian re-identification methods in embodiments 1 to 3 of the present application, and specific descriptions can be referred to the specific descriptions of the pedestrian re-identification methods in embodiments 1 to 3 above. The present embodiment will not be repeated here.

[0260] Embodiment 5:

[0261] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the pedestrian re-identification methods in embodiments 1 to 3 above are implemented.

[0262] The computer readable storage medium provided by the embodiments of the present application has the beneficial effects and implementation manners of the pedestrian re-identification methods of the embodiments 1 to 3 of the present application, and specific descriptions can be referred to the specific descriptions of the pedestrian re-identification methods of the embodiments 1 to 3 above. The present embodiment will not be described here again.

[0263] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the present application that come within the scope of the claims and that the terms limit such variations only to the specific embodiments described herein. It is intended that the application not be limited to the embodiments that are described but will include all embodiments that fall within the scope of the claims.

[0264] It can be understood that the above implementation manners are only exemplary implementation manners adopted for illustrating the principles of the present application, and the present application is not limited thereto. Various modifications and improvements can be made by those of ordinary skill in the art without departing from the spirit and principle of the present application, and these modifications and improvements are also considered as the protection scope of the present application.

Claims

1. A pedestrian re-identification method, characterized in that: include: Obtain the re-identification model input data corresponding to the original pedestrian image to be identified; The re-identification model input data at least includes an original pedestrian image; Inputting the re-identification model input data into a person re-identification model trained to convergence to generate corresponding first re-identification features; The pedestrian re-identification model includes at least a first visual Transformer model or a second visual Transformer model; the first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in the global branch, and a view perception module in the local branch; the second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module, and a multi-scale fusion module; Re-identification matching processing is performed according to the first re-identification feature to generate a corresponding re-identification result.

2. The method according to claim 1, characterized in that The pedestrian re-identification model is a first-line visual Transformer model; The step of obtaining re-identification model input data corresponding to the original pedestrian image to be identified includes: Performing semantic segmentation processing on the original pedestrian image using a pre-trained U-Net network to generate a corresponding semantic segmentation image; The original pedestrian image and the semantic segmentation image are determined as input data of the re-identification model.

3. The method according to claim 2, characterized in that Inputting the re-identification model input data into a person re-identification model trained to convergence to generate corresponding first re-identification features includes: Using the overlapping slice semantic feature extraction module to perform feature extraction on the semantic segmentation image and the original pedestrian image, generating a first semantic overlapping slice feature corresponding to the semantic segmentation image and a first original slice feature corresponding to the original pedestrian image; Using the foreground feature enhancement module to fuse the first semantic overlapping slice feature and the first original slice feature to generate a corresponding first global feature; Using the perspective perception module to extract perspective features from the first semantic overlapping slice features to generate corresponding first semantic perspective features; The first global feature and the first semantic perspective feature are fused to generate the corresponding first re-recognition feature.

4. The method according to claim 3, characterized in that Before inputting the re-identification model input data into a person re-identification model trained to convergence to generate corresponding first re-identification features, the method further includes: Using the U-Net network to perform semantic segmentation processing on the labeled pedestrian image training set to generate a corresponding semantic segmentation training set; Training a preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set; Using a triplet loss function to determine whether the preset first visual Transformer model converges; If it is determined that the preset first visual Transformer model converges, the preset first visual Transformer model is determined as a person re-identification model trained to convergence.

5. The method according to claim 4, characterized in that The training of the preset first visual Transformer model based on the semantic segmentation training set and the pedestrian image training set includes: Inputting the semantic segmentation training graph in the semantic segmentation training set and the pedestrian training image in the pedestrian image training set into a preset first visual Transformer model for iterative training; the pedestrian training image corresponds to the semantic segmentation training graph; For each semantic segmentation training image and the corresponding pedestrian training image, the preset first visual Transformer model performs the following processing: Using the overlapping slice semantic feature extraction module in the preset first visual Transformer model to perform feature extraction on the semantic segmentation training image and the pedestrian training image, to generate a second semantic overlapping slice feature corresponding to the semantic segmentation training image and a second original slice feature corresponding to the pedestrian training image; Using the foreground feature enhancement module in the preset first visual Transformer model to fuse the second semantic overlapping slice feature and the second original slice feature to generate a corresponding second global feature; Using the perspective perception module in the preset first visual Transformer model to extract perspective features from the second semantic overlapping slice features, and generate corresponding second semantic perspective features; The second global feature and the second semantic perspective feature are fused to generate a corresponding second recognition feature.

6. The method according to claim 1, characterized in that The pedestrian re-identification model is a second vision Transformer model; Before inputting the re-identification model input data into a person re-identification model trained to convergence to generate corresponding first re-identification features, the method further includes: The preset pedestrian image dataset is preprocessed using random grayscale block replacement and random shear mixing to generate the corresponding image enhancement training set; Training a preset second visual Transformer model based on the image enhancement training set; Jointly optimizing the preset second visual Transformer model using a triplet loss function and a cross entropy loss function until the preset second visual Transformer model converges; The preset second visual Transformer model is determined as a pedestrian re-identification model trained to convergence.

7. The method according to claim 6, characterized in that The training of the preset second visual Transformer model based on the image enhancement training set includes: Inputting the enhanced training image in the image enhancement training set into a preset second visual Transformer model for iterative training; For each enhanced training image, the preset second visual Transformer model performs the following processing: The slice enhancement module in the preset second visual Transformer model is used to extract and fuse slice features of different scales on the training images in the image enhancement training set to generate corresponding local semantic features; Use the auxiliary information embedding module in the preset second vision Transformer model to load pre-trained camera information and direction information; fusing the local semantic features, the camera information, and the direction information to generate corresponding view-invariant features; A multi-scale fusion module is used to perform shift and slice shuffling operations and feature scale division operations on the view-invariant features to generate multi-scale third-level recognition features.

8. A pedestrian re-identification device, characterized in that: include: An acquisition module is used to obtain the re-identification model input data corresponding to the original pedestrian image to be identified; The re-identification model input data at least includes an original pedestrian image; A first generating module is configured to input the re-identification model input data into a person re-identification model trained to convergence, to generate a corresponding first re-identification feature; The pedestrian re-identification model includes at least a first visual Transformer model or a second visual Transformer model; the first visual Transformer model includes an overlapping slice semantic feature extraction module, a foreground feature enhancement module in the global branch, and a view perception module in the local branch; the second visual Transformer model includes a slice enhancement module, an auxiliary information embedding module, and a multi-scale fusion module; The second generating module is used to perform re-identification matching processing according to the first re-identification feature to generate a corresponding re-identification result.

9. An electronic device, characterized in that: include: memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the pedestrian re-identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the pedestrian re-identification method according to any one of claims 1 to 7.

Images