Lightweight pedestrian re-identification method and system based on decoupling knowledge distillation

Abstract

The invention discloses a lightweight pedestrian re-identification method and system based on decoupling knowledge distillation, and the method comprises the steps: enabling processed to-be-re-identified image data to serve as the input of a lightweight pedestrian re-identification model, obtaining a feature layer through a convolution layer, enabling the feature layer to pass through an MV2 module, obtaining an expansion layer, and enabling the expansion layer to pass through an MV2 module; modeling a long-distance dependency relationship under a global visual field through a self-attention mechanism of a Transform module in the FB-MobileViT layer, repeatedly passing through the MV2 module and the FB-MobileViT module, and finally performing dimension raising through a convolutional layer to obtain a final image feature; the obtained image features, the individual IDs and the camera IDs are stored in a list, the relation between the image features, the individual IDs and the camera IDs is judged according to the distance, the first hit rate Rank-1 and the average accuracy are obtained through sorting according to the personnel IDs and the camera IDs of the query and candidate pedestrian library, and pedestrian re-identification is achieved; the lightweight model maintains high recognition precision while significantly reducing the calculation complexity, and is suitable for pedestrian re-recognition application in a resource-limited scene.

Description

Technical Field

[0001] The present invention belongs to the field of pedestrian re-identification, and specifically relates to a lightweight pedestrian re-identification method and system based on decoupled knowledge distillation. Background Art

[0002] Re-identification, or ReID, utilizes computer vision to determine whether specific human or machine targets exist in images or video sequences captured by multiple cameras. Traditional methods rely primarily on manually defined features, but the performance of these manually designed and extracted features is less than ideal. In recent years, the rapid development of deep learning technologies, such as convolutional neural networks, has enabled deep learning to extract deeper image features in the image domain. This has significantly advanced the state of the art in re-identification, a subcategory of image recognition, and has made re-identification increasingly effective in various real-world scenarios.

[0003] In the development of digital cities, leveraging massive amounts of surveillance information is crucial for achieving effective results. Identifying monitored objects, such as pedestrians, is a core area of ​​research in surveillance video. Re-identification technology plays a practical and effective role in locating individuals in traffic monitoring and identifying suspicious individuals in security, enabling it to contribute to the development of digital cities.

[0004] However, despite significant progress in re-ID, deep learning-based re-ID models still face challenges in practical implementation. Large parameters, high memory requirements, and limited throughput—the computational complexity that often prevents real-time performance—are unavoidable challenges for re-ID implementation. Therefore, practical re-ID applications require not only high accuracy and a high average precision, but also minimal memory and parameter requirements. To ensure speed, increased throughput is also necessary. Lightweighting is the solution to these problems.

[0005] Knowledge distillation is an effective method for lightweighting. It trains a large network model that performs relatively well but cannot be directly applied due to real-time reasons as a teacher model. The output of the teacher model is then used as a student's supervision signal to provide information to the student model. This information can provide more information than the simple classification of 0 and 1 of the true label. It can provide the teacher model with a series of probabilities for the entire classification. This information can help train the designed student re-identification model, so that the student model with significantly reduced parameter volume and computational complexity can maintain high performance, thereby effectively achieving model lightweighting, and then helping to deploy the re-identification model on resource-constrained inference devices, so that the re-identification model can be more widely used.

[0006] Therefore, the design and implementation of a lightweight re-ID model based on knowledge distillation can play a significant role in the fields of surveillance and security, expand the application scenarios of re-ID models, and facilitate the construction of digital cities. It is a highly research- and application-oriented work with broad prospects. According to the applicant's search and novelty search, the following related papers, belonging to the field of knowledge distillation and re-ID, were retrieved. Chinese Patent Publication No. CN112560631A provides a person re-ID method based on knowledge distillation. However, the teacher model is relatively primitive, leaving room for improvement in accuracy. The student model is also overly simple and not specifically designed for person re-ID. The distillation method used is complex and ineffective, and the distillation of features by fusion increases the model's distillation time. Chinese Patent Publication No. CN118351415A discloses a lightweight method for person re-ID in video scenes based on knowledge distillation. First, the MARS and iLIDS-VID video datasets are downloaded and processed. Then, the teacher model PiT is trained, including dataset preprocessing, teacher model feature extraction, loss calculation, and distilled feature extraction. Next, a student model with a small number of parameters, TinyViT, was constructed and a convolutional attention module (MCT) was inserted into the model. The MCT module, consisting of a convolutional attention (DCSA) module using depthwise separable convolutions and a RASA module based on dilated convolutions, extracts fused features from the video and enhances the network's accuracy in extracting image features. Finally, the student model undergoes distillation learning, which includes model initialization and iterative optimization, calculation of the distillation loss, joint optimization, and person re-identification matching. A knowledge distillation method was used based on the video dataset, but the distillation method was relatively primitive, and the outputs were not decoupled before distillation, which limited the distillation effect and the improvement in recognition accuracy. Furthermore, the accuracy of the selected teacher network itself still needs to be improved, and the feature fusion method also increases the model's runtime. Chinese patent publication number CN117351533A proposes a lightweight person re-identification method based on attention knowledge distillation. First, a color image is fed into the lightweight person re-identification feature extraction network. Then, a ResNet50 network and a dimensional interactive attention block are used to generate informative shallow and deep features from the input image. Next, bottlenecks are used to align the features of the first three layers with those of the fourth layer, resulting in aligned shallow and deep features. Attention transfer knowledge distillation is performed on the shallow features using the deep features. Decoupling knowledge distillation is then performed on the shallow features using the aligned deep features. Finally, if the specified number of training rounds is reached, training ends; otherwise, training continues. A ResNet50 network is used to fuse dimensional interactive attention blocks, but this fusion increases model parameters, weakening its lightweight advantage. Furthermore, while self-distillation is faster, the improvement in model accuracy is relatively limited, making it difficult to surpass the existing baseline model. Summary of the Invention

[0007] The purpose of the present invention is to provide a pedestrian re-identification method based on decoupled knowledge distillation; design a new FB-MobileViT module to replace the original MobileViT module; in terms of overall architecture optimization, PConv is used to replace the original traditional convolution to reduce computational complexity, SiLU activation function is used to replace ReLU activation function to improve the nonlinear expression ability of the model, and an inverted residual structure is added to enhance feature extraction and fusion; and by introducing the Clip-Reid teacher model and improving the original decoupled knowledge distillation method, knowledge distillation is used to transfer the teacher model knowledge to the student model to improve the accuracy of the student model. When the QPS is greatly increased, the accuracy drop is guaranteed to be within 2%, so as to facilitate deployment in resource-constrained inference devices.

[0008] To achieve the above objectives, in a first aspect, the present invention provides a lightweight person re-identification method based on decoupled knowledge distillation, comprising the following steps:

[0009] The processed image data to be re-identified is used as the input of the lightweight pedestrian re-identification model. It first passes through the convolution layer to obtain the feature layer, and then passes the feature layer through the MV2 module to obtain the inverted residual structure. The inverted residual structure first expands the feature dimension through convolution, then processes the features through separable convolution, and finally compresses it back to the original dimension through convolution. The residual connection and the original feature layer are then added to obtain the original and compressed information. The self-attention mechanism of the Transformer module in the FB-MobileViT layer is used to model long-distance dependencies from a global perspective. The model is then repeated through the MV2 module and the FB-MobileViT module, and finally the convolution layer is used to increase the dimension to obtain the final image features.

[0010] The obtained image features, individual IDs, and camera IDs are stored in a list respectively. Based on the distance judgment and the relationship with the image, the query and candidate pedestrian database’s person IDs and camera IDs are sorted to obtain the first hit rate and average precision to achieve pedestrian re-identification.

[0011] The lightweight person re-identification model is based on MobileViTv2 and replaces the original MobileViT module by designing the FB-MobileViT module. The FB-MobileViT module is used to extract local features, reduce feature dimensionality, and capture global dependencies. It uses a partial convolution structure to replace the original traditional convolution and replaces the ReLU activation function with the SiLU activation function.

[0012] The FB-MobileViT module consists of a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2.

[0013] Furthermore, the lightweight pedestrian re-identification model includes 1 conv3×3-SiLU module, 5 MV2 modules, 3 FB-MobileViT modules and 1 conv1×1-SiLU module.

[0014] Furthermore, the lightweight person re-identification model training includes the following steps:

[0015] S1, obtain and preprocess the pedestrian re-identification dataset, divide the dataset and perform data enhancement processing, where data enhancement includes operations such as horizontal flipping, vertical flipping and random cropping;

[0016] S2, the CLIP-ReID model is introduced as the teacher model and trained with each dataset. The teacher model is trained in two stages. In the first stage, the image encoder and text encoder are fixed and the required prompt words are trained. In the second stage, the prompt words are fixed and the image encoder is trained.

[0017] S3, based on the teacher model obtained in S2, uses a distillation method trained by decoupling knowledge distillation: when the output of the teacher model and the output of the student model are cross-entropy to obtain the loss, the loss is decomposed into two parts: target class knowledge distillation and non-target class knowledge distillation. By flexibly adjusting the weights of the two losses, the non-target class knowledge distillation loss suppressed by traditional distillation is enhanced, that is, the transfer of knowledge other than the target, which is important for performance, is enhanced to train the student model obtained in S2.

[0018] Furthermore, the Clip-Reid architecture introduced in S2 includes an image encoder, a text encoder, and a prompt word learner. Clip-Reid training includes:

[0019] Step 2.1: Keep the image encoder and text encoder unchanged. Input text and image are passed through the text and image encoders to obtain image features and text features respectively. Use text-to-image feature loss and image-to-text feature loss to train the prompt word.

[0020] In step 2.2, fix the text encoder and prompt word, and train the image encoder based on the input image and loss.

[0021] Furthermore, the formula for calculating the text-to-image feature loss is:

[0022]

[0023] Where: L t2i(i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, V a Represents the features of the a-th image;

[0024] The formula for calculating the image-to-text feature loss is:

[0025]

[0026] Where: L i2t (i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, T a Represents the features of the a-th character.

[0027] Furthermore, using ID loss L id and triplet loss L tri Optimizing image encoders:

[0028]

[0029] L tri =max(d p -d n +∝,0)

[0030] Among them, L id is the ID loss, p(y i |x i ) is for image x i Divided into y i The probability of pedestrians, L tri is the triplet loss, where the positive sample and the anchor sample belong to the same pedestrian, d p is the distance between the positive sample and the anchor sample, d n is the distance between negative samples and anchor samples;

[0031] For each image, we also use the text features obtained in the first training phase to calculate the image-to-text cross entropy L i2tce ;

[0032]

[0033] The losses used in the second stage are as follows:

[0034] L stage2 =L id +L tri +Lt2ice .

[0035] Furthermore, S3 specifically includes: in the feature extraction stage of the teacher model, the attention pooling module is connected after the backbone network. The attention pooling module enhances the features of key areas or channels by calculating the attention weights of the feature maps to generate the attention weighted feature maps att feat ,In the knowledge distillation process, the student model simultaneously learns the dual-branch prediction results of the teacher model, and the distillation loss function integrates the traditional prediction probability and the attention-enhanced prediction probability;

[0036] The improved decoupled knowledge distillation method is used for distillation. By adding distillation loss and supplementing the original loss to train the student model, a lightweight re-identification model is obtained.

[0037] In a second aspect, the present invention provides a lightweight pedestrian re-identification system based on decoupled knowledge distillation, including an image acquisition module, an image feature extraction module, and a re-identification module;

[0038] The image acquisition module is used to obtain the processed image data to be re-identified;

[0039] The image feature extraction module is used to take the image data to be re-identified as the input of the lightweight pedestrian re-identification model. It first passes the convolution layer to obtain the feature layer, and then passes the feature layer through the MV2 module to obtain the expansion layer. The expansion layer is used to increase the number of feature channels. Through the self-attention mechanism of the Transformer module in the FB-MobileViT layer, long-distance dependencies are modeled in a global perspective. The module is then repeatedly passed through the MV2 module and the FB-MobileViT module, and finally the convolution layer is used to increase the dimension to obtain the final features. The obtained features, individual ID and camera ID are stored in a list respectively. The relationship with the image is judged based on the distance. According to the person ID and camera ID of the query and candidate pedestrian library, the first hit rate Rank-1 and the average precision mAP are sorted to achieve pedestrian re-identification.

[0040] Among them, the lightweight pedestrian re-identification model is based on MobileViTv2 and replaces the original MobileViT module by designing the FB-MobileViT module. The FB-MobileViT module is used to realize local feature extraction, feature dimensionality reduction and fusion, and capture of global dependencies. It uses a partial convolution structure to replace the original traditional convolution and replaces the ReLU activation function with the SiLU activation function. The FB-MobileViT module includes a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2.

[0041] The re-identification module is used to store the obtained image features, individual ID and camera ID in a list respectively, judge the relationship with the image based on the distance, and sort the query and candidate pedestrian library according to the person ID and camera ID to obtain the first hit rate and average accuracy to achieve pedestrian re-identification.

[0042] At the same time, a computer device is provided, including a processor and a memory, the memory is used to store computer executable programs, the processor reads part or all of the computer executable programs from the memory and executes them, and when the processor executes part or all of the computer executable programs, the above-mentioned lightweight pedestrian re-identification method based on decoupled knowledge distillation can be implemented.

[0043] A computer-readable storage medium may also be provided, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned lightweight pedestrian re-identification method based on decoupled knowledge distillation can be implemented.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] A new lightweight model was designed, and Pconv was used to reduce the model's parameters and increase its speed; the SiLU activation function and inverted residual structure were used to ensure the model's accuracy; the FB-MobileVit module was proposed to ensure the model's accuracy while greatly increasing the throughput; the improvement of the knowledge distillation framework adopted the multimodal feature alignment capability of the Clip-Reid teacher model and constrained the feature distribution of the student model through cross-modal contrast loss; secondly, the traditional decoupling distillation method was improved, and attention transfer was implemented at the feature layer. Compared with Resnet34, the QPS was enhanced by 60% while the accuracy of the four data sets decreased by 0.5%.

[0046] We employ decoupled knowledge distillation and a Clip-Reid teacher model. This approach enhances the accuracy of the student model through advanced distillation methods and an excellent teacher model without changing the student model. Furthermore, by increasing the teacher model's prediction probability to two optimized decoupled knowledge distillations, these methods improve model accuracy by an average of 2% compared to traditional knowledge distillation methods and ResNet as the teacher model. Ultimately, our model achieves a 60% improvement in QPS, while maintaining an accuracy gap of less than 0.5% compared to a ResNet50 model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the algorithm flow chart of the present invention.

[0048] Figure 2 Schematic diagram of the lightweight pedestrian re-identification model described in the present invention.

[0049] Figure 3This is a schematic diagram of the identification results of the present invention. DETAILED DESCRIPTION

[0050] In order to enhance understanding of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and system implementation.

[0051] The present invention provides a person re-identification method based on decoupled knowledge distillation. The innovation lies in designing a new FB-MobileViT module to replace the original MobileViT module. In terms of overall architecture optimization, PConv is used to replace the original traditional convolution to reduce computational complexity, SiLU activation function is used to replace ReLU activation function to improve the nonlinear expression ability of the model, and an inverted residual structure is added to enhance feature extraction and fusion. In addition, by introducing the Clip-Reid teacher model and improving the original decoupled knowledge distillation method, knowledge distillation is used to transfer the teacher model knowledge to the student model to improve the accuracy of the student model. When the QPS is greatly increased, the accuracy drop is guaranteed to be within 2%, so as to facilitate deployment in resource-constrained inference devices.

[0052] As attached Figure 1 As shown, the present invention is a pedestrian re-identification method based on decoupled knowledge distillation, which includes three parts: designing a student model, training a Clip-Reid teacher model, and distilling the student model using a decoupled knowledge distillation method.

[0053] The technical solution of the present invention is described in detail below in conjunction with embodiments of specific applications.

[0054] Step 1: Process the datasets Market1501, MSMT17, CUHK03, and DukeMTMC-reID. Divide the datasets into training and test sets according to their structure, and perform data augmentation on the training set. The data augmentation methods include vertical flipping, horizontal flipping, and cropping.

[0055] Step 2, design a new lightweight re-identification model FB-MobileViT, in which based on MobileViTv2, the FB-MobileViT module is designed to replace the MobileViT module, and the 3×3 convolution in MobileViTv2 is replaced by partial convolution and 1×1 convolution, that is, the data first passes through the partial convolution layer, and then passes through the 1×1 convolution to obtain the feature layer, and the same effect as the original 3×3 convolution layer is obtained. The SILU activation function is used to replace the ReLU activation function in MobileViTv2, and the inverted residual structure is used more. In order to be lightweight, the number of FB-MobileViT modules and MV2 modules is reduced. The FB-MobileViT module includes a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2. The FB-MobileViT model includes 1 conv3×3-SiLU module, 5 MV2 modules, 3 FB-MobileViT modules and 1 conv1×1-SiLU module, as shown in Figure 2 As shown in the figure, the data first undergoes a 3×3 convolution layer to obtain a sufficient feature layer, and then the feature layer is passed through the MV2 module to obtain an inverted residual structure. The inverted residual structure first expands the feature dimension through a 1×1 convolution, such as from 64 dimensions to 256 dimensions, and then processes the features through a 3×3 depth-separable convolution, that is, processing features of the same dimension, and finally compresses it back to the original dimension through a 1× convolution, such as from 256 dimensions to 64 dimensions, obtaining the information of the original 256 dimensions, and then adds the residual connection and the original feature layer to obtain the original 64-dimensional and compressed 256-dimensional information; non-linear activation functions (such as SiLU) are introduced to Linear transformation is performed to enhance the semantic level of features. Then, through the self-attention mechanism of the Transformer module in the FB-MobileViT layer, long-distance dependencies are modeled from a global perspective. For example, the positional association between objects in the image is captured. By stacking multiple layers of Transformer encoders (such as 2 layers), the features are gradually abstracted into high-dimensional semantic vectors. For example, "object parts" are upgraded to "complete objects" or "scene categories", and then repeated through the MV2 module and the FB-MobileViT module. Finally, the dimensionality is increased through a 3×3 convolutional layer to obtain the final features.

[0056] Step 3: Introduce Clip-Reid as a teacher model and train it on each required dataset. The Clip-Reid architecture includes an image encoder, a text encoder, and a cue word learner. Clip-Reid training is divided into two stages:

[0057] Step 3.1: Fix the image encoder and text encoder unchanged, input text and image, and obtain the features of image and text features after the text encoder and image encoder respectively. Use text-to-image feature loss and image-to-text feature loss to train the prompt word. The text-to-image feature loss is calculated as:

[0058]

[0059] Where: L t2i (i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, V a Represents the features of the a-th image.

[0060] Among them, the calculation formula for image-to-text feature loss is:

[0061]

[0062] Where: L i2t (i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, T a Represents the features of the a-th character.

[0063] Step 3.2, in this stage, only the parameters in the image encoder are optimized. It is necessary to use the ID loss L id and triplet loss L tri , calculated as follows:

[0064]

[0065] L tri =max(d p -d n +∝,0)

[0066] Among them, L id is the ID loss, p(y i |x i ) is for image x i Divided into y i The probability of pedestrians, L tri is the triplet loss, where the positive sample and the anchor sample belong to the same pedestrian, but are different from the negative sample, d p is the distance between the positive sample and the anchor sample, d n is the distance between negative samples and anchor samples.

[0067] Through ID loss, the image encoder can accurately identify the corresponding pedestrian, and through L tri It is a triplet loss. The image increases the distance between each pedestrian. In order to make full use of CLIP, for each image, the text features obtained in the first training stage are also used to calculate the cross entropy L from image to text. i2tce .

[0068]

[0069] Finally, the loss used in the second stage looks like this:

[0070] L stage2 =L id +L tri +L t2ice

[0071] Step 4: Based on the teacher model obtained in S3, the distillation method of decoupled knowledge distillation training is used. That is, when the cross entropy loss is obtained by the output of the teacher model and the output of the student model, the loss is decomposed into target class knowledge distillation (TCKD) and non-target class knowledge distillation (NCKD). By adjusting the weights of the two losses, the student model obtained in S2 is trained on each required data set, and finally the prediction accuracy is obtained;

[0072] Step 4.1, the improved decoupled knowledge distillation method reconstructs the prediction mechanism of the teacher model by introducing the attention pooling module and embeds the feature enhancement module. In the feature extraction stage of the teacher model, the attention pooling module (such as spatial attention or channel attention mechanism) is connected after the backbone network. The attention pooling module calculates the attention weight of the feature map, enhances the features of the key areas or channels, suppresses irrelevant information, and thus generates an attention weighted feature map att feat In the knowledge distillation process, the student model simultaneously learns the dual-branch prediction results of the teacher model. The distillation loss function combines the traditional prediction probability and the attention-enhanced prediction probability. The specific formula is:

[0073] DKD=aTCKD(feat+att feat )+bNCKD(feat+att feat )

[0074] Among them, att feat is the predicted probability after the attention pooling module, feat is the predicted probability directly through the classifier without passing through the attention pooling module, TCKD is the target category knowledge distillation, and DCKD is the non-target category knowledge distillation.

[0075] Finally, the improved decoupled knowledge distillation method is used for distillation. By adding this distillation loss and supplementing it with the original loss to train the student model, we finally get a lightweight re-identification model with good performance.

[0076] Perform person re-identification:

[0077] First, the query data, such as part of the bounding_box_train data in the Market1501 dataset, and the data to be queried, such as part of the query data in the Market1501 dataset, are processed by the FB-MobileViT model to obtain corresponding features. The corresponding features, individual IDs, and camera IDs are stored in a list respectively. Then, the cosine distance between the query data features and the features of the data to be queried is calculated by cosine similarity. The relationship between the query data and the data to be queried is determined based on the cosine distance, that is, whether the person in the query image is consistent with the person in the query image. According to the person ID and camera ID of the query and candidate pedestrian library, they are sorted by similarity to obtain the first hit rate Rank-1 and the average precision mAP, thereby realizing pedestrian re-identification.

[0078] like Figure 3 The following figure shows the result after inputting a picture. There are a total of ten pictures. The upper one is the pedestrian that the model thinks does not match the pedestrian on the left, and the lower one is the pedestrian that the model thinks matches the pedestrian on the left. Red means that the pedestrian is actually the same as the pedestrian on the left, and blue means that the pedestrian is actually different from the pedestrian on the left.

[0079] In Example 2, the present invention further provides a lightweight pedestrian re-identification system based on decoupled knowledge distillation, comprising an image acquisition module, an image feature extraction module, and a re-identification module;

[0080] The image acquisition module is used to obtain the processed image data to be re-identified;

[0081] The image feature extraction module is used to use the image data to be re-identified as the input of the lightweight pedestrian re-identification model. It first passes through the convolution layer to obtain the feature layer, and then passes the feature layer through the MV2 module to obtain the inverted residual structure. The inverted residual structure first expands the feature dimension through convolution, then processes the features through separable convolution, and finally compresses it back to the original dimension through convolution. The residual connection and the original feature layer are added to obtain the original and compressed information. The self-attention mechanism of the Transformer module in the FB-MobileViT layer is used to model long-distance dependencies in a global perspective, and then repeatedly passes through the MV2 module and the FB-MobileViT module, and finally passes through the convolution layer to increase the dimension to obtain the final features; the obtained features, individual ID and camera ID are stored in a list respectively, and the relationship with the image is judged according to the distance, and the person ID and camera ID of the query and candidate pedestrian library are sorted to obtain the first hit rate Rank-1 and the average precision mAP to achieve pedestrian re-identification;

[0082] Among them, the lightweight pedestrian re-identification model is based on MobileViTv2 and replaces the original MobileViT module by designing the FB-MobileViT module. The FB-MobileViT module is used to realize local feature extraction, feature dimensionality reduction and fusion, and capture of global dependencies. It uses a partial convolution structure to replace the original traditional convolution and replaces the ReLU activation function with the SiLU activation function. The FB-MobileViT module includes a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2.

[0083] The re-identification module is used to store the obtained image features, individual ID and camera ID in a list respectively, judge the relationship with the image based on the distance, and sort the query and candidate pedestrian library according to the person ID and camera ID to obtain the first hit rate and average accuracy to achieve pedestrian re-identification.

[0084] On the other hand, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the lightweight pedestrian re-identification method based on decoupled knowledge distillation described in the present invention.

[0085] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the lightweight pedestrian re-identification method based on decoupled knowledge distillation described in the present invention.

[0086] The computer device may be a laptop computer, a desktop computer or a workstation.

[0087] The processor may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).

[0088] The memory of the present invention may be an internal storage unit of a laptop computer, desktop computer or workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0089] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSD) or optical disks, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).

[0090] The above content is only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A lightweight person re-identification method based on decoupled knowledge distillation, characterized by: The following steps are involved: The processed image data to be re-identified is used as the input of the lightweight pedestrian re-identification model. It first passes through the convolution layer to obtain the feature layer, and then passes the feature layer through the MV2 module to obtain the inverted residual structure. The inverted residual structure first expands the feature dimension through convolution, then processes the features through separable convolution, and finally compresses it back to the original dimension through convolution. The residual connection and the original feature layer are then added to obtain the original and compressed information. The self-attention mechanism of the Transformer module in the FB-MobileViT layer is used to model long-distance dependencies from a global perspective. The model is then repeated through the MV2 module and the FB-MobileViT module, and finally the convolution layer is used to increase the dimension to obtain the final image features. The obtained image features, individual IDs, and camera IDs are stored in a list respectively. Based on the distance judgment and the relationship with the image, the query and candidate pedestrian database’s person IDs and camera IDs are sorted to obtain the first hit rate and average precision to achieve pedestrian re-identification. The lightweight person re-identification model is based on MobileViTv2 and replaces the original MobileViT module by designing the FB-MobileViT module. The FB-MobileViT module is used to extract local features, reduce feature dimensionality, and capture global dependencies. It uses a partial convolution structure to replace the original traditional convolution and replaces the ReLU activation function with the SiLU activation function. The FB-MobileViT module consists of a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2.

2. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 1 is characterized in that: The lightweight pedestrian re-identification model consists of 1 conv3×3-SiLU module, 5 MV2 modules, 3 FB-MobileViT modules and 1 conv1×1-SiLU module.

3. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 1 is characterized in that: The training of the lightweight person re-identification model includes the following steps: S1, obtain and preprocess the pedestrian re-identification dataset, divide the dataset and perform data enhancement processing, where data enhancement includes operations such as horizontal flipping, vertical flipping and random cropping; S2, the CLIP-ReID model is introduced as the teacher model and trained with each dataset. The teacher model is trained in two stages. In the first stage, the image encoder and text encoder are fixed and the required prompt words are trained. In the second stage, the prompt words are fixed and the image encoder is trained. S3, based on the teacher model obtained in S2, uses a distillation method trained by decoupling knowledge distillation: when the output of the teacher model and the output of the student model are cross-entropy to obtain the loss, the loss is decomposed into two parts: target class knowledge distillation and non-target class knowledge distillation. By flexibly adjusting the weights of the two losses, the non-target class knowledge distillation loss suppressed by traditional distillation is enhanced, that is, the transfer of knowledge other than the target, which is important for performance, is enhanced to train the student model obtained in S2.

4. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 3 is characterized in that: The Clip-Reid architecture introduced in S2 consists of an image encoder, a text encoder, and a prompt word learner. Clip-Reid training involves: Step 2.1: Keep the image encoder and text encoder unchanged. Input text and image are passed through the text and image encoders to obtain image features and text features respectively. Use text-to-image feature loss and image-to-text feature loss to train the prompt word. In step 2.2, fix the text encoder and prompt word, and train the image encoder based on the input image and loss.

5. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 3 is characterized in that: in, The formula for calculating the text-to-image feature loss is: Where: L t2i (i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, V a Represents the features of the a-th image; The formula for calculating the image-to-text feature loss is: Where: L i2t (i) represents the i-th text-to-image loss, s represents the matrix point multiplication, V i Represents the features of the i-th image, T i represents the feature of the i-th character, B represents the number of a batch, a represents the a-th character in a batch, T a Represents the features of the a-th character.

6. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 3 is characterized in that: Use ID loss L id and triplet loss L tri Optimizing image encoders: L tri =max(d p -d n +∝,0) Among them, L id is the ID loss, p(y i |x i ) is for image x i Divided into y i The probability of pedestrians, L tri is the triplet loss, where the positive sample and the anchor sample belong to the same pedestrian, d p is the distance between the positive sample and the anchor sample, d n is the distance between negative samples and anchor samples; For each image, we also use the text features obtained in the first training phase to calculate the image-to-text cross entropy L i2tce ; The losses used in the second stage are as follows: L stage2 =L id +L tri +L t2ice 。 7. The lightweight person re-identification method based on decoupled knowledge distillation according to claim 3 is characterized in that: S3 includes: in the feature extraction stage of the teacher model, the attention pooling module is connected after the backbone network. The attention pooling module enhances the features of key areas or channels by calculating the attention weights of the feature map to generate the attention weighted feature map att feat ,In the knowledge distillation process, the student model simultaneously learns the dual-branch prediction results of the teacher model, and the distillation loss function integrates the traditional prediction probability and the attention-enhanced prediction probability; The improved decoupled knowledge distillation method is used for distillation. By adding distillation loss and supplementing the original loss to train the student model, a lightweight re-identification model is obtained.

8. A lightweight person re-identification system based on decoupled knowledge distillation, characterized by: Including image acquisition module, image feature extraction module, and re-identification module; The image acquisition module is used to obtain the processed image data to be re-identified; The image feature extraction module is used to use the image data to be re-identified as the input of the lightweight pedestrian re-identification model. It first passes through the convolution layer to obtain the feature layer, and then passes the feature layer through the MV2 module to obtain the inverted residual structure. The inverted residual structure first expands the feature dimension through convolution, then processes the features through separable convolution, and finally compresses it back to the original dimension through convolution. The residual connection and the original feature layer are added to obtain the original and compressed information. The self-attention mechanism of the Transformer module in the FB-MobileViT layer is used to model long-distance dependencies in a global perspective, and then repeatedly passes through the MV2 module and the FB-MobileViT module, and finally passes through the convolution layer to increase the dimension to obtain the final features; the obtained features, individual ID and camera ID are stored in a list respectively, and the relationship with the image is judged according to the distance, and the person ID and camera ID of the query and candidate pedestrian library are sorted to obtain the first hit rate Rank-1 and the average precision mAP to achieve pedestrian re-identification; Among them, the lightweight pedestrian re-identification model is based on MobileViTv2 and replaces the original MobileViT module by designing the FB-MobileViT module. The FB-MobileViT module is used to realize local feature extraction, feature dimensionality reduction and fusion, and capture of global dependencies. It uses a partial convolution structure to replace the original traditional convolution and replaces the ReLU activation function with the SiLU activation function. The FB-MobileViT module includes a PConv3×3-SiLU module, four Conv1×1-SiLU modules, and a Transformer module with a depth of 2. The re-identification module is used to store the obtained image features, individual ID and camera ID in a list respectively, judge the relationship with the image based on the distance, and sort the query and candidate pedestrian library according to the person ID and camera ID to obtain the first hit rate and average accuracy to achieve pedestrian re-identification.

9. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store computer executable programs, the processor reads part or all of the computer executable programs from the memory and executes them, and when the processor executes part or all of the computer executable programs, it can implement the lightweight pedestrian re-identification method based on decoupled knowledge distillation as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that A computer program is stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the lightweight pedestrian re-identification method based on decoupled knowledge distillation as described in any one of claims 1 to 7.

Images