Personnel re-identification model training method and personnel re-identification method

Abstract

The invention relates to a training method of a personnel re-identification model and a personnel re-identification method, and the training method comprises the steps: training a feature extractor, and freezing the parameters of the trained feature extractor; training a contour recognition module based on the trained feature extractor, and freezing parameters of the trained contour recognition module; based on the trained feature extractor and the contour recognition module, training the first feature extraction branch by using a data set in a person non-reloading scene, and freezing parameters of the trained first feature extraction branch; based on the trained feature extractor, the contour recognition module and the first feature extraction branch, training a second feature extraction branch by adopting a training set in a personnel reloading scene, and freezing parameters of the trained second feature extraction branch; wherein orthogonality loss between the second feature extraction branch and the first feature extraction branch is introduced into the training loss of the second feature extraction branch.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a training method for a person re-identification model and a person re-identification method. Background Art

[0002] With the rapid development of intelligent surveillance and video analysis technologies, the importance of person re-identification (ReID) in public safety, intelligent transportation, and retail analysis has become increasingly prominent. However, the performance of traditional ReID technology in complex scenarios still faces many challenges, especially when people change clothes, clothing features are completely invalid, and existing methods are difficult to extract stable and reliable features, resulting in a significant decrease in recognition effect. This problem is particularly prominent in multi-target, multi-scenario, and long-time span monitoring environments.

[0003] ReID tasks in clothing-changing scenarios present special difficulties: most existing methods are highly dependent on appearance features such as clothing color and texture, which change significantly after clothing is changed, resulting in the system being unable to effectively identify the same person. In addition, video environments are complex and diverse, with significant background interference factors, such as changes in ambient lighting, inconsistent camera perspectives, and changes in items carried by people. These factors together make it difficult for the system to extract comprehensive features that are not related to clothing, and it is impossible to completely remove the impact of clothing features on recognition results.

[0004] There is currently no effective solution to the problem of low recognition accuracy of current person re-identification methods in clothing-changing scenarios. Summary of the invention

[0005] The present invention provides a training method for a person re-identification model and a person re-identification method to solve the problem of low recognition accuracy of current person re-identification methods in a clothing change scenario.

[0006] In a first aspect, the present invention provides a method for training a person re-identification model, comprising:

[0007] Constructing a person re-identification model, the person re-identification model comprising a feature extractor, a contour recognition module, a first feature extraction branch and a second feature extraction branch, the input of the feature extractor being the input of the person re-identification module, the input of the contour recognition module being the output of the feature extractor, the input of the first feature extraction branch and the second feature extraction branch being the product of the input of the feature extractor and the input of the contour recognition module, and the inference result of the person re-identification model being generated according to the output of the first feature extraction branch and / or the second feature extraction branch;

[0008] Training the feature extractor and freezing the parameters of the trained feature extractor;

[0009] Based on the trained feature extractor, training the contour recognition module, and freezing the parameters of the trained contour recognition module;

[0010] Based on the trained feature extractor and the contour recognition module, a first feature extraction branch is trained using a data set in a scenario where the personnel do not change clothes, and parameters of the trained first feature extraction branch are frozen;

[0011] Based on the trained feature extractor, the contour recognition module and the first feature extraction branch, the second feature extraction branch is trained using a training set in a scene where a person changes clothes, and parameters of the trained second feature extraction branch are frozen;

[0012] The training loss of the second feature extraction branch introduces an orthogonality loss between the second feature extraction branch and the first feature extraction branch.

[0013] In a second aspect, the present invention provides a method for person re-identification, comprising:

[0014] Persons in video images are identified using a trained person re-identification model, wherein the person re-identification model is obtained using the training method for the person re-identification model described in the first aspect.

[0015] Compared with the related art, the present invention provides a training method for a person re-identification model, through which a person re-identification model that can be applied to a dressing-up scene can be obtained. When the person re-identification model is applied to a dressing-up scene, the global features with foreground enhancement of the input image are extracted through a feature extractor and a contour recognition module, and the non-clothing features of the person are further extracted from the aforementioned global features through a second feature extraction branch, and the inference result is finally obtained based on the non-clothing features of the person. At the same time, the person re-identification model can also be applied in non-dressing-up scenes, in which case the global features with foreground enhancement of the input image are extracted through a feature extractor and a contour recognition module, and the clothing features of the person are further extracted from the aforementioned global features through a first feature extraction branch, and the inference result is finally obtained based on the clothing features of the person.

[0016] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a training method for a person re-identification model provided in an embodiment of the present invention;

[0018] Figure 24 is an architecture diagram of a person re-identification model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] In an embodiment of the present invention, a method for training a person re-identification model is provided. Figure 1 is a flowchart of a training method for a person re-identification model provided in an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0022] Step S110: construct a person re-identification model, referring to Figure 2 The person re-identification model includes a feature extractor, a contour recognition module, a first feature extraction branch and a second feature extraction branch. The input of the feature extractor is the input of the person re-identification module, the input of the contour recognition module is the output of the feature extractor, the input of the first feature extraction branch and the second feature extraction branch are both the product of the input of the feature extractor and the contour recognition module, and the inference result of the person re-identification model is generated according to the output of the first feature extraction branch and / or the second feature extraction branch.

[0023] Among them, the feature extractor is the first part of the person re-identification model, which is used to extract the global features of the input image. The feature extractor can use the currently more mature feature extraction network. For example, the feature extractor can use the visual model (ViT) based on the Transformer architecture. The output of ViT is (W*H*C vit )’s high-dimensional feature map (global feature map) provides basic input for subsequent tasks.

[0024] As the second part of the person re-identification model, the contour recognition module is used to segment the foreground and background of the input image, thereby segmenting the person features in the image and removing the background features, which can minimize the interference of the image background area. The contour recognition module can use the currently mature semantic segmentation network. Exemplarily, the contour recognition module is the DeepLabv3+ model.

[0025] The input feature of the contour recognition module is the global feature map (W*H*C vit ), the output is a foreground probability mask map of the corresponding feature size (W*H*1). The foreground mask (W*H*1) is combined with the global feature map (W*H*C vit ) on each channel, by multiplying them element by element (i.e., dot product), we can retain only the image features of the foreground area and eliminate background interference.

[0026] The first feature extraction branch and the second feature extraction branch are the third part of the person re-identification model, which are used to extract deep features of the input image. The network structure of the first feature extraction branch and the second feature extraction branch is the same, both of which are dimensionality reduction networks. In the future, different methods can be used to make the two branches have different feature extraction capabilities. The two feature extraction branches can use the currently more mature dimensionality reduction network. For example, it is a dimensionality reduction network including a convolutional layer, a pooling layer, and a fully connected layer connected in sequence.

[0027] Exemplarily, the dimensionality reduction network structure used is as follows:

[0028] A. Initial dimension reduction convolution layer: Use 3*3 convolution with a stride of 1 and reduce the number of channels to C mid . That is, the input is W*H*C vit The feature map is output as W*H*C mid .

[0029] B. Spatial downsampling module: Use 3*3 convolution with a stride of 2 to downsample the spatial resolution and increase the channel dimension, that is, the input is W*H*C vit The feature map is output as W*H*C high .

[0030] C. ResNet residual module: extract higher-level semantic features.

[0031] Repeat the above A, B, and C processes to design appropriate feature values ​​such as stride, window, channel, etc. Finally, the output of the above single feature extraction branch is a one-dimensional feature of 1*1*C2.

[0032] When the two feature extraction branches are used together, the fused feature output is a one-dimensional feature of 1*1*2C2.

[0033] Finally, a fully connected layer is used to map the above one-dimensional features of 1*1*C2 or 1*1*2C2 to a final feature vector C of lower dimension. reid .

[0034] Step S120, training the feature extractor and freezing the parameters of the trained feature extractor.

[0035] This step is to train the feature extractor separately, and through training, the feature extractor is given priority to have a stronger ability to extract general image features.

[0036] In some embodiments, the feature extractor is a visual model (ViT) based on a Transformer architecture. Training the feature extractor includes: using an open source dataset and / or a local dataset as a training set to perform self-supervised training or unsupervised training on the feature extractor.

[0037] Since large models are gradually being widely used in the industry due to their excellent characteristics such as robustness and accuracy, in the re-identification task, in order to obtain initial image features with global context information and strong semantic representation capabilities, it is preferred to use the unsupervised training large model ViT (Vision Transformer) as a general feature extractor, using open source data and a trained model of an open source network framework.

[0038] For example, the ViT model can be trained using open source datasets (such as ImageNet, MS-COCO) or combined with local datasets through unsupervised or self-supervised methods (such as DINOv2, MAE) to obtain general image feature extraction capabilities. If a suitable open source model is already available, it can also be downloaded and used directly. After the training is completed, the parameters of the ViT model are frozen to ensure the stability and consistency of the general features in the subsequent training process.

[0039] Step S130: training the contour recognition module based on the trained feature extractor, and freezing the parameters of the trained contour recognition module.

[0040] This step is to train the contour recognition module separately. During the training process, the trained feature extractor (parameters are frozen) is needed to extract the global features in the image and give them to the contour recognition module.

[0041] Specifically, the contour recognition module is trained through open source datasets or combined with local datasets to extract the foreground area of ​​the person and remove background noise. The trained contour recognition module can generate a foreground mask and combine it with the global features provided by ViT to generate a feature representation of foreground enhancement.

[0042] Step S140, based on the trained feature extractor and contour recognition module, the first feature extraction branch is trained using a data set in a scene where the personnel have not changed clothes, and the parameters of the trained first feature extraction branch are frozen.

[0043] This step is to train the first feature extraction branch separately. During the training process, the trained feature extractor (parameter freezing) and contour recognition module (parameter freezing) need to provide the first feature extraction branch with the global features of the foreground enhancement of the input image. Among them, the data set in the scene where the person has not changed clothes means that the clothes of the same person are the same in any frame image, that is, the same person does not change clothes. In the scene without changing clothes, clothing features are the main feature basis for person re-identification (clothing features account for a high proportion of the total features of the person). By training the first feature extraction branch with the data set in the scene where the person has not changed clothes, the first feature extraction branch can have a stronger clothing feature extraction ability. Therefore, the first feature extraction branch can also be called the ReID branch without changing clothes.

[0044] Specifically, you can use open source ReID datasets (such as Market-1501, DukeMTMC-ReID) to train the ReID branch without changing clothes. Note that the selected dataset needs to ensure that the pedestrian's clothing remains unchanged and there is no change of clothing scene in the dataset. During the training process, the cross entropy loss (Softmax Loss) and triplet loss (Triplet Loss) commonly used in ReID training can be used for joint optimization to ensure that the first feature extraction branch has good recognition ability for clothing features. Finally, save the training model, including the complete network from the fully connected layer to the low-dimensional features, and save the model for separate use.

[0045] Step S150, based on the trained feature extractor, contour recognition module and first feature extraction branch, the second feature extraction branch is trained using a training set in a personnel dressing scene, and the parameters of the trained second feature extraction branch are frozen.

[0046] The training loss of the second feature extraction branch introduces the orthogonality loss between the second feature extraction branch and the first feature extraction branch.

[0047] This step is to train the second feature extraction branch separately. During the training process, the global features of the foreground enhancement of the input image need to be provided to the second feature extraction branch through the trained feature extractor (parameter freezing) and contour recognition module (parameter freezing). And the trained first feature extraction branch provides its own feature extraction results as the loss calculation basis of the second feature extraction branch. Among them, the training set in the scene of personnel changing clothes refers to the same person in at least one frame of the image. The clothing is different from that in other frames. In the scene of changing clothes, clothing features cannot be used as the feature basis for personnel re-identification (clothing features will change). Through the pre-order training, the first feature extraction branch has a strong clothing feature extraction ability. On this basis, the orthogonality loss between the second feature extraction branch and the first feature extraction branch is introduced in the training loss of the second feature extraction branch, so that the features of the two feature extraction branches are orthogonal, and then the clothing features and non-clothing features are decoupled, so that the second feature extraction branch has a strong non-clothing feature extraction ability. Therefore, the second feature extraction branch can also be called the ReID branch of changing clothes.

[0048] Specifically, the loss function of the second feature extraction branch is:

[0049] L total =a1L id +a2L triplet +a3L orth

[0050]

[0051] Among them, L total represents the total loss of the second feature extraction branch, L id represents the cross entropy loss of the second feature extraction branch, L triplet represents the triplet loss of the second feature extraction branch, L orth represents the orthogonality loss between the second feature extraction branch and the first feature extraction branch, a1, a2 and a3 are three loss weights respectively, F cloth represents the output of the first feature extraction branch, F id represents the output of the second feature extraction branch.

[0052] Through the above training, the second feature extraction branch can extract non-clothing features that can be used for person re-identification, which is convenient for person re-identification in the scene of changing clothes.

[0053] In summary, the present invention provides a training method for a person re-identification model, through which a person re-identification model that can be applied to a dressing scene can be obtained. When the person re-identification model is applied to a dressing scene, the global features with foreground enhancement of the input image are extracted through a feature extractor and a contour recognition module, and the non-clothing features of the person are further extracted from the aforementioned global features through a second feature extraction branch, and the inference result is finally obtained based on the non-clothing features of the person. At the same time, the person re-identification model can also be applied in non-dressing scenes, in which case the global features with foreground enhancement of the input image are extracted through a feature extractor and a contour recognition module, and the clothing features of the person are further extracted from the aforementioned global features through a first feature extraction branch, and the inference result is finally obtained based on the clothing features of the person.

[0054] Therefore, the present invention solves the problem of low recognition accuracy of current person re-identification methods in clothing-changing scenarios.

[0055] An embodiment of the present invention further provides a method for person re-identification, comprising:

[0056] The person in the video image is identified by using the trained person re-identification model, and the person re-identification model is obtained by the training method of the person re-identification model provided by the present invention.

[0057] The trained person re-identification model is used to identify people in video images, including:

[0058] When the video image is a video image of a scene in which a person has not changed clothes, an inference result of the person re-identification model is generated according to the output of the first feature extraction branch.

[0059] When the video image is a video image of a scene in which a person changes clothes, an inference result of the person re-identification model is generated according to the output of the second feature extraction branch.

[0060] When the video image is a video image of an unknown scene, an inference result of the person re-identification model is generated according to the fusion output of the first feature extraction branch and the second feature extraction branch.

[0061] As mentioned above, for three different scenarios, the present invention provides three different ways of using the person re-identification model. In the three cases, the final fully connected layer and the softmax activation function generate the final inference results according to different features (the output of the first feature extraction branch and / or the second feature extraction branch). Among them, the first feature extraction branch has a strong clothing feature extraction capability. In the scenario without changing clothes, the use of clothing features for person re-identification has the highest recognition accuracy. The second feature extraction branch has a strong non-clothing feature extraction capability. In the change of clothes scenario, the use of non-clothing features for person re-identification has the highest recognition accuracy. In an unknown scenario where it is impossible to determine whether a person has changed clothes, clothing features and non-clothing features can be used simultaneously for person re-identification, thereby obtaining a more stable recognition result.

[0062] A specific embodiment of the person re-identification method is described below.

[0063] 1. Model architecture design.

[0064] This embodiment proposes a dual-branch network architecture (person re-identification model architecture) for processing the scenes without changing clothes and changing clothes respectively in the person re-identification (ReID) task. Through the fine design of the general feature extractor, contour network and feature extraction branch, the recognition performance in the scene of changing clothes is effectively improved.

[0065] The following is the specific design plan:

[0066] 1. Use unsupervised trained ViT as a general feature extractor.

[0067] Large models are gradually being widely used in the industry for their excellent features such as robustness and accuracy. In ReID, in order to obtain initial image features with global context information and strong semantic representation capabilities, it is preferred to use the unsupervised large model ViT (Vision Transformer) as a general feature extraction module, using open source data and a trained model of an open source network framework.

[0068] 2. VIT is followed by a contour network.

[0069] In order to further enhance the foreground extraction capability of the human body area and avoid interference from the background area, an independent contour network branch is connected after VIT, and DeepLabv3+ can be used as the basic model of the contour network. The input feature is the global feature map (W*H*C vit ), the output is a foreground probability mask map of the corresponding feature size (W*H*1).

[0070] Combine the foreground mask (W*H*1) and the feature map (W*H*C vit) on each channel, by multiplying them element by element (i.e., dot product), we can retain only the image features of the foreground area and eliminate background interference.

[0071] 3.ReID feature output network.

[0072] Based on the foreground feature map generated by the contour network, two ReID feature extraction branches are designed to process the scenes without changing clothes and changing clothes respectively. The two branches maintain the same network structure. It is essentially a dimensionality reduction network, which can use multi-layer convolution + pooling, and finally add a fully connected layer to reduce the dimension of VIT features and finally output 1*1*C reid The following is an example of a network design:

[0073] A. Initial dimension reduction convolution layer: Use 3*3 convolution with a stride of 1 and reduce the number of channels to C mid . That is, the input is W*H*C vit The feature map is output as W*H*C mid .

[0074] B. Spatial downsampling module: Use 3*3 convolution with a stride of 2 to downsample the spatial resolution and increase the channel dimension, that is, the input is W*H*C vit The feature map is output as W*H*C high .

[0075] C. Further introduce the ResNet residual module to extract higher-level semantic features.

[0076] D. Repeat the above A, B, and C processes, design appropriate feature values ​​such as stride, window, channel, etc., and finally convert the above single-branch features into one-dimensional features of 1*1*C2.

[0077] When the combined training of the dressed-up and non-dressed branches is used, the two feature branches are concatenated to output a one-dimensional feature of 1*1*2C2.

[0078] 4. Fully connected layer design:

[0079] Through a fully connected layer, the feature vector is mapped to a lower-dimensional final feature vector C rei .

[0080] 2. Model training steps.

[0081] This embodiment also proposes the training steps of the above network architecture:

[0082] 1. V I T model training.

[0083] Use open source datasets (such as ImageNet, MS-COCO) or combine with local datasets to train the ViT model through unsupervised or self-supervised methods (such as DINOv2, MAE) to obtain general image feature extraction capabilities. If there is a suitable open source model, it can also be downloaded and used directly. After training, freeze the parameters of the ViT model to ensure the stability and consistency of general features in subsequent training. The output of ViT is W*H*C vit The high-dimensional feature map provides basic input for subsequent tasks.

[0084] 2. Contour network training.

[0085] The human contour extraction network is trained through open source datasets or combined with local datasets to extract pedestrian foreground areas and remove background noise. The trained contour network can generate foreground masks and combine them with the features provided by ViT to generate feature representations of foreground enhancement.

[0086] 3. Train the ReID branch without changing clothes separately.

[0087] Use open source ReID datasets (such as Market-1501 and DukeMTMC-ReID) to train the ReID branch without changing clothes. Note that the selected dataset needs to ensure that the pedestrian's clothing remains unchanged and there is no scene of changing clothes in the dataset. During the training process, the cross entropy loss (Softmax Loss) and triplet loss (Triplet Loss) commonly used in ReID training can be used for joint optimization to ensure that the branch has good recognition ability for clothing features. Finally, save the training model, including the complete network from the fully connected layer to the low-dimensional features, and save the model for separate use.

[0088] 4. Train the ReID branch for changing clothes separately.

[0089] A dataset of dressing scenes (such as LTCC and PRCC) is selected to train the dressing ReID branch. During the training process, an orthogonality constraint loss is introduced to make the features extracted by the dressing branch orthogonal to the features of the non-dressing branch, thereby achieving decoupling from clothing features. During the training process, the cross entropy loss and triplet loss are set with the identity ID. At the same time, the orthogonality loss with the non-dressing branch is added, and the training model is saved. The orthogonality loss forces the features extracted by the two branches to be independent, thereby improving the recognition robustness of the dressing branch.

[0090] 3. Model usage process.

[0091] In actual use, the system provides at least two query modes, allowing users to switch models according to their needs to complete different tasks. The first is a ReID query mode based on clothing consistency, which mainly uses the non-clothing branch model (the clothing branch is disabled) to quickly complete short-term matching and searching by analyzing the characteristics of a person's clothing (including color, texture, and local details). This mode is suitable for scenarios where the monitoring time interval is short and the person's clothing remains consistent.

[0092] The second is a comprehensive ReID query mode, which loads the fused comprehensive model, combines the identity features such as gait, body shape, and head shape extracted by the dressing branch, and performs a joint analysis with clothing features to handle situations where people's clothing changes. This mode is suitable for people re-identification tasks with a longer time span and can complete accurate matching when people may change their clothing.

[0093] Switching between the two modes is achieved by loading different branch models or complete models, ensuring that the system can flexibly adapt to needs in both short-term search and long-term historical retrieval scenarios.

[0094] The above embodiments have the following technical effects:

[0095] 1. Improve recognition robustness in dressing-up scenarios: Through the dual-branch network design, this embodiment achieves feature extraction optimization for non-dressing and dressing-up scenarios. The non-dressing branch performs fast matching in a short period of time based on clothing features, and the dressing branch effectively decouples clothing feature interference by extracting features unrelated to clothing such as gait, body shape, and head shape. The introduction of orthogonality loss further enhances the independence of dressing-up features and clothing features, significantly improving recognition performance in dressing-up scenarios.

[0096] 2. Flexible query mode and model switching: The above embodiment supports two query modes: clothing consistency mode in a short period of time and comprehensive mode suitable for a long span of time. By switching different models, users can quickly match target personnel in a short period of time according to their needs, or perform long-term historical retrieval that adapts to clothing changes. This design provides a flexible way of use to meet the monitoring needs of multiple scenarios.

[0097] 3. End-to-end efficient feature extraction and large model advantages: The above embodiments combine the global feature modeling capabilities of large models such as ViT and the foreground extraction capabilities of the contour network to build an end-to-end feature extraction and re-identification framework that can simultaneously learn spatial and semantic global features. The introduction of large models significantly improves the representation capabilities of features and ensures robustness in complex backgrounds and long time spans. The phased independent training and subsequent joint training strategies optimize computational efficiency, and the unified framework design facilitates future expansion, such as adapting new feature extraction techniques or data enhancement solutions.

[0098] In an embodiment of the present invention, a training device for a person re-identification model and a person re-identification device are also provided, which are used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "sub-unit", etc. used below may be a combination of software and / or hardware that implements predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0099] A training device for a person re-identification model, comprising:

[0100] A model building module is used to build a person re-identification model. The person re-identification model includes a feature extractor, a contour recognition module, a first feature extraction branch and a second feature extraction branch. The input of the feature extractor is the input of the person re-identification module, the input of the contour recognition module is the output of the feature extractor, the input of the first feature extraction branch and the second feature extraction branch are both the product of the input of the feature extractor and the contour recognition module, and the inference result of the person re-identification model is generated based on the output of the first feature extraction branch and / or the second feature extraction branch.

[0101] The first training module is used to train the feature extractor and freeze the parameters of the trained feature extractor.

[0102] The second training module is used to train the contour recognition module based on the trained feature extractor and freeze the parameters of the trained contour recognition module.

[0103] The third training module is used to train the first feature extraction branch based on the trained feature extractor and contour recognition module using a data set in a scenario where the personnel have not changed their clothes, and freeze the parameters of the trained first feature extraction branch.

[0104] The fourth training module is used to train the second feature extraction branch based on the trained feature extractor, contour recognition module and first feature extraction branch using a training set in a personnel dressing scenario, and freeze the parameters of the trained second feature extraction branch.

[0105] The training loss of the second feature extraction branch introduces the orthogonality loss between the second feature extraction branch and the first feature extraction branch.

[0106] A person re-identification device, comprising:

[0107] The personnel identification module is used to identify personnel in the video image through a trained personnel re-identification model, and the personnel re-identification model is obtained through the training method of the personnel re-identification model provided by the present invention.

[0108] The above two devices can be specifically described with reference to their respective corresponding method embodiments.

[0109] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0110] In an embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the training method of the person re-identification model or the person re-identification method provided by the present invention.

[0111] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0112] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

Claims

1. A training method for a person re-identification model, characterized in that: include: Constructing a person re-identification model, the person re-identification model comprising a feature extractor, a contour recognition module, a first feature extraction branch and a second feature extraction branch, the input of the feature extractor being the input of the person re-identification module, the input of the contour recognition module being the output of the feature extractor, the input of the first feature extraction branch and the second feature extraction branch being the product of the input of the feature extractor and the input of the contour recognition module, and the inference result of the person re-identification model being generated according to the output of the first feature extraction branch and / or the second feature extraction branch; Training the feature extractor and freezing the parameters of the trained feature extractor; Based on the trained feature extractor, training the contour recognition module, and freezing the parameters of the trained contour recognition module; Based on the trained feature extractor and the contour recognition module, a first feature extraction branch is trained using a data set in a scenario where the personnel do not change clothes, and parameters of the trained first feature extraction branch are frozen; Based on the trained feature extractor, the contour recognition module and the first feature extraction branch, the second feature extraction branch is trained using a training set in a scene where a person changes clothes, and parameters of the trained second feature extraction branch are frozen; The training loss of the second feature extraction branch introduces an orthogonality loss between the second feature extraction branch and the first feature extraction branch.

2. The training method of the person re-identification model according to claim 1, characterized in that: The feature extractor is a visual model based on the Transformer architecture; Training the feature extractor includes: An open source dataset and / or a local dataset is used as a training set to perform self-supervised training or unsupervised training on the feature extractor.

3. The training method of the person re-identification model according to claim 1, characterized in that: The contour recognition module is a DeepLabv3+ model.

4. The method for training a person re-identification model according to claim 1, characterized in that: The network structures of the first feature extraction branch and the second feature extraction branch are the same, both being dimension reduction networks; The dimension reduction network includes a convolutional layer, a pooling layer and a fully connected layer which are connected in sequence.

5. The method for training a person re-identification model according to claim 1, characterized in that: The loss function of the second feature extraction branch is: <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> total <h2 style=";text-align:left;direction:ltr"> =a1L<h2 style=";text-align:left;direction:ltr"> id <h2 style=";text-align:left;direction:ltr"> +a2L<h2 style=";text-align:left;direction:ltr"> triplet <h2 style=";text-align:left;direction:ltr"> +a3L<h2 style=";text-align:left;direction:ltr"> orth Among them, L total represents the total loss of the second feature extraction branch, L id represents the cross entropy loss of the second feature extraction branch, L triplet represents the triplet loss of the second feature extraction branch, L orth represents the orthogonality loss between the second feature extraction branch and the first feature extraction branch, a1, a2 and a3 are three loss weights respectively, F cloth represents the output of the first feature extraction branch, F id represents the output of the second feature extraction branch.

6. A person re-identification method, characterized in that: include: Persons in video images are identified by using a trained person re-identification model, wherein the person re-identification model is obtained by the training method of the person re-identification model described in any one of claims 1-5.

7. The person re-identification method according to claim 6, characterized in that: The trained person re-identification model is used to identify people in video images, including: When the video image is a video image of a scene in which a person does not change clothes, generating an inference result of the person re-identification model according to the output of the first feature extraction branch; When the video image is a video image of a scene in which a person changes clothes, generating an inference result of the person re-identification model according to the output of the second feature extraction branch; When the video image is a video image of an unknown scene, an inference result of the person re-identification model is generated according to the fusion output of the first feature extraction branch and the second feature extraction branch.

8. A training device for a person re-identification model, characterized in that: include: A model building module, used to build a person re-identification model, the person re-identification model includes a feature extractor, a contour recognition module, a first feature extraction branch and a second feature extraction branch, the input of the feature extractor is the input of the person re-identification module, the input of the contour recognition module is the output of the feature extractor, the input of the first feature extraction branch and the second feature extraction branch are both the product of the input of the feature extractor and the input of the contour recognition module, and the inference result of the person re-identification model is generated according to the output of the first feature extraction branch and / or the second feature extraction branch; A first training module, used for training the feature extractor and freezing the parameters of the trained feature extractor; A second training module, used for training the contour recognition module based on the trained feature extractor, and freezing the parameters of the trained contour recognition module; A third training module is used to train the first feature extraction branch based on the trained feature extractor and the contour recognition module using a data set in a scene where the personnel do not change clothes, and freeze the parameters of the trained first feature extraction branch; a fourth training module, configured to train the second feature extraction branch based on the trained feature extractor, the contour recognition module and the first feature extraction branch using a training set in a personnel changing outfit scenario, and freeze parameters of the trained second feature extraction branch; The training loss of the second feature extraction branch introduces an orthogonality loss between the second feature extraction branch and the first feature extraction branch.

9. A person re-identification device, characterized in that: include: A person identification module is used to identify persons in a video image by using a trained person re-identification model, wherein the person re-identification model is obtained by the training method of the person re-identification model described in any one of claims 1-5.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the training method of the person re-identification model described in any one of claims 1 to 5 or the person re-identification method described in any one of claims 6 to 7.

Images