A method for constructing a person re-identification dataset for general scenarios

Abstract

The present invention belongs to the technical field of pedestrian re-identification, and discloses a method for constructing a pedestrian re-identification dataset for general scenarios. The method for constructing a pedestrian re-identification dataset for general scenarios includes: S1, collecting video image data in different scenarios and different monitoring angles; S2, constructing a pedestrian target detection network, taking the video image data as input, performing two-dimensional target detection, and obtaining pedestrian source image data; S3, performing multi-scene data enhancement on the source image to construct a pedestrian re-identification dataset for general scenarios. By applying the method for constructing a pedestrian re-identification dataset for general scenarios described in the present invention, it is possible to enhance the pedestrian re-identification data in different scenarios, unify the pedestrian re-identification data in different scenarios, and construct a pedestrian re-identification dataset for general scenarios, thereby overcoming the deployment and application problems of a single-scenario pedestrian re-identification model in practical applications, and having positive significance for improving the efficiency and quality of pedestrian re-identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian re-identification, and in particular, relates to a method for constructing a pedestrian re-identification dataset for general scenarios. Background Art

[0002] Person Re-identification (ReID) is a technology that uses computer vision technology to identify and match the same target person (pedestrian) within the field of view of different cameras. Supervised person re-identification is an important branch task in the field of computer vision. It mainly trains the model to accurately identify the same pedestrian in different scenarios based on given pedestrian data with labeled information. Simply put, it allows the computer to determine whether the target person appearing at different times, places or camera perspectives is the same person by learning from images or videos of pedestrians with marked identities.

[0003] For supervised person re-identification tasks, since it is necessary to mark the same pedestrian in different scenarios as the same pedestrian ID, a large amount of manual work is required for data collection, screening and labeling, which consumes a lot of human and material resources. Currently, unsupervised (self-supervised) person re-identification is receiving more and more attention.

[0004] Therefore, there is an urgent need to provide a method for constructing a pedestrian re-identification dataset that can be applied to general scenarios of unsupervised pedestrian re-identification to overcome the above problems.

[0005] In view of this, this application is hereby filed. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the deficiency in the prior art that the pedestrian re-identification task requires manual labeling, which leads to huge consumption of manpower and material resources. The purpose is to provide a method for constructing a pedestrian re-identification dataset for general scenarios that can be applied to unsupervised pedestrian re-identification.

[0007] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is: a method for constructing a pedestrian re-identification dataset for a general scenario, which is applied to unsupervised pedestrian re-identification, comprising the steps of:

[0008] S1, collect video image data in different scenes and different monitoring angles;

[0009] S2, constructing a pedestrian target detection network, taking the video image data as input, performing two-dimensional target detection, and obtaining pedestrian source image data;

[0010] S3. Perform multi-scene data enhancement on the source images to build a pedestrian re-identification dataset for general scenarios.

[0011] According to one embodiment of the present invention, the pedestrian target detection network includes: a feature extraction network part and a target detection network part.

[0012] According to one embodiment of the present invention, for the feature extraction network part, a convolutional neural network is applied, which takes the video image data as input and outputs a feature representation of the image.

[0013] According to an embodiment of the present invention, for the target detection network part, a multi-category detection method is applied to output the category and location information of the target;

[0014] The output is defined as:

[0015] O i =[c i ,x i ,y i ,w i ,h i ]

[0016] O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangle, w i ,h i are the length and width of the target respectively.

[0017] According to an embodiment of the present invention, the multiple scenes include: illumination change scene, blur change scene, pedestrian changing clothes change scene, occlusion change scene, and different modal data change scene.

[0018] According to an embodiment of the present invention, for a lighting transformation scene;

[0019] By adjusting the brightness and contrast of the source image data, different lighting scenes can be simulated and lighting transformation can be achieved.

[0020] According to an embodiment of the present invention, for a blur transformation scene;

[0021] The source image data is enhanced using a Gaussian blur operation to produce a blurred image that contains resolution variations.

[0022] According to an embodiment of the present invention, for a clothes changing scene;

[0023] By leveraging an open source generative adversarial model, pedestrian clothing features are extracted and then replaced.

[0024] According to an embodiment of the present invention, for an occlusion scene;

[0025] By using the copy and paste method to crop different categories of targets and then paste them into the source image, the pedestrians can be occluded by different categories of targets. For the acquisition of different categories of targets, the mask areas of different categories of targets are acquired by using the semantic segmentation algorithm, and then cropped according to the mask position and pasted according to the size ratio of the source image and the mask image.

[0026] According to an embodiment of the present invention, for a multimodal scenario;

[0027] Grayscale data is used to simulate infrared images of cross-modal scenes, and the source image data is processed in grayscale.

[0028] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0029] By applying the method for constructing a pedestrian re-identification dataset for general scenarios described in the present invention, pedestrian re-identification data enhancement in different scenarios is achieved, which is used for self-supervised multi-scenario pedestrian re-identification tasks, covering multiple scene types such as conventional scenes, pedestrian changing clothes scenes, dark and blurred scenes, occlusion scenes, cross-modal data scenes, etc., solving the problem of data collection and labeling for pedestrian re-identification tasks, and at the same time unifying the pedestrian re-identification data in different scenarios to construct pedestrian re-identification for general scenarios, overcoming the deployment and application problems of single-scenario pedestrian re-identification models in practical applications, which has a positive significance for improving the efficiency and quality of pedestrian re-identification.

[0030] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are part of the present invention and are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but do not constitute an improper limitation of the present invention. Obviously, the drawings described below are only some embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the accompanying drawings:

[0032] Figure 1 The present invention is a flowchart of a method for constructing a pedestrian re-identification dataset for a general scenario in an embodiment of the present invention.

[0033] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0035] In the description of the present invention, it should be noted that the directions or positional relationships indicated by terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “vertical”, “inside” and “outside” are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0036] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] Contrastive learning in unsupervised learning refers to comparing positive samples with similar semantics and negative samples with dissimilar semantics. By constructing a model and a contrast loss function, the representations corresponding to samples with similar semantics are closer in the representation space, while samples with dissimilar semantics are farther apart in the corresponding representation space. For the task of pedestrian re-identification in different scenes, data enhancement of different scenes is performed on pedestrian images in a certain regular scene, such as realizing pedestrian changing clothes, pedestrian occlusion, etc. The pedestrian images after data enhancement are used as positive samples of the source pedestrian images in the regular scene, and other pedestrians are used as negative samples. This allows the model to represent the features of pedestrians in different scenes more closely, thereby realizing re-identification of the same pedestrian in different scenes.

[0038] The unsupervised learning method uses contrastive learning to help the model learn to distinguish different pedestrian instance IDs by maximizing the similarity between different enhanced views of the same source image. No manual annotation is required, but data enhancement of different scenes of the source image is required to realize the construction of contrastive learning data.

[0039] Therefore, the present invention proposes a method for constructing a dataset for multi-scenario pedestrian re-identification, which is suitable for self-supervised (unsupervised) pedestrian re-identification, and realizes data enhancement for pedestrian re-identification in different multiple scenarios, covering multiple scenarios such as conventional scenes, pedestrian changing clothes scenes, dark and blurred scenes, occlusion scenes, cross-modal data scenes, etc., which solves the problem of data collection and labeling for pedestrian re-identification tasks, and at the same time unifies the pedestrian re-identification data in different scenarios to construct a dataset for general pedestrian re-identification, overcoming the deployment and application problems of single-scenario pedestrian re-identification models in practical applications.

[0040] like Figure 1 As shown, the method for constructing a pedestrian re-identification dataset for a general scenario described in the present invention is applied to unsupervised pedestrian re-identification, comprising the steps of:

[0041] S1. Collect video image data in different scenes and different monitoring angles.

[0042] Specifically, the different scenarios include: camera data in different seasons, different time periods, different weather conditions, different lighting conditions, different urban areas, etc.;

[0043] Specifically, the different monitoring viewing angles represent different camera installation positions, such as parallel installation, overhead installation, etc.

[0044] S2. Construct a pedestrian target detection network, use the video image data as input, perform two-dimensional target detection, and obtain pedestrian source image data.

[0045] By applying two-dimensional object detection, it is possible to locate and identify the target in the two-dimensional image, and determine its category information and position information.

[0046] Specifically, the pedestrian target detection network includes a feature extraction network part and a target detection network part;

[0047] 1) For the feature extraction part, a commonly used backbone network is used, including but not limited to convolutional neural networks such as ResNet, MobileNet, and VGG, with the input being an image (the video image data) and the output being a feature representation of the image;

[0048] 2) For the target detection network part, including but not limited to using commonly used multi-category detection methods such as YOLO, SSD, CenterNet, etc., the output is the category and location information of the target;

[0049] The output is defined as:

[0050] O i =[c i ,x i ,y i ,wi ,h i ]

[0051] O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangular box, w i ,h i are the length and width of the target respectively;

[0052] Specifically, according to the target detection result, the coordinate position of the pedestrian two-dimensional target detection box is obtained, and the pedestrian image is extracted from the image as the source image data of the pedestrian re-identification dataset.

[0053] Specifically, the pedestrian image is extracted from the image, the width and height of the extracted image are calculated, and the longer side is taken. The size of the longer side is fixed to 256, and then the shorter side (relative to the longer side) is reduced in the same proportion. Finally, the size of the shorter side is expanded to 128, and the insufficient part is filled with pixel value 0. Finally, the size of the pedestrian (target person) image is unified to 256*128.

[0054] S3. Perform multi-scene data enhancement on the source image.

[0055] The multiple scenes specifically include: illumination change scene, blur change scene, pedestrian changing clothes change scene, occlusion change scene, and different modal data change scene.

[0056] 1) For lighting changes.

[0057] Specifically, in actual application scenarios, there are different light transformations. By adjusting the brightness and contrast of source image data, simulation of different lighting scenes is achieved to realize the lighting transformation.

[0058] 2) For fuzzy transformation.

[0059] Specifically, in actual application scenarios, image blur may occur and the image resolution is low. In order to simulate the blur transformation scenario, Gaussian blur (digital image processing) operation is used to enhance the source image data to generate a blurred image with resolution changes.

[0060] 3) For the clothing changing scene.

[0061] Specifically, the scene of pedestrians changing clothes is an important challenge in the task of pedestrian re-identification. Pedestrians changing clothes usually occurs in video data spanning a long period of time, which makes data collection difficult. The data collected by data collectors who bring their own clothes to change are limited.

[0062] Specifically, by utilizing open source generative adversarial models (e.g., StyleGAN, DCGAN, CycleGAN, Pix2Pix, DragGAN, PaddleGAN), we can extract pedestrian clothing features and then replace them, including the color and material of pedestrian clothing, greatly increasing the diversity of pedestrian clothing change data.

[0063] 4) For occlusion scenes.

[0064] Specifically, for the situation where pedestrians are occluded in the pedestrian re-identification task, the different categories of targets are cropped and then pasted into the source image data by copying and pasting to achieve the occlusion of pedestrians by different categories of targets. For the acquisition of different categories of targets, the mask areas of different categories of targets are acquired by utilizing the semantic segmentation algorithm, and then cropped according to the mask position and pasted according to the size ratio of the source image and the mask image to achieve data enhancement of the occluded scene.

[0065] 5) For multimodal scenarios.

[0066] Specifically, multimodal data acquisition mainly solves pedestrian re-identification in dark light scenes, mainly using infrared cameras. Infrared cameras can generate well-perceived night images at night by using infrared thermal imaging. In night scenes, when the perception ability of visible light cameras decreases, using infrared perception images for visual perception can improve the accuracy of perception. However, it is difficult to collect infrared camera data, but infrared images look similar to grayscale images to some extent. Therefore, grayscale data is used to simulate infrared images of cross-modal scenes, and the source image data is grayscale processed to enhance multimodal scene data.

[0067] By applying the method for constructing a pedestrian re-identification dataset for general scenarios described in the present invention, pedestrian re-identification data enhancement in different scenarios is achieved, which is used for self-supervised (unsupervised) multi-scenario pedestrian re-identification tasks, covering multiple scene types such as conventional scenes, pedestrian changing scenes, dark and blurred scenes, occlusion scenes, cross-modal data scenes, etc., solving the problem of data collection and labeling for pedestrian re-identification tasks, and at the same time unifying the pedestrian re-identification data in different scenarios to construct pedestrian re-identification for general scenarios, overcoming the deployment and application problems of single-scenario pedestrian re-identification models in practical applications, which has positive significance for improving the efficiency and quality of pedestrian re-identification.

[0068] The above description is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this patent can make some changes or modifications to equivalent embodiments of equivalent changes by using the technical content suggested above without departing from the scope of the technical solution of the present invention. The implementation scheme in the above embodiment can also be further combined or replaced. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the solution of the present invention.

Claims

1. A method for constructing a person re-identification dataset for general scenarios, applied to unsupervised person re-identification, characterized in that: Includes steps: S1, collect video image data in different scenes and different monitoring angles; S2, constructing a pedestrian target detection network, taking the video image data as input, performing two-dimensional target detection, and obtaining pedestrian source image data; S3. Perform multi-scene data enhancement on the source images to build a pedestrian re-identification dataset for general scenarios.

2. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 1, characterized in that: The pedestrian target detection network includes: a feature extraction network part and a target detection network part.

3. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 2, characterized in that: For the feature extraction network part, a convolutional neural network is applied, which takes the video image data as input and outputs the feature representation of the image.

4. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 2 or 3, characterized in that: For the target detection network part, a multi-category detection method is applied to output the category and location information of the target; The output is defined as: O i =[c i ,x i ,y i ,w i ,h i ] O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangle, w i ,h i are the length and width of the target respectively.

5. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 1, characterized in that: The multiple scenes include: illumination change scene, blur change scene, pedestrian changing clothes change scene, occlusion change scene, and different modal data change scene.

6. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 5, characterized in that: For lighting change scenes; By adjusting the brightness and contrast of the source image data, different lighting scenes can be simulated and lighting transformation can be achieved.

7. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 5, characterized in that: For blurry transformation scenes; The source image data is enhanced using a Gaussian blur operation to produce a blurred image that contains resolution variations.

8. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 5, characterized in that: For the clothing change scene; By leveraging an open source generative adversarial model, pedestrian clothing features are extracted and then replaced.

9. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 5, characterized in that: For occlusion scenes; By using the copy and paste method to crop different categories of targets and then paste them into the source image, the pedestrians can be occluded by different categories of targets. For the acquisition of different categories of targets, the mask areas of different categories of targets are acquired by using the semantic segmentation algorithm, and then cropped according to the mask position and pasted according to the size ratio of the source image and the mask image.

10. The method for constructing a pedestrian re-identification dataset for a general scenario according to claim 5, characterized in that: For multimodal scenarios; Grayscale data is used to simulate infrared images of cross-modal scenes, and the source image data is processed in grayscale.

Images