A random coloring data augmentation method for cross-modal person re-identification

Abstract

The present invention relates to a random color data enhancement method suitable for cross-modal person re-identification. The method first randomly crops a local region of a visible light image and performs a random grayscale transformation on the cropped local region. It also randomly crops a local region of an infrared light image and performs a random colorization transformation on the cropped local region. The transformed local region is then replaced with the original cropped region to obtain randomly color-enhanced visible light and infrared light images for cross-modal person re-identification. At the image level, the method mitigates the style differences between visible light and infrared light images at a low computational cost, thereby improving the cross-modal person re-identification model's adaptability to modal changes and, consequently, the accuracy of cross-modal person re-identification.

Description

Technical Field

[0001] The present invention relates to image processing and machine vision technologies in the field of intelligent video surveillance, and in particular to a random coloring data enhancement method suitable for cross-modal pedestrian re-identification. Background Art

[0002] In all-weather video surveillance scenarios, surveillance cameras operate in the visible light mode under normal lighting conditions and in the infrared light mode under low-light conditions. This inevitably leads to the problem of cross-modal person re-identification (RID). Due to the different imaging band characteristics of visible and infrared light, the styles of pedestrian images in the two modalities differ significantly. While image style transfer methods based on generative adversarial learning exist, these methods involve multiple complex subnetworks, are computationally intensive, and carry the risk of model collapse, making training difficult. This limits the accuracy of RID algorithms and, consequently, their practical application in intelligent video surveillance systems. Summary of the Invention

[0003] The purpose of the present invention is to propose a random color data augmentation method for improving the performance of cross-modal pedestrian re-identification algorithms, thereby improving the practical capabilities of pedestrian re-identification. The present invention first randomly crops a local area of ​​the visible light image and performs a random grayscale transformation, and randomly crops a local area of ​​the infrared light image and performs a random color transformation; then the content of the original cropped area is replaced with the transformed local area, thereby alleviating the style difference between visible light and infrared light images at the image level at the beginning of discriminative feature learning with a light computational cost, thereby facilitating subsequent discriminative feature learning. The specific contents of the present invention are as follows:

[0004] Step 1: Randomly crop local areas of the visible light image and perform random grayscale enhancement

[0005] For visible light images Randomly cropping to obtain local areas , and for local areas Perform random grayscale to obtain ,Will After channel duplication and expansion, the cropped area in the original visible light image is replaced to obtain a visible light image enhanced by random coloring. .

[0006] The local area obtained by random cropping , calculated as shown in formula (1):

[0007] ,(1)

[0008] in, RC represents a random cropping operation, Indicates that it is used to control RC The extent of the clipping area. Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

[0009] The local area after random grayscale , calculated as shown in formula (2):

[0010] ,(2)

[0011] in, is a random grayscale operation, Respectively of , and Channel value, is a random value, , .

[0012] The enhanced visible light image after random color placement , calculated as shown in formula (3):

[0013] (3)

[0014] in, Indicates the vertical and horizontal coordinates of the pixel.

[0015] Step 2: Randomly crop local areas of the infrared image and perform random color enhancement

[0016] Infrared image after channel replication and expansion Randomly cropping to obtain local areas , for this local area Random colorization is performed ,Will Replace the cropped area in the original infrared image to obtain an enhanced infrared image after random coloring ;

[0017] The local area obtained by random cropping , calculated as shown in formula (4):

[0018] ,(4)

[0019] in, RC represents a random cropping operation, Indicates that it is used to control RC The extent of the clipping area. Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

[0020] The randomly colored local area , calculated as shown in formula (5):

[0021] ,(5)

[0022] in, is a random colorization operation, Respectively The values ​​on the three channels, is a random value, , .

[0023] The infrared image enhanced after random color placement , calculated as shown in formula (6): (6)

[0024] in, Indicates the vertical and horizontal coordinates of the pixel.

[0025] Step 3: Obtain randomly colored visible light and infrared light images for cross-modal pedestrian re-identification. That is, use the enhanced images to drive the deep network to learn image features, calculate the distance between images based on the features, sort the registered set images in ascending order according to the distance, and select the top-ranked registered set images as the re-identification results.

[0026] The present invention has the following technical advantages:

[0027] The present invention does not simply average the three channels of the visible light image, but randomly colors the visible light image and the infrared light image to achieve data enhancement, thereby better training the cross-modal pedestrian re-identification model. The present invention randomly crops and randomly grayscales the visible light image, making full use of the values ​​of the three channels, rather than being limited to a single channel; at the same time, the infrared light image is replicated and expanded through channels and randomly cropped and randomly colored, which can alleviate the style difference between the two modal images and achieve data enhancement of the two modalities of visible light and infrared light images. The data enhancement of the present invention does not fuse the visible light image and the infrared light image, but fully considers the local characteristics of the image, and randomly crops and grayscales the visible light image to give it some grayscale characteristics; replicates and expands the infrared light image through channels and then randomly crops and randomly colorizes it, so that it has some color characteristics, which is beneficial to the subsequent learning of discriminative features for pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the calculation process of the present invention. DETAILED DESCRIPTION

[0029] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0030] The following description sets forth many specific details to facilitate a full understanding of the present invention. The embodiments described are merely some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0032] The terms "first," "second," and "third," etc., in the specification and claims of the present invention and the accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "comprise," "comprising," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0033] Example 1

[0034] like Figure 1 The present invention relates to a random coloring data enhancement method suitable for cross-modal pedestrian re-identification, which specifically includes the following steps:

[0035] Step 1: Randomly crop local areas of the visible light image and perform random grayscale enhancement, as follows:

[0036] Step 1.1: Visible light image Randomly cropping to obtain local areas , calculated as shown in formula (1):

[0037] ,(1)

[0038] in, RC represents a random cropping operation, Indicates that it is used to control RC The extent of the clipping area. Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

[0039] Step 1.2: For local area Perform random grayscale to obtain , calculated as shown in formula (2):

[0040] ,(2)

[0041] in, is a random grayscale operation, Respectively of , and Channel value, is a random value, , .

[0042] Step 1.3, After channel duplication and expansion, the cropped area in the original visible light image is replaced to obtain a visible light image enhanced by random coloring. , calculated as shown in formula (3):

[0043] (3)

[0044] in, Indicates the vertical and horizontal coordinates of the pixel.

[0045] Step 2: Randomly crop local areas of the infrared image and perform random color enhancement, as follows:

[0046] Step 2.1: Copy the expanded infrared image for the channel Randomly cropping to obtain local areas , calculated as shown in formula (4):

[0047] ,(4)

[0048] in, RC represents a random cropping operation, Indicates that it is used to control RC The extent of the clipping area. Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

[0049] Step 2.2: For this local area Random colorization is performed , calculated as shown in formula (5):

[0050] ,(5)

[0051] in, is a random colorization operation, Respectively The values ​​on the three channels, is a random value, , .

[0052] Step 2.3, Replace the cropped area in the original infrared image to obtain an enhanced infrared image after random coloring , calculated as shown in formula (6):

[0053] (6)

[0054] in, Indicates the vertical and horizontal coordinates of the pixel.

[0055] Step 3: Obtain randomly colored visible light and infrared light images for cross-modal pedestrian re-identification. That is, use a deep network to extract features from the image, calculate the image distance based on the features, and use the sorted registration set images as the re-identification results.

[0056] Step 3.1: Training phase

[0057] Obtain a training set of randomly colored visible and infrared images to train a deep network for person re-identification model training.

[0058] like Figure 1As shown, this embodiment uses a dual-branch residual network ResNet50 as the backbone and combines it with a generalized-mean pooling layer (Generalized-mean Pooling, GeP), a batch normalization layer (Batch Normalization, BN) and a fully connected layer (Fully Connected, FC) to form a deep network. In this embodiment, the dual-branch construction method is specifically that the stem (Stem) of ResNet50, the residual groups Layer1 and Layer2 do not share parameters, while Layer3 and Layer4 share parameters, and the loss function uses label smoothed cross entropy loss. After training is completed, a cross-modal pedestrian re-identification model based on random color data enhancement is obtained.

[0059] Step 3.2, Testing Phase

[0060] For the query images and registered images in the test set, the cross-modal pedestrian re-identification model obtained in the training phase is used to extract features, that is, the output of the fully connected layer in the cross-modal pedestrian re-identification model is used as the features of the query image and the feature vector of the registered image; based on the extracted feature vector, the Euclidean distance between the query image and the registered image is calculated, and the images are sorted in ascending order according to the Euclidean distance. The top-ranked registered images are images that are suspected to have the same identity attributes as the query image. That is, as the results of the cross-modal pedestrian re-identification model, the suspected images are further manually verified according to actual needs.

[0061] The key to this invention lies in its ability to mitigate style differences between visible and infrared images at the image level through random color augmentation at a low computational cost. This improves the adaptability of the cross-modal person re-identification model to modal changes, thereby enhancing the accuracy of cross-modal person re-identification. Therefore, this invention can be widely applied to intelligent video surveillance systems, contributing to the development of smart transportation, smart security, and smart cities.

[0062] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units or modules as needed, that is, the internal structure of the storage device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of this application.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A random coloring data augmentation method for cross-modal person re-identification, characterized by: The steps include: Step 1: Randomly crop local areas of the visible light image and perform random grayscale enhancement For visible light images Randomly cropping to obtain local areas , and for local areas Perform random grayscale to obtain ,Will After channel duplication and expansion, the cropped area in the original visible light image is replaced to obtain a visible light image enhanced by random coloring. ; Step 2: Randomly crop local areas of the infrared image and perform random color enhancement Infrared image after channel replication and expansion Randomly cropping to obtain local areas , for this local area Random colorization is performed ,Will Replace the cropped area in the original infrared image to obtain an enhanced infrared image after random coloring ; Step 3: Obtain randomly colored visible light and infrared light images for cross-modal pedestrian re-identification. That is, use a deep network to extract features from the image, calculate the image distance based on the features, and use the sorted registration set images as the re-identification results.

2. The random coloring data augmentation method for cross-modal person re-identification according to claim 1, characterized in that: The local area obtained by random cropping , calculated as shown in formula (1): ,(1) in, RC represents a random cropping operation, Indicates that it is used to control RC Clipping area range, Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

3. The random coloring data augmentation method for cross-modal person re-identification according to claim 2, characterized in that: The local area after random grayscale , calculated as shown in formula (2): ,(2) in, is a random grayscale operation, Respectively of and Channel value, is a random value, , .

4. The random coloring data augmentation method for cross-modal person re-identification according to claim 3, characterized in that: The enhanced visible light image after random color placement , calculated as shown in formula (3): (3) in, Indicates the vertical and horizontal coordinates of the pixel.

5. The random coloring data augmentation method for cross-modal person re-identification according to claim 1, characterized in that: The local area obtained by random cropping , calculated as shown in formula (4): ,(4) in, RC represents a random cropping operation, Indicates that it is used to control RC Clipping area range, Indicates the vertical coordinate of the upper left corner of the cropping area. Indicates the horizontal coordinate of the upper left corner of the cropping area. Indicates the length of the cropping area. Indicates the width of the cropping area.

6. The random coloring data augmentation method for cross-modal person re-identification according to claim 5, characterized in that: The randomly colored local area , calculated as shown in formula (5): ,(5) in, is a random colorization operation, Respectively of and Channel value, is a random value, , .

7. The random coloring data augmentation method for cross-modal person re-identification according to claim 6, characterized in that: The infrared image enhanced after random color placement , calculated as shown in formula (6): (6) in, Indicates the vertical and horizontal coordinates of the pixel.

Images