Training method, background fusion method, system and control device of reid model
By segmenting and fusing background images to generate multiple training images, the problems of insufficient data and cross-domain issues in ReID models are solved, improving the model's generalization performance and cross-domain capability, and reducing annotation costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU YUNCONG INFORMATION TECH CO LTD
- Filing Date
- 2023-05-08
- Publication Date
- 2026-05-29
AI Technical Summary
ReID model training suffers from problems such as insufficient data, weak generalization ability, and errors in learning information, especially performance degradation in cross-domain scenarios.
By segmenting the original image into pedestrian mask images and fusing them with the background image, multiple fused images are generated. An image harmonization model is used to adjust the style and background similarity of the fused images, thereby improving the diversity and quality of the training data.
It improves the generalization performance of the ReID model, enhances its cross-domain capability, reduces the risk of model overfitting, improves the mAP metric, and reduces the annotation cost.
Smart Images
Figure CN116563184B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically providing a training method, background fusion method, system, and control device for a ReID model. Background Technology
[0002] ReID, also known as "cross-camera tracking technology," is a popular research area in perception algorithms, primarily addressing the identification and retrieval of pedestrians across different cameras and scenes. Currently, ReID model training faces several challenges:
[0003] 1) Insufficient data. ReID data requires cross-camera footage, which is difficult to collect. Cross-camera data is very scarce when building datasets from real-world scenes, requiring the installation of multiple camera positions for filming, which places high demands on the data acquisition team. Insufficient data directly leads to the model being prone to overfitting and having weak generalization performance.
[0004] 2) Cross-domain issues. Models trained in the source domain and tested directly in the target domain experience a significant performance drop. Cross-domain issues are prevalent in machine learning, but the relatively homogeneous nature of ReID datasets exacerbates this problem. Furthermore, the backgrounds of IDs collected from a single scene are often similar, making it easy for models to learn background information rather than the pedestrian's own characteristics during training. This can lead to issues like trajectories being misaligned in downstream tracking tasks.
[0005] Common data augmentation methods such as geometric transformations and color transformations can effectively improve the generalization ability of models and alleviate the problem of insufficient data to some extent. However, most of these methods transform the original image and it is difficult to introduce new information.
[0006] Accordingly, a new background fusion method is needed in this field to address the above problems. Summary of the Invention
[0007] To overcome the above-mentioned shortcomings, this invention is proposed to provide a solution or at least a partial solution to the technical problems of insufficient training data, weak generalization ability, and erroneous learning information in the training of ReID models in the prior art.
[0008] In a first aspect, the present invention provides a background fusion method, comprising the steps of:
[0009] The original image is input into the trained segmentation model to obtain the pedestrian mask image;
[0010] Obtain the background image set;
[0011] The first fused image is obtained by fusing any background image from the set of background images with the pedestrian mask image.
[0012] One technical solution of the aforementioned background fusion method also includes:
[0013] The first fused image and the pedestrian mask image are input into the image harmonization model to obtain the second fused image.
[0014] In one technical solution of the aforementioned background fusion method, the image harmonization model includes a feature extractor, a fusion unit, N fully connected layers, and N filters.
[0015] The input of the feature extractor receives a pedestrian mask image and a first fused image, and the output of the feature extractor outputs the original features.
[0016] The input of the first fully connected layer receives the original feature, and the output of the first fully connected layer outputs the first feature. The input of the i-th fully connected layer receives the feature obtained by concatenating the original feature with the (i-1)-th feature, and the output of the i-th fully connected layer outputs the i-th feature. The input of the N-th fully connected layer receives the feature obtained by concatenating the original feature with the (N-1)-th feature, and the output of the N-th fully connected layer outputs the N-th feature, where 1 < i < N.
[0017] The input of the first filter receives the first fused image and the first feature. The output of the first filter is connected to the input of the second filter. The input of the j-th filter receives the j-th feature. The output of the j-th filter is connected to the input of the (j+1)-th filter. The input of the N-th filter is used to receive the N-th feature. The output of the N-th filter is used to connect to the input of the fusion unit, where 1 < j < N.
[0018] The input of the fusion unit is used to receive pedestrian mask images, and the output of the fusion unit is used to output a second fused image.
[0019] In one technical solution of the above background fusion method, the number of fully connected layers and filters are both four, and the four filters are a brightness filter, a contrast filter, a color temperature filter, and a saturation filter.
[0020] In one technical solution of the aforementioned background fusion method, the training method of the segmentation model includes,
[0021] Step a: Input multiple unsegmented original images into the segmentation model to obtain pedestrian mask images;
[0022] Step b: Filter out the original images whose segmentation results do not meet the requirements for manual annotation, and input the manually annotated original images into the segmentation model for retraining;
[0023] Step c: Repeat steps a to b a certain number of times to obtain the trained segmentation model.
[0024] In one technical solution of the above background fusion method, fusing any background image in the background image set with the pedestrian mask image includes:
[0025] Obtain any background image from the set of background images;
[0026] The background image is cropped using a fixed aspect ratio;
[0027] The pedestrian mask image is fused with the cropped background image to obtain the first fused image.
[0028] In one technical solution of the above background fusion method, cropping the background image using a fixed aspect ratio includes:
[0029] The background image is cropped using a fixed aspect ratio, retaining the upper region of the background image.
[0030] In one technical solution of the above background fusion method, fusing the pedestrian mask image with the cropped background image includes:
[0031] The pedestrian mask image is merged with the upper region of the cropped background image.
[0032] In one technical solution of the aforementioned background fusion method, the original image is input into the trained segmentation model and run in offline mode.
[0033] In a second aspect, the present invention provides a method for training a ReID model, comprising the following steps:
[0034] Obtain the original image set;
[0035] Each original image in the original image set is processed using the background fusion method described above to obtain multiple first fused images;
[0036] The ReID model is trained using the multiple first fused images;
[0037] or
[0038] The multiple first fused images and the pedestrian mask image are input into the image harmonization model to obtain multiple second fused images;
[0039] The ReID model is trained using the multiple second fused images.
[0040] In a third aspect, the present invention provides a background fusion system, comprising:
[0041] The first processing module is used to input the original image into the trained segmentation model to obtain the pedestrian mask image;
[0042] The acquisition module is used to acquire a set of background images;
[0043] The second processing module is used to coarsely fuse any background image in the background image set with the pedestrian mask image to obtain a first fused image.
[0044] In a fourth aspect, a control device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the aforementioned background fusion method or ReID model training method.
[0045] In a fifth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the aforementioned background fusion method or ReID model training method.
[0046] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:
[0047] In implementing the technical solution of this invention, by segmenting the original image to obtain pedestrian mask images, and then fusing the pedestrian mask images with background images from a background image set, the effect of obtaining multiple first fused images from a single original image can be achieved, solving the problem of insufficient data volume in this invention. Correspondingly, this makes the model less prone to overfitting and improves generalization performance. The multiple first fused images obtained have the same pedestrian information but different background information, which is beneficial for the model to learn the correct information. Attached Figure Description
[0048] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:
[0049] Figure 1 This is a schematic flowchart of the main steps of a background fusion method according to an embodiment of the present invention;
[0050] Figure 2 This is a schematic diagram of the segmentation model training steps according to an embodiment of the present invention;
[0051] Figure 3 This is a schematic diagram of the background fusion steps according to an embodiment of the present invention;
[0052] Figure 4This is a schematic flowchart of the main steps of a background fusion method according to an embodiment of the present invention;
[0053] Figure 5 This is a main structural block diagram of a background fusion system according to an embodiment of the present invention;
[0054] Figure 6 This is a main structural block diagram of a control device according to an embodiment of the present invention.
[0055] List of reference numerals :
[0056] 11: First processing module; 12: Acquisition module; 13: Second processing module. Detailed Implementation
[0057] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0058] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.
[0059] The directional terms used in this article, such as "front," "front side," "front part," "rear," "rear side," and "rear part," are all based on the front-rear direction of the vehicle after the component is installed. The terms "longitudinal," "longitudinal direction," and "longitudinal section" mentioned in this article are based on the front-rear direction after the component is installed in the vehicle, while "transverse," "lateral," and "cross section" indicate the direction perpendicular to the longitudinal direction.
[0060] Here we will first explain some of the terms involved in this invention.
[0061] ReID: Pedestrian Re-identification, a technology that uses computer technology to determine whether a specific pedestrian exists in an image or video sequence.
[0062] Semantic segmentation: Segmenting each pixel in an image to achieve pixel-level classification.
[0063] Mask: A binary image generated by the pedestrian segmentation model. 1 represents the foreground, i.e., the main information of the pedestrian, and 0 represents the background.
[0064] Cross-domain: The training dataset and the test dataset come from different scenarios, which is called cross-domain.
[0065] Image harmonization: Through model learning, the foreground of the fused image is adjusted to be similar to the background in terms of style, tone, etc., resulting in a more natural visual effect.
[0066] mAP: A commonly used evaluation metric for object detection and ReID, measuring the accuracy of model predictions.
[0067] ID: A tag identifying a pedestrian.
[0068] In this embodiment of the invention, the original image is processed using a background fusion method to ensure that the pedestrian foreground information of the same original image obtains different backgrounds, generating more fused images and enriching the training samples. Simultaneously, the training samples obtained after background fusion have greater background differences, making it easier for the ReID model to learn pedestrian information during training and optimizing the training results.
[0069] This invention proposes a background fusion method. Figure 1 This is a schematic flowchart illustrating the main steps of a background fusion method according to an embodiment of the present invention. Figure 1 As shown, it mainly includes the following steps S11-S14.
[0070] Step S11: Input the original image into the trained segmentation model to obtain the pedestrian mask image.
[0071] In this embodiment, high accuracy is required for pedestrian segmentation, and a larger segmentation model is generally used to obtain more accurate segmentation results. If the segmentation model is used to obtain pedestrian mask information during ReID model training, it will inevitably reduce the training efficiency of the ReID model and lengthen the training cycle. Therefore, the generation of pedestrian masks from the original image is done offline by pre-training a segmentation model to perform segmentation, which does not affect the training efficiency of the ReID model. At the same time, the background fusion method of this invention has the same advantages when applied to the training of other similar models.
[0072] In a further embodiment of this method, since the ReID model only needs to segment pedestrians from the dataset, other objects can be considered as background. Furthermore, items carried by pedestrians, such as backpacks, mobile phones, and hats, are important attributes identifying the ID and need to be segmented into the pedestrian subject. However, the original segmentation model is a multi-class segmentation. If only the pre-trained weights of the original segmentation model are fine-tuned, the original model will segment the objects carried by pedestrians into other categories, almost never achieving ideal segmentation results. Therefore, further improvements are needed to the training process of the segmentation model. Figure 2 This is a schematic diagram of the segmentation model training steps according to an embodiment of the present invention, as follows: Figure 2 As shown, the training method for the segmentation model includes:
[0073] Step a: Input multiple unsegmented original images into the segmentation model to obtain pedestrian mask images;
[0074] Step b: Filter out the original images whose segmentation results do not meet the requirements for manual annotation, and input the manually annotated original images into the segmentation model for retraining;
[0075] Step c: Repeat steps a to b a certain number of times to obtain the trained segmentation model.
[0076] In this embodiment, after 3 to 4 rounds of iteration, the trained segmentation model basically meets the requirements of the ReID model for pedestrian segmentation.
[0077] Step S12: Obtain the background image set.
[0078] Step S13: Fuse any background image in the background image set with the pedestrian mask image to obtain a first fused image.
[0079] In this embodiment, background fusion is performed online, meaning that step S30 is executed during ReID model training. If background fusion were done offline, this solution would become an offline data augmentation strategy. The biggest problem with offline methods is their lack of flexibility. It requires pre-fusing and saving all pedestrian mask data needed by the ReID model with the background, consuming significant storage space and placing higher demands on hardware configuration.
[0080] The pedestrian mask information does not change during ReID model training, making offline processing the most suitable approach. However, the background needs to be randomly replaced, making online processing a more reasonable choice. Background replacement is a low-cost data augmentation strategy; while acquiring the original image is difficult, background images can be easily scraped from the internet. By combining offline image segmentation with online background fusion, a balance between flexibility and efficiency is achieved.
[0081] In a further embodiment of the method, Figure 3 This is a schematic diagram of the background fusion steps according to an embodiment of the present invention, as shown below. Figure 3 As shown, step S30 includes:
[0082] Step S131: Obtain any background image from the set of background images.
[0083] Each pedestrian mask image needs to be fused with multiple background images from the background image set, so that one original image can generate multiple training images.
[0084] Step S132: Crop the background image using a fixed aspect ratio.
[0085] In this embodiment, since the background images in the set vary in size, preprocessing is required to crop them to the original image size. The background images are cropped using a fixed aspect ratio to ensure that the cropped background image information is not compressed as much as possible. Furthermore, regardless of whether the background image is outdoor or indoor, the top of the image often represents the sky or roof; directly merging the pedestrian mask image with these areas would be problematic. Therefore, we select the lower 80% of the background image for cropping to avoid the sky or roof as much as possible. The upper area of the background image is retained.
[0086] Step S133: Fuse the pedestrian mask image with the cropped background image to obtain the first fused image.
[0087] The first fused image obtained from steps S11-S13 can be used as a training image for the ReID model. Experiments show that, under the same configuration, the first fused image achieves a 0.5% higher performance metrics than training with the original images. However, the foreground and background of the first fused image exhibit significant differences in tone and style. Therefore, a lightweight image harmonization network model is needed to refine the results of the first fused image, narrowing the distance between the foreground and background.
[0088] Step S14: Input the first fused image and the pedestrian mask image into the image harmonization model to obtain the second fused image.
[0089] In one embodiment, the image harmonization model includes a feature extractor, a fusion unit, four fully connected layers, and four filters, namely a brightness filter, a contrast filter, a color temperature filter, and a saturation filter.
[0090] The input of the feature extractor receives a pedestrian mask image and a first fused image, and the output of the feature extractor outputs the original features.
[0091] The input of the first fully connected layer receives the original feature, and the output of the first fully connected layer outputs the first feature. The input of the second fully connected layer receives the feature obtained by concatenating the original feature and the first feature, and the output of the second fully connected layer outputs the second feature. The input of the third fully connected layer receives the feature obtained by concatenating the original feature and the second feature, and the output of the third fully connected layer outputs the third feature. The input of the fourth fully connected layer receives the feature obtained by concatenating the original feature and the third feature, and the output of the fourth fully connected layer outputs the fourth feature.
[0092] The input of the first filter receives the first fused image and the first feature. The output of the first filter is connected to the input of the second filter. The input of the second filter receives the second feature. The output of the second filter is connected to the input of the third filter. The input of the third filter receives the third feature. The output of the third filter is connected to the input of the fourth filter. The input of the fourth filter is used to receive the fourth feature. The output of the fourth filter is used to connect to the input of the fusion unit.
[0093] The input of the fusion unit is used to receive pedestrian mask images, and the output of the fusion unit is used to output a second fused image.
[0094] In this embodiment, assuming the original image is I, the background image is B, and the pedestrian mask information is M, the first fused image result I is obtained by using Alpha fusion. coarse =M·I+(1-M)·B
[0095] Filter parameters are learned through cascading to control the brightness, contrast, color temperature, and saturation of the fused image. The output of the current filter is obtained by filtering the output of the previous filter and the parameters learned in the current layer: I i =F i (I i-1 θ i ), i∈[1,4]
[0096] The result of the second fused image obtained through image harmonization processing I refine The final output from the fusion unit is: I refine =M·I⁴ + (1-M)·I coarse
[0097] The second fused image, processed by the image harmonization model, exhibits more harmonious foreground and background, resulting in a more stylistically similar image. During ReID model training, original images from different scenes are fused with images of the same background according to a certain probability. After harmonization, the two different scene datasets share the same background information, and their image styles become more similar, effectively narrowing the inter-domain gap between the two datasets. The second fused image trained using image harmonization enhances the ReID model's cross-domain capabilities.
[0098] In the experiment, 200 categories of indoor and outdoor scene images were crawled from the internet as the background image set. This approach was tested on a ReID original image set of 130,000 IDs and nearly 3 million images. The mAP metric improved by nearly 1% compared to the training results on the original data. This improvement even surpassed the improvement in the accuracy of labeling a new batch of data, effectively reducing labeling costs.
[0099] The background fusion method in this embodiment of the invention is a universal data augmentation method that can be extended to other supervised learning tasks. Taking face recognition as an example, only one face segmentation model needs to be trained to segment the face mask from the original image, and the subsequent training strategy can follow steps S12 to S14 of this method.
[0100] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.
[0101] Furthermore, this invention provides a training method for a ReID model. Figure 4 This is a schematic flowchart illustrating the main steps of a background fusion method according to an embodiment of the present invention. Figure 4 As shown, steps S21 to S23 are included:
[0102] Step S21: Obtain the original image set.
[0103] The four most commonly used public ReID datasets are shown in Table 1, with a total of 8,881 valid IDs and only 208,263 images. This is on a completely different scale compared to face datasets with millions of IDs. Insufficient data directly leads to the model being prone to overfitting and exhibiting weak generalization performance.
[0104] Table 1. Statistics on four commonly used public datasets for ReID.
[0105]
[0106] In addition, most of the publicly available datasets shown in Table 1 were collected on campuses. Market-1501 was collected from domestic campuses in the summer, while DukeMTMC-reID was collected from foreign campuses in the winter. The difference in clothing styles in different seasons caused a significant inter-domain gap between the two datasets.
[0107] Step S22: Process each original image in the original image set using the background fusion method described above to obtain multiple second fused images.
[0108] Step S23: Train the ReID model using the multiple second fused images.
[0109] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.
[0110] Furthermore, the present invention also provides a background fusion system.
[0111] See appendix Figure 5 , Figure 5 This is a main structural block diagram of a background fusion system according to an embodiment of the present invention. Figure 5 As shown, the background fusion system in this embodiment of the invention mainly includes a first processing module 11, an acquisition module 12, and a second processing module 13. The first processing module is used to input the original image into a trained segmentation model to obtain a pedestrian mask image; the acquisition module is used to acquire a set of background images; and the second processing module is used to coarsely fuse any background image in the set of background images with the pedestrian mask image to obtain a first fused image.
[0112] In some embodiments, one or more of the first processing module 11, the acquisition module 12, and the second processing module 13 may be combined into a single module.
[0113] In one embodiment, the specific functions of the first processing module 11 can be described in step S11. The specific functions of the acquisition module 12 can be described in step S12. The specific functions of the second processing module 13 can be described in step S13.
[0114] The aforementioned background fusion system is used for execution Figure 1The background fusion method embodiments shown are similar in technical principle, the technical problems they solve, and the technical effects they produce. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the background fusion system can be found in the embodiments of the background fusion method, which will not be repeated here.
[0115] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0116] Furthermore, the present invention also provides a control device. For example... Figure 6 As shown, in one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the background fusion method or the ReID model training method of the above-described method embodiments. The processor can be configured to execute the program in the storage device, which includes, but is not limited to, a program for executing the background fusion method or the ReID model training method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The control device can be a control device device comprising various electronic devices.
[0117] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program that performs the background fusion method or the ReID model training method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described background fusion method or ReID model training method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.
[0118] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, a part of its hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.
[0119] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.
[0120] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A background fusion method, characterized in that, Including the following steps: The original image is input into the trained segmentation model to obtain the pedestrian mask image; Obtain the background image set; The first fused image is obtained by fusing any background image from the set of background images with the pedestrian mask image; The first fused image and the pedestrian mask image are input into the image harmonization model to obtain the second fused image; The image harmonization model includes a feature extractor, a fusion unit, N fully connected layers, and N filters. The input of the feature extractor receives a pedestrian mask image and a first fused image, and the output of the feature extractor outputs the original features. The input of the first fully connected layer receives the original feature, and the output of the first fully connected layer outputs the first feature. The input of the i-th fully connected layer receives the feature obtained by concatenating the original feature with the (i-1)-th feature, and the output of the i-th fully connected layer outputs the i-th feature. The input of the N-th fully connected layer receives the feature obtained by concatenating the original feature with the (N-1)-th feature, and the output of the N-th fully connected layer outputs the N-th feature, where 1 < i < N. The input of the first filter receives the first fused image and the first feature. The output of the first filter is connected to the input of the second filter. The input of the j-th filter receives the j-th feature. The output of the j-th filter is connected to the input of the (j+1)-th filter. The input of the N-th filter is used to receive the N-th feature. The output of the N-th filter is used to connect to the input of the fusion unit, where 1 < j < N. The input of the fusion unit is used to receive pedestrian mask images, and the output of the fusion unit is used to output a second fused image.
2. The background fusion method according to claim 1, characterized in that, The number of fully connected layers and filters are both four, and the four filters are a brightness filter, a contrast filter, a color temperature filter, and a saturation filter.
3. The background fusion method according to claim 1, characterized in that, The training method for the segmentation model includes, Step a: Input multiple unsegmented original images into the segmentation model to obtain pedestrian mask images; Step b: Filter out the original images whose segmentation results do not meet the requirements for manual annotation, and input the manually annotated original images into the segmentation model for retraining; Step c: Repeat steps a to b a certain number of times to obtain the trained segmentation model.
4. The background fusion method according to claim 1, characterized in that, The step of fusing any background image from the set of background images with the pedestrian mask image includes: Obtain any background image from the set of background images; The background image is cropped using a fixed aspect ratio; The pedestrian mask image is fused with the cropped background image to obtain the first fused image.
5. The background fusion method according to claim 4, characterized in that, The method of cropping the background image using a fixed aspect ratio includes: The background image is cropped using a fixed aspect ratio, retaining the upper region of the background image.
6. The background fusion method according to claim 4, characterized in that, The step of fusing the pedestrian mask image with the cropped background image includes: The pedestrian mask image is merged with the upper region of the cropped background image.
7. The background fusion method according to any one of claims 1-6, characterized in that, The original image is input into the trained segmentation model, which runs in offline mode.
8. A training method for a ReID model, characterized in that, Including the following steps: Obtain the original image set; Each original image in the original image set is processed by the background fusion method as described in any one of claims 1-7 to obtain multiple first fused images; The ReID model is trained using the multiple first fused images; or The multiple first fused images and the pedestrian mask image are input into the image harmonization model to obtain multiple second fused images; The ReID model is trained using the multiple second fused images.
9. A background blending system, characterized in that, include: The first processing module is used to input the original image into the trained segmentation model to obtain the pedestrian mask image; The acquisition module is used to acquire a set of background images; The second processing module is used to coarsely fuse any background image in the background image set with the pedestrian mask image to obtain a first fused image, and input the first fused image and the pedestrian mask image into an image harmonization model to obtain a second fused image. The image harmonization model includes a feature extractor, a fusion unit, N fully connected layers, and N filters. The input of the feature extractor receives a pedestrian mask image and a first fused image, and the output of the feature extractor outputs the original features. The input of the first fully connected layer receives the original feature, and the output of the first fully connected layer outputs the first feature. The input of the i-th fully connected layer receives the feature obtained by concatenating the original feature with the (i-1)-th feature, and the output of the i-th fully connected layer outputs the i-th feature. The input of the N-th fully connected layer receives the feature obtained by concatenating the original feature with the (N-1)-th feature, and the output of the N-th fully connected layer outputs the N-th feature, where 1 < i < N. The input of the first filter receives the first fused image and the first feature. The output of the first filter is connected to the input of the second filter. The input of the j-th filter receives the j-th feature. The output of the j-th filter is connected to the input of the (j+1)-th filter. The input of the N-th filter is used to receive the N-th feature. The output of the N-th filter is used to connect to the input of the fusion unit, where 1 < j < N. The input of the fusion unit is used to receive pedestrian mask images, and the output of the fusion unit is used to output a second fused image.
10. A control device, comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the background fusion method of any one of claims 1 to 7 or the training method of the ReID model of claim 8.
11. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the background fusion method of any one of claims 1 to 7 or the training method of the ReID model of claim 8.