Data set expansion method and device
Through semantic segmentation model, and scene migration model is used for scene migration, the problem of difficult instance sample-level data expansion in the existing technology is solved, efficient expansion and quality improvement of data sets are achieved, and more efficient training of autonomous driving perception models is supported.
Patent Information
- Application Number
- CN202311320096.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-12
- Publication Date
- 2025-04-15
AI Technical Summary
The prior art is difficult to augment data at the sample level, and it is impossible to effectively improve the quality and richness of data sets in the field of autonomous driving.
The example sample pictures are extracted by inputting the image containing the instance samples into the pre-constructed semantic segmentation model, and fusing them with the pre-acquisitioned site pictures, and then using the scene migration model to perform scene migration processing on the fused pictures to generate pictures that are consistent with the picture scenes in the target domain image set, thereby building the expanded data set.
It realizes the extraction of instance samples, completes data expansion at the instance sample level, improves the quality and richness of the data set, and supports more efficient training of autonomous driving perception models.
Smart Images
Figure CN119832352A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for expanding a data set. Background Art
[0002] In the field of autonomous driving, the quality and richness of the data set used for training will directly affect the performance of the related perception model. Therefore, the data set needs to be expanded to improve its quality and richness.
[0003] Among the existing dataset expansion schemes that can be applied on a large scale, deep learning models such as neural style transfer and generative adversarial networks are mainly used to transfer the style of images in existing datasets and expand existing datasets. However, this method cannot generate instance samples and it is difficult to perform instance-level data expansion. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a method and device for expanding a data set, so as to achieve the purpose of extracting instance samples and completing data expansion at the instance sample level.
[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0006] A first aspect of an embodiment of the present invention discloses a method for expanding a data set, comprising:
[0007] Input the image containing the instance sample into the pre-built semantic segmentation model for extraction to obtain the instance sample image;
[0008] Merging the example sample image with a pre-collected site image to obtain a first image;
[0009] Inputting the first picture into a scene migration model pre-constructed based on a target domain picture set, performing scene migration processing on the first picture, and obtaining a second picture whose scene is consistent with the scene of the picture in the target domain picture set; the target domain picture set is composed of pictures pre-collected in the same scene, and the scene includes at least daytime scene, night scene, sunny scene and rainy scene;
[0010] A picture dataset is constructed based on the first picture and the second picture.
[0011] Preferably, the process of constructing the semantic segmentation model includes:
[0012] Build the semantic segmentation model to be trained;
[0013] Inputting a training picture pre-labeled with instance annotation information into the semantic segmentation model, extracting and outputting instance sample pictures from the training picture based on the instance annotation information using the semantic segmentation model, and obtaining a trained semantic segmentation model;
[0014] Inputting a test image not labeled with the instance annotation information into the trained semantic segmentation model;
[0015] If the trained semantic segmentation model correctly outputs the instance sample image, it is determined that the training of the semantic segmentation model is completed;
[0016] If the trained semantic segmentation model cannot correctly output the instance sample picture, continue to input the training picture into the semantic segmentation model for training until the test picture is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample picture.
[0017] Preferably, the step of fusing the example sample image with a pre-collected site image to obtain a first image includes:
[0018] Modify the size of the example sample picture;
[0019] The resized instance sample image is fused with the pre-collected site image to obtain a first image.
[0020] Preferably, the step of fusing the example sample image with a pre-collected site image to obtain a first image includes:
[0021] Performing horizontal flipping and / or rotation processing on the example sample image;
[0022] The instance sample image after horizontal flipping or rotation processing is fused with the pre-collected site image to obtain a first image.
[0023] Preferably, the process of constructing the scene migration model includes:
[0024] Build a scene migration model including the Cycle GAN network;
[0025] Inputting original domain pictures in an original domain picture set and target domain pictures in a corresponding target domain picture set into the scene migration model, and constructing a mapping network between the original domain picture set and the target domain picture set in the scene migration model; the original domain picture set is composed of pictures pre-collected in the same scene; the target domain picture set is composed of pictures pre-collected in the same scene, and the scene at least includes daytime scene, night scene, sunny scene and rainy scene;
[0026] When the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.
[0027] A second aspect of an embodiment of the present invention discloses a data set expansion device, comprising:
[0028] An extraction unit, used for inputting a picture containing an instance sample into a pre-built semantic segmentation model for extraction to obtain an instance sample picture;
[0029] A fusion unit, configured to fuse the example sample image with a pre-collected site image to obtain a first image;
[0030] a migration unit, configured to input the first picture into a scene migration model pre-constructed based on a target domain picture set, perform scene migration processing on the first picture, and obtain a second picture whose scene is consistent with the scene of the pictures in the target domain picture set; the target domain picture set is composed of pictures pre-collected in the same scene, and the scenes include at least daytime scenes, night scenes, sunny scenes, and rainy scenes;
[0031] The first construction unit is configured to construct a picture dataset based on the first picture and the second picture.
[0032] Preferably, it also includes:
[0033] The second construction unit is used to construct a semantic segmentation model to be trained; input a training picture pre-annotated with instance annotation information into the semantic segmentation model, and use the semantic segmentation model to extract and output instance sample pictures in the training picture based on the instance annotation information to obtain a trained semantic segmentation model; input a test picture not annotated with the instance annotation information into the trained semantic segmentation model; if the trained semantic segmentation model correctly outputs the instance sample picture, it is determined that the training of the semantic segmentation model is completed; if the trained semantic segmentation model cannot correctly output the instance sample picture, continue to input the training picture into the semantic segmentation model for training until the test picture is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample picture.
[0034] Preferably, the fusion unit comprises:
[0035] A modification subunit, used to modify the size of the example sample picture;
[0036] The first fusion subunit is used to fuse the resized instance sample image with the pre-collected site image to obtain a first image.
[0037] Preferably, the fusion unit comprises:
[0038] A processing subunit, configured to perform horizontal flipping and / or rotation processing on the example sample image;
[0039] The second fusion subunit is used to fuse the instance sample image after horizontal flipping or rotation processing with the pre-collected site image to obtain a first image.
[0040] Preferably, it also includes:
[0041] The third construction unit is used to construct a scene migration model including a Cycle GAN network; the original domain pictures in the original domain picture set and the target domain pictures in the corresponding target domain picture set are input into the scene migration model, and a mapping network between the original domain picture set and the target domain picture set is constructed in the scene migration model; the original domain picture set is composed of pictures pre-collected in the same scene; the target domain picture set is composed of pictures pre-collected in the same scene, and the scenes include at least daytime scenes, night scenes, sunny scenes and rainy scenes; when the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.
[0042] Based on a method and device for expanding a data set provided by the above-mentioned embodiment of the present invention, a picture containing an instance sample is input into a pre-built semantic segmentation model for extraction to obtain an instance sample picture; the instance sample picture is fused with a pre-collected site picture to obtain a first picture; the first picture is input into a scene migration model pre-built based on a target domain picture set, and the first picture is subjected to scene migration processing to obtain a second picture whose scene is consistent with that of the pictures in the target domain picture set; the target domain picture set is composed of pre-collected pictures of the same scene, and the scenes include at least daytime scenes, night scenes, sunny scenes, and rainy scenes; a picture data set is constructed based on the first picture and the second picture. In this solution, a pre-built semantic segmentation model is used to extract instance sample pictures, and then the instance sample pictures are fused with other pictures and scene migration is performed to achieve the purpose of extracting instance samples and completing data expansion at the instance sample level. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0044] Figure 1 A flowchart of a method for expanding a data set disclosed in an embodiment of the present invention;
[0045] Figure 2 This is an example diagram of an example sample picture disclosed in an embodiment of the present invention;
[0046] Figure 3 This is an example diagram of sample picture fusion disclosed in an embodiment of the present invention;
[0047] Figure 4 This is an example diagram of a scene migration disclosed in an embodiment of the present invention;
[0048] Figure 5 A flowchart of a method for constructing a semantic segmentation model disclosed in an embodiment of the present invention;
[0049] Figure 6 This is an architecture diagram of a Cycle GAN network disclosed in an embodiment of the present invention;
[0050] Figure 7 A structural diagram of a data set expansion device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.
[0053] As can be seen from the background technology, existing data set expansion methods cannot obtain instance samples and are difficult to perform instance-level data expansion.
[0054] Therefore, an embodiment of the present invention discloses a method and device for expanding a data set, which uses a pre-built semantic segmentation model to extract instance sample images, and then fuses the instance sample images with other images and performs scene migration to achieve the purpose of extracting instance samples and completing data expansion at the instance sample level.
[0055] like Figure 1FIG. 1 is a flowchart of a method for expanding a data set disclosed in an embodiment of the present invention, and the method comprises the following steps:
[0056] Step S101: Input a picture containing an instance sample into a pre-built semantic segmentation model for extraction to obtain an instance sample picture.
[0057] In step S101, the semantic segmentation model can be a high-precision model such as SegNeXt or Transfiner. Specifically, the semantic segmentation model is a supervised learning model, that is, training images with manual annotation information are required to train the semantic segmentation model so that it can meet the actual task requirements. For the specific training semantic segmentation model construction and training process, please refer to Figure 5 The embodiment shown.
[0058] The instance samples may be vehicles or people, etc., without any specific limitation.
[0059] like Figure 2 , which is an example diagram of an example sample picture disclosed in an embodiment of the present invention.
[0060] Specifically, Figure 2 The example sample shown is of type vehicle.
[0061] In one embodiment, the type of instance samples extracted by the semantic segmentation model depends on the manually labeled information in the training images used to train the semantic segmentation model. For example, if the training images used to train the semantic segmentation model carry manually labeled information representing the type of vehicle, then when the trained semantic segmentation model extracts instance samples, it will only extract instance samples of the vehicle type.
[0062] In the specific implementation process of step S101, the picture containing the instance sample is input into the pre-built semantic segmentation model for extraction, the instance sample is extracted, and the instance sample is output in the form of a picture to obtain the instance sample picture.
[0063] Step S102: Fusing the instance sample image with the pre-collected site image to obtain a first image.
[0064] In order to more clearly illustrate the fusion process in step S102, Figure 3 , which is an example diagram of an example sample picture fusion disclosed in an embodiment of the present invention.
[0065] Specifically, Figure 3 The left side shows the pre-collected site image. It can be seen that there is no instance sample of the vehicle type in the site image. Figure 3 After the example sample images in are fused to the position indicated by the rectangular box, we get Figure 3The fused picture on the right is the first picture mentioned in this application.
[0066] It should be noted that Figure 3 The fusion position indicated by the rectangular box is only for illustration and is not limiting. In other embodiments, the example sample image can be fused at any position in the site image.
[0067] In one embodiment, the size of the example sample image is first modified, and then the example sample image after the modified size is merged with the pre-collected site image to obtain the first image.
[0068] For example, Figure 3 After enlarging or reducing the instance sample image in the scene, it is fused to any position in the scene image to obtain the first image.
[0069] In one embodiment, the example sample image is first horizontally flipped and / or rotated, and the example sample image after the horizontal flipping or rotation processing is merged with the pre-collected site image to obtain a first image.
[0070] Step S103: input the first picture into a scene migration model pre-constructed based on the target domain picture set, perform scene migration processing on the first picture, and obtain a second picture whose scene is consistent with the picture scene in the target domain picture set.
[0071] In step S103, the scene migration model includes a Cycle GAN network. The scene migration model is obtained by training the original domain picture set and the target domain picture set. The specific construction and training process includes the following steps:
[0072] Step S11: construct a scene migration model including a Cycle GAN network.
[0073] In step S11, CycleGAN is an advanced open source algorithm model. The specific architecture of the Cycle GAN network can be found in Figure 6 The embodiment shown.
[0074] Step S12: inputting the original domain pictures in the original domain picture set and the corresponding target domain pictures in the target domain picture set into the scene migration model, and constructing a mapping network between the original domain picture set and the target domain picture set in the scene migration model.
[0075] In step S12, the original domain picture set is composed of pictures pre-collected in the same scene, and the target domain picture set is also composed of pictures pre-collected in the same scene, and the scenes at least include daytime scenes, night scenes, sunny scenes and rainy scenes.
[0076] It should be noted that the scenes in the original domain picture set and the target domain picture set selected for training are set according to actual needs.
[0077] Step S13: When the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.
[0078] In step S13, assuming that the original domain image is a daytime scene and the target domain image is a nighttime scene, when the scene of the image output by the scene migration model becomes a nighttime scene, it is determined that the construction of the scene migration model is completed.
[0079] In order to more clearly illustrate the scene migration process in step S103, the present invention selects Figure 3 The first fused image shown on the right is used as an example of the object of scene migration. Figure 4 , which is an example diagram of a scene migration disclosed in an embodiment of the present invention.
[0080] Specifically, the scene transfer model is trained using the original domain image set of daytime scenes and the target domain image set of nighttime scenes.
[0081] in, Figure 4 The picture on the lower left corner is a daytime scene. Figure 4 After the lower left corner picture is input into the scene migration model for scene migration, the migrated picture is obtained. Figure 4 As can be seen from the picture in the lower right corner, the scene of the picture after migration is a night scene.
[0082] Step S104: constructing a picture data set based on the first picture and the second picture.
[0083] In one embodiment, when an instance sample image is fused with a site image, multiple first images can be obtained by changing the size of the instance sample image, the fusion position, and flipping or rotating the instance sample image. Correspondingly, multiple second images can be obtained by performing scene migration on the multiple first images, and an image data set is constructed based on each first image and each second image.
[0084] Based on the method for expanding a data set disclosed in the above-mentioned embodiment of the present invention, a picture containing an instance sample is input into a pre-built semantic segmentation model for extraction to obtain an instance sample picture, the instance sample picture is fused with a site picture to obtain a first picture, and then a pre-built scene migration model is used to perform scene migration on the first picture to obtain a second picture, and a picture data set is constructed based on the first picture and the second picture, i.e., an expanded data set. In this solution, an instance sample picture is extracted using a pre-built semantic segmentation model, and then the instance sample picture is fused with other pictures and scene migration is performed to achieve the purpose of extracting instance samples and completing data expansion at the instance sample level.
[0085] Based on the method for expanding a data set disclosed in the above embodiment of the present invention, Figure 5 FIG. 1 is a flowchart of a method for constructing a semantic segmentation model disclosed in an embodiment of the present invention. The method mainly includes the following steps:
[0086] Step S501: construct a semantic segmentation model to be trained.
[0087] In step S501, the semantic segmentation model may be a SegNeXt model or a Transfiner model, or other high-precision models with the same functions.
[0088] Step S502: input the training picture pre-labeled with instance annotation information into the semantic segmentation model, and use the semantic segmentation model to extract and output instance sample pictures in the training picture based on the instance annotation information to obtain a trained semantic segmentation model.
[0089] In step S502, instance annotation information is manually annotated in the training image in advance, and the instance annotation information is used to represent instance samples that need to be extracted by the semantic segmentation model.
[0090] In the specific implementation process of step S502, the training pictures with instance annotation information in the training set are input into the semantic segmentation model in sequence and continuously. The semantic segmentation model is used to extract and output instance sample pictures in the training pictures based on the instance annotation information, so as to obtain a set of instance sample pictures and a trained semantic segmentation model.
[0091] It should be noted that whether the semantic segmentation model training is successful, that is, whether the accuracy meets the requirements, needs to be verified through steps S503 to S505.
[0092] Step S503: Input the test image that is not labeled with instance annotation information into the trained semantic segmentation model.
[0093] In step S503 , the test image not labeled with instance annotation information contains instance samples of the same type as the training image, for example, instance samples of the vehicle type.
[0094] Step S504: If the trained semantic segmentation model correctly outputs the instance sample image, it is determined that the semantic segmentation model training is completed.
[0095] In step S504, in order to illustrate how to determine whether the semantic segmentation model correctly outputs the instance sample image, the following example is provided:
[0096] Assuming that the training images contain instance samples of vehicle types, and the test images also contain instance samples of vehicle types, then only when the semantic segmentation model extracts and outputs instance samples of all vehicle types in the test images, is it considered that the semantic segmentation model correctly outputs the instance sample images.
[0097] Step S505: If the trained semantic segmentation model cannot correctly output the instance sample image, continue to input the training image into the semantic segmentation model for training until the test image is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample image.
[0098] In step S505, it is determined whether the semantic segmentation model correctly outputs the instance sample image. Please refer to the explanation in step S504. When the semantic segmentation model cannot correctly output the instance sample image, it means that the accuracy of the semantic segmentation model does not meet the requirements, and return to step S502 to continue training.
[0099] Based on the method for constructing a semantic segmentation model disclosed in the above embodiment of the present invention, a semantic segmentation model to be trained is constructed, and the semantic segmentation model is trained using training pictures pre-labeled with instance annotation information until the semantic segmentation model can correctly output instance samples in the test pictures. In this solution, the semantic segmentation model is constructed and trained so that it can extract and output instance sample pictures from pictures containing instance samples, providing conditions for instance-level data expansion.
[0100] Based on the method for expanding a data set disclosed in the above embodiment of the present invention, Figure 6 As shown, it is an architecture diagram of a Cycle GAN network disclosed in an embodiment of the present invention. The network mainly includes a data reading module, a network training module, a network reasoning module and a data storage module.
[0101] The data reading module is used to input original domain images and target domain images into the network training module, and to input images that need to be migrated into the network reasoning module.
[0102] In the network training module, IMG(A) is the original domain image, IMG(B) is the target domain image, G(A2B) and G(B2A) are two generators with the same network structure, D(A) and D(B) are two discriminators with the same structure, and Loss is the cycle consistency function.
[0103] Among them, G(A2B) is used to convert the original domain image into the target domain image, and G(B2A) is used to convert the target domain image into the original domain image. The two are a pair of inverse mappings. The discriminators D(A) and D(B) are used to score the generator respectively, and then the network weights of the generator are obtained by solving the minmax optimization problem. In addition, CycleGAN adds cycle consistency loss on the basis of adversarial loss to maintain the consistency of semantic information in the generated image and the original image.
[0104] In the network reasoning module, IMG(A) is the original domain image that needs scene migration, IMG(B) is the target domain image, and G(A2B) and G(B2A) are trained generators.
[0105] Among them, G(A2B) is used to convert the original domain images that require scene migration into the target domain images, and G(B2A) is used to convert the target domain images into the original domain images that require scene migration.
[0106] It should be noted that the conversion mentioned in the embodiment of the present invention is a scene migration operation, for example, converting the scene of the original domain image into a target domain image that is consistent with the scene in the target domain image.
[0107] The data storage module is used to store the images output by the network inference module.
[0108] It can be understood that, correspondingly, in the data set expansion method disclosed in the above-mentioned embodiment of the present invention, the generator G(A2B) is generally used to convert the pictures that need scene migration into target domain pictures.
[0109] Based on the architecture of a Cycle GAN network disclosed in the above-mentioned embodiment of the present invention, the Cycle GAN network includes a data reading module, a network training module, a network reasoning module and a data storage module, wherein the data reading module is used to input original domain images and target domain images into the network training module, and input images that need to be migrated to the network reasoning module, the network training module is used to train the generator, the network reasoning module is used to perform scene migration, and the data storage module is used to store images output by the network reasoning module. In this solution, the Cycle GAN network can realize scene migration operations, providing conditions for the expansion of images in the direction of scene style.
[0110] Based on the method for expanding a data set disclosed in the above embodiment of the present invention, Figure 7 , which is a structural diagram of a data set expansion device disclosed in an embodiment of the present invention, the device includes: an extraction unit 701, a fusion unit 702, a migration unit 703 and a first construction unit 704.
[0111] The extraction unit 701 is used to input the picture containing the instance sample into the pre-built semantic segmentation model for extraction to obtain the instance sample picture.
[0112] In one embodiment, the data set expansion device further includes:
[0113] The second construction unit is used to construct a semantic segmentation model to be trained; input a training picture pre-annotated with instance annotation information into the semantic segmentation model, use the semantic segmentation model to extract and output instance sample pictures in the training picture based on the instance annotation information, and obtain a trained semantic segmentation model; input a test picture not annotated with instance annotation information into the trained semantic segmentation model; if the trained semantic segmentation model correctly outputs the instance sample picture, it is determined that the training of the semantic segmentation model is completed; if the trained semantic segmentation model cannot correctly output the instance sample picture, the training picture continues to be input into the semantic segmentation model for training until the test picture is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample picture.
[0114] The fusion unit 702 is used to fuse the instance sample image with the pre-collected site image to obtain a first image.
[0115] In one embodiment, the fusion unit 702 includes:
[0116] The modification subunit is used to modify the size of the instance sample image.
[0117] The first fusion subunit is used to fuse the resized instance sample image with the pre-collected site image to obtain a first image.
[0118] In one embodiment, the fusion unit 702 includes:
[0119] The processing subunit is used to perform horizontal flipping and / or rotation processing on the example sample image.
[0120] The second fusion subunit is used to fuse the instance sample image after horizontal flipping or rotation processing with the pre-collected site image to obtain a first image.
[0121] The migration unit 703 is used to input the first picture into a scene migration model pre-constructed based on a target domain picture set, perform scene migration processing on the first picture, and obtain a second picture whose scene is consistent with the picture scene in the target domain picture set; the target domain picture set is composed of pictures pre-collected in the same scene, and the scenes include at least daytime scenes, night scenes, sunny scenes and rainy scenes.
[0122] The first constructing unit 704 is configured to construct a picture data set based on the first picture and the second picture.
[0123] In one embodiment, the data set expansion device further includes:
[0124] The third construction unit is used to construct a scene migration model including a Cycle GAN network, input the original domain pictures in the original domain picture set and the target domain pictures in the corresponding target domain picture set into the scene migration model, and construct a mapping network between the original domain picture set and the target domain picture set in the scene migration model, the original domain picture set is composed of pictures pre-collected in the same scene, and the target domain picture set is composed of pictures pre-collected in the same scene.
[0125] The scenes include at least daytime scenes, nighttime scenes, sunny scenes and rainy scenes; when the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.
[0126] Based on the apparatus for expanding a data set disclosed in the above-mentioned embodiment of the present invention, a picture containing an instance sample is input into a pre-built semantic segmentation model for extraction to obtain an instance sample picture, the instance sample picture is fused with a scene picture to obtain a first picture, and then a pre-built scene migration model is used to perform scene migration on the first picture to obtain a second picture, and a picture data set is constructed based on the first picture and the second picture, i.e., an expanded data set. In this solution, an instance sample picture is extracted using a pre-built semantic segmentation model, and then the instance sample picture is fused with other pictures and scene migration is performed to achieve the purpose of extracting instance samples and completing data expansion at the instance sample level.
[0127] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0128] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0129] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for expanding a data set, characterized in that: include: Input the image containing the instance sample into the pre-built semantic segmentation model for extraction to obtain the instance sample image; Merging the example sample image with a pre-collected site image to obtain a first image; Inputting the first picture into a scene migration model pre-constructed based on a target domain picture set, performing scene migration processing on the first picture, and obtaining a second picture whose scene is consistent with the scene of the picture in the target domain picture set; the target domain picture set is composed of pictures pre-collected in the same scene, and the scene includes at least daytime scene, night scene, sunny scene and rainy scene; A picture dataset is constructed based on the first picture and the second picture.
2. The method according to claim 1, characterized in that The process of constructing the semantic segmentation model includes: Build the semantic segmentation model to be trained; Inputting a training picture pre-labeled with instance annotation information into the semantic segmentation model, extracting and outputting instance sample pictures from the training picture based on the instance annotation information using the semantic segmentation model, and obtaining a trained semantic segmentation model; Inputting a test image not labeled with the instance annotation information into the trained semantic segmentation model; If the trained semantic segmentation model correctly outputs the instance sample image, it is determined that the training of the semantic segmentation model is completed; If the trained semantic segmentation model cannot correctly output the instance sample picture, continue to input the training picture into the semantic segmentation model for training until the test picture is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample picture.
3. The method according to claim 1, characterized in that The step of fusing the example sample image with a pre-collected site image to obtain a first image includes: Modify the size of the example sample picture; The resized instance sample image is fused with the pre-collected site image to obtain a first image.
4. The method according to claim 1, characterized in that: The step of fusing the example sample image with a pre-collected site image to obtain a first image includes: Performing horizontal flipping and / or rotation processing on the example sample image; The instance sample image after horizontal flipping or rotation processing is fused with the pre-collected site image to obtain a first image.
5. The method according to any one of claims 1 to 4, characterized in that: The construction process of the scene migration model includes: Build a scene migration model including the Cycle GAN network; Inputting original domain pictures in an original domain picture set and target domain pictures in a corresponding target domain picture set into the scene migration model, and constructing a mapping network between the original domain picture set and the target domain picture set in the scene migration model; the original domain picture set is composed of pictures pre-collected in the same scene; the target domain picture set is composed of pictures pre-collected in the same scene, and the scene at least includes daytime scene, night scene, sunny scene and rainy scene; When the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.
6. A data set expansion device, characterized in that: include: An extraction unit, used for inputting a picture containing an instance sample into a pre-built semantic segmentation model for extraction to obtain an instance sample picture; A fusion unit, configured to fuse the example sample image with a pre-collected site image to obtain a first image; a migration unit, configured to input the first picture into a scene migration model pre-constructed based on a target domain picture set, perform scene migration processing on the first picture, and obtain a second picture whose scene is consistent with the scene of the pictures in the target domain picture set; the target domain picture set is composed of pictures pre-collected in the same scene, and the scenes include at least daytime scenes, night scenes, sunny scenes, and rainy scenes; The first construction unit is configured to construct a picture dataset based on the first picture and the second picture.
7. The device according to claim 6, characterized in that Also includes: The second construction unit is used to construct a semantic segmentation model to be trained; input a training picture pre-annotated with instance annotation information into the semantic segmentation model, and use the semantic segmentation model to extract and output instance sample pictures in the training picture based on the instance annotation information to obtain a trained semantic segmentation model; input a test picture not annotated with the instance annotation information into the trained semantic segmentation model; if the trained semantic segmentation model correctly outputs the instance sample picture, it is determined that the training of the semantic segmentation model is completed; if the trained semantic segmentation model cannot correctly output the instance sample picture, continue to input the training picture into the semantic segmentation model for training until the test picture is input into the semantic segmentation model and the semantic segmentation model correctly outputs the instance sample picture.
8. The device according to claim 6, characterized in that The fusion unit comprises: A modification subunit, used to modify the size of the example sample picture; The first fusion subunit is used to fuse the resized instance sample image with the pre-collected site image to obtain a first image.
9. The device according to claim 6, characterized in that The fusion unit comprises: A processing subunit, configured to perform horizontal flipping and / or rotation processing on the example sample image; The second fusion subunit is used to fuse the instance sample image after horizontal flipping or rotation processing with the pre-collected site image to obtain a first image.
10. The device according to any one of claims 6 to 9, characterized in that: Also includes: The third construction unit is used to construct a scene migration model including a Cycle GAN network; The original domain pictures in the original domain picture set and the target domain pictures in the corresponding target domain picture set are input into the scene migration model, and a mapping network between the original domain picture set and the target domain picture set is constructed in the scene migration model; the original domain picture set is composed of pictures pre-collected in the same scene; the target domain picture set is composed of pictures pre-collected in the same scene, and the scenes at least include daytime scenes, night scenes, sunny scenes and rainy scenes; when the scene of the picture output by the scene migration model is consistent with the scene of the target domain picture, it is determined that the construction of the scene migration model is completed.