Method, system, computer device and storage medium for increasing difficult sample data

Abstract

The application discloses a method and system for increasing difficult sample data, a computer device and a storage medium, and relates to the technical field of data recognition.The method comprises the following steps: obtaining an original training set with multiple simple sample images and difficult sample images; training an initial segmentation model by using the original training set to obtain a target segmentation model; screening the original training set to obtain difficult sample images, and training the target segmentation model by using the difficult sample images to obtain an optimized target segmentation model; inputting a target image into the optimized target segmentation model for identification, and judging whether the identification result is a difficult sample image; and if the identification result is a difficult sample image, saving the target image as a new difficult sample image in the original training set. The application trains a conventional initial segmentation model to obtain a target segmentation model for identifying difficult sample data, realizes the rapid identification of difficult sample data, improves the identification efficiency, and reduces the acquisition cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a method and system for increasing difficult sample data, a computer device and a storage medium. BACKGROUND

[0002] With the rapid development of video special effects, various instance segmentation gradually enters the public view. At present, various instance segmentation methods are mainly based on semantic and image matting. Both methods have good performance for regular scenes, but in actual application, the problem of difficult sample scarcity is often encountered.

[0003] To solve the problem of difficult sample scarcity, there are currently two solutions. One is to modify the model structure by adding a detailed processing branch to solve the problem of poor segmentation effect of a certain type of sample. The other is to increase the number of data annotations of difficult samples to improve the accuracy of the model on difficult samples. The solution of modifying the model and adding a detailed processing branch often increases the inference time of the model, making it difficult to meet the real-time processing requirements in real-time video segmentation tasks. Increasing the number of data annotations of difficult samples often brings higher development costs. For example, in the case of scarce fine annotation data, the cost of annotating a fine picture is about 7-20 yuan. SUMMARY

[0004] The embodiments of the present application provide a method and system for increasing difficult sample data, a computer device and a storage medium, aiming to solve the problem of high cost and long time consumption of difficult sample data acquisition in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a method for increasing difficult sample data, comprising:

[0006] obtaining an original training set with multiple simple sample images and difficult sample images, training an initial segmentation model using the original training set to obtain a target segmentation model;

[0007] screening the original training set to obtain difficult sample images, and training the target segmentation model using the difficult sample images to obtain an optimized target segmentation model;

[0008] inputting a target image into the optimized target segmentation model for recognition, and determining whether the recognition result is a difficult sample image;

[0009] if the recognition result is a difficult sample image, saving the target image as a new difficult sample image in the original training set.

[0010] In a second aspect, the embodiments of the present application provide a system for increasing difficult sample data, comprising:

[0011] The target segmentation model acquisition unit is configured to acquire an original training set including multiple simple sample images and difficult sample images, train an initial segmentation model by using the original training set, and obtain a target segmentation model.

[0012] The target segmentation model optimization unit is configured to screen the original training set to obtain difficult sample images, and train the target segmentation model by using the difficult sample images to obtain an optimized target segmentation model.

[0013] The difficult sample image identification unit is configured to input a target image into the optimized target segmentation model for identification, and determine whether the identification result is a difficult sample image.

[0014] The original training set updating unit is configured to save the target image as a new difficult sample image in the original training set if the identification result is a difficult sample image.

[0015] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for increasing difficult sample data according to the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program causes a processor to execute the method for increasing difficult sample data according to the first aspect when the processor executes the computer program.

[0017] The method for increasing difficult sample data, the system, the computer device, and the storage medium provided by the embodiments of the present application include: acquiring an original training set including multiple simple sample images and difficult sample images, training an initial segmentation model by using the original training set, and obtaining a target segmentation model; screening the original training set to obtain difficult sample images, and training the target segmentation model by using the difficult sample images to obtain an optimized target segmentation model; inputting a target image into the optimized target segmentation model for identification, and determining whether the identification result is a difficult sample image; and saving the target image as a new difficult sample image in the original training set if the identification result is a difficult sample image. The embodiments of the present application train a conventional initial segmentation model to obtain a target segmentation model for identifying difficult sample data, realize fast identification of difficult sample data, improve the identification efficiency, and reduce the acquisition cost. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0019] Figure 1 The flowchart of the method for increasing difficult sample data provided by the embodiment of the present application is shown in the figure.

[0020] Figure 2 The sub-flowchart of the method for increasing difficult sample data provided by the embodiment of the present application is shown in the figure.

[0021] Figure 3 The sub-flowchart of the method for increasing difficult sample data provided by the embodiment of the present application is shown in the figure.

[0022] Figure 4 The sub-flowchart of the method for increasing difficult sample data provided by the embodiment of the present application is shown in the figure.

[0023] Figure 5 The sub-flowchart of the method for increasing difficult sample data provided by the embodiment of the present application is shown in the figure.

[0024] Figure 6 The schematic block diagram of the system for increasing difficult sample data provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the present application.

[0026] It should be understood that the terms "include" and "contain" as used in the specification and the appended claims indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or collections thereof.

[0027] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should be further understood that the term "and / or" used in the description and claims of the application herein is used to mean any one and / or any combination of the associated listed items in the combination, and includes all possible combinations.

[0029] Referring to Figure 1 , Figure 1 A flowchart of a method for increasing difficult sample data is provided for an embodiment of the application. The method comprises steps S101-S104.

[0030] S101, obtaining an original training set having multiple simple sample images and difficult sample images, training an initial segmentation model using the original training set, and obtaining a target segmentation model;

[0031] S102, screening the original training set to obtain difficult sample images, and training the target segmentation model using the difficult sample images to obtain an optimized target segmentation model;

[0032] S103, inputting a target image into the optimized target segmentation model for recognition, and determining whether the recognition result is a difficult sample image;

[0033] S104, if the recognition result is a difficult sample image, saving the target image as a new difficult sample image in the original training set.

[0034] In this embodiment, the initial segmentation model is first trained using the original training set to obtain a target segmentation model, and then difficult sample images are screened from the original training set to train the target segmentation model, and an optimized target segmentation model is obtained. The target image is recognized using the optimized target segmentation model, so as to screen the difficult sample images and save them in the original training set.

[0035] The original training set includes pre-labeled sample images and unlabeled sample images. When the initial segmentation model is trained by using the original training set, the pre-labeled sample images are input into the initial segmentation model for training, so as to obtain a target segmentation model. Then, difficult sample images that are labeled in the pre-labeled sample images are screened out, the target segmentation model is optimized, and the unlabeled sample images are input into the optimized target segmentation model for identification, so as to label according to the identification result and save to the original training set. The hair level refined image data is used as a difficult sample image. All the hair level refined image data is screened out from the labeled sample images in the original training set, to form a difficult sample data set. Then, the difficult sample data set is used to continue training the target segmentation model to obtain an optimized target segmentation model. Finally, the optimized target segmentation model is used to identify target images, and the hair level refined image data that meets the requirements is screened out from the identification result and added to the original training set as training data.

[0036] In an embodiment, referring to Figure 2 , the step S101 comprises:

[0037] S201, inputting the sample images in the original training set into a semantic estimation module for convolution processing, and calculating the loss of the semantic estimation module by using an L2 loss function, to obtain a semantic processing result;

[0038] S202, inputting the sample images in the original training set into a detail prediction module for encoding-decoding operation, and calculating the loss by using a cross-entropy loss function, to obtain a detail processing result;

[0039] S203, performing feature splicing on the semantic processing result and the detail processing result, to obtain a final segmentation result, and calculating the final segmentation result by using an L1 loss function, to obtain a target segmentation model.

[0040] In the embodiment, the sample images in the original training set are respectively input into the semantic estimation module and the detail prediction module for corresponding processing, and the respective losses are calculated, to obtain the semantic processing result and the detail processing result. Then, the semantic processing result and the detail processing result are subjected to feature splicing, to obtain a final segmentation result. Finally, the L1 loss function is used for convergence, to obtain a target segmentation model.

[0041] In an embodiment, referring to Figure 3 , the step S201 comprises:

[0042] S301, inputting the sample images in the original training set into a plurality of layers of continuous convolution layers for convolution, to obtain the convolution result of each layer;

[0043] S302, input the convolution result of the last layer to a global pooling layer for pooling processing, and input the pooling result to two continuous fully connected layers for convolution, and finally input to a sigmod layer for activation processing to obtain a first feature map, and perform feature addition on the first feature map and the convolution result of the last layer to obtain a target feature map;

[0044] S303, perform feature multiplication processing on the target feature map and the convolution result of each layer, and perform convolution processing on the feature multiplication result to obtain a semantic processing result.

[0045] In this embodiment, after inputting the sample images in the original training set into a plurality of continuous convolution layers for convolution processing, performing pooling processing on the convolution result of the last layer, inputting the pooling result into a fully connected layer for processing, and finally activating by using a sigmod function to obtain a first feature map, the first feature map is added with the convolution result of the last layer to obtain a target feature map; then the target feature map is multiplied with the convolution result of each layer, and the feature multiplication result is convolved to obtain a semantic processing result.

[0046] In an embodiment, referring to Figure 4 , the step S202 comprises:

[0047] S401, performing down-sampling processing on the sample images in the original training set, and performing feature splicing on the down-sampling result and the convolution result of the last layer to obtain a first splicing result;

[0048] S402, inputting the first splicing result into a plurality of continuous encoding layers for encoding processing to obtain an encoded feature map;

[0049] S403, up-sampling the semantic processing result, performing feature splicing on the up-sampling result and the encoded feature map to obtain a second splicing result, and inputting the second splicing result into a plurality of continuous decoding layers for feature decoding to obtain a feature decoding map;

[0050] S404, inputting the feature decoding map and the first splicing result into a convolution layer for convolution to obtain a detail processing result.

[0051] In the embodiment, the input sample image is first down-sampled, and the down-sampled result is spliced with the convolution result of the last layer to obtain a first spliced result; then the first spliced result is input into an encoding unit with multiple continuous encoding layers for feature encoding to obtain a corresponding encoding feature map, which is spliced with the semantic processing result after up-sampling to obtain a second spliced result, and the second spliced result is taken as the input of a decoding unit with multiple continuous decoding layers for feature decoding to obtain a corresponding feature decoding map; then the feature decoding map is input into the convolution layer together with the first spliced result for convolution to obtain a detail processing result.

[0052] In an embodiment, referring to Figure 5 The splicing of the semantic processing result and the detail processing result to obtain the final segmentation result includes:

[0053] S501, input the semantic processing result into a convolution layer for convolution processing, up-sample the convolution result, and splice the up-sampled result with the feature decoding map to obtain a third spliced result;

[0054] S502, input the third spliced result into multiple continuous convolution layers for convolution processing to obtain the final segmentation result.

[0055] In the embodiment, the semantic processing result is first input into a convolution layer for convolution processing, then up-sampled, and the up-sampled result is spliced with the feature decoding map to obtain a third spliced result, and the third spliced result is subjected to multiple convolution processes to obtain the final segmentation result.

[0056] In a specific embodiment, the target segmentation model can also be obtained by the following steps:

[0057] The sample images in the original training set are input into 6 layers of continuous encoding layers for feature coding to obtain the coding feature maps corresponding to each layer of the encoding layers; the coding feature map of the last layer of the encoding layers and the coding feature map of the second last layer of the encoding layers are fused and then up-sampled, and the up-sampling result is input into the first layer of the decoding layers for feature decoding; the decoding feature map of the first layer of the decoding layers and the coding feature map of the third layer of the encoding layers are fused and then up-sampled, and the up-sampling result is input into the second layer of the decoding layers for feature decoding; the decoding feature map of the fourth layer of the decoding layers and the coding feature map of the first layer of the encoding layers are fused and then up-sampled, and the up-sampling result is input into the fifth layer of the decoding layers for feature decoding to obtain the decoding feature map of the fifth layer of the decoding layers; the coding feature map of the last layer of the encoding layers and the decoding feature map of each layer of the decoding layers are respectively up-sampled, fused, and activated by a sigmoid function, and then up-sampled to obtain a target feature map; finally, the target feature map is trained by back propagation using an L1 loss function to obtain a target segmentation model.

[0058] Referring to Figure 6 , Figure 6 A schematic block diagram of a system for increasing difficult sample data is provided for an embodiment of the present application. The system 200 for increasing difficult sample data comprises:

[0059] A target segmentation model acquisition unit 201 is configured to acquire an original training set having a plurality of simple sample images and difficult sample images, train an initial segmentation model using the original training set, and obtain a target segmentation model.

[0060] A target segmentation model optimization unit 202 is configured to filter the original training set to obtain difficult sample images, and train the target segmentation model using the difficult sample images to obtain an optimized target segmentation model.

[0061] A difficult sample image identification unit 203 is configured to input a target image into the optimized target segmentation model for identification, and determine whether the identification result is a difficult sample image.

[0062] An original training set updating unit 204 is configured to save the target image as a new difficult sample image in the original training set if the identification result is a difficult sample image.

[0063] In an embodiment, the target segmentation model acquisition unit 201 comprises:

[0064] The semantic processing result acquisition unit is configured to input the sample image in the original training set into a semantic estimation module for convolution processing, and calculate a loss of the semantic estimation module by using an L2 loss function to obtain a semantic processing result.

[0065] The detail processing result acquisition unit is configured to input the sample image in the original training set into a detail prediction module for encoding-decoding operation, and calculate a loss by using a cross-entropy loss function to obtain a detail processing result.

[0066] The feature splicing unit is configured to splice the semantic processing result and the detail processing result to obtain a final segmentation result, and calculate the final segmentation result by using an L1 loss function to obtain a target segmentation model.

[0067] In an embodiment, the semantic processing result acquisition unit comprises:

[0068] The multi-layer convolution result acquisition unit is configured to input the sample image in the original training set into a plurality of continuous convolution layers for convolution to obtain a convolution result of each layer.

[0069] The target feature map acquisition unit is configured to input the convolution result of the last layer into a global pooling layer for pooling processing, and input the pooled result into two continuous fully connected layers for convolution, and finally input into a sigmod layer for activation processing to obtain a first feature map, and perform feature addition on the first feature map and the convolution result of the last layer to obtain a target feature map.

[0070] The feature multiplication processing unit is configured to perform feature multiplication processing on the target feature map and the convolution result of each layer, and perform convolution processing on the feature multiplication result to obtain a semantic processing result.

[0071] In an embodiment, the detail processing result acquisition unit comprises:

[0072] The first splicing result acquisition unit is configured to perform down-sampling processing on the sample image in the original training set, and perform feature splicing on the down-sampled result and the convolution result of the last layer to obtain a first splicing result.

[0073] The encoded feature map acquisition unit is configured to input the first splicing result into a plurality of continuous encoding layers for encoding processing to obtain an encoded feature map.

[0074] The feature decoding map acquisition unit is configured to perform up-sampling on the semantic processing result, perform feature splicing on the up-sampled result and the encoded feature map to obtain a second splicing result, and input the second splicing result into a plurality of continuous decoding layers for feature decoding to obtain a feature decoding map.

[0075] A convolution processing unit is configured to input the feature decoding graph and the first splicing result into a convolution layer to perform convolution to obtain a detail processing result.

[0076] In an embodiment, the feature splicing unit comprises:

[0077] A feature splicing unit is configured to input the semantic processing result into a convolution layer to perform convolution processing, to perform up-sampling processing on the convolution result, and to perform feature splicing on the up-sampling result and the feature decoding graph to obtain a third splicing result.

[0078] A segmentation result acquisition unit is configured to input the third splicing result into a plurality of continuous convolution layers to perform convolution processing to obtain a final segmentation result.

[0079] The embodiment of the present application also provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor implements the method for increasing difficult sample data when the computer program is executed.

[0080] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for increasing difficult sample data.

[0081] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part. It should be pointed out that, for those skilled in the art, without departing from the principle of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0082] It should also be noted that in the specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

Claims

1. A method for increasing difficult sample data, characterized in that, The application relates to a method for training a segmentation model. The method comprises the following steps: acquiring an original training set with multiple simple sample images and difficult sample images, training an initial segmentation model by using the original training set, and obtaining a target segmentation model; the original training set comprises pre-labeled sample images and unlabeled sample images, and the images trained in the initial segmentation model are pre-labeled sample images; difficult sample images are obtained by screening the original training set, and the target segmentation model is trained by using the difficult sample images, so as to obtain an optimized target segmentation model; the difficult sample images are labeled difficult sample images screened from the pre-labeled sample images; a target image is input into the optimized target segmentation model for identification, and it is judged whether the identification result is a difficult sample image; if the identification result is a difficult sample image, the target image is saved as a new difficult sample image in the original training set; the original training set is input into a semantic estimation module for convolution processing, and a loss of the semantic estimation module is calculated by using an L2 loss function, so as to obtain a semantic processing result; the sample images in the original training set are input into a detail prediction module for encoding-decoding operation, and a loss is calculated by using a cross-entropy loss function, so as to obtain a detail processing result; the semantic processing result and the detail processing result are spliced to obtain a final segmentation result, and the final segmentation result is calculated by using an L1 loss function, so as to obtain the target segmentation model; 2. The method of claim 1, wherein, the sample images in the original training set are input into a plurality of continuous convolution layers for convolution, so as to obtain the convolution result of each layer; the convolution result of the last layer is input into a global pooling layer for pooling processing, and the pooling result is input into two continuous fully-connected layers for convolution, and finally input into a sigmod layer for activation processing, so as to obtain a first feature map; the first feature map and the convolution result of the last layer are added in feature, so as to obtain a target feature map; the target feature map and the convolution result of each layer are multiplied in feature, and the feature multiplication result is convolved, so as to obtain the semantic processing result. the sample images in the original training set are down-sampled, and the down-sampling result and the convolution result of the last layer are spliced in feature, so as to obtain a first splicing result; the first splicing result is input into a plurality of continuous encoding layers for encoding processing, so as to obtain an encoding feature map; the semantic processing result is up-sampled, and the up-sampling result and the encoding feature map are spliced in feature, so as to obtain a second splicing result which is input into a plurality of continuous decoding layers for feature decoding, so as to obtain a feature decoding map; and the feature decoding map is up-sampled, and the up-sampling result and the target feature map are spliced in feature, so as to obtain a final feature map. The feature decoding graph and the first splicing result are input into a convolution layer for convolution to obtain a detail processing result.

3. The method of claim 2, wherein the difficult sample data is increased by, The semantic processing result and the detail processing result are spliced to obtain a final segmentation result, and the final segmentation result is obtained. The semantic processing result is input into a convolution layer for convolution processing, and the convolution result is up-sampled, and the up-sampled result is spliced with a feature decoding graph to obtain a third splicing result. The third splicing result is input into a plurality of continuous convolution layers for convolution processing to obtain a final segmentation result.

4. A system for increasing difficult sample data, the system comprising: It comprises: A target segmentation model acquisition unit is configured to acquire an original training set including a plurality of simple sample images and difficult sample images, train an initial segmentation model using the original training set, and obtain a target segmentation model. The original training set includes pre-labeled sample images and unlabeled sample images, and the initial segmentation model is trained using pre-labeled sample images. A target segmentation model optimization unit is configured to filter difficult sample images from the original training set, and train the target segmentation model using the difficult sample images to obtain an optimized target segmentation model. The difficult sample images are pre-labeled difficult sample images selected from the pre-labeled sample images. A difficult sample image identification unit is configured to input a target image into the optimized target segmentation model for identification, and determine whether the identification result is a difficult sample image. An original training set update unit is configured to save the target image as a new difficult sample image in the original training set if the identification result is a difficult sample image. The target segmentation model acquisition unit comprises: A semantic processing result acquisition unit is configured to input a sample image in the original training set into a semantic estimation module for convolution processing, calculate the loss of the semantic estimation module using an L2 loss function, and obtain a semantic processing result. A detail processing result acquisition unit is configured to input a sample image in the original training set into a detail prediction module for encoding-decoding operation, calculate the loss using a cross-entropy loss function, and obtain a detail processing result. A feature splicing unit is configured to feature splice the semantic processing result and the detail processing result to obtain a final segmentation result, and calculate the final segmentation result using an L1 loss function to obtain a target segmentation model. The semantic processing result acquisition unit comprises: A multi-layer convolution result acquisition unit is configured to input a sample image in the original training set into a plurality of continuous convolution layers for convolution to obtain a convolution result of each layer. A target feature map acquisition unit is configured to input the convolution result of the last layer into a global pooling layer for pooling processing, input the pooled result into two continuous fully connected layers for convolution, and finally input into a sigmod layer for activation processing to obtain a first feature map. The first feature map is added to the convolution result of the last layer to obtain a target feature map. A feature multiplication processing unit is configured to multiply the target feature map with the convolution result of each layer, and convolve the feature multiplication result to obtain a semantic processing result.

5. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method for increasing difficult sample data according to any one of claims 1 to 3 when executing the computer program.

6. A computer readable storage medium characterized by The computer readable storage medium stores a computer program, and the computer program causes the processor to execute the method for increasing difficult sample data according to any one of claims 1 to 3 when the processor executes the computer program.

Images