Model training method and device, storage medium and electronic device

By generating expanded fault image samples in photovoltaic image fault detection, the training accuracy is improved using deep learning models, the low-precision problem caused by the scarcity of fault image samples is solved, and more efficient photovoltaic fault recognition is achieved.

CN119006954APending Publication Date: 2024-11-22HUANENG JILIN POWER GENERATION JIUTAI ELECTRIC FACTORY +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411094279.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The deep learning image recognition model has low training accuracy in photovoltaic image failure detection, mainly because the faulty image samples are far fewer than normal image samples, resulting in poor training results of data-driven model.

Method used

By generating an expanded fault image sample in the training set, the method includes extracting the foreground image of each image and adding a noise image to generate a third training set for training the target model and improving the training accuracy of the model.

Benefits of technology

By augmenting the combination of faulty image samples and normal image samples, the training accuracy of deep learning image recognition models is improved, and the low-precision problem caused by the scarcity of faulty image samples is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006954A_ABST
    Figure CN119006954A_ABST
Patent Text Reader

Abstract

The invention discloses a model training method and device, a storage medium and an electronic device.The method comprises the steps that a second training set is determined in a first training set, and the second training set comprises a plurality of first images; respectively extracting a foreground image corresponding to each first image, and adding each first noise image into the foreground image corresponding to each first image to generate a third training set; and training a target model according to the first training set and the third training set. Through the method, the problem that the training precision of a deep learning image recognition model is low due to the fact that fault image samples are far less than normal image samples in the prior art can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to a model training method and device, a storage medium, and an electronic device. Background Art

[0002] With the rapid increase in the proportion of photovoltaic power generation, it is increasingly important to realize unmanned fault inspection of photovoltaic stations. An important way to realize unmanned fault inspection is to collect images of photovoltaic equipment, analyze and process them based on artificial intelligence technology, and realize image-based photovoltaic fault identification.

[0003] However, since the number of fault image samples is far less than that of normal image samples, and the cost of manually labeling fault images is high, the scarce sample size and unbalanced ratio of abnormal and normal image samples are very unfavorable for the training of data-driven deep learning image recognition models. Therefore, although deep learning image algorithms have obvious technical advantages over traditional feature engineering methods, they still face difficulties in the practical application of photovoltaic image fault detection.

[0004] There is no effective solution to the problem of low training accuracy of deep learning image recognition models in related technologies due to the fact that the number of fault image samples is far less than that of normal image samples. Summary of the invention

[0005] The embodiments of the present application relate to a model training method and device, a storage medium and an electronic device. Through the embodiments of the present application, the problem of low training accuracy of deep learning image recognition models in related technologies caused by the fact that the number of faulty image samples is far less than that of normal image samples can be solved.

[0006] According to an embodiment of the embodiments of the present application, a model training method is provided, comprising: determining a second training set from a first training set, wherein the second training set includes: multiple first images; extracting foreground images corresponding to each first image respectively, and adding each first noise image to the foreground image corresponding to each first image to generate a third training set; training a target model according to the first training set and the third training set.

[0007] In an exemplary embodiment, respectively extracting the foreground image corresponding to each first image includes: extracting the foreground image corresponding to each first image by one of the following methods: extracting the foreground image corresponding to each first image by a pixel threshold corresponding to each first image; extracting the foreground image corresponding to each first image by a foreground extraction segmentor.

[0008] In an exemplary embodiment, extracting a foreground image corresponding to each first image by using a pixel threshold corresponding to each first image includes: setting a pixel threshold corresponding to each first image; determining a second image that is less than or equal to the pixel threshold in each first image; and determining each second image as a foreground image corresponding to each first image.

[0009] In an exemplary embodiment, extracting a foreground image corresponding to each first image by a foreground extraction segmentor includes: pre-training the foreground extraction segmentor by a general image semantic segmentation dataset; inputting each first image into the pre-trained foreground extraction segmentor so that the pre-trained foreground extraction segmentor outputs the foreground image corresponding to each first image.

[0010] In an exemplary embodiment, before adding each first noise image to the foreground image corresponding to each first image, the method further includes: generating a plurality of second noise images by a noise generator; binarizing each second noise image to generate a plurality of third noise images; flipping each third noise image, and determining each flipped third noise image as each first noise image.

[0011] In an exemplary embodiment, each first noise image is added to a foreground image corresponding to each first image to generate a third training set, including: determining the coordinates of any position in the foreground image corresponding to each first image; inserting each first noise image into the position corresponding to the coordinates, and determining each first image inserted into each first noise image as an image in the third training set.

[0012] In an exemplary embodiment, training a target model according to the first training set and the third training set includes: pre-training the target model according to the first training set and the third training set to obtain a pre-trained target model; freezing parameters corresponding to the target network structure in the pre-trained target model, and training the target model with frozen parameters according to the first training set.

[0013] According to another embodiment of the embodiments of the present application, a model training device is also provided, including: a determination module, used to determine a second training set in a first training set, wherein the second training set includes: multiple first images; an extraction module, used to respectively extract the foreground image corresponding to each first image, and add each first noise image to the foreground image corresponding to each first image to generate a third training set; a training module, used to train the target model according to the first training set and the third training set.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0015] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to perform the method.

[0017] In an embodiment of the present application, a second training set including a plurality of first images is determined in the first training set; the foreground images corresponding to each first image are extracted respectively, and each first noise image is added to the foreground image corresponding to each first image to generate a third training set; and the target model is trained according to the first training set and the third training set. That is, in the embodiment of the present application, a plurality of first images are selected in the first training set, and each first noise image is added to the foreground image corresponding to each first image to generate each first image to which the first noise image is added, and the first image to which the first noise image is added is a fault image sample. Then, the target model is trained by the fault image samples and the images in the first training set, that is, the normal image samples. Through the embodiment of the present application, the first fault image is added to the foreground image corresponding to the first image to expand the fault image samples, which can solve the problem of low training accuracy of the deep learning image recognition model in the related art due to the fact that the fault image samples are far less than the normal image samples, and the training accuracy of the target model can be improved by training the target model with the expanded fault image samples and the original normal image samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal of a model training method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for training a model according to an embodiment of the present application;

[0021] Figure 3 is a comparison diagram of a fault image and a non-fault image according to an optional embodiment of the present application;

[0022] Figure 4 is a general framework diagram of a data-enhanced photovoltaic fault visual recognition method according to an optional embodiment of the present application;

[0023] Figure 5 is a flow chart of expanding negative samples in a training set according to an optional embodiment of the present application;

[0024] Figure 6 is a network architecture diagram of an anomaly identification network according to an optional embodiment of the present application;

[0025] Figure 7 It is a structural block diagram of a training device for a model according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices; "plurality" means two or more.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device or cloud platform or independent physical server or software platform, wherein the above software platform is run through one or more servers. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for a model training method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1Only one is shown in the figure) processor 102 and memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the methods in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned methods are implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a model training method is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flow chart of a method for training a model according to an embodiment of the present application, the process comprising the following steps:

[0032] Step S202, determining a second training set in the first training set, wherein the second training set includes: a plurality of first images;

[0033] The first training set may include: a plurality of normal image samples and a small number of fault image samples. Since the number of fault image samples in the related art is far less than that of normal image samples, a plurality of first images may be randomly determined in the first training set, and fault image samples may be generated based on the first images to solve the problem that the number of fault image samples is far less than that of normal image samples.

[0034] The first image in the second training set is a normal image sample.

[0035] Step S204, respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set;

[0036] Step S206: training a target model according to the first training set and the third training set.

[0037] Among them, the above-mentioned target model can be an image recognition model.

[0038] In an embodiment of the present application, a second training set including a plurality of first images is determined in the first training set; the foreground images corresponding to each first image are extracted respectively, and each first noise image is added to the foreground image corresponding to each first image to generate a third training set; and the target model is trained according to the first training set and the third training set. That is, in the embodiment of the present application, a plurality of first images are selected in the first training set, and each first noise image is added to the foreground image corresponding to each first image to generate each first image to which the first noise image is added, and the first image to which the first noise image is added is a fault image sample. Then, the target model is trained by the fault image samples and the images in the first training set, that is, the normal image samples. Through the embodiment of the present application, the first fault image is added to the foreground image corresponding to the first image to expand the fault image samples, which can solve the problem of low training accuracy of the deep learning image recognition model in the related art due to the fact that the fault image samples are far less than the normal image samples, and the training accuracy of the target model can be improved by training the target model with the expanded fault image samples and the original normal image samples.

[0039] Optionally, extracting the foreground image corresponding to each first image in step S204 includes: extracting the foreground image corresponding to each first image in one of the following ways:

[0040] (1) extracting a foreground image corresponding to each first image by using a pixel threshold corresponding to each first image; comprising:

[0041] A pixel threshold corresponding to each of the first images is set; a second image less than or equal to the pixel threshold is determined in each of the first images; and each of the second images is determined as a foreground image corresponding to each of the first images.

[0042] (2) Extracting a foreground image corresponding to each of the first images by a foreground extraction segmentor.

[0043] The foreground extraction segmentor is pre-trained using a general image semantic segmentation dataset; each of the first images is input into the pre-trained foreground extraction segmentor so that the pre-trained foreground extraction segmentor outputs a foreground image corresponding to each of the first images.

[0044] It can be understood that the foreground image corresponding to each first image can be determined by the above two methods, specifically:

[0045] (1) A pixel threshold corresponding to each first image can be set, and a portion of each first image that is less than or equal to the pixel threshold is determined as a foreground image corresponding to each first image, and a portion of each first image that is greater than the pixel threshold is determined as a background image or an irrelevant image corresponding to each first image.

[0046] (2) The foreground image segmentor can be pre-trained using a general image semantic segmentation dataset. The general image semantic segmentation dataset marks the targets to be identified as foreground images, and the rest as background images. The foreground segmentation extractor can use classic deep semantic segmentation models such as Fully Convolutional Network (FCN), U-type network (UNet), and Segmentation Network (SegNet). Then, the pre-trained foreground extraction segmentor is used with each first image as input to output the image area where the photovoltaic panel is located (i.e., the foreground image corresponding to each first image in the present application).

[0047] Optionally, before adding each first noise image to the foreground image corresponding to each first image in the above step S204, the method further includes: generating multiple second noise images by a noise generator; binarizing each second noise image to generate multiple third noise images; flipping each third noise image, and determining each flipped third noise image as each first noise image.

[0048] It is understandable that before adding each first noise image to the foreground image corresponding to each first image, each first noise image needs to be determined. Specifically, a plurality of second noise images need to be generated by a noise generator, and then each second noise image needs to be processed, for example, by binarization, flipping, etc., to obtain a plurality of first noise images.

[0049] Optionally, the above step S204 of adding each first noise image to the foreground image corresponding to each first image includes: determining the coordinates of any position in the foreground image corresponding to each first image; inserting each first noise image into the position corresponding to the coordinates, and determining each first image inserted into each first noise image as an image in the third training set.

[0050] It can be understood that the first noise image can be added to any position in the foreground image corresponding to each first image. By determining the coordinates of any position in the foreground image, each first noise image can be inserted into the position corresponding to the coordinates in the foreground image to generate an image in the third training set, i.e., a fault image sample.

[0051] Optionally, the above-mentioned step S206 of training the target model according to the first training set and the third training set includes: pre-training the target model according to the first training set and the third training set to obtain a pre-trained target model; freezing parameters corresponding to the target network structure in the pre-trained target model, and training the target model with frozen parameters according to the first training set.

[0052] It can be understood that after obtaining enough normal image samples and fault image samples, the target model can be trained according to the first training set and the second training set. Specifically:

[0053] The target model may be trained for the first time according to the first training set and the third training set, the parameters of the target network structure in the target model after the first training may be frozen, and then the target model may be trained for the second time through the images in the first training set.

[0054] It can be understood that the first training set may include: a plurality of normal image samples and a small number of fault image samples.

[0055] In order to better understand the process of the training method of the above-mentioned model, the training method of the above-mentioned model is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0056] The optional embodiment of the present application defines a data-enhanced photovoltaic fault visual recognition method, which can automatically generate abnormal photovoltaic images to expand the number of fault image samples in the training data set, so that the fault recognition model can obtain better abnormality recognition results, thereby improving the fault recognition accuracy.

[0057] Figure 3 is a comparison diagram of a fault image and a non-fault image according to an optional embodiment of the present application; Figure 3 As shown:

[0058] The main purpose of the optional embodiment of the present application is to train the image recognition model. First, it is necessary to generate photovoltaic images with abnormalities (i.e., images in the third training set of the present application). Figure 3 The input part is the normal photovoltaic image, and the output part is the abnormal photovoltaic image.

[0059] The data enhanced photovoltaic image fault identification method proposed in the optional embodiment of the present application is mainly divided into two parts:

[0060] The first part is an image enhancement method of foreground image extraction and random fault generation. By applying this image enhancement method, the number of fault image samples in the training data set can be automatically expanded, and an expanded data set expanded based on this method can be obtained.

[0061] The second part is the anomaly recognition pre-trained network and the fault detection network. First, the anomaly recognition pre-trained network is constructed and trained in the expanded data set. However, the main purpose of the image enhancement method is to enable the anomaly recognition pre-trained network to learn abnormal information that is inconsistent with normal photovoltaic images and the ability to identify abnormal information. Image enhancement cannot replace the actual photovoltaic fault image samples. Therefore, after obtaining the anomaly recognition pre-trained network, this part freezes some model parameters of the previously pre-trained anomaly recognition pre-trained network, and performs training and fine-tuning on actual photovoltaic fault image samples to finally achieve photovoltaic fault recognition.

[0062] Specifically, Figure 4 is a general framework diagram of a data-enhanced photovoltaic fault visual recognition method according to an optional embodiment of the present application, such as Figure 4 As shown:

[0063] (1) Part I: Randomly extract a certain number of samples (i.e., the second training set of the present application) from the normal samples in the original data set (i.e., the first training set of the present application), add noise to these samples to simulate photovoltaic failures, and then add the samples with added noise to the failure samples in the data set to achieve the expansion of negative samples in the training set. Specifically, Figure 5 is a flowchart of expanding negative samples in a training set according to an optional embodiment of the present application, such as Figure 5 As shown:

[0064] Step S501, extracting picture A from normal samples of the original data set;

[0065] Step S502, foreground extraction to obtain a photovoltaic image area (ie, a foreground image in this application);

[0066] Through foreground extraction, the image area where the photovoltaic panels are located is extracted. There are two foreground extraction methods: 1) Set a pixel threshold, and the image within a certain threshold range is the photovoltaic panel area, and the rest is the background or irrelevant image; 2) Pre-train a foreground extraction segmentor on a general image semantic segmentation dataset. In the general image semantic segmentation dataset, the targets to be identified are marked as foregrounds, and the rest are marked as backgrounds. The foreground segmentation extractor can use classic deep semantic segmentation models such as FCN, UNet, SegNet, and then use the pre-trained foreground extraction segmentor, take picture A as input, and output the image area where the photovoltaic panels are located.

[0067] Step S503, using a Perlin noise generator (Perlin noise generator, i.e., the noise generator of the present application) to generate a random noise texture image N (i.e., the second noise image of the present application);

[0068] Step S504, randomly scaling the image N and determining a random coordinate position (i.e., the coordinates of any position in the present application) in the photovoltaic image area;

[0069] Step S505, removing the mask from the image N, binarizing the image N, and then flipping the image N to obtain a corresponding label image (i.e., the first noise image of the present application);

[0070] A random noise texture image N is generated using a Perlin noise generator, and N is randomly scaled to 1 / 2 to 1 / 20 of the size of image A. Then, a coordinate position is randomly determined in the foreground area extracted in step S502, and the scaled noise image N is inserted into the random coordinate position of image A and added to the augmented data set as a fault sample.

[0071] It is also possible to generate a label image corresponding to the fault sample. The method for generating the label image is as follows: binarize the noise image N, and then flip it, that is, all pixels where the pixel is 0 become 1, and all pixels where the pixel is 1 become 0. Initialize a pure 0 pixel value image B of the same size as image A, and insert the flipped binary image N into the random coordinate position of image B. The resulting image is the label image corresponding to the generated fault sample.

[0072] Step S506, inserting image N into a random coordinate position of image A;

[0073] Step S507: adding the generated fault image and the corresponding label image (the fault image and the label image are both fault image samples) to the expanded data set.

[0074] (2) Part II: Building an anomaly recognition network (ADNet, the target model of this application), Figure 6 is a network architecture diagram of an abnormality identification network according to an optional embodiment of the present application, such as Figure 6 As shown:

[0075] The anomaly recognition network includes: convolution layer, deconvolution layer, etc.

[0076] Table 1 is a detailed structure of an abnormality recognition network according to an optional embodiment of the present application, as shown in Table 1:

[0077] Table 1

[0078]

[0079]

[0080] ADNet is trained on the expanded data set to obtain the pre-trained anomaly recognition network Pre-ADNet. Subsequently, the parameters of the network structure of convolutional layer 1 to convolutional layer 6 in Table 1 (i.e., the target network structure of the present application) are frozen. Then, the original data set (i.e., the first training set of the present application) is used to retrain and fine-tune Pre-ADNet, and a photovoltaic fault detection network is obtained after training. The photovoltaic fault detection network can realize the fault area recognition of photovoltaic images.

[0081] An optional embodiment of the present application proposes a data-enhanced photovoltaic image fault recognition method, designs an image enhancement method for foreground image extraction and random generation of anomalies, which can automatically generate photovoltaic images with anomalies to expand the number of fault image samples in the training data set; and proposes a method for pre-training anomaly recognition network in the expanded data set, then freezing the feature extraction layer, and re-training and fine-tuning the upsampling layer in the original data set to obtain better photovoltaic fault recognition results.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0083] The present application also provides a structural block diagram of a model training device. Figure 7 is a structural block diagram of a training device for a model according to an embodiment of the present application; Figure 7 As shown, including:

[0084] A determination module 72 is configured to determine a second training set from the first training set, wherein the second training set includes: a plurality of first images;

[0085] An extraction module 74, configured to respectively extract a foreground image corresponding to each first image, and add each first noise image to the foreground image corresponding to each first image, so as to generate a third training set;

[0086] The training module 76 is used to train the target model according to the first training set and the third training set.

[0087] Through the above device, a second training set including multiple first images is determined in the first training set; the foreground images corresponding to each first image are extracted respectively, and each first noise image is added to the foreground image corresponding to each first image to generate a third training set; the target model is trained according to the first training set and the third training set. That is to say, the embodiment of the present application selects multiple first images in the first training set, and each first noise image is added to the foreground image corresponding to each first image to generate each first image to which the first noise image is added, and the first image to which the first noise image is added is the fault image sample. Then, the target model is trained by the fault image samples and the images in the first training set, that is, the normal image samples. Through the embodiment of the present application, the first fault image is added to the foreground image corresponding to the first image to expand the fault image samples, which can solve the problem of low training accuracy of the deep learning image recognition model in the related art due to the fact that the fault image samples are far less than the normal image samples. The training accuracy of the target model can be improved by training the target model with the expanded fault image samples and the original normal image samples.

[0088] In an exemplary embodiment, the extraction module 74 is also used to extract the foreground image corresponding to each first image by one of the following methods: extracting the foreground image corresponding to each first image by a pixel threshold corresponding to each first image; extracting the foreground image corresponding to each first image by a foreground extraction segmentor.

[0089] In an exemplary embodiment, the extraction module 74 is further used to set a pixel point threshold corresponding to each of the first images; determine a second image that is less than or equal to the pixel point threshold in each of the first images; and determine each second image as a foreground image corresponding to each of the first images.

[0090] In an exemplary embodiment, the extraction module 74 is also used to pre-train the foreground extraction segmentor using a general image semantic segmentation dataset; input each of the first images into the pre-trained foreground extraction segmentor so that the pre-trained foreground extraction segmentor outputs a foreground image corresponding to each of the first images.

[0091] In an exemplary embodiment, the extraction module 74 is further used to generate a plurality of second noise images through a noise generator; binarize each second noise image to generate a plurality of third noise images; flip each third noise image, and determine each flipped third noise image as each first noise image.

[0092] In an exemplary embodiment, the extraction module 74 is also used to determine the coordinates of any position in the foreground image corresponding to each first image; insert each first noise image into the position corresponding to the coordinates, and determine each first image inserted into each first noise image as an image in the third training set.

[0093] In an exemplary embodiment, the training module 76 is also used to pre-train the target model according to the first training set and the third training set to obtain a pre-trained target model; freeze the parameters corresponding to the target network structure in the pre-trained target model, and train the target model with frozen parameters according to the first training set.

[0094] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0095] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0096] S1, determining a second training set in the first training set, wherein the second training set includes: a plurality of first images;

[0097] S2, respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set;

[0098] S3: Train a target model according to the first training set and the third training set.

[0099] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0100] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0101] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0102] S1, determining a second training set in the first training set, wherein the second training set includes: a plurality of first images;

[0103] S2, respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set;

[0104] S3: Train a target model according to the first training set and the third training set.

[0105] An embodiment of the present application further provides a computer program product, including a computer program, which is used by a processor to execute the steps in any of the above method embodiments.

[0106] Optionally, in this embodiment, the computer program product may be executed by a processor to perform the following steps:

[0107] S1, determining a second training set in the first training set, wherein the second training set includes: a plurality of first images;

[0108] S2, respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set;

[0109] S3: Train a target model according to the first training set and the third training set.

[0110] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0111] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0112] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0113] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A model training method, characterized in that: include: Determine a second training set in the first training set, wherein the second training set includes: a plurality of first images; Respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set; A target model is trained according to the first training set and the third training set.

2. The training method of the model according to claim 1, characterized in that: Extracting the foreground image corresponding to each first image respectively includes: Extract the foreground image corresponding to each first image by one of the following methods: Extracting a foreground image corresponding to each first image by using a pixel threshold corresponding to each first image; The foreground image corresponding to each first image is extracted by a foreground extraction segmentor.

3. The training method of the model according to claim 2, characterized in that: Extracting a foreground image corresponding to each first image by using a pixel threshold corresponding to each first image includes: Setting a pixel threshold corresponding to each first image; Determine a second image in each of the first images that is less than or equal to the pixel threshold; Each second image is determined as a foreground image corresponding to each first image.

4. The training method of the model according to claim 2, characterized in that: Extracting a foreground image corresponding to each first image by a foreground extraction segmentor includes: Pre-training the foreground extraction segmentor using a general image semantic segmentation dataset; Each of the first images is input into a pre-trained foreground extraction segmentor, so that the pre-trained foreground extraction segmentor outputs a foreground image corresponding to each of the first images.

5. The training method of the model according to claim 1, characterized in that: Before adding each first noise image to the foreground image corresponding to each first image, the method further includes: generating a plurality of second noise images by a noise generator; Each second noise image is binarized to generate a plurality of third noise images; each third noise image is flipped, and each flipped third noise image is determined as each first noise image.

6. The training method of the model according to claim 1, characterized in that: Adding each first noise image to a foreground image corresponding to each first image to generate a third training set includes: Determining the coordinates of any position in the foreground image corresponding to each of the first images; Each of the first noise images is inserted into a position corresponding to the coordinates, and each first image inserted into the first noise image is determined as an image in the third training set.

7. The model training method according to claim 1, characterized in that: Training a target model according to the first training set and the third training set includes: Pre-training the target model according to the first training set and the third training set to obtain a pre-trained target model; Freeze the parameters corresponding to the target network structure in the pre-trained target model, and train the target model with frozen parameters according to the first training set.

8. A model training device, characterized in that: include: A determination module, configured to determine a second training set in the first training set, wherein the second training set includes: a plurality of first images; An extraction module, used for respectively extracting a foreground image corresponding to each first image, and adding each first noise image to the foreground image corresponding to each first image, so as to generate a third training set; A training module is used to train a target model according to the first training set and the third training set.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.