Image generation method, training method, device, equipment and storage medium
By using a noise prediction network and an iterative generation method of Gaussian noise, image data other than those with existing labels was generated, which solves the limitations of existing image generation models and improves the robustness and accuracy of image processing models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU KEDA TECH
- Filing Date
- 2023-03-24
- Publication Date
- 2026-05-29
AI Technical Summary
Existing image generation models struggle to generate images beyond those already labeled, failing to meet the learning needs of practical applications and resulting in insufficient robustness and accuracy of image processing models.
Image data is generated iteratively by using a noise prediction network and Gaussian noise. The noise vector sampled from the second Gaussian noise is used as the identity identifier of the target image label by utilizing the number of input feature channels of the feature extraction layer, thus generating image data with an image label set.
The labeling of image data has been expanded, which improves the robustness and accuracy of image processing models, especially in target recognition and target tracking tasks.
Smart Images
Figure CN116468673B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more particularly to an image generation method, training method, apparatus, device, and storage medium. Background Technology
[0002] In the field of image processing, to improve the robustness and accuracy of image processing models such as object recognition and object tracking, a large number of sample images of different types are needed to train the image processing models. Since the number of images in image libraries is limited, and there may be obstacles to real-time capture and acquisition of large numbers of images, some current image generation models can generate multiple images based on existing labels.
[0003] However, in practical applications of image processing models, there is often a need to learn from sample images other than those with existing labels, making it difficult for existing image generation models to meet these needs.
[0004] Therefore, how to generate images while ensuring that the labels of the generated images are not limited to existing labels is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In order to overcome the deficiencies of the prior art, this application provides an image generation method, training method, apparatus, device, and storage medium, so that while generating an image, the labels of the generated image are not limited to existing labels.
[0006] According to one aspect of this application, an image generation method is provided, comprising:
[0007] The first Gaussian noise is used as the quasi-image data;
[0008] The image generation steps are performed iteratively N times, where N is an integer greater than 1. The image generation steps include:
[0009] The target image label and the quasi-image data are input into a trained noise prediction network, which includes multiple feature extraction layers.
[0010] In response to the fact that the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer;
[0011] Obtain the output vector of the noise prediction network;
[0012] Randomly generate third Gaussian noise;
[0013] The quasi-image data is updated based on the quasi-image data, the output vector, and the third Gaussian noise.
[0014] The updated quasi-image data obtained after N image generation steps is used as image data with the target image label.
[0015] In some embodiments of this application, each feature extraction layer corresponds to an embedding layer, the embedding layer containing the embedding vectors of each image label in the image label set, and the step of inputting the target image label and the quasi-image data into the trained noise prediction network and obtaining the output vector of the noise prediction network includes:
[0016] In response to the target image label belonging to the image label set of the noise prediction network, the embedding vector corresponding to the target image label is extracted from the embedding layer corresponding to each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer.
[0017] In some embodiments of this application, the image generation method is performed by sampling different first Gaussian noises to obtain multiple image data with the target image label.
[0018] In some embodiments of this application, the image generation method is performed using different target image labels to obtain image data with different target image labels.
[0019] In some embodiments of this application, the first Gaussian noise, the second Gaussian noise, and the third Gaussian noise are standard Gaussian noise.
[0020] In some embodiments of this application, the quasi-image data g1 is updated according to the following formula:
[0021] g1=α*(g1-β*g2)-γ*g3
[0022] Where g2 is the output vector, g3 is the third Gaussian noise, and α, β and γ are hyperparameters.
[0023] In some embodiments of this application, the noise prediction network is trained according to the following steps:
[0024] Fourth Gaussian noise is randomly added to the sample images, and the sample images have image labels from the image label set;
[0025] The sample images with added noise and their corresponding image labels are input into a noise prediction network. Each feature extraction layer of the noise prediction network corresponds to an embedding layer, and the embedding layer contains the embedding vectors of each image label in the image label set.
[0026] The embedding vectors of the image labels corresponding to the sample images are extracted from the embedding layers corresponding to each feature extraction layer and used as the input data of the corresponding feature extraction layer.
[0027] Obtain the fifth noise output by the noise prediction network;
[0028] The loss of the noise prediction network is calculated based on the difference between the fourth Gaussian noise and the fifth noise.
[0029] The model parameters of the noise prediction network and the embedding vector of the embedding layer are updated based on the loss of the noise prediction network.
[0030] According to another aspect of this application, a method for training a classification model is also provided, comprising:
[0031] Image data generated using the image generation method described above, along with their associated image labels, are used as classification samples.
[0032] The image data of the classified samples is input into the classification model;
[0033] Obtain the classification labels output by the classification model;
[0034] The model parameters of the classification model are updated based on the differences between the classification labels and the image labels.
[0035] According to another aspect of this application, an image generation apparatus is also provided, comprising:
[0036] The quasi-image data module is configured to use the first Gaussian noise as quasi-image data;
[0037] The iterative module is configured to iteratively execute the image generation steps N times, where N is an integer greater than 1. The image generation steps include:
[0038] The target image label and the quasi-image data are input into a trained noise prediction network, which includes multiple feature extraction layers.
[0039] In response to the fact that the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer;
[0040] Obtain the output vector of the noise prediction network;
[0041] Randomly generate third Gaussian noise;
[0042] The quasi-image data is updated based on the quasi-image data, the output vector, and the third Gaussian noise.
[0043] The image data module is configured to use the updated quasi-image data obtained through N image generation steps as image data with the target image label.
[0044] According to another aspect of this application, a training apparatus for a classification model is also provided, comprising:
[0045] The classification sample acquisition module is configured to use the image data generated by the image generation method described above, and use the generated image data and its image labels as classification samples;
[0046] The input module is configured to input the image data of the classified samples into the classification model;
[0047] The output module is configured to obtain the classification labels output by the classification model;
[0048] The update module is configured to update the model parameters of the classification model based on the difference between the classification label and the image label.
[0049] According to another aspect of this application, an electronic device is also provided, the electronic device comprising: a processor; and a storage medium having a computer program stored thereon, the computer program being executed by the processor to perform the steps described above.
[0050] According to another aspect of this application, a storage medium is also provided, on which a computer program is stored, the computer program being executed by a processor to perform the steps described above.
[0051] Therefore, the solution provided in this application has the following advantages compared with the prior art:
[0052] When the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, which is then used as the input data for the corresponding feature extraction layer. This allows image data with target image labels outside the image label set to be generated based on this input data. Thus, while generating images, the labels of the generated images are not limited to existing labels, expanding the labels of the generated image data. This, in turn, improves the robustness and accuracy of image processing models trained based on the generated image data and its labels, such as those for target recognition and target tracking. Attached Figure Description
[0053] The above and other features and advantages of this application will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.
[0054] Figure 1A flowchart of an image generation method according to an embodiment of this application is shown.
[0055] Figure 2 A flowchart illustrating the iteratively executed image generation steps in the image generation method according to an embodiment of this application is shown.
[0056] Figure 3 A schematic diagram of a noise prediction network according to an embodiment of this application is shown.
[0057] Figure 4 A flowchart illustrating a training method for a classification model according to an embodiment of this application is shown.
[0058] Figure 5 A block diagram of an image generation apparatus according to an embodiment of this application is shown.
[0059] Figure 6 A block diagram of a training apparatus for a classification model according to an embodiment of this application is shown.
[0060] Figure 7 This illustration schematically depicts a computer-readable storage medium according to an exemplary embodiment of the present disclosure.
[0061] Figure 8 The schematic diagram illustrates an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation
[0062] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0063] Furthermore, the accompanying drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0064] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.
[0065] In order to overcome the defects of the prior art, this application provides an image generation method, training method, apparatus, device and storage medium, so that the labels of the generated images are not limited to existing labels.
[0066] First see Figure 1 and Figure 2 , Figure 1 A flowchart of an image generation method according to an embodiment of this application is shown. Figure 2 A flowchart illustrating the iteratively executed image generation steps in an image generation method according to an embodiment of this application is shown. The image generation method provided in this application includes:
[0067] Step S110: Use the first Gaussian noise as quasi-image data.
[0068] Specifically, the quasi-image data and target image labels are used as input data for the N iterative image generation steps in step S120. In the N iterative image generation steps, a trained noise prediction network is used to process the image data. The noise prediction network can be trained using an image label set and images with labels from within the image label set. The target image label can be an image label from the image label set, or it may not belong to the image label set. Image labels within the image label set can be named according to their content or marked with different serial numbers. In an embodiment where image labels are marked by serial numbers, if the number of target image labels is greater than the number of image labels in the image label set, the target image label can be considered not to belong to the image label set, thus improving the efficiency of identifying whether a target image label belongs to the image label set.
[0069] Specifically, the first Gaussian noise can be standard Gaussian noise with a given image size. The pixel value x of the Gaussian noise follows a Gaussian distribution, as shown in the following formula:
[0070]
[0071] Where f(x) is the probability density of pixel value x, σ is the variance of x, and μ is the mean and expected value of x. In a standard Gaussian distribution, μ = 0; σ 2 =1. Therefore, different first Gaussian noises can be generated based on Gaussian random numbers.
[0072] Step S120: Iteratively execute the image generation step N times, where N is an integer greater than 1. The image generation step includes:
[0073] Step S121: Input the target image label and the quasi-image data into a trained noise prediction network, which includes multiple feature extraction layers.
[0074] Specifically, see Figure 3 , Figure 3 The diagram schematically illustrates the structure of a noise prediction network 200. The noise prediction network 200 may include multiple feature extraction layers 201, each of which can perform upsampling or downsampling operations on the data input to that layer. Furthermore, each feature extraction layer 201 may correspond to an embedding layer 202, and the feature extraction layer 201 can incorporate the embedding vectors from the embedding layer 202 into its vector extraction and encoding process.
[0075] Each embedding layer 202 includes an embedding vector for an image label within the corresponding image label set. The width (or length) of the embedding vector is equal to the number of input channels of the feature extraction layer 201 corresponding to the embedding layer 202. In other words, the size of the embedding vector of the embedding layer 202 is the number of image labels in the image label set multiplied by the number of input channels of the feature extraction layer 201 corresponding to the embedding layer 202. The embedding vectors of the embedding layer 202 are updated during the training of the noise prediction network 200.
[0076] Step S122: In response to the target image label not belonging to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer.
[0077] Specifically, as mentioned above, each feature extraction layer may correspond to an embedding layer. The embedding layer contains the embedding vectors of each image label in the image label set. In response to the target image label belonging to the image label set of the noise prediction network, the embedding vector corresponding to the target image label can be extracted from the embedding layer corresponding to each feature extraction layer as the identity identifier of the target image label, and used as input data for the corresponding feature extraction layer. For example, the embedding vectors of the same embedding layer may include image label A, embedding vector A; image label B, embedding vector B; and image label C, embedding vector C, where image label A, image label B, and image label C belong to the image label set. When the target image label is image label B, the corresponding embedding vector B (the identity identifier of image label B) can be found in the embedding layer using the target image label as an index, and used as input data for the corresponding feature extraction layer.
[0078] Specifically, the second Gaussian noise can be standard Gaussian noise with a set image size. In some embodiments, the second Gaussian noise can be the same as the first Gaussian noise, but this application is not limited thereto. The second Gaussian noise can also be Gaussian noise generated based on Gaussian random numbers.
[0079] Specifically, when the target image label does not belong to the image label set of the noise prediction network, the noise vector of the second Gaussian noise can be sampled as the identity of the target image label. Thus, based on the identity, subsequent steps can be performed in a manner similar to that of the target image label belonging to the image label set of the noise prediction network to complete the update of the quasi-image data.
[0080] Step S123: Obtain the output vector of the noise prediction network.
[0081] Step S124: Randomly generate third Gaussian noise.
[0082] Specifically, the third Gaussian noise can be standard Gaussian noise with a set image size. Alternatively, the third Gaussian noise can be Gaussian noise generated based on random Gaussian numbers.
[0083] Step S125: Update the quasi-image data according to the quasi-image data, the output vector, and the third Gaussian noise.
[0084] Specifically, the quasi-image data g1 is updated according to the following formula:
[0085] g1=α*(g1-β*g2)-γ * g3
[0086] Where g2 is the output vector, g3 is the third Gaussian noise, and α, β, and γ are hyperparameters. Furthermore, α, β, and γ can be updated through learning and training. This application is not limited thereto, and other methods for updating quasi-image data g1 are also within the scope of this application.
[0087] Step S130: Use the updated quasi-image data obtained after N image generation steps as image data with the target image label.
[0088] Specifically, in each iteration, the image is further reconstructed based on the target image label using quasi-image data, third Gaussian noise, and the output vector of the noise prediction network. The accuracy and clarity of the quasi-image data can be fed back during the iteration process, making the noise (output vector) predicted based on the quasi-image data more accurate, thus resulting in higher accuracy and clarity of the updated quasi-image data. Therefore, the larger the number of iterations, the higher the clarity and accuracy of the obtained quasi-image data. To ensure the clarity and accuracy of the image data, the number of iterations can be greater than 1. In some preferred embodiments, to balance the number of iterations and the clarity and accuracy of the image data, the number of iterations can be 1000. This application can implement many more variations, which will not be elaborated here.
[0089] Specifically, by sampling different first Gaussian noises and executing the image generation method, multiple image data sets with the target image label can be obtained. By using different target image labels and executing the image generation method, image data sets with different target image labels can be obtained. Thus, multiple image data sets with image labels both within and outside the image label set can be obtained, thereby expanding the image labels and image data set.
[0090] Therefore, in the image generation method provided in this application, when the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer. This enables the generation of image data with target image labels outside the image label set based on this input data. Thus, while generating the image, the labels of the generated image are not limited to existing labels, expanding the labels of the generated image data, thereby improving the robustness and accuracy of image processing models such as target recognition and target tracking trained based on the generated image data and its labels.
[0091] Further, the noise prediction network can be trained according to the following steps: Randomly adding fourth Gaussian noise n1 to a sample image, the sample image having image labels from the image label set; inputting the noise-added sample image and its image labels into the noise prediction network, each feature extraction layer of the noise prediction network corresponding to an embedding layer, the embedding layer containing the embedding vectors of each image label from the image label set; extracting the embedding vectors of the image labels corresponding to the sample image from the embedding layers corresponding to each feature extraction layer, as input data for the corresponding feature extraction layer; obtaining the fifth noise output by the noise prediction network; calculating the loss of the noise prediction network based on the difference between the fourth Gaussian noise and the fifth noise; updating the model parameters of the noise prediction network and the embedding vectors of the embedding layers based on the loss of the noise prediction network.
[0092] Specifically, the fourth Gaussian noise can be Gaussian noise with a set image size. The fourth Gaussian noise can be Gaussian noise generated based on random Gaussian numbers. Furthermore, during training, the above steps can be repeated based on different image labels, different sample images, and different fourth Gaussian noise until the loss of the noise prediction network no longer decreases, thus completing the training of the noise prediction network.
[0093] See below. Figure 4 , Figure 4 A flowchart illustrating a training method for a classification model according to an embodiment of this application is shown. The training method for the classification model includes:
[0094] Step S101: Using the image generation method described above, the generated image data and its image labels are used as classification samples;
[0095] Step S102: Input the image data of the classification samples into the classification model;
[0096] Step S103: Obtain the classification labels output by the classification model;
[0097] Step S104: Update the model parameters of the classification model based on the difference between the classification label and the image label.
[0098] The trained classification model can be used to predict the classification label of the input image data.
[0099] Specifically, by sampling different first Gaussian noises and performing the image generation method, multiple image data sets with the target image label can be obtained. By using different target image labels and performing the image generation method, image data sets with different target image labels can be obtained. Thus, multiple image data sets with image label sets both within and outside the set can be obtained, thereby expanding the image labels and image data. Therefore, by expanding the labels of the classification samples, the robustness and accuracy of the classification model trained based on the generated image data and its labels are improved.
[0100] The above examples illustrate multiple implementations of this application. This application is not intended to limit itself. In each implementation, the addition, omission, or change of the order of steps are all within the protection scope of this application. Each implementation can be implemented individually or in combination.
[0101] The following is combined Figure 5 The image generation apparatus 300 provided in this application is described. The image generation apparatus 300 includes a quasi-image data module 310, an iteration module 320, and an image data module 330.
[0102] The quasi-image data module 310 is configured to use the first Gaussian noise as quasi-image data;
[0103] The iteration module 320 is configured to perform the image generation step N times, where N is an integer greater than 1. The image generation step includes:
[0104] The target image label and the quasi-image data are input into a trained noise prediction network, which includes multiple feature extraction layers.
[0105] In response to the fact that the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer;
[0106] Obtain the output vector of the noise prediction network;
[0107] Randomly generate third Gaussian noise;
[0108] The quasi-image data is updated based on the quasi-image data, the output vector, and the third Gaussian noise.
[0109] The image data module 330 is configured to use the updated quasi-image data obtained after N image generation steps as image data with the target image label.
[0110] In the image generation apparatus provided in this application, when the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer. This enables the generation of image data with target image labels outside the image label set based on this input data. Therefore, while generating the image, the labels of the generated image are not limited to existing labels, expanding the labels of the generated image data. This improves the robustness and accuracy of image processing models such as target recognition and target tracking trained based on the generated image data and its labels.
[0111] The following is combined Figure 6 The training apparatus 400 for the classification model provided in this application is described. The training apparatus 400 for the classification model includes a classification sample acquisition module 410, an input module 420, an output module 430, and an update module 440.
[0112] The classification sample acquisition module 410 is configured to use the image data generated by the image generation method described above, and use the generated image data and its image labels as classification samples;
[0113] Input module 420 is configured to input the image data of the classified samples into the classification model;
[0114] The output module 430 is configured to obtain the classification labels output by the classification model;
[0115] The update module 440 is configured to update the model parameters of the classification model based on the difference between the classification label and the image label.
[0116] This application can realize the image generation device 300 and the classification model training device 400 through software, hardware, firmware and any combination thereof. Figure 5 and Figure 6 The image generation device 300 and the classification model training device 400 provided in this application are merely schematic illustrations. Without departing from the concept of this application, the splitting, merging, and addition of modules are all within the protection scope of this application.
[0117] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by, for example, a processor, can implement the steps of the image generation method and / or classification model training method described in any of the above embodiments. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps according to the various exemplary embodiments of this application described in the above-described section on the image generation method and / or classification model training method.
[0118] refer to Figure 7 As shown, a program product 800 for implementing the above-described method according to an embodiment of this application is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0119] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0120] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0121] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the tenant's computing device, partially on the tenant's device, as a standalone software package, partially on the tenant's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the tenant's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0122] In exemplary embodiments of this disclosure, an electronic device is also provided, which may include a processor and a memory for storing executable instructions of the processor. The processor is configured to perform the steps of the image generation method and / or classification model training method described in any of the above embodiments by executing the executable instructions.
[0123] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."
[0124] The following reference Figure 8 To describe an electronic device 600 according to this embodiment of the present application. Figure 8 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0125] like Figure 8 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), a display unit 640, etc.
[0126] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the above-described image generation method and / or classification model training method section of this specification, according to various exemplary embodiments of this application. For example, the processing unit 610 can perform, as follows: Figure 1 The steps are shown in the figure.
[0127] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only memory unit (ROM) 6203.
[0128] The storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0129] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0130] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable tenants to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0131] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described image generation method and / or classification model training method according to the embodiments of this disclosure.
[0132] Therefore, the solution provided in this application has the following advantages compared with the prior art:
[0133] When the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, which is then used as the input data for the corresponding feature extraction layer. This allows image data with target image labels outside the image label set to be generated based on this input data. Thus, while generating images, the labels of the generated images are not limited to existing labels, expanding the labels of the generated image data. This, in turn, improves the robustness and accuracy of image processing models trained based on the generated image data and its labels, such as those for target recognition and target tracking.
[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. An image generation method, characterized in that, include: The first Gaussian noise is used as the quasi-image data; The image generation steps are performed iteratively N times, where N is an integer greater than 1. The image generation steps include: The target image label and the quasi-image data are input into a trained noise prediction network, which includes multiple feature extraction layers. In response to the fact that the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer; Obtain the output vector of the noise prediction network; Randomly generate third Gaussian noise; The quasi-image data is updated based on the quasi-image data, the output vector, and the third Gaussian noise. The updated quasi-image data obtained after N image generation steps is used as image data with the target image label.
2. The image generation method as described in claim 1, characterized in that, Each of the aforementioned feature extraction layers corresponds to an embedding layer, the embedding layer containing the embedding vectors of each image label in the image label set. The process of inputting the target image labels and the quasi-image data into the trained noise prediction network, and before obtaining the output vector of the noise prediction network, includes: In response to the target image label belonging to the image label set of the noise prediction network, the embedding vector corresponding to the target image label is extracted from the embedding layer corresponding to each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer.
3. The image generation method as described in claim 1, characterized in that, The image generation method is performed by sampling different first Gaussian noises to obtain multiple image data with the target image label.
4. The image generation method as described in claim 1, characterized in that, The image generation method is performed using different target image labels to obtain image data with different target image labels.
5. The image generation method as described in claim 1, characterized in that, The quasi-image data g1 is updated according to the following formula: g1=α*(g1-β*g2)-γ*g3 Where g2 is the output vector, g3 is the third Gaussian noise, and α, β and γ are hyperparameters.
6. The image generation method as described in claim 1, characterized in that, The noise prediction network is trained according to the following steps: Fourth Gaussian noise is randomly added to the sample images, and the sample images have image labels from the image label set; The sample images with added noise and their corresponding image labels are input into a noise prediction network. Each feature extraction layer of the noise prediction network corresponds to an embedding layer, and the embedding layer contains the embedding vectors of each image label in the image label set. The embedding vectors of the image labels corresponding to the sample images are extracted from the embedding layers corresponding to each feature extraction layer and used as the input data of the corresponding feature extraction layer. Obtain the fifth noise output by the noise prediction network; The loss of the noise prediction network is calculated based on the difference between the fourth Gaussian noise and the fifth noise. The model parameters of the noise prediction network and the embedding vector of the embedding layer are updated based on the loss of the noise prediction network.
7. A method for training a classification model, characterized in that, include: Image data generated using the image generation method as described in any one of claims 1 to 6, and the generated image data and its image labels are used as classification samples; The image data of the classified samples is input into the classification model; Obtain the classification labels output by the classification model; The model parameters of the classification model are updated based on the differences between the classification labels and the image labels.
8. An image generation apparatus, characterized in that, include: The quasi-image data module is configured to use the first Gaussian noise as quasi-image data; The iterative module is configured to iteratively execute the image generation steps N times, where N is an integer greater than 1. The image generation steps include: The target image label and the quasi-image data are input into a trained noise prediction network, which includes multiple feature extraction layers. In response to the fact that the target image label does not belong to the image label set of the noise prediction network, a noise vector is sampled from the second Gaussian noise according to the number of input feature channels of each feature extraction layer as the identity identifier of the target image label, and used as the input data of the corresponding feature extraction layer; Obtain the output vector of the noise prediction network; Randomly generate third Gaussian noise; The quasi-image data is updated based on the quasi-image data, the output vector, and the third Gaussian noise. The image data module is configured to use the updated quasi-image data obtained through N image generation steps as image data with the target image label.
9. An electronic device, characterized in that, The electronic device includes: processor; A memory, on which a computer program is stored, is executed by the processor during runtime: The image generation method as described in any one of claims 1 to 6; and / or The training method for the classification model as described in claim 7.
10. A storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor: The image generation method as described in any one of claims 1 to 6; and / or The training method for the classification model as described in claim 7.