Image data augmentation method and device based on generative adversarial network

Abstract

The image data augmentation method and device based on the generative adversarial network of the present disclosure can extract the underlying semantic features of training images to form visual vocabulary of the training images; can label target annotation words of a target scene according to the connotation of the target scene, and the connotation and extension of the visual vocabulary; can construct a generative adversarial network, and train the generative adversarial network by using the training images, the target annotation words and random noise until the image data output by the generative adversarial network meets the requirements, and obtain augmented image data. The method can break through the conversion bottleneck of sample semantic scenes, solve the small sample image classification and recognition problem of multi-source information, improve the quality of augmented image data, and improve the performance of artificial intelligence algorithms.

Description

Technical Field

[0001] This invention belongs to the field of image data augmentation technology, and particularly relates to an image data augmentation method, system, device and storage medium based on generative adversarial networks. Background Technology

[0002] In recent years, artificial intelligence (AI) technologies, represented by deep learning, have made continuous breakthroughs and played a significant role in an increasing number of application scenarios, quietly changing people's daily lives. AI technology is also highly favored in specialized application fields, particularly in areas such as specialized image recognition and classification, where there is a huge demand for its application.

[0003] In the field of artificial intelligence applications, computing power, algorithms, and data are inextricably linked. With advancements in hardware technology and the emergence of various optimization algorithms, the issues of computing power and algorithms have been largely resolved. The key to whether artificial intelligence technology can achieve breakthroughs in a particular field lies in the amount of data. For artificial intelligence, the more data the better; small amounts of data will significantly limit the capabilities of algorithmic models, greatly diminishing their effectiveness or even rendering them ineffective.

[0004] Similarly, a major bottleneck in implementing artificial intelligence technology for various spacecraft and special application scenarios is the constraint of experimental data. On the one hand, experimental data is highly valuable, and data obtained through flight tests, low-dynamic experiments, and other experimental methods require extremely high investment in funding, personnel, and time. On the other hand, data obtained through simulation methods has limited application value due to the difficulty in simulating the characteristics of the scenarios. To overcome these bottlenecks, it is necessary to explore more usable data in specific fields, especially image data, and to conduct model research suitable for accumulating small amounts of data.

[0005] According to publicly available literature both domestically and internationally, there has been some research on image data augmentation methods. For example, methods based solely on multi-source semantic scenes fully utilize graphic semantic information, forming a unified paradigm for semantics, scene, and target. However, the core drawback of this method is that it uses manually constructed features, resulting in limited feature representation and semantic information expression capabilities. Especially for special applications where samples are scarce, manually constructed features have limitations, and the model is prone to underfitting, leading to generally mediocre results. Summary of the Invention

[0006] This invention overcomes one of the shortcomings of the prior art and provides an image data augmentation method, system, device and storage medium based on generative adversarial networks. It can break through the bottleneck of sample semantic scene conversion and solve the problem of small sample image classification and recognition with multi-source information, improve the quality of augmented image data and improve the performance of artificial intelligence algorithms.

[0007] According to one aspect of this disclosure, an image data augmentation method based on generative adversarial networks is proposed, the method comprising:

[0008] Extract the low-level semantic features of the training images to form the visual vocabulary of the training images;

[0009] Based on the target scene connotation of the training images, and the connotation and extension of the visual vocabulary, the target annotation words of the target scene are labeled;

[0010] A generative adversarial network (GAN) is constructed. The GAN is trained using the training images, target labels, and random noise until the image data output by the GAN meets the requirements, thus obtaining augmented image data.

[0011] In one possible implementation, the connotation of the target scene is the relationship between the target label words of the training image and the target scene, the connotation of the visual vocabulary is the relationship between the visual vocabulary and the target label words, and the extension of the visual vocabulary is the probability of the relationship between the visual vocabulary and the target scene concept.

[0012] In one possible implementation, forming the visual vocabulary of the training images based on the extracted low-level semantic features includes:

[0013] Extract the low-level semantic features of the training image to obtain the low-level semantic feature vector of the training image, and each low-level semantic feature vector corresponds to a SIFT interest point vector;

[0014] The K-means clustering algorithm is used to cluster the low-level semantic feature vectors of the training images, and the set of cluster centers is used as a visual dictionary.

[0015] Each SIFT interest point vector is mapped to a visual word in the visual dictionary to obtain the visual vocabulary for each training image.

[0016] In one possible implementation, the step of labeling the target scene with target tags based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary includes:

[0017] Read the semantic information of the training images;

[0018] Based on the semantic information, the connotation and extension of the target scene, visual vocabulary, are determined;

[0019] The connotation and extension of the target scene and visual vocabulary are input into the target annotation model to obtain the target annotation words of the target scene.

[0020] In one possible implementation, the generative adversarial network (GAN) is trained using the training images, target labeled words, and random noise until the image data output by the GAN meets the requirements, resulting in augmented image data, including:

[0021] P1: Initialize the parameters of the generator network and the discriminator network of the generative adversarial network;

[0022] P2: Input the fused data of the random noise and the target labeled words into the generation network, output the generated image, input the generated image and the training image into the discriminator network, train the discriminator network, and update the parameters of the discriminator network;

[0023] P3: Keeping the parameters of the discriminant network unchanged, input the output loss error value of the discriminant network into the generator network to update the parameters of the generator network;

[0024] P4: Repeat steps P2-P3 until the generated image output by the generating network meets the image data requirements, that is, the generated image data is amplified image data.

[0025] In one possible implementation, the SIFT interest point vector is a 128-dimensional vector, and the visual vocabulary is a 1-dimensional vector.

[0026] According to another aspect of this disclosure, an image data augmentation device based on a generative adversarial network is proposed, the device comprising:

[0027] The semantic representation module is used to extract the low-level semantic features of training images to form the visual vocabulary of training images;

[0028] The target annotation module is used to annotate the target scene with target annotation words based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary;

[0029] The data augmentation module is used to construct a generative adversarial network. It trains the generative adversarial network using the training images, target labeled words, and random noise until the image data output by the generative adversarial network meets the requirements, thus obtaining augmented image data.

[0030] According to another aspect of this disclosure, an electronic device is proposed, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0031] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method described above.

[0032] This disclosed image data augmentation method based on generative adversarial networks (GANs) extracts low-level semantic features from training images to form visual vocabulary for the training images. Based on the target scene connotation of the training images and the connotation and extension of the visual vocabulary, target labels for the target scene are annotated. A GAN is constructed and trained using the training images, target labels, and random noise until the image data output by the GAN meets the requirements, resulting in augmented image data. This method overcomes the bottleneck of sample semantic scene conversion and solves the problem of small-sample image classification and recognition with multi-source information, improving the quality of augmented image data and enhancing the performance of artificial intelligence algorithms. Attached Figure Description

[0033] The accompanying drawings are used to provide a further understanding of the technical solutions of this application or the prior art, and constitute a part of the specification. The drawings illustrating embodiments of this application, together with the embodiments of this application, are used to explain the technical solutions of this application, but do not constitute a limitation on the technical solutions of this application.

[0034] Figure 1 A flowchart of an image data augmentation method based on a generative adversarial network according to an embodiment of the present disclosure is shown;

[0035] Figure 2 A flowchart illustrating an example of step S1 according to an embodiment of the present disclosure is shown;

[0036] Figure 3 A flowchart illustrating an example of step S2 according to an embodiment of the present disclosure is shown;

[0037] Figure 4 A flowchart illustrating an example of step S3 according to an embodiment of the present disclosure is shown;

[0038] Figure 5 A schematic diagram of the structure of an image data augmentation device based on a generative adversarial network according to an embodiment of the present disclosure is shown.

[0039] Figure 6 A schematic diagram of the structure of an electronic device 3 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0040] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples, so that the process of how the present invention uses technical means to solve technical problems and achieve corresponding technical effects can be fully understood and implemented accordingly. The embodiments of this application and the various features in the embodiments can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of the present invention.

[0041] Furthermore, the steps illustrated in the flowcharts of the accompanying drawings can be executed in a computer, such as a set of computer-executable instructions. Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that presented here.

[0042] Figure 1 A flowchart illustrating an image data augmentation method based on a generative adversarial network according to an embodiment of this disclosure is shown. This method can be applied to technical fields such as aerospace and special applications. Figure 1 As shown, the method may include:

[0043] Step S1: Extract the low-level semantic features of the training images to form the visual vocabulary of the training images.

[0044] Visual vocabulary connects the features of training images with the object words in those images, imbuing them with semantic information. For example, if the concepts of certain objects co-occur frequently with certain scenes, it indicates a high degree of correlation between them. By analyzing latent semantic information (semantic relevance), objects with different labeled words in different scenes but potentially representing the same concept can be identified. For instance, when a vehicle and a road appear together in an image, the latent semantic concept of "movement" describes the vehicle's motion state, while when a parking sign and a vehicle appear simultaneously, the latent semantic concept of "stationary" characterizes the vehicle's state. In this invention, such objects are referred to as "gray samples." These "gray samples" can be used to enrich the training data of artificial intelligence algorithm models, increase samples of semantically related categories, or improve the model's generalization ability through learned knowledge, thereby enhancing the quality of image data augmentation.

[0045] Figure 2 A flowchart illustrating an example of step S1 according to an embodiment of the present disclosure is shown.

[0046] like Figure 2 As shown, in step S1, forming the visual vocabulary of the training images based on the extracted low-level semantic features may include:

[0047] Step S11: Extract the low-level semantic features of the training image to obtain the low-level semantic feature vector of the training image, wherein each low-level semantic feature vector corresponds to a SIFT interest point vector.

[0048] SIFT (Scale-invariant feature transform) is a feature extraction algorithm (image local feature extraction algorithm) in computer vision tasks. It includes an interest point detector and descriptors, which can detect key points in an image and is a type of local feature descriptor. SIFT interest points (key points) can be located using the difference of Gaussian function. The SIFT interest point vector is a 128-dimensional vector.

[0049] Step S12: Use the K-means clustering algorithm to cluster the low-level semantic feature vectors of the training images, and use the set of cluster centers as the visual dictionary. For example, set the number of cluster centers Z, where Z is an integer, and use the K-means clustering algorithm to cluster the low-level semantic feature vectors of the training images to obtain Z cluster centers. The set of Z cluster centers is called the visual dictionary.

[0050] Step S13: Map each SIFT interest point vector to a visual word in the visual dictionary to obtain the visual vocabulary for each training image. After obtaining the visual dictionary in step S12, mapping each 128-dimensional SIFT interest point vector to a 1-dimensional visual word in the visual vocabulary allows the 128-dimensional SIFT interest point vector to be represented by a 1-dimensional visual word. This means each training image can be represented by a set of visual words, which can describe or explain the training image from a certain perspective.

[0051] Step S2: Label the target scene with target words based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary.

[0052] Wherein, the connotation of the target scene is the relationship between the target annotation words of the training image and the target scene, the connotation of the visual vocabulary is the relationship between the visual vocabulary and the target annotation words, and the extension of the visual vocabulary is the probability of the relationship between the visual vocabulary and the target scene concept.

[0053] Figure 3 A flowchart illustrating an example of step S2 according to an embodiment of the present disclosure is shown.

[0054] like Figure 3 As shown, in step S2, the target annotation words for the target scene, based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary, may include:

[0055] Step S21: Read the semantic information of the training image.

[0056] Step S22: Determine the connotation of the target scene and the connotation and extension of visual vocabulary based on the semantic information.

[0057] Step S23: Input the connotation of the target scene, the connotation and extension of the visual vocabulary into the target annotation model to obtain the target annotation words of the target scene.

[0058] Image semantics can be divided into three layers: visual, object, and conceptual. The visual layer, commonly understood as the bottom layer, includes features such as color, texture, and shape; these features are called low-level semantic features. The object layer, the intermediate layer, typically contains attribute features, representing the state of an object at a given moment. The conceptual layer is the highest layer, representing what the image expresses that is closest to human understanding. For example, in an image containing sand, blue sky, and seawater, the visual layer distinguishes individual areas, the object layer represents the sand, blue sky, and seawater, and the conceptual layer represents the beach—this is the semantic meaning expressed by the image.

[0059] Image semantic information can include low-level semantic information, intermediate-level semantic information, and high-level semantic information. Leveraging the strong semantic relationships between objects in natural scene images can address the problem of small sample sizes (insufficient image data). For example, in a "desert" scene, the object "boat" is highly unlikely to appear, while the probability of "sand" appearing is very high, and it also occupies a large proportion of the entire image. In this invention, by analyzing the semantic information expressed by the scene semantics and the combination of objects contained in the scene, and using methods similar to semantic analysis in text, semantic relationships between scene categories and object categories, as well as semantic relationships between objects and the visual words they contain, are established. Through these semantic relationships, data from other categories can be learned to be trained on categories with less training data, thus addressing the small sample size problem and further guiding image classification.

[0060] In one example, if we denote b as a visual word, a as a target label word, y as the extension of visual word b, z as the connotation of target label word a, and c as the target concept in the training image, then the connotation of the target scene is denoted as P(a, s), the connotation of the visual word is denoted as p(y, a), and the extension of the visual word is denoted as P(b|y).

[0061] Suppose there are r+t target concepts, S original scenarios, and the rest are target scenarios, where T is the number of iterations and Y is the size of the visual dictionary.

[0062] For each scenario, the target annotation words for the target scenario are obtained using the target annotation model as follows:

[0063] Step 1: If Proceed to step 2 if necessary, otherwise end.

[0064] Step 2: Put p (0) (b|y, c) is initialized to p(b|y), and p is randomly initialized. (0) (a|z,c) and p (0)(y, z);

[0065] Step 3: Let k = 1;

[0066] Step 4: For the target concept c from 1 to s+t, update p. (k) (y, z|b, a, c),

[0067]

[0068] Step 5: For the target concept c, from r+1 to s+t,

[0069] according to Update p (k) (a|z,c);

[0070] according to Update p (k) (y, z);

[0071] according to Update p (k) (c);

[0072] Step 6: Let K = k + 1;

[0073] Step 7: If r <T,

[0074] Step 8: According to

[0075]

[0076] The calculation results are output as target annotation words for the target scene.

[0077] See above for the meaning of the symbols used in each step.

[0078] Step S3: Construct a generative adversarial network (GAN). Train the GAN using the training images, target labels, and random noise until the image data output by the GAN meets the requirements, thus obtaining augmented image data.

[0079] Figure 4 A flowchart illustrating an example of step S3 according to an embodiment of the present disclosure is shown.

[0080] like Figure 4 As shown, step S3 may include:

[0081] P1: Initialize the parameters of the generator network and the discriminator network of the generative adversarial network;

[0082] P2: Input the fused data of the random noise and the target labeled words into the generation network, output the generated image, input the generated image and the training image into the discriminator network, train the discriminator network, and update the parameters of the discriminator network.

[0083] P3: Keeping the parameters of the discriminant network unchanged, input the output loss error value of the discriminant network into the generator network to update the parameters of the generator network;

[0084] P4: Repeat steps P2-P3 until the generated image output by the generating network meets the image data requirements, that is, the generated image data is amplified image data.

[0085] In one example, first, the convolutional neural networks for the generator (generator network), discriminator (discriminator network), and classifier of the generative adversarial network are determined and the network model parameters are initialized. Then, random noise can be fused with target annotations and latent coding vectors to obtain generative adversarial network (GAN) fused semantic information. This GAN fused semantic information is used as input to the generator (generator network) to generate images. The generated image data and the original image data are then input to the discriminator (discriminator network) to train the discriminator. The discriminator outputs the loss difference between the generated and original image data, adjusting the generator parameters. Finally, keeping the discriminator parameters unchanged, the loss difference between the generated and original image data is input into the generator to train it again, updating the generator parameters and performing a preliminary update. The GAN fused semantic information is then input into the generator to generate new image data. This new generated image data, along with the original image data, is used to further train the discriminator and update its parameters. This process is repeated to update both the generator and discriminator until the final image data provided by the generator meets the requirements. At this point, the data augmentation process is complete, and the data augmentation module exits, yielding high-quality augmented image data.

[0086] This disclosed image data augmentation method based on generative adversarial networks (GANs) extracts low-level semantic features from training images to form visual vocabulary for the training images. Based on the target scene connotation of the training images and the connotation and extension of the visual vocabulary, target labeling words for the target scene are assigned. A GAN is constructed and trained using the training images, target labeling words, and random noise until the image data output by the GAN meets the requirements, resulting in augmented image data. This method effectively utilizes the semantics of scene category labeling words to mine commonalities and distinctions within semantics. It not only effectively solves the small sample size problem in specific domains but also enables learning across multiple scene data during image augmentation (cross-scene learning) through commonalities. Therefore, applying this technology to practical engineering, especially in artificial intelligence and aerospace fields, is of great significance.

[0087] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0088] Figure 5 A schematic diagram of an image data augmentation apparatus based on a generative adversarial network according to an embodiment of the present disclosure is shown; as follows: Figure 5 As shown, the image data augmentation device includes:

[0089] The semantic representation module 501 is used to extract the low-level semantic features of the training images to form the visual vocabulary of the training images;

[0090] The target annotation module 502 is used to annotate the target annotation words of the target scene according to the target scene connotation of the training image and the connotation and extension of the visual vocabulary;

[0091] The data augmentation module 503 is used to construct a generative adversarial network. It trains the generative adversarial network using the training images, target labeled words, and random noise until the image data output by the generative adversarial network meets the requirements, thus obtaining augmented image data.

[0092] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] Figure 6 This is a schematic diagram of the structure of the electronic device 3 provided in an embodiment of this application. Figure 6As shown, the electronic device 3 of this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various device embodiments described above.

[0094] For example, computer program 303 may be divided into one or more modules / units, which are stored in memory 302 and executed by processor 301 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 303 in electronic device 3.

[0095] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0096] Processor 301 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0097] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 302 can include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments claimed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0105] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image data augmentation method based on generative adversarial networks, characterized in that, The method includes: Extract the low-level semantic features of the training images to form the visual vocabulary of the training images; Based on the target scene connotation of the training images, and the connotation and extension of the visual vocabulary, the target annotation words of the target scene are labeled; A generative adversarial network is constructed, and the training image, target labeled words, and random noise are used to train the generative adversarial network until the image data output by the generative adversarial network meets the requirements, thus obtaining augmented image data; The connotation of the target scene is the relationship between the target annotation words of the training image and the target scene; the connotation of the visual vocabulary is the relationship between the visual vocabulary and the target annotation words; and the extension of the visual vocabulary is the probability of the relationship between the visual vocabulary and the target scene concept. The step of annotating the target scene with target label words based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary includes: Read the semantic information of the training images; Based on the semantic information, the connotation and extension of the target scene, visual vocabulary, are determined; The connotation and extension of the target scene and visual vocabulary are input into the target annotation model to obtain the target annotation words of the target scene; The generative adversarial network (GAN) is trained using the training images, target labeled words, and random noise until the image data output by the GAN meets the requirements, resulting in augmented image data, including: P1: Initialize the parameters of the generator network and the discriminator network of the generative adversarial network; P2: Input the fused data of the random noise and the target labeled words into the generation network, output the generated image, input the generated image and the training image into the discriminator network, train the discriminator network, and update the parameters of the discriminator network; P3: Keeping the parameters of the discriminant network unchanged, input the output loss error value of the discriminant network into the generator network to update the parameters of the generator network; P4: Repeat steps P2-P3 until the generated image output by the generating network meets the image data requirements, that is, the generated image data is amplified image data.

2. The image data augmentation method according to claim 1, characterized in that, The step of forming the visual vocabulary of the training images based on the extracted low-level semantic features includes: Extract the low-level semantic features of the training image to obtain the low-level semantic feature vector of the training image, and each low-level semantic feature vector corresponds to a SIFT interest point vector; The K-means clustering algorithm is used to cluster the low-level semantic feature vectors of the training images, and the set of cluster centers is used as a visual dictionary. Each SIFT interest point vector is mapped to a visual word in the visual dictionary to obtain the visual vocabulary for each training image.

3. The image data augmentation method according to claim 2, characterized in that, The SIFT interest point vector is a 128-dimensional vector, and the visual vocabulary is a 1-dimensional vector.

4. An image data augmentation device based on generative adversarial networks, used in the image data augmentation method based on generative adversarial networks as described in claim 1, characterized in that, The device includes: The semantic representation module is used to extract the low-level semantic features of training images to form the visual vocabulary of training images; The target annotation module is used to annotate the target scene with target annotation words based on the target scene connotation of the training image and the connotation and extension of the visual vocabulary; The data augmentation module is used to construct a generative adversarial network. It trains the generative adversarial network using the training images, target labeled words, and random noise until the image data output by the generative adversarial network meets the requirements, thus obtaining augmented image data.

5. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 3.

Images