Method for generating composite image generator, electronic device, medium, and program product

Abstract

Embodiments of the present application mainly relate to the field of image generation, and in particular, to a method for generating a composite image generator, an electronic device, a medium, and a program product. The method comprises: receiving a carrier for describing an AI task and input by a user; on the basis of the carrier for describing the AI task, generating, by means of a task recipe generator, a structured text corresponding to a task recipe; on the basis of a name corresponding to the structured text of the task recipe, performing mapping by means of a compiler to obtain a corresponding asset; and on the basis of the asset and the structured text, performing data alignment and connection by means of the compiler, and compiling and packaging task graph executors in the compiler together to generate a composite image generator.

Description

Methods, electronic devices, media, and program products for generating synthetic images Technical Field

[0001] This application primarily relates to the field of image generation, and more particularly to a method, electronic device, medium, and program product for generating synthetic images. Background Technology

[0002] In today's rapidly developing field of Artificial Intelligence (AI), the importance of data as the driving force for algorithm training is self-evident. However, in practical applications, especially in industry cases, acquiring large amounts of high-quality, highly relevant, and realistic datasets presents numerous challenges. These challenges include, but are not limited to, the complexity of data collection, the difficulty of data annotation, and data privacy considerations. These issues limit the performance and generalization ability of AI algorithms and increase the workload of data scientists and AI developers.

[0003] Currently, synthetic data is gaining increasing attention as a solution. Synthetic data is generated by simulating real-world environments or scenarios. It not only solves the problem of obtaining realistic data but also offers advantages such as low cost and flexible control. However, existing synthetic data generation methods typically involve multiple complex steps, such as resource preparation, scene construction, rendering, and dataset generation, each requiring significant configuration and preparation work. This not only increases the difficulty of data generation but also reduces its efficiency.

[0004] Summary of the Invention

[0005] This application provides a method, electronic device, medium, and program product for generating synthetic image generators, which can generate synthetic image generators quickly and with high quality.

[0006] In a first aspect, a method for generating a synthetic image generator is provided, comprising: receiving a carrier describing an AI task input by a user; generating structured text corresponding to a task recipe using a task recipe generator based on the carrier describing the AI ​​task; mapping corresponding assets according to the names corresponding to the structured text of the task recipe using a compiler; and performing data alignment and connection using the compiler based on the assets and the structured text, and compiling and packaging the task graph executor in the compiler together to generate the synthetic image generator.

[0007] In a second aspect, an electronic device is provided, comprising: at least one memory configured to store computer-readable code; and at least one processor configured to invoke the computer-readable code to perform the steps of the method provided in the first aspect.

[0008] Thirdly, a computer-readable medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, cause the processor to perform the steps of the method provided in the first aspect.

[0009] Fourthly, a computer program product is provided, the computer program product being tangibly stored on a computer-readable medium and including computer-executable instructions that, when executed, cause at least one processor to perform the steps of the method provided in the first aspect. Attached Figure Description

[0010] The following figures are intended only to illustrate and explain the embodiments of this application, and do not limit the scope of the embodiments of this application. Wherein:

[0011] Figure 1 is a flowchart of a method for generating a synthetic image generator according to an embodiment of this application;

[0012] Figure 2 is a schematic diagram of an electronic device according to an embodiment of this application.

[0013] Figure reference numerals: 100: Method for generating a composite image generator; 101-104: Method steps; 200: Electronic device; 201: Processor; 202: Communication interface; 203: Memory; 204: Communication bus; 205: Program. Detailed Implementation

[0014] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of the embodiments of this application. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0015] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0016] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0017] Figure 1 is a schematic diagram of a method for generating a composite image generator according to an embodiment of this application. As shown in Figure 1, the method 100 for generating a composite image generator includes:

[0018] Step 101: Receive the user's input describing the AI ​​task.

[0019] Step 102: Based on the carrier describing the AI ​​task, generate the structured text corresponding to the task recipe through the task recipe generator.

[0020] Optionally, the structured text may be in formats such as JSON, XML, etc.

[0021] In one embodiment, before step 102, a mapping relationship is established in the task recipe generator for at least one scene description, task graph, static resources, and executable file description corresponding to the AI ​​task. Next, the carrier describing the AI ​​task is converted into a text file, which includes at least one of the following: the type of AI task, its usage scenario, data format, etc. Based on the text file, a corresponding scene description is generated, a corresponding task graph is constructed, and configurable items and parameters are extracted. Based on the scene description and the task graph, the corresponding scene configuration, static resources, program code files, and executable file descriptions are obtained through mapping, and combined with the extracted configurable items and parameters, to generate structured text corresponding to the task recipe.

[0022] In one embodiment, the AI ​​task type is, for example, "visual AI workpiece recognition," the use case is, for example, "workpiece quality inspection," and the data format is, for example, "generating a synthetic 2D image with a resolution of 1920*1080, and the synthetic dataset can be in the standard COCO data format (Common Objects in Context, COCO)." For further explanation, the listed COCO format includes images and image annotation information, and this dataset format can be directly used for AI model training. The scenario description is, for example, "a workshop workpiece inspection station, equipped with an independent camera and an independent light source." The task graph is used to represent the workflow required to generate the dataset. Assuming that in a scenario, the task graph can represent the complete workflow required to construct the synthetic dataset in the form of task nodes.

[0023] Scene configuration includes: camera configuration, light source configuration, and background configuration; static resources include: images and configuration files.

[0024] Configurable options and parameters include: light source type, number of cameras, and data augmentation type.

[0025] The description of the executable file specifically includes: relevant configurations, executable binary files, etc. The carriers describing the AI ​​task include: text, PDFs, or images describing the AI ​​task.

[0026] In one embodiment, the relevant mapping can leverage the capabilities of a large language model (LLM) to perform mapping that better meets application requirements. By adding semantically descriptive tags to relevant scene descriptions and task graphs, a corresponding LLM database is formed, and the mapping action can be completed through the LLM to output the structured text corresponding to the task recipe.

[0027] Step 103: Based on the name corresponding to the task recipe structured text, obtain the corresponding asset through compiler mapping.

[0028] The name includes the AI ​​task name or a unique code (Identity Document, ID).

[0029] Optionally, prior to step 103, a mapping relationship is established between the asset names contained in the structured text and the corresponding assets in the compiler.

[0030] Step 104: Based on the assets and structured text, data alignment and connection are performed by the compiler, and the task graph executor in the compiler is compiled and packaged to generate a synthetic image generator.

[0031] Specifically, the task graph executor can perform corresponding operations based on the task graph. Assets can include: at least one or more images, program code files, executable binary files, configuration files, or UI code, etc. The relevant assets are used to represent the external dependencies of the task graph executor during execution. Assuming a scenario of "workshop workpiece inspection station, configured with independent cameras and independent light sources", the assets can specifically include, for example, "scene background images, camera and light source configuration files, configuration diagrams, configuration programs, and corresponding UI code".

[0032] In one embodiment, when a user outputs a natural language description of the actual AI task to the synthetic image generator, the generator can output a large number of related synthetic images. Furthermore, the user can also use the generator to output related scene or hardware configurations.

[0033] This application embodiment transforms the carrier describing the AI ​​task into structured text corresponding to the task recipe, then determines the corresponding assets, aligns and connects the obtained structured text and assets, and compiles and packages them together with the task graph executor to generate the corresponding executable program. Through this executable program, the corresponding synthetic image can be output based on the actual task description input by the user.

[0034] The method for generating synthetic images provided in this application significantly improves the efficiency and flexibility of synthetic data generation, and can be precisely and effectively optimized for specific application scenarios. This technological achievement greatly reduces the time data scientists need to invest in preparing datasets. This application aims to provide an easy-to-use synthetic image generator that requires no repetitive adjustments and operations and can be effectively optimized for specific application scenarios to meet the needs of different industries for high-quality synthetic data.

[0035] Through automated and intelligent processing, the synthetic image generator in this embodiment significantly reduces the tedious manual operations and repetitive work in traditional methods. This means that data scientists can generate the required datasets more quickly. Furthermore, this embodiment allows users to flexibly adjust the generator according to different application scenarios. This high degree of customizability enables the generated synthetic data to more accurately reflect the complexity and diversity of the real world.

[0036] Figure 2 is a schematic diagram of an electronic device provided in an embodiment of this application. The specific implementation of the electronic device is not limited by the specific embodiments of this application. Referring to Figure 2, the electronic device 200 provided in this embodiment includes: a processor 202, a communications interface 204, a memory 206, and a communication bus 208. Wherein:

[0037] The processor 202, communication interface 204, and memory 206 communicate with each other via communication bus 208.

[0038] Communication interface 204 is used to communicate with other electronic devices or servers.

[0039] The processor 202 is used to execute program 210, specifically to execute the relevant steps in the aforementioned method 100 embodiment for generating a synthetic image generator.

[0040] Specifically, program 210 may include program code that includes computer operation instructions.

[0041] The processor 202 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0042] Memory 202 is used to store program 210. Memory 206 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0043] This application also provides a computer-readable storage medium storing instructions for causing a machine to execute the method 100 for generating a synthetic image generator as described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0044] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0045] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0046] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0047] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0048] This application also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions. When executed, the computer-executable instructions cause at least one processor to perform the method 100 for generating a synthetic image generator provided in the above embodiments. It should be understood that the solutions in this embodiment have the corresponding technical effects in the above method embodiments, which will not be repeated here.

[0049] It should be noted that not all steps and modules in the above processes and system structure diagrams are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0050] In this patent application, nouns and pronouns relating to people are not limited to specific genders.

[0051] In the above embodiments, the hardware modules can be implemented mechanically or electrically. For example, a hardware module may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operations. The hardware module may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operations. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0052] The present application has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present application is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art will know that more embodiments of the present application can be obtained by combining the code review methods in the different embodiments above. These embodiments are also within the protection scope of the present application.

[0053] In this patent application, nouns and pronouns relating to people are not limited to specific genders.

Claims

A method for generating a synthetic image generator, comprising: - A carrier that receives (101) user input describing the AI ​​task; -Based on the carrier describing the AI ​​task, the structured text corresponding to the task recipe is generated by the task recipe generator (102); -Based on the name corresponding to the structured text of the task recipe, the corresponding asset is obtained through compiler mapping (103); -Based on the assets and the structured text, data alignment (104) and concatenation (104) are performed by the compiler, and the task graph executor in the compiler is compiled and packaged (104) to generate a synthetic image generator. According to the method of claim 1, wherein, - Before generating the structured text corresponding to the task recipe (102) through the task recipe generator based on the carrier describing the AI ​​task, the method further includes: - In the task recipe generator, at least one scene description, task graph, static resource, and executable file description corresponding to the AI ​​task are mapped together; -Based on the carrier describing the AI ​​task, the task recipe generator generates (102) structured text corresponding to the task recipe, including: - Convert the carrier describing the AI ​​task into a text file; wherein the file includes at least one of the following: the type of AI task, the use case, and the data format; - Based on the text format file, generate the corresponding scene description, construct the corresponding task graph, and extract configurable items and parameters; - Based on the scenario description and the task graph, the corresponding scenario configuration, static resources, program code files, and executable file descriptions are obtained through mapping, and combined with the extracted configurable items and parameters, to generate the structured text corresponding to the task recipe. According to the method of claim 1, wherein, Before obtaining the corresponding asset through compiler mapping (103), the method further includes: - The compiler establishes a mapping relationship between the asset names contained in the structured text and the corresponding assets. The method according to claim 1 or 3, wherein, The assets include: at least one or more images, program code files, executable binary files, configuration files, or UI code. According to the method of claim 1, wherein, The carriers describing AI tasks include: - A text, PDF, or image describing the AI ​​task. An electronic device (200) includes: The processor (201), communication interface (202), memory (203), and communication bus (204) communicate with each other through the communication bus (204). The memory (203) is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method of generating a synthetic image generator as described in any one of claims 1-5. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating a synthetic image generator as described in any one of claims 1-5. A computer program product, tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform a method for generating a synthetic image generator according to any one of claims 1-5.

Images