Ai model training method, and electronic device, medium and computer program product

Abstract

The embodiments of the present application mainly relate to the field of artificial intelligence, and particularly relate to an AI model training method, and an electronic device, a medium and a computer program product. The method comprises: acquiring a plurality of images of a real scene; generating a plurality of composite images corresponding to the real scene; determining, from among the plurality of composite images, composite images that are closest to each of the plurality of images of the real scene, so as to obtain a first group of composite images; deleting the first group of composite images from the plurality of composite images, so as to obtain a second group of composite images; and inputting the second group of composite images and the plurality of images of the real scene into an AI model together for training.

Description

AI model training method, electronic device, medium and computer program product Technical Field

[0001] The embodiments of the present application mainly relate to the field of artificial intelligence, and in particular to a training method, electronic device, medium and computer program product for an artificial intelligence (AI) model. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, it has been widely applied in various fields. The performance of AI models is undoubtedly crucial in their application, especially in industrial scenarios. It is well known that the performance of AI models is closely related to the AI ​​model structure and the corresponding training dataset. However, AI model structures are generally open source and relatively fixed. Therefore, improving the quality of training datasets and, in turn, the performance of AI models is a pressing issue. Currently, real-world images are commonly used as training datasets. However, this method is not only time-consuming and labor-intensive, but also often contains unintentional biases during the data collection process, resulting in poor performance of the trained AI models.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide an AI model training method, electronic device, medium, and computer program product. Through the embodiments of the present application, a high-quality training data set can be quickly obtained to train a high-performance AI model.

[0005] In a first aspect, a method for training an AI model is provided, comprising: acquiring multiple images of a real scene; generating multiple synthetic images corresponding to the real scene; determining, among the multiple synthetic images, a synthetic image that is closest to each image in the multiple images of the real scene to obtain a first set of synthetic images; deleting the first set of synthetic images from the multiple synthetic images to obtain a second set of synthetic images; and inputting the second set of synthetic images and the multiple images of the real scene into the AI ​​model for training.

[0006] In a second aspect, another method for training an AI model is provided, including: acquiring multiple images of a real scene; generating multiple synthetic images corresponding to the real scene; determining, among the multiple synthetic images, a synthetic image that is closest to each image in the multiple images of the real scene to obtain a first set of synthetic images; deleting the first set of synthetic images from the multiple synthetic images to obtain a second set of synthetic images; based on the rendering conditions corresponding to each synthetic image in the second set of synthetic images, acquiring a rendering condition range corresponding to the second set of synthetic images; based on the rendering condition range, acquiring a first set of real images; and inputting the first set of real images and the multiple images of the real scene into the AI ​​model for training.

[0007] In a third aspect, an electronic device is provided, comprising: at least one memory configured to store computer-readable code; and at least one processor configured to call the computer-readable code and execute each step of the method provided in the first aspect or the second aspect.

[0008] In a fourth aspect, a computer-readable medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor executes each step in the method provided in the first aspect or the second aspect.

[0009] In a fifth aspect, a computer program product is provided, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which, when executed, cause at least one processor to perform the steps of the method provided in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The following figures are intended only to illustrate and explain the embodiments of the present application and are not intended to limit the scope of the embodiments of the present application.

[0011] FIG1 is a flowchart of a method for training an AI model according to an embodiment of the present application;

[0012] FIG2 is a flowchart of another AI model training method according to an embodiment of the present application;

[0013] FIG3 is a schematic diagram of an electronic device according to an embodiment of the present application.

[0014] Description of reference numerals 100: AI model training method 101-105: method steps 200: AI model training method 201-207: method steps 300: electronic device 301: processor 302: communication interface 303: memory 304: communication bus 305: program DETAILED DESCRIPTION

[0015] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that discussing these embodiments is merely to enable those skilled in the art to better understand and implement the subject matter described herein, and is not intended to limit the scope of protection, applicability, or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the embodiments of the present application. Various examples may omit, replace, or add various processes or components as needed. For example, the described method may be performed in an order different from the described order, and various steps may be added, omitted, or combined. In addition, the features described relative to some examples may also be combined in other examples.

[0016] As used herein, the term "including" and its variations are open terms meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0017] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0018] FIG1 is a flow chart of a method for training an AI model according to an embodiment of the present application. As shown in FIG1 , the AI ​​model training method 100 includes:

[0019] Step 101: Acquire multiple images of a real scene.

[0020] Step 102: Generate multiple synthetic images corresponding to the real scene.

[0021] Specifically, relevant metadata such as rendering parameters and related CAD models can be used to model the real scene, that is, to build a digital twin of the real scene, and then perform corresponding rendering to obtain a synthetic image corresponding to the real scene.

[0022] Step 103 : Determine, among the multiple synthetic images, a synthetic image that is closest to each of the multiple images of the real scene, to obtain a first group of synthetic images.

[0023] Optionally, feature maps corresponding to multiple images of a real scene are generated to obtain a first set of feature maps, and each feature map in the first set of feature maps is compressed into a corresponding vector to obtain a first set of vectors. Feature maps corresponding to multiple synthetic images are generated to obtain a second set of feature maps, and each feature map in the second set of features is compressed into a corresponding vector to obtain a second set of vectors. The center of the second set of vectors is aligned with the first set of vectors to obtain an aligned second set of vectors. Specifically, the mean of the first set of vectors and the mean of the second set of vectors can be calculated respectively. Based on the difference between the mean of the first set of vectors and the mean of the second set of vectors, the second set of vectors is translated to align the centers of the first set of vectors and the second set of vectors.

[0024] Next, for each vector in the aligned second set of vectors, the closest vector in the first set of vectors is determined. When the distance between a first vector in the aligned second set of vectors and the closest vector in the first set of vectors is below a specified threshold, the composite image corresponding to the first vector is determined as the first set of composite images.

[0025] Optionally, the feature map corresponding to the real scene image / synthetic image can be generated by inputting the relevant image into an artificial intelligence model trained based on a large visual database of visual object recognition software research to output the corresponding feature map.

[0026] In one embodiment, a synthetic image that is closest to each of the multiple images of the real scene may be determined by using a perceptual hashing algorithm or a structural similarity method.

[0027] Step 104 : Delete the first group of composite images from the multiple composite images to obtain a second group of composite images.

[0028] The remaining second group of synthetic images can represent the relevant images corresponding to the conditions that are lacking behind the real image dataset in the real scene.

[0029] In step 105 , the second set of synthetic images and multiple images of real scenes are input into the AI ​​model for training.

[0030] The embodiment of the present application determines the synthetic image set that is closest to the real image in the real scene in the synthetic image dataset, and then deletes the closest synthetic image set in the synthetic image dataset. In this way, it can infer the conditions that are lacking behind the real image dataset in the real scene, such as the lack of bright light or the camera being too close to the object, etc. Then, the remaining synthetic image dataset and the original real image dataset in the real scene are input into the AI ​​model for training. The embodiment of the present application determines the defective parts in the original training dataset before training the AI ​​model, that is, the image dataset corresponding to the conditions that are lacking in the images directly obtained from the real scene. By expanding the original training dataset into a training dataset with wider coverage to train the AI ​​model, it can not only improve the stability and generalization ability of the AI ​​model, but also save the time and energy that would otherwise be needed to repeatedly train the AI ​​model.

[0031] FIG2 is a flow chart of another AI model training method according to an embodiment of the present application. As shown in FIG2 , the AI ​​model training method 200 includes:

[0032] Step 201: Acquire multiple images of a real scene.

[0033] Step 202: Generate multiple synthetic images corresponding to the real scene.

[0034] Specifically, the real scene can be modeled first to obtain a corresponding model, and then the model can be rendered accordingly to obtain a corresponding synthetic image.

[0035] Step 203 : Determine, among the multiple synthetic images, a synthetic image that is closest to each of the multiple images of the real scene, to obtain a first group of synthetic images.

[0036] Step 204: Delete the first group of composite images from the multiple composite images to obtain a second group of composite images.

[0037] Step 205 : Based on the rendering condition corresponding to each synthetic image in the second set of synthetic images, obtain a rendering condition range corresponding to the second set of synthetic images.

[0038] The rendering conditions may include: background settings, camera setting requirements, and light setting requirements. Optionally, the union of the rendering conditions corresponding to each synthetic image in the second set of synthetic images is calculated to obtain the range of rendering conditions corresponding to the second set of synthetic images. Assume that the second set of synthetic images contains 5 synthetic images, wherein the camera angle corresponding to the first synthetic image is 1°, the camera angle corresponding to the second synthetic image is 2°, the camera angle corresponding to the third synthetic image is 3°, the camera angle corresponding to the fourth synthetic image is 4°, and the camera angle corresponding to the fifth synthetic image is 5°. By calculating the corresponding union, one of the rendering conditions corresponding to the second set of synthetic images can be obtained, that is, the range of camera angles is [1° to 5°].

[0039] Step 206 : Acquire a first set of real images based on the rendering condition range.

[0040] Alternatively, assuming that the rendering condition range includes the range of rendering condition A, the range of rendering condition B, and the range of rendering condition C, the first set of real images may be acquired by limiting the range to the ranges of rendering condition A, rendering condition B, and rendering condition C. Alternatively, the first set of real images may be acquired by limiting the range to at least one of the ranges of rendering condition A, rendering condition B, and rendering condition C.

[0041] In step 207 , the first set of real images and multiple images of real scenes are input into the AI ​​model for training.

[0042] The embodiment of the present application determines the synthetic image set that is closest to the real image in the real scene in the synthetic image dataset, and then deletes the closest synthetic image set in the synthetic image dataset. In this way, it is possible to infer the conditions that are lacking behind the real image dataset in the real scene. Then, within the obtained condition range, another set of real image datasets is obtained, and the newly obtained real image dataset and the original real image dataset in the real scene are input into the AI ​​model for training, thereby improving the prediction performance of the trained AI model.

[0043] FIG3 is a schematic diagram of an electronic device according to an embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the electronic device. As shown in FIG3 , the electronic device 300 may include: a processor 301, a communications interface 302, a memory 303, and a communication bus 304.

[0044] The processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0045] The communication interface 302 is used to communicate with other electronic devices or servers.

[0046] The processor 301 is configured to execute the program 302 , and specifically may execute the relevant steps in any one of the aforementioned method embodiments.

[0047] Specifically, the program 305 may include program codes, which include computer operation instructions.

[0048] Processor 301 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0049] The memory 303 is used to store the program 305. The memory 203 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0050] The program 305 can be specifically used to enable the processor 301 to execute any one of the multiple method embodiments in the aforementioned embodiments.

[0051] The specific implementation of each step in program 305 can refer to the corresponding description of the corresponding steps and units in the aforementioned AI model training method embodiment, and will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment, and will not be repeated here.

[0052] The present application also provides a computer-readable storage medium storing instructions for causing a machine to perform any of the multiple method embodiments described herein. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and a computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.

[0053] In this case, the program code read from the storage medium itself can realize the function of any one of the above embodiments, so the program code and the storage medium storing the program code constitute part of this application.

[0054] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0055] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to perform any corresponding operation in the above-mentioned multiple method embodiments.

[0056] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0057] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or can be implemented as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0058] It should be noted that not all steps and modules in the above processes and system structure diagrams are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0059] In the above embodiments, the hardware module can be implemented mechanically or electrically. For example, a hardware module can include a permanent dedicated circuit or logic (such as a dedicated processor, FPGA or ASIC) to complete the corresponding operation. The hardware module can also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which can be temporarily set by software to complete the corresponding operation. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0060] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A method for training an AI model, comprising: - acquiring (101) a plurality of images of a real scene; - generating (102) a plurality of synthetic images corresponding to the real scene; - determining (103) among the plurality of synthetic images a synthetic image that is closest to each of the plurality of images of the real scene, to obtain a first set of synthetic images; - deleting (104) a first group of composite images from the plurality of composite images to obtain a second group of composite images; - Inputting (105) the second set of composite images and the plurality of images of the real scene into the AI ​​model for training.

2. The method according to claim 1, wherein: Determining (103) a synthetic image that is closest to each of the multiple images of the real scene among the multiple synthetic images to obtain a first set of synthetic images includes: - generating feature maps corresponding to a plurality of images of the real scene to obtain a first set of feature maps, and compressing each feature map in the first set of feature maps into a corresponding vector to obtain a first set of vectors; - generating feature maps corresponding to a plurality of synthetic images respectively, obtaining a second set of feature maps, and compressing each feature map in the second set of features into a corresponding vector, obtaining a second set of vectors; - Align the centers of the second set of vectors with the first set of vectors to obtain an aligned second set of vectors; - for each vector in the aligned second set of vectors, determining the closest vector in the first set of vectors; - When the distance between a first vector in the aligned second group of vectors and the closest vector in the first group of vectors is lower than a specified threshold, determining the synthetic image corresponding to the first vector as a first group of synthetic images.

3. The method according to claim 2, wherein: The aligning the centers of the second set of vectors with the first set of vectors comprises: - Calculate the mean of the first set of vectors and the mean of the second set of vectors respectively; - Based on a difference between a mean value of the first group of vectors and a mean value of the second group of vectors, translating the second group of vectors to align centers of the first group of vectors and the second group of vectors.

4. A method for training an AI model, comprising: - acquiring (201) a plurality of images of a real scene; - generating (202) a plurality of synthetic images corresponding to the real scene; - determining (203) among the plurality of synthetic images a synthetic image that is closest to each of the plurality of images of the real scene, to obtain a first set of synthetic images; - deleting (204) a first group of composite images from the plurality of composite images to obtain a second group of composite images; - based on the rendering condition corresponding to each synthetic image in the second set of synthetic images, obtaining (205) a rendering condition range corresponding to the second set of synthetic images; - based on the rendering condition range, obtaining (206) a first set of real images; - Inputting (207) the first set of real images and the plurality of images of the real scene into the AI ​​model for training.

5. The method according to claim 4, wherein: The obtaining (205) a rendering condition range corresponding to the second set of synthetic images based on the rendering condition corresponding to each synthetic image in the second set of synthetic images includes: - Calculating the union of rendering conditions corresponding to each synthetic image in the second set of synthetic images to obtain a rendering condition range corresponding to the second set of synthetic images.

6. The method according to claim 4, wherein: The rendering conditions include: -Background settings, camera settings and lighting settings.

7. An electronic device (300), comprising: A processor (301), a communication interface (302), a memory (303) and a communication bus (304), wherein the processor (301), the memory (303) and the communication interface (302) communicate with each other via the communication bus (304); The memory (303) is used to store at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the training method of the AI ​​model as described in any one of claims 1 to 6.

8. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method of the AI ​​model as described in any one of claims 1 to 6.

9. A computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which, when executed, cause at least one processor to perform the training method of the AI ​​model according to any one of claims 1-6.

Images