Image dataset generation method, electronic device, medium, and program product

By adding texture maps to the CAD model and simulating defective objects, and generating image data sets from multiple camera perspectives, the problem of difficulty in obtaining defective image data in AI model training is solved, and efficient and accurate defect detection is achieved.

WO2024229659A1PCT designated stage expired Publication Date: 2024-11-14SIEMENS AG +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/092803
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing AI models require a large number of real defect-containing objects for training in defect detection, but collecting and labeling these images is time-consuming and labor-intensive, and it is difficult to obtain a sufficient amount of image data containing defect information.

Method used

By acquiring the CAD model, adding texture maps and adjusting to simulate defect objects, rendering from multiple camera perspectives generates image datasets, and generating corresponding labels in the renderings, achieving rapid acquisition of high-quality image datasets containing defect information .

Benefits of technology

It realizes the rapid and efficient generation of high-quality image data sets, improves the accuracy of AI models for defect recognition, and can replace manual use in defect detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023092803_14112024_PF_FP_ABST
    Figure CN2023092803_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide an image dataset generation method, comprising: acquiring a first CAD model; adding a first texture map to the first CAD model; acquiring a first defect object; adjusting the first CAD model on the basis of the first defect object to obtain a second CAD model; rendering the second CAD model from a plurality of camera perspectives to obtain a plurality of render graphs; and generating in each of the plurality of render graphs a tag corresponding to the first defect object.
Need to check novelty before this filing date? Find Prior Art

Description

Method for generating image data set, electronic device, medium and program product Technical Field

[0001] The embodiments of the present application mainly relate to the field of artificial intelligence, and in particular to a method for generating an image dataset, an electronic device, a medium, and a program product. Background Art

[0002] In recent years, demand for AI-based defect detection in products and parts has grown rapidly. However, these AI models require a large dataset of relevant images of objects containing defects. However, collecting these real-world images is not only time-consuming and labor-intensive, but also requires manual labeling of defect locations. Furthermore, in real-world scenarios, it is often difficult to collect sufficient image data containing defect information. Therefore, there is an urgent need for methods to quickly acquire relevant image datasets.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a method for generating an image dataset, an electronic device, a medium, and a program product. Through the embodiments of the present application, a high-quality image dataset containing defect information and with label information can be quickly obtained.

[0005] In a first aspect, a method for generating an image dataset is provided, comprising: obtaining a first CAD model; adding a first texture map to the first CAD model; obtaining a first defect object; adjusting the first CAD model based on the first defect object to obtain a second CAD model; rendering the second CAD model under multiple camera perspectives to obtain multiple renderings; and generating a label corresponding to the first defect object in each of the multiple renderings.

[0006] In a second aspect, an electronic device is provided, comprising: at least one memory configured to store computer-readable code; and at least one processor configured to call the computer-readable code and execute each step of the method provided in the first aspect.

[0007] According to a third aspect, a computer-readable medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor executes the steps of the method provided in the first aspect.

[0008] In a fourth aspect, a computer program product is provided, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which, when executed, cause at least one processor to perform the steps of the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The following figures are intended only to illustrate and explain the embodiments of the present application and are not intended to limit the scope of the embodiments of the present application.

[0010] FIG1 is a flowchart of a method for generating an image dataset according to an embodiment of the present application;

[0011] FIG2 is a schematic diagram of an electronic device according to an embodiment of the present application.

[0012] DESCRIPTION OF REFERENCE NUMERALS 100 : Method for generating an image data set 101 - 106 : Method steps 200 : Electronic device 201 : Processor 202 : Communication interface 203 : Memory 204 : Communication bus 205 : Program DETAILED DESCRIPTION

[0013] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that discussing these embodiments is merely to enable those skilled in the art to better understand and implement the subject matter described herein, and is not intended to limit the scope of protection, applicability, or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the embodiments of the present application. Various examples may omit, replace, or add various processes or components as needed. For example, the described method may be performed in an order different from the described order, and various steps may be added, omitted, or combined. In addition, the features described relative to some examples may also be combined in other examples.

[0014] As used herein, the term "including" and its variations are open terms meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0015] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0016] FIG1 is a flow chart of a method for generating an image dataset according to an embodiment of the present application. As shown in FIG1 , the method 100 for generating an image dataset includes:

[0017] Step 101: Obtain a first CAD model.

[0018] Optionally, the first CAD model may include only one CAD model, or may include multiple different / same CAD models.

[0019] Step 102: Add a first texture map to the first CAD model.

[0020] The first texture map is added to the first CAD model in a texture mapping manner, wherein the texture mapping manner may include: spherical mapping, cubic surface mapping, or the first texture map includes texture coordinates, and the first texture map is mapped based on the texture coordinates.

[0021] Step 103: Acquire the first defect object.

[0022] Step 104: Based on the first defect object, adjust the first CAD model to obtain a second CAD model.

[0023] Step 105 : Render the second CAD model under multiple camera viewing angles to obtain multiple renderings.

[0024] Optionally, the second CAD model is rendered under multiple camera perspectives and under each of multiple sets of rendering conditions. Each of the multiple sets of rendering conditions includes a background image, lighting parameters, camera parameters, and a position and attitude of the second CAD model. The position refers to the three-dimensional coordinates of the model, and the attitude refers to the yaw, roll, and pitch angles of the model.

[0025] Step 106 : Generate a label corresponding to the first defect object in each of the plurality of rendering images.

[0026] The embodiment of the present application adds a texture map to a CAD model to make the object represented by the CAD model more realistic. The CAD model is then adjusted by the acquired first defect object, that is, the defect information is reflected on the CAD model. For the CAD model with defects, rendering is performed under multiple camera perspectives, and labels corresponding to the defective objects are generated in each rendering image to obtain a corresponding image data set. Through the embodiment of the present application, an image data set can be generated quickly and with high quality for training of relevant AI models, thereby improving the accuracy of the relevant AI models for defect identification. In a practical application link, the relevant AI model can be built into a robot, and in conjunction with a machine vision system, it can replace manual labor to sort out defective workpieces.

[0027] The embodiments of the present application can be applied to various defect types, including surface defects, missing parts, and mis-assembly defects.

[0028] In one embodiment, first, a first CAD model is obtained. A first texture map is added to the first CAD model. Then, a first defect object can be obtained by one of the following two methods:

[0029] First, a defect library is established, wherein the defect library includes a plurality of preset surface defect texture maps, and then a first surface defect texture map is obtained from the defect library.

[0030] Second, a shape map of the surface defect is generated, the shape map including the first shape. At least one texture map is generated, such as a color texture map, a normal texture map, a roughness texture map, and / or a metallic texture map. The pixel values ​​of the texture maps can be completely random or follow a distribution, such as a Gaussian distribution. The shape map of the surface defect is combined with each of the at least one texture maps to generate a first defect texture map.

[0031] Optionally, the shape image of the surface defect is a binary image. Specifically, the defect shape can be, for example, a scratch or a crack, whose pixel value is set to 255, while the rest of the shape image is set to 0. In this case, the first shape refers to the area corresponding to the pixel value of 255.

[0032] Alternatively, the shape of the surface defect can be randomly generated by using a region growing algorithm. Alternatively, the shape of the surface defect can be obtained by using a preset template.

[0033] Next, the first defect texture map is added to the first texture map to adjust the texture of the first CAD model, thereby obtaining a second CAD model. Optionally, the first defect texture map can be further modified to obtain a modified first defect texture map. Such modifications may include scaling, rotating, copying, and / or translating. The modified first defect texture map is then added to the first texture map.

[0034] Optionally, in the case of multiple shape maps and multiple texture maps of surface defects, the specific method may be: randomly generate multiple shape maps of surface defects, and then generate multiple texture maps. Combine each shape map in the multiple shape maps of surface defects with each texture map in the multiple texture maps to obtain multiple groups of defect texture maps, wherein each group of defect texture maps has the same shape as each other but contains different textures. Then, based on the multiple groups of defect texture maps, adjust the first CAD model separately. Assuming that the first shape map and the second shape map are randomly generated, and the generated texture map includes a metalness texture map and a roughness texture map, then the multiple groups of defect texture maps obtained by combination include: the first multiple groups of defect texture maps: the first shape map including the metalness texture, the first shape map including the roughness texture; and the second multiple groups of defect texture maps: the second shape map including the metalness texture, the second shape map including the roughness texture.

[0035] Again, the second CAD model is rendered under multiple camera perspectives to obtain multiple renderings.

[0036] Finally, in each of the multiple renderings, all triangular facets covered by the first shape in the first defect texture image on the second CAD model are framed to generate corresponding labels. Optionally, when multiple triangular facets involved in the first defect texture image on the second CAD model are connected, they can be framed with the same frame. Optionally, after obtaining the relevant graphic data set, the image data that does not contain label information in the image data set can be deleted, and the remaining image data can be used as the final graphic data set. By filtering out valid data, the training time of the relevant AI model can be further reduced.

[0037] The embodiment of the present application is particularly suitable for generating image data sets corresponding to CAD models with surface defects. Not only is the generation efficiency high, but the synthetic images obtained by simulating real surface defect situations and a rich combination of rendering conditions can be closer to real images.

[0038] In one embodiment, first, a first CAD model is obtained. A first texture map is added to the first CAD model. Secondly, a first component is obtained, which is a part of the first CAD model. Optionally, the first component can be obtained in a user-specified manner or in a randomly determined manner on the first CAD model. Again, the first component is deleted from the first CAD model to obtain a second CAD model. Finally, the second CAD model is rendered under multiple camera perspectives to obtain multiple renderings. In each of the multiple renderings, the position where the first component is missing in the second CAD model is framed to generate a corresponding label. Optionally, after obtaining the relevant graphic data set, the image data that does not contain label information in the image data set can be deleted, and the remaining image data can be used as the final graphic data set.

[0039] The embodiment of the present application is particularly suitable for generating image data sets corresponding to CAD models with missing components. Not only is the generation efficiency high, but the synthetic images obtained by simulating real missing situations and combining rich rendering conditions can be closer to real images.

[0040] FIG2 is a schematic diagram of an electronic device according to an embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the electronic device. As shown in FIG2 , the electronic device 200 may include: a processor 201, a communications interface 202, a memory 203, and a communication bus 204. Among them:

[0041] The processor 201 , the communication interface 202 , and the memory 203 communicate with each other via the communication bus 204 .

[0042] The communication interface 202 is used to communicate with other electronic devices or servers.

[0043] The processor 201 is configured to execute the program 205 , and specifically may execute the relevant steps in any one of the aforementioned method embodiments.

[0044] Specifically, the program 205 may include program codes, which include computer operation instructions.

[0045] The processor 201 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0046] The memory 203 is used to store the program 205. The memory 203 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0047] The program 205 can be specifically used to enable the processor 201 to execute any one of the multiple method embodiments in the aforementioned embodiments.

[0048] The specific implementation of each step in program 205 can be found in the corresponding descriptions of the corresponding steps and units in the aforementioned data interaction model updating method embodiment, and will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the aforementioned method embodiment, and will not be repeated here.

[0049] The present application also provides a computer-readable storage medium storing instructions for causing a machine to perform any of the multiple method embodiments described herein. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and a computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.

[0050] In this case, the program code read from the storage medium itself can realize the function of any one of the above embodiments, so the program code and the storage medium storing the program code constitute part of this application.

[0051] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0052] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to perform any corresponding operation in the above-mentioned multiple method embodiments.

[0053] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0054] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or can be implemented as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0055] It should be noted that not all steps and modules in the above processes and system structure diagrams are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0056] In the above embodiments, the hardware module can be implemented mechanically or electrically. For example, a hardware module can include a permanent dedicated circuit or logic (such as a dedicated processor, FPGA or ASIC) to complete the corresponding operation. The hardware module can also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which can be temporarily set by software to complete the corresponding operation. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0057] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A method for generating an image dataset, comprising: - Obtaining (101) a first CAD model; - adding (102) a first texture map to the first CAD model; - Obtain (103) a first defect object; - based on the first defect object, adjusting (104) the first CAD model to obtain a second CAD model; - rendering (105) the second CAD model under multiple camera viewing angles to obtain multiple renderings; - In each of the plurality of renderings, generating (106) a label corresponding to the first defect object.

2. The method according to claim 1, wherein: Adding (102) a first texture map to the first CAD model includes: - adding a first texture map to the first CAD model in the form of texture mapping.

3. The method according to claim 1, wherein: The obtaining (103) of the first defect object comprises: - establishing a defect library, wherein the defect library includes a plurality of preset surface defect texture maps; - Obtaining a first surface defect texture map from the defect library.

4. The method according to claim 1, wherein: The obtaining (103) of the first defect object comprises: - generating a shape map of the surface defect, the shape map comprising a first shape; -Generate at least one texture map; - Combining the shape map of the surface defect with at least one texture map to obtain a first defect texture map.

5. The method according to claim 4, wherein: Generating at least one texture image comprises: -Generate a color texture map, a normal texture map, a roughness texture map and / or a metalness texture map.

6. The method according to claim 4 or 5, wherein: The adjusting (104) the first CAD model based on the first defect object to obtain a second CAD model includes: - Adding the first defect texture map to the first texture map to adjust the texture of the first CAD model to obtain a second CAD model.

7. The method according to claim 6, wherein: Adding the first defect texture map to the first texture map comprises: - changing the first defect texture map to obtain a changed first defect texture map; wherein the change includes: scaling, rotating, copying and / or translating; -Adding the changed first defect texture map to the first texture map.

8. The method according to claim 6, wherein: In each of the plurality of renderings, generating (106) a label corresponding to the first defect object includes: - In each of the plurality of renderings, all triangular facets covered by the first shape in the first defective texture image on the second CAD model are framed to generate corresponding labels.

9. The method according to claim 1, wherein: - The obtaining (103) of the first defect object comprises: - obtaining a first component, wherein the first component is a part of the first CAD model; - Based on the first defect object, adjusting (104) the first CAD model to obtain a second CAD model comprises: - Deleting the first component from the first CAD model to obtain a second CAD model.

10. The method according to claim 9, wherein: In each of the plurality of renderings, generating (106) a label corresponding to the first defect object includes: - In each of the plurality of renderings, a position in the second CAD model where the first component is missing is framed to generate a corresponding label.

11. The method according to claim 1, wherein: The rendering (105) of the second CAD model under multiple camera viewing angles comprises: - Rendering the second CAD model under multiple camera perspectives and under each of multiple sets of rendering conditions; wherein each of the multiple sets of rendering conditions includes: a background image, lighting parameters, camera parameters, and a position and posture of the second CAD model.

12. An electronic device (200), comprising: A processor (201), a communication interface (202), a memory (203) and a communication bus (204), wherein the processor (201), the memory (203) and the communication interface (202) communicate with each other via the communication bus (204); The memory (203) is used to store at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the method for generating an image data set according to any one of claims 1 to 11.

13. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for generating an image data set according to any one of claims 1 to 11 is implemented.

14. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions which, when executed, cause at least one processor to perform the method for generating an image dataset according to any one of claims 1-11.

Citation Information

Patent Citations

  • Texture mapping method, apparatus, computer device, and storage medium

    CN109410302A

  • Object spatial data augmentation method and system based on three-dimensional reconstruction

    CN111932673A

  • Defect sample data generation method and device

    CN113706540A

  • Ceramic cup defect sample amplification method and system based on adversarial neural network

    CN114529689A

  • Sample data generation method and device, electronic equipment and storage medium

    CN115601616A