Image data generation apparatus, image data generation method, and computer program

Abstract

This invention relates to an image data generation apparatus, an image data generation method, and a computer program. The image data generation apparatus includes: an object acquisition unit that acquires a learning object created as CG (Computer Graphics); a virtual space generation unit that generates a virtual space in which the learning object is arranged; a background setting unit that sets a background image captured in physical space as the background in the virtual space; and an image data generation unit that generates image data using a captured image obtained by capturing the learning object arranged in the virtual space.

Description

Technical Field

[0001] This disclosure relates to an image data generation apparatus, an image data generation method, and a computer program for generating image data for machine learning. Background Technology

[0002] Various methods for generating image data for machine learning are known. For example, Japanese Patent Application Publication No. 2024-64413 discloses generating multiple virtual spaces by changing various parameters that constitute a virtual space in which learning objects are configured. Furthermore, Japanese Patent Application Publication No. 2024-64413 discloses generating image data by photographing each virtual space from various positions and directions. Summary of the Invention

[0003] Background images in virtual space used to generate image data for machine learning are typically created using CG (Computer Graphics). However, it is often difficult to completely reproduce the background in physical space using CG. Therefore, because the background in physical space differs from the background image created as CG, the accuracy of detecting learning objects in virtual space may be reduced.

[0004] This disclosure can be implemented in the following ways.

[0005] (1) According to one aspect of the present disclosure, an image data generation apparatus for generating image data for machine learning is provided. The image data generation apparatus comprises: The object acquisition section acquires the learning object created as CG (Computer Graphics); The virtual space generation unit generates a virtual space configured with the learning object; The background setting unit sets a background image captured in the physical space as the background in the virtual space; and The image data generation unit generates the image data using images obtained by photographing the learning object disposed in the virtual space.

[0006] According to this method, the background setting unit sets the background image captured in physical space as the background in virtual space. Therefore, compared to the background in virtual space being a structure created by CG, it is possible to more faithfully reproduce physical space in virtual space. Consequently, the accuracy of detecting learning objects in virtual space is improved.

[0007] (2) In the above-described manner, the image data generating apparatus may also include a light source adjustment unit that adjusts parameters related to a virtual light source configured in the virtual space. The image data generation unit generates multiple image data using images captured multiple times at different positions of the virtual light source.

[0008] According to this method, the image data generation unit generates multiple image data by taking multiple images at different positions of the virtual light source. Therefore, in virtual space, image data can be generated using diverse learning objects with different shadow projection methods. Thus, compared to structures that do not change the position of the virtual light source, learning objects in physical space can be reproduced more faithfully in virtual space, further improving the accuracy of detecting learning objects in virtual space.

[0009] (3) Alternatively, in the above method, the light source adjustment unit may change the position of the virtual light source along a predetermined trajectory. According to this method, since the light source adjustment unit changes the position of the virtual light source along a predetermined trajectory, the position changes of the virtual light source in the physical space can be reproduced more faithfully in the virtual space. Therefore, the changes in the way the shadow of the learning object is projected in the physical space can be reproduced more faithfully in the virtual space, and the accuracy of detecting the learning object in the virtual space is further improved. Attached Figure Description

[0010] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which like reference numerals denote like elements, and wherein: Figure 1 This is a block diagram of the image data generation apparatus in this embodiment; Figure 2 These are explanatory diagrams illustrating the generation of image data using the virtual space described in this embodiment; and Figure 3 This is a flowchart illustrating the image data generation process in this embodiment. Detailed Implementation

[0011] A. Implementation method: A1. Structure of the image data generation device 100: Figure 1 This is a block diagram of the image data generation apparatus 100 in this embodiment. The image data generation apparatus 100 generates image data for machine learning. Hereinafter, the final image data for machine learning will be referred to simply as "image data". The image data in this embodiment is used for machine learning in the prototype manufacturing process of a vehicle to automatically determine whether the vehicle's specifications meet predetermined benchmarks through image recognition.

[0012] Figure 2This is an explanatory diagram used to illustrate the generation of image data using the virtual space VS in this embodiment. For example... Figure 2 As shown, image data is generated using a virtual space (VS). A virtual light source (VL) and a learning object (LT) are configured in the virtual space (VS). The learning object LT, configured in the virtual space (VS), is photographed whenever the position of the virtual light source (VL) is changed little by little. Furthermore, the photographing mentioned here refers to taking pictures by a virtual camera (not shown) configured in the virtual space (VS). Image data is generated using the images obtained through this photographing. The detailed process of generating image data will be described later.

[0013] like Figure 1 As shown, the image data generation apparatus 100 includes a processor PR and a memory MM. The processor PR functions as the object acquisition unit 10, the virtual space generation unit 20, the background setting unit 30, the light source adjustment unit 40, and the image data generation unit 50 by executing programs pre-stored in the memory MM. The memory MM is an example of a storage medium. The functional units will be described below.

[0014] The object acquisition unit 10 acquires the learning object LT created as CG (Computer Graphics).

[0015] The virtual space generation unit 20 generates a virtual space VS containing the learning object LT. The virtual space generation unit 20 sets various parameters such as texture, color, position, size, and orientation of the learning object LT in the virtual space VS to generate the virtual space VS.

[0016] Background setting unit 30 sets the background image actually captured in physical space as the background BG in virtual space VS. "Physical space" refers to space that is not virtual space VS, that is, space that actually exists in the body.

[0017] The light source adjustment unit 40 adjusts parameters related to the virtual light source VL configured in the virtual space VS. More specifically, the light source adjustment unit 40 adjusts parameters related to the properties of light emitted from the virtual light source VL, such as brightness, hue, lightness, and saturation, as well as parameters related to the position of the virtual light source VL in the virtual space VS.

[0018] The image data generation unit 50 generates image data. Image data is generated by capturing images of the learning object LT positioned in the virtual space VS.

[0019] A2. Image data generation steps: Figure 3This is a flowchart illustrating the image data generation process in this embodiment. Image data generation begins when the user instructs the image data generation apparatus 100 to perform image data generation. Hereinafter, steps S1 to S5 of the image data generation process will be described.

[0020] In S1, the object acquisition unit 10 acquires the learning object LT, which is created as a CG. The learning object LT is created as 3DCG (3D Computer Graphics). The learning object LT is created using design aids such as CAD (Computer Aided Design). In this embodiment, the learning object LT is a vehicle that becomes the object of specification inspection at the vehicle prototype site.

[0021] In S2, the virtual space generation unit 20 generates a virtual space VS containing the learning object LT created as a 3DCG. The virtual space generation unit 20 sets parameters related to the texture, color tone, etc., of the learning object LT in the virtual space VS in a way that allows for a more faithful reproduction of the learning object LT in the physical space. Furthermore, the virtual space generation unit 20 sets the position, size, and orientation of the learning object LT in the virtual space VS based on the position, size, and orientation of the learning object LT in the image recognition scene.

[0022] In S3, the background setting unit 30 sets the background image captured in the physical space as the background BG in the virtual space VS. In this embodiment, the background image used is an image actually captured inside the factory, which serves as the prototype production site for the vehicle.

[0023] In S4, the light source adjustment unit 40 adjusts the parameters related to the virtual light source VL configured in the virtual space VS. When light emitted from the virtual light source VL shines on the learning object LT, the shadow SD of the learning object LT is projected into the background BG. Therefore, the shadow projected in the physical space can also be reproduced in the virtual space VS.

[0024] In this embodiment, the light emitted from the virtual light source VL reproduces sunlight reaching the factory from windows or other openings in the factory, which serves as the prototype production site. Therefore, as the aforementioned "parameters related to the virtual light source VL," the light source adjustment unit 40 pre-sets various parameters related to the properties of light, such as brightness, hue, lightness, and saturation, emitted from the virtual light source VL, based on the properties of sunlight. Furthermore, the light source adjustment unit 40 sets the trajectory TR of the virtual light source VL based on the temporal variation of sunlight imagined in physical space, and causes the position of the virtual light source VL to change along the set trajectory TR. As the position of the virtual light source VL changes, the position of the shadow SD projected onto the background BG also changes.

[0025] Imagine sunlight entering a factory through multiple windows, being reflected by equipment within the factory, and then illuminating the learning object LT. Therefore, the light source adjustment unit 40 considers these multiple factors to set the trajectory TR of the virtual light source VL. For example... Figure 3 As shown, in this embodiment, the light source adjustment unit 40 sets a spiral trajectory TR and causes the virtual light source VL to change along it. The trajectory TR of the virtual light source VL is set, for example, based on the result of simulating the change of sunlight illuminating the learning object LT in physical space over time.

[0026] In S5, the learning object LT, configured in the virtual space VS, is photographed whenever the position of the virtual light source VL is changed little by little. That is, the learning object LT is photographed multiple times when the position of the virtual light source VL is different from each other. The image data generation unit 50 uses the multiple photographed images obtained from the learning object LT configured in the virtual space VS to generate multiple image data. The multiple photographed images are processed by brightness correction, cropping, etc., to generate the final image data. The generated multiple image data are used to construct an image recognition system using machine learning.

[0027] According to the image data generation apparatus 100 described above, the background setting unit 30 sets the background image actually captured in physical space as the background BG in the virtual space VS. Therefore, compared with the background BG in the virtual space VS being a structure created as CG, physical space can be reproduced more faithfully in the virtual space VS. Therefore, the accuracy of detecting the learning object LT in the virtual space VS is improved.

[0028] Furthermore, the image data generation unit 50 generates multiple image data using images captured multiple times at different positions of the virtual light source VL. Therefore, in the virtual space VS, image data can be generated using diverse learning object LTs with different shadow SD projection methods. Thus, compared to a structure that does not change the position of the virtual light source VL, the learning object LT in the physical space can be reproduced more faithfully in the virtual space VS, further improving the accuracy of detecting the learning object LT in the virtual space VS.

[0029] Furthermore, the light source adjustment unit 40 changes the position of the virtual light source VL along a predetermined trajectory TR, thus enabling a more faithful reproduction of the position changes of the virtual light source VL in the physical space within the virtual space VS. Therefore, the changes in the projection method of the shadow SD of the learning object LT in the physical space can be more faithfully reproduced within the virtual space VS, further improving the accuracy of detecting the learning object LT in the virtual space VS.

[0030] B. Other implementation methods: (B1) In this embodiment, the learning object LT is a vehicle that is subject to specification inspection at the vehicle prototype site. The background BG in the virtual space VS uses a background image actually taken at the vehicle prototype site, i.e., within the factory. However, this disclosure is not limited to this. The type of learning object LT can be arbitrarily selected depending on the object for which image recognition is desired. Furthermore, the background image set as the background BG can be arbitrarily selected depending on the location where image recognition of the learning object LT is desired. In this way, the image data generation apparatus 100 can also be used in various scenarios for generating image data for machine learning.

[0031] (B2) In this embodiment, the object acquisition unit 10 acquires the learning object LT created as a CG, but this disclosure is not limited to this. The object acquisition unit 10 may also have the function of creating the learning object LT as a CG.

[0032] (B3) In this embodiment, the light source adjustment unit 40 changes the position of the virtual light source VL along a predetermined trajectory TR, but this disclosure is not limited to this. For example, the light source adjustment unit 40 may also change the position of the virtual light source VL randomly. In this case, compared with a structure that does not change the position of the virtual light source VL, it is possible to generate image data using a variety of learning object LTs with different shadow SD projection methods.

[0033] (B4) In this embodiment, a virtual light source VL is configured in the virtual space VS, but this disclosure is not limited to this. It is also envisioned that sunlight reaching the factory is reflected by equipment configured in the factory and shines on the learning object LT from multiple directions. In this case, two or more virtual light sources VL may also be set in the virtual space VS. In addition, the positions of two or more virtual light sources VL may be changed separately along a predetermined trajectory TR.

[0034] (B5) In this embodiment, various parameters related to the properties of light, such as brightness, hue, lightness, and saturation, emitted from the virtual light source VL are predetermined based on the properties of sunlight, but this disclosure is not limited thereto. The parameters related to the properties of sunlight, such as brightness and hue, actually vary according to the time of day, so the parameters such as brightness and hue of the light emitted from the virtual light source VL can also be varied accordingly.

[0035] (B6) In this embodiment, the image data generation apparatus 100 includes a light source adjustment unit 40, but it can also be omitted in this disclosure. For example, if it is assumed that sunlight reaching the factory will not be considered, but rather that only light emitted from lighting fixtures located in the factory will illuminate the learning object LT, then it is not necessary to adjust the various parameters of the virtual light source VL. Therefore, the light source adjustment unit 40 can be omitted.

[0036] (B7) In this embodiment, the image data is generated in the order of S1 to S5, but it is not limited to this. For example, after the light source adjustment unit 40 sets the parameters related to the virtual light source VL, the virtual space generation unit 20 may set the parameters related to the learning object LT in the virtual space VS. Alternatively, after setting the background BG in the virtual space VS, the learning object LT created as CG may be placed in the virtual space VS.

[0037] In addition to being implemented as an image data generation apparatus 100, this disclosure can also be implemented in various ways, such as as an image data generation method, a computer program, etc.

[0038] This disclosure is not limited to the embodiments described above, and can be implemented in various structures without departing from its spirit. For example, the technical features in the embodiments corresponding to the technical features in the various methods described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-mentioned problems. Alternatively, to achieve some or all of the above-mentioned effects, appropriate replacements or combinations can be made. In addition, if a technical feature is not described as an essential feature in this specification, it can be appropriately deleted.

Claims

1. An image data generation apparatus for generating image data for machine learning, wherein, The image data generation device includes: The object acquisition section acquires learning objects created as CG, i.e., computer graphics. The virtual space generation unit generates a virtual space configured with the learning object; The background setting unit sets a background image captured in the physical space as the background in the virtual space; as well as The image data generation unit generates the image data using images obtained by photographing the learning object disposed in the virtual space.

2. The image data generation apparatus according to claim 1, wherein, The image data generation device also includes a light source adjustment unit, which adjusts parameters related to the virtual light source configured in the virtual space. The image data generation unit generates multiple image data using images captured multiple times at different positions of the virtual light source.

3. The image data generation apparatus according to claim 2, wherein, The light source adjustment unit causes the position of the virtual light source to change along a predetermined trajectory.

4. An image data generation method for generating image data for machine learning, wherein, The image data generation method includes: The steps to obtain a learning object created as CG, or computer graphics; The step of generating a virtual space configured with the learning object; The step of setting a background image captured in physical space as the background in the virtual space; and The step of generating the image data using images obtained by photographing the learning object configured in the virtual space.

5. A computer program for generating image data for machine learning, wherein, The computer program enables the computer to: To acquire the functionality of learning objects created as CG, i.e., computer graphics; The function of generating a virtual space configured with the learning object; The function of setting a background image captured in physical space as the background in the virtual space; as well as The function of generating image data by taking pictures of the learning object configured in the virtual space.

Images