Image generation device and image generation method

Abstract

To make it possible to generate an image for training for creating a machine learning model with high detection accuracy.SOLUTION: An image generation device 100 includes an image generation unit 115 for generating an image of a three-dimensional model including an object to which a texture is attached. The texture is an image obtained by processing an extracted texture that is an image extracted from an actual image (see an original image database 210) of an object. The image generation unit 100 includes a plurality of extracted texture, and the image in a region where the plurality of extracted texture overlap may include a texture synthesis unit 113 that synthesizes the processed image based on transparency or a synthetic ratio set for each extracted texture.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image generating device and an image generating method for generating an image of a three-dimensional model to which a texture has been applied. [Background technology]

[0002] Due to labor shortages and the retirement of skilled workers, visual maintenance inspections of facilities and equipment are becoming increasingly automated and mechanized. Automated and mechanized maintenance inspections use machine learning models trained using images of defects, problems, and abnormalities. Training machine learning models requires a large number of images (training images, teacher data, learning data, training data). However, it is difficult to obtain a large number of images of defects, problems, and abnormalities. If the defects, problems, and abnormalities are specific to a facility or equipment, environment, or region, it becomes even more difficult to obtain a large number of images. There are few images of defects, problems, and abnormalities that are specific to a facility or equipment, environment, or region, making it extremely difficult to generate a machine learning model with high detection accuracy.

[0003] As a solution to the lack of training images, there is a method to generate (synthesize) images and use them to train machine learning models. One method of image generation is GAN (Generative Adversarial Network). However, GAN is a machine learning technology and requires a certain amount of images. In addition, there is a problem that the image generation process cannot be precisely controlled.

[0004] Another method of generating images is to use 3D models. In this method, a 3D model with defects, problems, and abnormalities is first generated based on a 3D model of the target facility or equipment, the background, and the texture to be applied (pasted) on the surface of the 3D model. Training images are generated by rendering this 3D model as a 2D image. A large number of images can be generated by adjusting the camera (viewpoint and field of view), lighting, and background when rendering.

[0005] Patent Document 1 and Non-Patent Document 1 disclose inventions as image generation technologies using 3D models. The image generation method disclosed in Patent Document 1 is provided with a library of textures, background images, 3D models, and camera settings, and generates a large number of images using a drawing engine. The quality assessment system disclosed in Non-Patent Document 1 detects rust, cracks, and deformations using 3D models. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US Patent Application Publication No. 2020 / 342652 [Non-patent literature]

[0007] [Non-Patent Document 1] Kim, Alexander, et al., "Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding," Applied Sciences 12 (19) (2022): 10097. [online], [Retrieved May 1, 2023], Internet<URL: https: / / doi.org / 10.3390 / app121910097> Summary of the Invention [Problem to be solved by the invention]

[0008] The machine learning model of CNN (Convolutional Neural Network) is biased towards texture. In other words, the detection process of CNN is highly dependent on texture. For this reason, it is important to use textures that resemble the real thing when applying textures to 3D models. For example, rust as a texture is important in the training data of a machine learning model that detects rust on facilities and equipment. Rust varies greatly depending on the metal properties, environment, and region (e.g., closeness to the sea). For this reason, it is necessary to prepare the correct rust texture. Since abnormalities such as rust and oil leaks occur over time, even if you visit a facility where facilities and equipment are located, there is not necessarily an abnormality. For this reason, it is necessary to collect images of abnormalities that have been acquired in the past, taking into account the facilities, equipment, region, etc., and use textures that resemble the real thing based on these images (rust and oil leaks). It is also desirable to avoid the texture including the shape (contour) of the original facility or equipment.

[0009] In the above example, the rust itself is the detection target, but this is not limited to this. Because the CNN machine learning model is biased towards texture, it is preferable for the texture of the facility or equipment to be as close to the real thing as possible, even if the machine learning model is intended to detect defects, problems, or abnormalities in facilities or equipment other than rust.

[0010] In the image generation method described in Patent Document 1, there is no description of texture generation. In the quality assessment system described in Non-Patent Document 1, textures are generated using a 3D computer graphics creation tool, and random noise is used. As described above, in existing technologies, the texture of a 3D model is not based on a real texture. For this reason, a machine learning model trained using generated images may have false positives or negatives, resulting in low detection accuracy.

[0011] The present invention has been made in consideration of the above background, and has an objective of providing an image generation device and an image generation method that enable the generation of training images for creating a machine learning model with high detection accuracy. [Means for solving the problem]

[0012] In order to solve the above-mentioned problems, the image generating device of the present invention includes an image generating unit that generates an image of a three-dimensional model including an object to which a texture has been applied, and the texture is an image that has been processed from an extracted texture, which is an image extracted from an actual image of an object. Effect of the Invention

[0013] According to the present invention, it is possible to provide an image generating device and an image generating method that enable the generation of training images for creating a machine learning model with high detection accuracy. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a functional block diagram of an image generating apparatus according to an embodiment of the present invention. [Diagram 2] 1A and 1B are diagrams illustrating a composite pattern according to the present embodiment. [Diagram 3] FIG. 2 is a diagram showing a three-dimensional model of a transformer, which is an example of the three-dimensional model according to the present embodiment. [Figure 4] 1A and 1B are diagrams showing attachment patterns of conservators according to the present embodiment. [Diagram 5] 5A to 5C are diagrams showing a pattern of attaching a heat sink according to the embodiment; [Figure 6] 11 is a flowchart of a texture extraction process according to the present embodiment. [Figure 7] 11 is a flowchart of a texture synthesis process according to the embodiment. [Figure 8] 10 is a flowchart of an image generating process according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] <Image Generation Device Overview> An image generating device in a mode (embodiment) for carrying out the present invention will be described below. The image generating device in this embodiment generates training images (teacher data) for a machine learning model that detects abnormal locations where rust has occurred in a transformer. More specifically, the image generating device extracts and collects rust textures from images showing the actual transformer. Next, the image generating device combines the extracted textures to generate a composite texture. Next, the image generating device pastes the composite texture onto a three-dimensional model of the transformer, renders the three-dimensional model, and generates an image.

[0016] Machine learning models such as CNN are easily affected by texture. A machine learning model trained using images of transformers with textures different from the real thing detects anomalies due to the influence of the texture, increasing the possibility of false positives and false negatives. The image of a transformer generated by an image generation device has a texture similar to the real thing. A machine learning model trained using such images is expected to have high detection accuracy.

[0017] In this embodiment, an image generating device that generates an image of a transformer will be described as an example, but the present invention is not limited to this and may be an image generating device that generates images of other objects including other facilities and equipment. In addition, the present invention is described using rust as an example of a texture, but the present invention is not limited to this and may be various textures such as oil leakage. For example, the present invention may be an image for training a machine learning model for detecting anomalies including oil leakage on the surface of a transformer or under a transformer.

[0018] Configuration of the image generating device 1 is a functional block diagram of an image generating device 100 according to this embodiment. The image generating device 100 is a computer, and includes a control unit 110, a storage unit 120, and an input / output unit 180. User interface devices such as a display, a keyboard, and a mouse are connected to the input / output unit 180. The input / output unit 180 may include a communication device, and may be capable of transmitting and receiving data to and from other devices. A media drive may also be connected to the input / output unit 180, and data may be exchanged using a recording medium.

[0019] <Image generating device: memory unit> The storage unit 120 includes storage devices such as a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), etc. The storage unit 120 stores an original image database 210, an extraction target information database 220, an extracted texture database 230, a composite pattern database 240, a composite texture database 250, a three-dimensional model database 260, a pasting pattern database 270, a generated image database 280, and a program 128.

[0020] Images showing facilities / equipment are stored in the original image database 210. In this embodiment, the original image database 210 stores images showing transformers. The extraction target information database 220 stores the names of components from which textures are to be extracted among the components of the facility / device. In this embodiment, the extraction target information database 220 stores the names of parts / devices that rust occurs among the parts / devices that make up the transformer. The extraction target information database 220 also stores information such as the color (color range) and brightness (brightness range) of the texture to be extracted.

[0021] The extracted texture database 230 stores textures extracted from components of the images stored in the original image database 210, whose names are stored in the extraction target information database 220. The synthesis pattern database 240 stores synthesis patterns 410 (see FIG. 2, which will be described later) that are patterns of combinations (synthesis) of the textures stored in the extracted texture database 230. The synthetic texture database 250 stores synthetic textures that are generated by combining (synthesizing) the textures stored in the extracted texture database 230 .

[0022] 2 is a diagram showing a synthesis pattern 410 according to this embodiment. Synthesis pattern 410 is a pattern for synthesizing three textures. The area of ​​the first texture is an area surrounded by a thick dotted line, and is made up of areas 412 to 414. The area of ​​the second texture is an area surrounded by a thick dashed line, and is made up of areas 413 to 417. The area of ​​the third texture is an area surrounded by a thick dashed line, and is made up of areas 416 to 418.

[0023] Area 411 is an area without texture. Areas 413 and 414 are areas where a first texture and a second texture overlap. Areas 416 and 417 are areas where a second texture and a third texture overlap. In each of areas 411 to 418, a transparency is set for each texture. For example, in area 412, 0% is set as the transparency of the first texture. When a composite texture synthesized using synthesis pattern 410 is applied to a three-dimensional model object, the first texture appears in the portion of the object surface corresponding to area 412.

[0024] For example, in area 413, the transparency of the first texture is set to 0% and the transparency of the second texture is set to 50%. When a composite texture synthesized using synthesis pattern 410 is pasted onto a three-dimensional model object, the second texture with a transparency of 50% is pasted onto the first texture in the portion of the object surface corresponding to area 412.

[0025] By setting the synthesis pattern 410 in this way, the user of the image generating device 100 can synthesize a rust texture that suits the characteristics of the device / component to which the synthetic texture is to be applied, the environment of the installation location, and the like.

[0026] The transparency may be replaced with the composition ratio of the texture. For example, in the area 413, the composition ratio of the first texture is 30%, and the composition ratio of the second texture is 40%. Then, when a composite texture synthesized using the composition pattern 410 is pasted onto a three-dimensional model object, the texture synthesized by combining the color (texture) of the object at 30%, the first texture at 30%, and the second texture at 40% is pasted onto the part of the object surface corresponding to the area 413.

[0027] Returning to FIG. 1, the description of the storage unit 120 continues. The 3D model database 260 stores 3D models of facilities / equipment. It is desirable to store 3D models of facilities / equipment of various types / models. Fig. 3 is a diagram showing a 3D model 470 of a transformer, which is an example of a 3D model according to this embodiment. The 3D model 470 is composed of a bushing object 471, a heat sink object 472, a conservator object 473, and the like.

[0028] Returning to FIG. 1, the description of the storage unit 120 continues. The pasting pattern database 270 stores pasting patterns 420, 430 (see FIGS. 4 and 5, described later) that indicate the locations / portions of an object to which a texture is pasted.

[0029] 4 is a diagram showing an attachment pattern 420 of a conservator (see object 473 in FIG. 3) according to this embodiment. The conservator is a container that is installed on top of an oil-filled transformer to store oil in order to prevent oil deterioration in the transformer. In the attachment pattern 420, the side surface of a cylinder is hatched, indicating that a texture showing rust will be attached to the side surface of the conservator on the cylinder.

[0030] 5 is a diagram showing an attachment pattern 430 of a heat sink (see object 472) according to this embodiment. The heat sink is a device for dissipating heat generated by a winding and an iron core in an oil-filled self-cooling transformer. In the attachment pattern 430, the upper part of the heat sink is hatched, indicating that a texture showing rust will be attached to the upper part of the heat sink.

[0031] Returning to FIG. 1, the description of the storage unit 120 continues. The generated image database 280 stores images that are output from the image generating device 100. These images are images that depict a three-dimensional model that includes an object to which a texture is applied, and are training images for a machine learning model. The program 128 includes descriptions of texture extraction processing (see FIG. 6 described later), texture synthesis processing (see FIG. 7 described later), and image generation processing (see FIG. 8 described later).

[0032] <Image generating device: control unit> The control unit 110 includes a CPU (Central Processing Unit) and is provided with a texture extraction unit 111, an image quality improvement unit 112, a texture synthesis unit 113, a three-dimensional model generation unit 114, and an image generation unit 115. The control unit 110 may include a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc.

[0033] The texture extraction unit 111 extracts texture from the images of the facility / equipment stored in the original image database 210, and stores the texture in the extracted texture database 230. Information on the color, brightness, etc. of the texture to be extracted is stored in the extraction target information database 220. Detailed processing of the texture extraction unit 111 will be described later with reference to FIG.

[0034] As described above, the image generating device 100 includes a texture extraction unit 111 that extracts an area of ​​a specified color, an area of ​​a specified luminance, or an area of ​​a specified color and specified luminance (see extraction target information database 220) from an actual image of an object (see original image database 210) as an extracted texture.

[0035] The image quality improving unit 112 performs image quality improvement processing on the image. Image quality improvement methods include, for example, a method using deep learning and bilinear interpolation.

[0036] The texture synthesis unit 113 synthesizes (combines) the textures stored in the extracted texture database 230, using a synthesis pattern 410 (see FIG. 2) stored in the synthesis pattern database 240. The texture synthesis unit 113 stores the synthesized texture (synthetic texture) in the synthesis texture database 250.

[0037] As described above, the image generating device 100 includes the texture synthesis unit 113 that synthesizes a processed image (synthetic texture). The processed image is an image that includes a plurality of extracted textures. The image of the area in which the multiple extracted textures overlap within the processed image is an image in which the multiple extracted textures are superimposed based on the transparency or blending ratio set for each extracted texture.

[0038] The texture synthesis unit 113 synthesizes the processed image based on a synthesis pattern 410 (see Figure 2), which is a template in which the arrangement of the extracted textures in the processed image and the transparency or synthesis ratio of the extracted textures are set for areas where multiple extracted textures overlap.

[0039] The three-dimensional model generating unit 114 pastes the synthetic texture onto objects 472, 473 included in a three-dimensional model 470 of the facility / equipment (see FIG. 3), based on the paste patterns 420, 430 stored in the paste pattern database 270.

[0040] As described above, the image generating device 100 includes a three-dimensional model generating unit 114 that pastes a processed image onto the object 472, 473 in accordance with the pasting patterns 420, 430, which are templates that set the locations where the processed image (synthetic texture) is to be pasted in the object 472, 473 included in the three-dimensional model 470.

[0041] The image generation unit 115 renders a three-dimensional model to generate an image, and stores the image in the generated image database 280 .

[0042] As described above, the image generating device 100 includes the image generating unit 115 that generates an image of the three-dimensional model 470 including the objects 472 and 473 to which textures (synthetic textures) have been applied. A texture (synthesized texture) is an image obtained by processing an extracted texture (see extracted texture database 230), which is an image extracted (by texture synthesis unit 113) from an actual image of an object (see original image database 210).

[0043] <Texture extraction process> 6 is a flowchart of the texture extraction process according to this embodiment. The texture extraction process is a process that is repeatedly executed for each image stored in the original image database 210. By the texture extraction process, textures are extracted from the images stored in the original image database 210 and stored in the extracted texture database 230.

[0044] In step S11, the texture extraction unit 111 cuts out an area (image) in which an object whose name is stored in the extraction target information database 220 appears, from an image stored in the original image database 210. The object in this embodiment is a rusty conservator or radiator in a transformer. The texture extraction unit 111 identifies the area in which the object appears, for example, by using deep learning technology. The texture extraction unit 111 may identify the area in which the object appears in response to an instruction from a user of the image generating device 100.

[0045] In step S12, the image quality improving section 112 starts the process of repeating steps S13 to S21 for each area cut out in step S11. Hereinafter, the image in the area that is the target of this repeated process will be referred to as a processing target image. In step S13, the image quality improving section 112 improves the image quality of the processing target image. In step S14, the texture extraction unit 111 extracts texture from the processing target image whose image quality has been improved in step S13. To explain in detail, the texture extraction unit 111 specifies an area of ​​the image that matches the color and brightness of the texture to be extracted stored in the extraction target information database 220, and extracts the texture.

[0046] In step S15, if the size of the texture extracted in step S14 is equal to or larger than a predetermined value (step S15→YES), texture extraction unit 111 advances the process to step S16. If the size is less than the predetermined value (step S15→NO), texture extraction unit 111 returns the process to step S13 and processes the next image to be processed.

[0047] In step S16, if the texture extraction unit 111 determines that the shape of the texture extracted in step S14 is to be retained (step S16→YES), the process proceeds to step S17. If the texture extraction unit 111 determines that the shape of the texture is not to be retained (step S16→NO), the process proceeds to step S18.

[0048] In this embodiment, the texture is a rust texture and has no specific shape, so the texture extraction unit 111 proceeds to step S18. Note that a bird's nest is an example of a problem / abnormality with a shape related to a transformer. When generating training images for a machine learning model that detects bird's nests, the texture extraction unit 111 proceeds to step S17. A user of the image generating device 100 may set the branch at step S16 in advance.

[0049] In step S17, the texture extraction unit 111 stores the texture in the extracted texture database 230. To explain in detail, the texture extraction unit 111 stores a pair of the image with high image quality in step S13 and the mask that leaves the area of ​​the texture extracted in step S14 in the extracted texture database 230 as a texture.

[0050] In step S18, the texture extraction unit 111 identifies the texture area with the longest contour among areas showing one or more textures included in the image to be processed that has been made high quality. In step S19, the texture extraction unit 111 divides the rectangular region surrounding the region of the texture identified in step S into a plurality of rectangular regions. For example, the texture extraction unit 111 divides the rectangular region into M×N rectangular divided regions.

[0051] In step S20, the texture extraction unit 111 starts the process of repeating step S21 for each of the divided regions obtained in step S20. In step S21, the texture extraction unit 111 masks the texture of the divided region. In other words, the texture extraction unit 111 assumes that there is no texture in the divided region (that is, that the texture was not extracted in step S14). Next, the texture extraction unit 111 stores the texture (a pair of the high-quality image and the mask) in the extracted texture database 230.

[0052] In step S11, the texture extraction unit 111 cuts out an object region (image) from the original image. Therefore, the texture does not include information related to objects or the background other than the object from which the texture was extracted that appears in the current image. Furthermore, in step S21, texture extraction unit 111 removes the divided regions from the texture, so that the texture does not include information related to the contour of the object from which the texture was extracted.

[0053] In this way, the texture extraction unit 111 removes context information of the texture in the current image (information related to the outline (shape information) of the object from which the texture was extracted, objects other than the object from which the texture was extracted, and information related to the background) when extracting the texture. Therefore, the texture does not include the context information contained in the current image, and is a generic texture without information specific to the current image.

[0054] As described above, the texture extraction unit 111 removes context information related to the extracted texture in an image in which an object appears. The context information is any one of the contour of an object in an image in which the object appears, an object different from the object in the image, and the background.

[0055] <Texture synthesis processing> 7 is a flowchart of the texture synthesis process according to this embodiment. The textures stored in the extracted texture database 230 are synthesized (combined) and stored in the synthesized texture database 250 by the texture synthesis process.

[0056] In step S31, the texture synthesis unit 113 calculates a combination of multiple textures from the textures stored in the extracted texture database 230. In this embodiment, the textures represent rust, and the combination of textures is calculated taking into consideration the characteristics of the metal on which the rust occurs, the environment, the region, and other conditions. In other words, rust textures that match the conditions are combined and calculated. For example, combinations of rust textures on different metals are not calculated.

[0057] In step S32, the texture synthesis unit 113 starts the process of repeatedly executing steps S33 and S34 for each combination calculated in step S31. In step S33, the texture synthesis unit 113 starts the process of repeatedly executing step S for each synthesis pattern 410 (see FIG. 2) stored in the synthesis pattern database 240. In step S 34 , the texture synthesis unit 113 uses the synthesis pattern 410 to synthesize the textures included in the combination to generate a synthetic texture, and stores the synthetic texture in the synthetic texture database 250 .

[0058] <Image generation processing> 8 is a flowchart of the image generation process according to this embodiment. The image generation process is a process for generating an image of a 3D model of the facility / equipment stored in the 3D model database 260 and a 3D model with texture applied based on a synthetic texture stored in the synthetic texture database 250.

[0059] In step S41, the three-dimensional model generating unit 114 starts the process of repeatedly executing steps S42 to S44 for each three-dimensional model of the facility / equipment stored in the three-dimensional model database 260. In step S42, the three-dimensional model generating unit 114 starts the process of repeatedly executing step S43 for each object included in the three-dimensional model.

[0060] In step S43, the three-dimensional model generation unit 114 refers to pasting patterns 420, 430 (see Figs. 4 and 5) corresponding to the object, and pastes a synthetic texture stored in the synthetic texture database 250 onto the object. The three-dimensional model generation unit 114 selects a synthetic texture to be pasted onto the object, taking into consideration the material (metal) of the device / part indicated by the object and the environment and region in which the object is placed. If there is no pasting pattern 420, 430 corresponding to the object, the three-dimensional model generation unit 114 skips step S43. In step S44, the image generating unit 115 renders the three-dimensional model to generate an image, and stores the image in the generated image database 280.

[0061] <Features of the image generating device> The image generating device 100 extracts and collects textures from actual images of facilities / equipment. The image generating device 100 pastes the collected textures onto the facilities / equipment objects by referring to the pasting patterns 420, 430. When the user sets the pasting patterns 420, 430 in accordance with the texture of the actual object, the image generating device 100 can generate an image of the facilities / equipment with a texture similar to the actual object.

[0062] Furthermore, the image generating device 100 generates a synthetic texture by referring to a synthetic pattern 410. When a user sets various synthetic patterns 410, the image generating device 100 can generate a large number of synthetic textures. A machine learning model trained using images generated by such an image generating device 100 can be expected to have high detection accuracy.

[0063] <<Modification: Image Generation>> In the process of drawing a three-dimensional model and generating an image (see step S44), the image generating unit 115 may generate an image by drawing the same three-dimensional model while changing the position, height, direction, viewing angle, etc. of the viewpoint (of the camera). Furthermore, the image generating unit 115 may generate an image by drawing while changing the lighting (number, position, brightness, etc. of light sources) and background. By drawing under different conditions in this way, the image generating device 100 can generate a large number of images. In turn, by using a large number of training images, it becomes possible to generate a machine learning model with higher detection accuracy.

[0064] Furthermore, when there are multiple pasting patterns 420, 430 for one object, a three-dimensional model may be generated by pasting textures according to the respective pasting patterns 420, 430, and an image may be generated. The image generating device 100 can generate a large number of images by generating images while changing the synthetic texture, pasting pattern, viewpoint, and lighting.

[0065] <<Variation: Texture Extraction>> The texture extraction unit 111 extracts texture according to conditions such as the color (color range) and brightness (brightness range) of the texture to be extracted stored in the extraction target information database 220. The extraction target information database 220 stores the specified color of the equipment / component from which the texture is to be extracted, and the texture extraction unit 111 may extract an area of ​​the equipment / component that deviates from the specified color as a texture. In this way, the texture extraction unit 111 can extract discolored parts of the equipment / component. As a result, the image generating device 100 can generate an image of the facility / equipment that reflects discoloration due to aging.

[0066] As described above, the image generating device 100 includes a texture extracting section 111 that extracts an image of a discolored portion of an object as an extracted texture.

[0067] <<Variation: Texture synthesis>> In the above embodiment, the texture synthesis unit 113 generates a synthetic texture by combining textures extracted from real images. The texture synthesis unit 113 may generate a synthetic texture by combining synthetic textures. In this way, the image generating device 100 can generate a large number of images.

[0068] <<Variation: 3D model>> In the above embodiment, the 3D model of the facility / equipment to which the texture is applied is stored in the 3D model database 260. The objects constituting the 3D model and the rules for combining the objects may be set in advance, and the 3D model generating unit 114 may generate a 3D model and apply the texture to the generated 3D model. In this way, the image generating device 100 can generate a large number of images.

[0069] Other variations Although several embodiments and modifications of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0070] 100 Image generating device 111 Texture Extraction Unit 112 High-definition image processing section 113 Texture synthesis unit 114 3D model generation unit 115 Image Generation Unit 410 Composite Pattern 420,430 Paste pattern 470 3D models 471,472,473 objects

Claims

1. an image generating unit that generates an image of a three-dimensional model including an object to which a texture is applied; The texture is an image extracted from a real image of an object, and the extracted texture is processed. Image generating device.

2. a texture synthesis unit that synthesizes the processed image, the processed image is an image including a plurality of the extracted textures, The image of the area where the multiple extracted textures overlap among the areas of the processed image is an image in which the multiple extracted textures are overlapped based on the transparency or blending ratio set for each of the extracted textures.

2. The image generating device of claim 1.

3. The texture synthesis unit includes: The processed image is synthesized based on a synthesis pattern, which is a template in which the arrangement of the extracted texture in the processed image and the transparency or synthesis ratio of the extracted texture are set for a region in which a plurality of the extracted textures overlap.

3. The image generating device of claim 2.

4. A texture extraction unit is provided for extracting, from the actual image of the object, an area of ​​a specified color, an area of ​​a specified luminance, or an area of ​​a specified color and a specified luminance as the extracted texture.

2. The image generating device of claim 1.

5. A texture extraction unit is provided for extracting an image of a discolored portion of the object as the extracted texture.

2. The image generating device of claim 1.

6. The texture extraction unit excluding context information related to the extracted texture in the image in which the object appears 6. An image generating device according to claim 4 or 5.

7. The context information comprises: In an image in which the object appears, the object is either the outline of the object, an object other than the object that appears in the image, or the background.

7. The image generating device of claim 6.

8. a three-dimensional model generating unit that pastes the processed image onto the object in accordance with a pasting pattern that is a template in which a portion to paste the processed image onto the object included in the three-dimensional model is set; 2. The image generating device of claim 1.

9. An image generating device, generating an image of the three-dimensional model including the textured object; The texture is an image extracted from a real image of an object, and the extracted texture is processed. Image generation method.

Images