Program, system and method for generating image

Abstract

To enable a general user not having an illustration-creating skill to easily create illustration content of an original model.SOLUTION: A program for generating an image causes a processor included in a computer to realize: a model acquiring function to acquire a 3D model of an object, an external-feature image generating function to generate an external-feature image showing an external feature of the 3D model based on a pose of the 3D model, and a 2D-model image generating function to generate a 2D-model image as a 2D image on which the 3D model is reflected, based on the external feature image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] At least one embodiment of the present invention relates to an image generating program, an image generating system, and an image generating method. [Background technology]

[0002] Avatars are used in virtual spaces such as social networking sites and the metaverse. When the virtual space is a 3D space, avatars expressed as 3D models may be used.

[0003] Patent Document 1, for example, describes a technology for generating a 3D avatar. The avatar display device includes an acquisition unit that acquires image data of a person, a second acquisition unit that acquires depth data or image data of the person from the image data, a generation unit that synthesizes a 3D model based on the depth data or image data to generate an avatar, and a display unit that displays the avatar in an image display area of ​​an information communication device, the avatar following the movements of the person. This allows a 3D avatar that represents the user to be generated. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-094549 Summary of the Invention [Problem to be solved by the invention]

[0005] Users are increasingly creating original 3D models, including avatars, which can be used in virtual spaces such as social networking sites and the Metaverse.

[0006] However, for a user to create illustration content of an original model such as an original character requires illustration creation skills, which makes it difficult for general users who do not have such skills.

[0007] An object of at least one embodiment of the present invention is to solve the above-mentioned problems and provide an image generation program, an image generation system, and an image generation method that allow ordinary users who do not have illustration creation skills to easily create illustration content of original models. [Means for solving the problem]

[0008] From a non-limiting perspective, an image generation program according to one embodiment of the present invention is intended to cause a processor of a computer to realize a model acquisition function that acquires a 3D model of an object, an appearance feature image generation function that generates an appearance feature image that shows the appearance features of the 3D model based on the pose of the 3D model, and a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image.

[0009] From a non-limiting perspective, an image generation system according to one embodiment of the present invention is an image generation system including at least a processor and a memory, and is configured to cause the processor to realize a model acquisition function that acquires a 3D model of an object, an appearance feature image generation function that generates an appearance feature image that shows the appearance features of the 3D model based on the pose of the 3D model, and a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image.

[0010] From a non-limiting perspective, an image generation method according to one embodiment of the present invention is an image generation method performed by an image generation system including at least a processor and a memory, and includes a model acquisition process that acquires a 3D model of an object, an appearance feature image generation process that generates an appearance feature image that shows the appearance features of the 3D model based on the pose of the 3D model, and a 2D model image generation process that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image. [Effects of the Invention]

[0011] Each embodiment of the present application addresses one or more of the deficiencies. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of an image generation system corresponding to at least one of the embodiments of the present invention. [Figure 2] FIG. 1 is a block diagram showing a configuration of a server according to at least one of the embodiments of the present invention. [Figure 3] FIG. 1 is a block diagram showing an example of the hardware configuration of a server according to at least one of the embodiments of the present invention. [Figure 4] 10 is a flowchart showing an example of processing of an image generating program corresponding to at least one of the embodiments of the present invention. [Figure 5] FIG. 1 illustrates a 3D model according to at least one embodiment of the present invention. [Figure 6] 10A-10C are diagrams illustrating generated 2D model images in accordance with at least one embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a display on a user terminal according to at least one of the embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. Note that the various components in the examples of the embodiments described below can be combined as appropriate to the extent that no inconsistencies or the like arise. Furthermore, content described as an example of one embodiment may be omitted in other embodiments. Furthermore, the content of operations and processes unrelated to the characteristic parts of each embodiment may be omitted. Furthermore, the order of various processes constituting the various flows and sequences described below may be in any order to the extent that no inconsistencies or the like arise in the processing content.

[0014] The following description will be given taking as an example an image generation program executed on a server, which is an example of a computer. However, the computer may be another device, such as a user terminal. Alternatively, the image generation system as a whole may execute the image generation program.

[0015] 1 is a block diagram showing an example of the configuration of an image generation system corresponding to at least one embodiment of the present invention. The image generation system 1 includes a server 10 and a user terminal 20 used by a user of the image generation system 1. Each of the user terminals 20A, 20B, and 20C is an example of the user terminal 20. The configuration of the image generation system 1 is not limited to this. For example, the image generation system 1 may be configured such that multiple users use a single user terminal. The image generation system 1 may also include multiple servers.

[0016] The server 10 and the user terminal 20 are examples of computers. The server 10 and the user terminal 20 are each communicatively connected to a communication network 30 such as the Internet. The connection between the communication network 30 and the server 10 and the connection between the communication network 30 and the user terminal 20 may be a wired connection or a wireless connection. For example, the user terminal 20 may be connected to the communication network 30 by performing data communication with a base station managed by a telecommunications carrier via a wireless communication line.

[0017] The image generation system 1 includes a server 10 and a user terminal 20, and thereby realizes various functions for executing various processes in response to user operations.

[0018] The server 10 includes a processor 11, a memory 12, and a storage device 13. The processor 11 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the server 10 includes a GPU (Graphics Processing Unit), some of the calculations and controls may be performed by the GPU. The server 10 uses the data read into the memory 12 to execute various information processing operations using the processor 11, and stores the obtained processing results in the storage device 13 as necessary.

[0019] The storage device 13 functions as a storage medium for storing various types of information. The configuration of the storage device 13 is not particularly limited, but may be configured to store all of the various types of information necessary for the control performed by the image generation system 1, from the viewpoint of reducing the processing load on the user terminal 20. Examples of such a configuration include an HDD and an SSD. However, the storage device that stores the various types of information only needs to have a storage area accessible by the server 10, and may be configured to have a dedicated storage area outside the server 10, for example.

[0020] The user terminal 20 is managed by a user. Examples of the user terminal 20 include a mobile phone terminal, a smartphone, a PDA (Personal Digital Assistant), a personal computer, a tablet, a stationary game device, a portable game device, etc. The user terminal 20 may be VR goggles, AR glasses, smart glasses, AR contacts, or other wearable devices that can be operated by the user.

[0021] The user terminal 20 is equipped with hardware and software for connecting to the communication network 30 and performing various processes by communicating with the server 10. Each of the multiple user terminals 20 may be configured to be able to communicate directly with each other without going through the server 10.

[0022] The user terminal 20 may have a built-in display device. The display device may also be connected to the user terminal 20 wirelessly or via a wire. Note that the display device is an extremely common configuration and is therefore not shown here. The display device is an example of an output device. Display devices include, for example, a holographic display device capable of displaying holograms, a projection device that projects images (including game screens and information processing screens) onto a screen, and a device that performs XR display. XR includes VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality), among others.

[0023] The user terminal 20 includes a processor 21, a memory 22, and a storage device 23. The processor 21 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and control. Furthermore, if the user terminal 20 includes a GPU (Graphics Processing Unit), some of the various calculations and control may be performed by the GPU. The user terminal 20 uses the data read into the memory 22 to execute various information processing in the processor 21, and stores the obtained processing results in the storage device 23 as necessary. The storage device 23 functions as a storage medium that stores various information.

[0024] The user terminal 20 may have a built-in input device. The input device may also be connected to the user terminal 20 wirelessly or via a wire. The input device accepts operational inputs from the user. In response to the operational inputs from the user, a processor provided in the server 10 or a processor provided in the user terminal 20 executes various control processes. Examples of input devices include touch panel screens provided in smartphones, tablets, etc., keyboards, mice, game pads, joysticks, and other controllers. A camera built into or connected to the user terminal 20 may also serve as an input device. The user performs operational inputs by making gestures such as moving their hands in front of the camera (gesture input). The user terminal 20 may have a microphone. The user can perform voice input by speaking words in a natural language into the microphone.

[0025] Additionally, the user terminal 20 may be equipped with other output devices such as a speaker, etc. The other output devices output sounds, vibrations, and various other types of information to the user.

[0026] 2 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. The server 10 includes a model acquisition unit 101, an appearance feature image generation unit 102, a 2D model image generation unit 103, an appearance feature description generation unit 104, and an appearance feature description acquisition unit 105. The processor included in the server 10 references an image generation program stored in a storage device and executes the program to functionally realize the model acquisition unit 101, the appearance feature image generation unit 102, the 2D model image generation unit 103, the appearance feature description generation unit 104, and the appearance feature description acquisition unit 105.

[0027] The model acquisition unit 101 has a function of acquiring a 3D model of an object. The appearance feature image generation unit 102 has a function of generating an appearance feature image that shows the appearance features of a 3D model based on the pose of the 3D model. The 2D model image generation unit 103 has a function of generating a 2D model image that is a 2D image in which the 3D model is reflected based on the appearance feature image. The appearance feature description generation unit 104 has a function of generating an appearance feature description that is information that describes the appearance features of the 3D model based on the appearance feature image. The appearance feature description acquisition unit 105 has a function of acquiring the appearance feature description.

[0028] An object refers to a physical object in real space or a virtual object that can be placed in a virtual space. Examples of objects include, but are not limited to, a user's virtual avatar, a virtual animal, a virtual vehicle, and a virtual building. For ease of explanation, the following description will be made assuming that an object is an avatar, which is a humanoid character that can be placed in a virtual space.

[0029] The 3D model of the object may be a 3D model in a format such as VRM or fbx, but may also be a 3D model in a format other than these.

[0030] The pose of a 3D model refers to the state in which the shape of the 3D model is changed to a specific shape, if the shape of the 3D model is changeable. For example, if the 3D model is a humanoid character that can be placed in a virtual space, the 3D model may have multiple bones. The joints between the bones correspond to the joints in a human body in the real world. By moving these bones, the shape of the 3D model can be changed. Therefore, a 3D model that is a humanoid character can be posed in various ways, such as crouching, turning around, or bending its arms.

[0031] An appearance feature image showing the appearance features of a 3D model based on the pose of the 3D model may be generated, for example, by capturing an image of the 3D model with a virtual camera placed at a predetermined position in a virtual space while the 3D model is in a predetermined pose. In other words, the appearance feature image may be a 2D image obtained by capturing an image of the 3D model in a predetermined pose from a predetermined direction in the virtual space.

[0032] An appearance feature description refers to information describing features that indicate the appearance of the 3D model, such as "legs crossed" or "hair is red." The appearance feature description may be, for example, a word, phrase, or sentence. The appearance feature description is not limited to being written in Japanese, and may be written in a foreign language, such as "red hair." The appearance feature description may be written in a natural language. Here, "described" means that the information called the appearance feature description is expressed or recorded in that format.

[0033] The appearance feature description may also be written in a format other than natural language. For example, it may be written as a string of characters such as code, whose meaning is not immediately understandable to humans but which a program can interpret. "Written" here means that the information called the appearance feature description is expressed or recorded in that format.

[0034] Fig. 3 is a block diagram showing an example of the hardware configuration of a server corresponding to at least one embodiment of the present invention. The hardware configuration shown in Fig. 3 shows an embodiment in which the image generation system of the present disclosure is implemented using various types of AI. However, the image generation system 1 of the present disclosure does not necessarily have to be implemented using AI.

[0035] The server 10 includes a processor 11, a memory 12, a storage device 13, and an input / output device 14. The processor 11, the memory 12, the storage device 13, and the input / output device 14 are connected to each other via a main bus 15 so as to be able to communicate with each other.

[0036] In the hardware configuration shown in FIG. 3, the processor 11 is equipped with a feature extraction engine 111, an image generation engine 112, a feature explanation engine 113, an image feature instruction generation engine 114, and an appearance feature image creation engine 115.

[0037] The appearance characteristic image creation engine 115 has a function of inputting data of a 3D model in a predetermined pose and generating an appearance characteristic image showing the appearance characteristics of the 3D model. In other words, the appearance characteristic image creation engine 115 is an implementation of the above-mentioned appearance characteristic image generation unit 102.

[0038] The feature extraction engine 111 has a function of extracting appearance feature data indicating the appearance features of a model from an appearance feature image. The feature explanation engine 113 has a function of generating an appearance feature description based on the appearance feature data extracted by the feature extraction engine 111. In other words, the feature extraction engine 111 and the feature explanation engine 113 are implemented as one form of the appearance feature description generator 104 described above.

[0039] The image feature instruction generation engine 114 has a function of generating image feature instructions to be input to the image generation engine 112 based on the appearance feature description. The image feature instruction is information indicating characteristics when issuing a generation instruction to the AI, such as a prompt that can be input to the AI. The image generation engine 112 has a function of generating a 2D model image based on at least one of the appearance feature image, the appearance feature description, and the image feature instruction. For example, the image generation engine 112 generates a 2D model image based on the image feature instruction and the appearance feature image output from the image feature instruction generation engine 114 based on the appearance feature description. In other words, the image feature instruction generation engine 114 and the image generation engine 112 are implemented as one form of the 2D model image generation unit 103.

[0040] The memory 12 temporarily stores, for example, an image feature instruction 121, a 3D model 122, and a 3D model placement instruction 123. The image feature instruction 121 is, for example, a prompt output from the image feature instruction generation engine 114 described above. The 3D model 122 is a 3D model that is the basis for generating a 2D model image. The 3D model placement instruction 123 is information for specifying the pose to be taken by the 3D model. The 3D model placement instruction 123 may include, for example, information indicating the positions and angles of bones of the 3D model. The 3D model placement instruction 123 may also include other information.

[0041] The storage device 13 stores appearance feature data 131 and appearance images 132. As the appearance feature data 131, appearance feature data extracted by the feature extraction engine 111 is stored. As the appearance image 132, an appearance feature image generated by the appearance feature image creation engine 115 is stored.

[0042] FIG. 4 is a flowchart showing an example of processing of an image generating program corresponding to at least one of the embodiments of the present invention.

[0043] The model acquisition unit 101 acquires a 3D model of an object (St101). Based on a 3D model placement instruction, the appearance feature image generation unit 102 generates an appearance feature image that shows the appearance features of the 3D model based on the pose of the 3D model (St102).

[0044] The processor 11 may search the storage device 13 or the like for an appearance feature image that has already been generated and that matches the request content of the image creation request and the conditions indicated by the 3D model placement instruction (St103). If the search finds an appearance feature image that matches the request content of the received image creation request and the conditions indicated by the 3D model placement instruction, the processing of step St102 is unnecessary. The 3D model placement instruction may be an instruction based on a user input via the user terminal 20. The image creation request may be a request based on a user input via the user terminal 20.

[0045] The processor 11 generates an image feature instruction based on the received image creation request (St104). Note that the processor 11 may also acquire an image feature instruction manually input by the user via the user terminal 20.

[0046] The 2D model image generation unit 103 generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image (St105). Note that in this image generation, the 2D model image may be generated based further on image feature instructions.

[0047] The processor 11 transmits the generated 2D model image to the user (St106). Here, transmitting to the user may mean transmitting to the user terminal 20.

[0048] Figure 5 is a diagram illustrating a 3D model according to at least one embodiment of the present invention. Figure 6 is a diagram illustrating a generated 2D model image according to at least one embodiment of the present invention.

[0049] Figure 5 shows a 3D model of an animal that can be placed in a virtual space, and Figure 6 shows four 2D model images generated based on the 3D animal model.

[0050] In step St102 shown in FIG. 4, the appearance feature image generation unit 102 generates an appearance feature image showing the appearance features of the 3D model while the 3D model is posed. Then, in step St105, the 2D model image generation unit 103 generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image. Therefore, as shown in FIG. 6, it is possible to generate 2D model images with different compositions even when based on the same 3D model. Note that a 2D image in which a 3D model is reflected means that the 2D 3D model is included in the 2D image. The contours, etc., of the 3D model and the 2D 3D model do not need to strictly match. The 2D 3D model may be deformed to the extent that a user can recognize it as the same object as the 3D model.

[0051] Furthermore, when using a generative AI, the 2D model image generation unit 103 may generate a 2D model image based not only on an appearance feature image but also on an image feature instruction. In this case, the instruction content indicated in the image feature instruction is reflected in the generated 2D model image. For example, if the prompt "Christmas" is input as an image feature instruction, a fir tree will be reflected in the background of the generated 2D model image (see FIG. 6). If the prompt "swing" is input as an image feature instruction, an animal riding on a swing will be included in the generated 2D model image (see FIG. 6).

[0052] (2D model image using appearance feature description) The above describes an embodiment in which a 2D model image is generated from an appearance feature image. However, the image generation system 1 of the present disclosure may generate an appearance feature description from the appearance feature image, and generate a 2D model image further based on the appearance feature description.

[0053] The appearance feature description generator 104 generates an appearance feature description based on the appearance feature image. For example, descriptions such as "looking back," "peace sign," "fist pump," and "blue hair" are generated based on the appearance feature image. The description generation may be performed using, for example, the feature extraction engine 111 and feature explanation engine 113 described above.

[0054] The 2D model image generation unit 103 generates a 2D model image based on the appearance feature image and the appearance feature description. As a result, the generated 2D model image is generated as an image that more appropriately reflects the features included in the appearance feature image.

[0055] The 2D model image generating unit 103 may generate an image of the 2D model based on the appearance feature image, the appearance feature description, and the image feature instruction.

[0056] (Variations related to the source of appearance feature description) Although the embodiment in which an appearance feature description is generated from an appearance feature image has been described above, the processor 11 may obtain the appearance feature description based on information other than the appearance feature image.

[0057] The appearance feature description acquisition unit 105 has a function of acquiring the appearance feature description. The 2D model image generation unit 103 may generate a 2D model image further based on the appearance feature description acquired by the appearance feature description acquisition unit 105.

[0058] The appearance feature description acquisition unit 105 may acquire appearance feature descriptions that have been registered in advance in association with the 3D model. For example, a correspondence table that associates the identification IDs of 3D models with appearance feature descriptions corresponding to the 3D models indicated by the identification IDs may be stored in the storage device 13 of the server 10. The correspondence table may be stored in a storage device of an external device that can be accessed from the server 10 via the communication network 30. The correspondence table may be stored in the memory 22 or storage device 23 of the user terminal 20. The appearance feature description acquisition unit 105 acquires appearance feature descriptions that have been registered in advance in association with the 3D model from these storage devices.

[0059] Furthermore, the appearance feature description acquisition unit 105 may acquire a description of an appearance feature based on a user input. For example, the appearance feature description acquisition unit 105 acquires an appearance feature description input by a user via the user terminal 20.

[0060] FIG. 7 is a diagram showing an example of a display on a user terminal according to at least one embodiment of the present invention.

[0061] In this example, the user terminal 20 is a smartphone. The smartphone is equipped with a touch panel that serves as both an input device and an output device. At least a 3D model selection area 501, an appearance feature description display area 502, an image feature instruction area 503, and a generated image display area 504 are displayed on the touch panel.

[0062] The 3D model selection area 501 displays one or more selectable 3D models that can be used by the user who manages the user terminal 20. In this example, the available 3D models are avatars that can be placed in virtual space. In the illustrated example, the rightmost 3D model is selected, and a rectangular frame indicating the selection is displayed surrounding the selected 3D model. Note that the user can add selectable 3D models by tapping the + button.

[0063] Appearance feature description display area 502 displays appearance feature descriptions that have been registered in advance in association with the currently selected 3D model. In this example, the appearance feature descriptions are written in English, and appearance feature descriptions such as "1girl" and "looking at viewer" have been registered in advance in association with the selected 3D model. The user can temporarily delete an appearance feature description by tapping an x ​​button located near the displayed appearance feature description.

[0064] A text box is arranged in the image feature instruction area 503. The user manually inputs image feature instructions (prompts) into the text box. In this example, the image feature instructions are input in English. The user inputs one image feature instruction and another image feature instruction separated by a comma. The image feature instructions input into the text box are used to generate a 2D model image.

[0065] When the user taps the generate button, a 2D model image based on the selected 3D model is generated using the appearance feature description and image feature instructions displayed on the user terminal 20. Note that the 2D model image in this example is an illustration, which is a 2D image in which the 3D model is reflected.

[0066] A generated image display area 504 displays a 2D model image that has already been generated.

[0067] As described above, each embodiment of the present application solves one or more deficiencies. Note that the effects of each embodiment are non-limiting effects or examples of effects.

[0068] In each of the above-described embodiments, the user terminal 20 and the server 10 execute the above-described various processes in accordance with various control programs (e.g., image generation programs) stored in their own storage devices. Also, other computers, not limited to the user terminal 20 and the server 10, may execute the above-described various processes in accordance with various control programs (e.g., image generation programs) stored in their own storage devices.

[0069] Furthermore, the configuration of the image generation system 1 is not limited to the configuration described as an example of the above-mentioned embodiment. For example, a configuration may be adopted in which a server executes some or all of the processes described as processes executed by a user terminal, or a configuration may be adopted in which a user terminal executes some or all of the processes described as processes executed by a server. Also, a configuration may be adopted in which the user terminal includes some or all of the storage unit (storage device) included in the server. In other words, a configuration may be adopted in which the user terminal or the server in the image generation system 1 includes some or all of the functions included in the other.

[0070] Furthermore, the program may be configured to cause a part or all of the functions described as examples of each of the above-mentioned embodiments to be realized by a single device that does not include a communication network.

[0071] [Note] The above-described embodiments have been described so that at least the following invention can be implemented by a person having ordinary skill in the art to which the invention pertains.

[0072] [1] The processor in the computer A model acquisition function to acquire a 3D model of an object, an appearance feature image generation function that generates an appearance feature image showing the appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generation program to achieve this.

[0073] According to the image generation program, even general users who do not have the skills to create illustrations can easily create illustration content of original models.

[0074] [2] The processor is further configured to realize an appearance feature description generation function of generating an appearance feature description, which is information describing the appearance features of the 3D model, based on the appearance feature image; the 2D model image generation function generates the 2D model image based on the appearance feature image and the appearance feature description; [1] The image generation program according to [1]. This makes it possible to generate a 2D model image that reflects the contents of the appearance feature description based on the appearance feature image.

[0075] [3] The processor further implements an appearance feature description acquisition function for acquiring the appearance feature description; The 2D model image generation function generates the 2D model image further based on the appearance feature description acquired by the appearance feature description acquisition function. [2] The image generation program according to [2]. This makes it possible to generate a 2D model image that reflects the contents of the appearance feature description specified by a means separate from the appearance feature image.

[0076] [4] The image generation program according to [3], wherein the appearance feature description acquisition function acquires an appearance feature description that has been registered in advance in association with the 3D model. This allows us to generate 2D model images that reflect the contents of the appearance feature descriptions appropriate for each 3D model.

[0077] [5] The image generation program according to [3], wherein the appearance feature description acquisition function acquires an appearance feature description based on user input. This allows us to generate a 2D model image that reflects the appearance feature description entered by the user.

[0078] [6] The appearance feature description is written as a character string. [2] to [5]. An image generating program according to any one of [2] to [5].

[0079] [7] The appearance feature description is written in a natural language. [2] to [5]. An image generating program according to any one of [2] to [5].

[0080] The image generation program described above allows appearance features to be expressed in text or natural language, and a 2D model image can be generated that reflects the content of the appearance feature technology that describes the appearance features.

[0081] [8] 1. An image generation system including at least a processor and a memory, the processor, A model acquisition function to acquire a 3D model of an object, an appearance feature image generation function that generates an appearance feature image showing the appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generation system to achieve this.

[0082] According to the image generation system described above, ordinary users who do not have the skills to create illustrations can easily create illustration content of original models.

[0083] [9] 1. An image generation method performed by an image generation system including at least a processor and a memory, comprising: A model acquisition process for acquiring a 3D model of the object; an appearance feature image generation process for generating an appearance feature image showing appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation process for generating a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generating method comprising:

[0084] According to the image generation method described above, ordinary users who do not have the skills to create illustrations can easily create illustration content of original models. [Industrial Applicability]

[0085] According to one embodiment of the present invention, an image generation program, an image generation system, and an image generation method are useful, which allow general users who do not have illustration creation skills to easily create illustration content of original models. [Explanation of symbols]

[0086] 1. Image generation system 10 Servers 11 processors 12 Memory 13 Storage device 14 Input / Output Devices 15 Main Bus 20, 20A, 20B User terminal 21 processors 22 Memory 23 Storage device 30 Communication Network 101 Model Acquisition Department 102 Appearance feature image generation unit 103 Model image generation unit 104 Appearance feature description generation unit 105 Appearance feature description acquisition unit 111 Feature Extraction Engine 112 Image Generation Engine 113 Feature Description Engine 114 Image Feature Directed Generation Engine 115 Appearance feature image creation engine 121 Image feature indication 122 3D models 123 3D model placement instructions 131 Appearance characteristics data 132 Appearance image 501 3D model selection area 502 Appearance characteristics description display area 503 Image Feature Indication Region 504 Generated image display area

Claims

1. The processor in the computer A model acquisition function to acquire a 3D model of an object; an appearance feature image generation function that generates an appearance feature image showing appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generation program to achieve this.

2. The processor is further configured to realize an appearance feature description generation function of generating an appearance feature description, which is information describing the appearance features of the 3D model, based on the appearance feature image; the 2D model image generation function generates the 2D model image based on the appearance feature image and the appearance feature description; The image generating program according to claim 1 .

3. The processor further implements an appearance feature description acquisition function for acquiring the appearance feature description; the 2D model image generation function generates the 2D model image further based on the appearance feature description acquired by the appearance feature description acquisition function; The image generating program according to claim 2 .

4. The image generating program according to claim 3 , wherein the appearance feature description acquisition function acquires an appearance feature description that is registered in advance in association with the 3D model.

5. 4. The image generating program according to claim 3, wherein the appearance feature description acquisition function acquires an appearance feature description based on a user input.

6. The appearance feature description is written as a character string.

6. The image generating program according to claim 2.

7. The appearance feature description is written in a natural language.

6. The image generating program according to claim 2.

8. 1. An image generation system including at least a processor and a memory, the processor, A model acquisition function to acquire a 3D model of an object; an appearance feature image generation function that generates an appearance feature image showing appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation function that generates a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generation system to achieve this.

9. 1. An image generation method performed by an image generation system including at least a processor and a memory, comprising: a model acquisition process for acquiring a 3D model of an object; an appearance feature image generation process for generating an appearance feature image showing appearance features of the 3D model based on the pose of the 3D model; a 2D model image generation process for generating a 2D model image, which is a 2D image in which the 3D model is reflected, based on the appearance feature image; An image generating method comprising:

Images