Generation apparatus, generation method, and generation program

Abstract

To generate the desired silhouette data. [Solution] The generation apparatus according to this embodiment includes an acquisition unit, a determination unit, and a generation unit. The acquisition unit acquires generation category information, which is information about the type of silhouette to be generated. The determination unit determines a template for the shape. The generation unit inputs the template and the generation category information into a trained model and outputs silhouette data from the trained model corresponding to the shape specified by the template.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a generation device, a generation method, and a generation program.

Background Art

[0002] With the recent progress of machine learning, a trained model called so-called generative AI (Artificial Intelligence), which can generate an image based on information input by a user, has been widely used. Also, many studies have been conducted on methods for controlling image generation in image generation processing using a trained model. However, in image generation processing using a trained model, generating only the silhouette of a certain object or region is not often assumed. This is because, for example, when trying to train a machine learning model for generating a silhouette, if the target silhouette itself can be prepared as learning data, there is no need to generate the silhouette using a trained model. Therefore, there is a problem that it is not known what information should be input to the trained model in order to generate an image of a desired single silhouette.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

[0005] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is the generation of desired silhouette data. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] The generation apparatus according to this embodiment includes an acquisition unit, a determination unit, and a generation unit. The acquisition unit acquires generation category information, which is information about the type of silhouette to be generated. The determination unit determines a template for the shape. The generation unit inputs the template and the generation category information into a trained model and outputs silhouette data from the trained model corresponding to the shape specified by the template. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a block diagram showing the generating apparatus according to this embodiment. [Figure 2] Figure 2 is a flowchart showing an example of the operation of the generation apparatus according to this embodiment. [Figure 3] Figure 3 shows an example of a template according to this embodiment. [Figure 4] Figure 4 shows another example of the template according to this embodiment. [Figure 5] Figure 5 shows an example of silhouette generation using the generation apparatus according to this embodiment. [Modes for carrying out the invention]

[0008] The generation apparatus, learning method, and learning program according to this embodiment will be described below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate. One embodiment will be described below with reference to the drawings.

[0009] The generating apparatus according to this embodiment will be described with reference to the block diagram in Figure 1. The generation device 10 according to this embodiment includes a processing circuit 11, a memory 12, an input interface 13, and a communication interface 14.

[0010] The processing circuit 11 includes an acquisition function 111, a determination function 112, a generation function 113, and an output function 114. The processing circuit 11 has a processor as a hardware resource (not shown).

[0011] The acquisition function 111 acquires generation category information, which is information about the type of silhouette to be generated. The template is graphic data that serves as auxiliary information for determining the shape of the silhouette. Specifically, the generation category information is information about the object or area within the object that the user wants to output as a silhouette, and is expected to be text, images, or videos that specify the object or area within the object.

[0012] The decision function 112 determines a template for the shape by determining, for example, the shape or arrangement of the figure to be assumed as the silhouette shape in response to user input.

[0013] The generation function 113 inputs the template and generation category information into the trained model 101 and outputs silhouette data from the trained model 101 according to the shape specified in the template. Silhouette data is data represented by the contour lines of an object or region, regardless of whether the inside of the contour lines is filled with a single color or multiple colors.

[0014] The output function 114 outputs the silhouette data generated by the generation function 113 to the outside via the communication interface 14.

[0015] Furthermore, the various functions of the processing circuit 11 may be stored in memory 12 in the form of programs that can be executed by a computer. In this case, the processing circuit 11 can be said to be a processor that realizes the functions corresponding to each program by reading and executing the programs corresponding to these various functions from memory 12. In other words, the processing circuit 11 in the state in which each program has been read will have multiple functions, etc., as shown in the processing circuit 11 of Figure 1.

[0016] In Figure 1, these various functions are explained as being realized by a single processing circuit 11, but it is also possible to configure the processing circuit 11 by combining multiple independent processors, and each processor realizes the functions by executing a program. In other words, each of the above functions may be configured as a program, and one processing circuit may execute each program, or a specific function may be implemented in a dedicated, independent program execution circuit.

[0017] Memory 12 stores templates, generated category information, trained models 101, etc. Memory 12 can be a semiconductor memory element such as RAM (Random Access Memory) or flash memory, a hard disk drive (HDD), a solid state drive (SSD), or an optical disc. Memory 12 may also be a drive device that reads and writes various information to and from portable storage media such as CD-ROM drives, DVD drives, or flash memory. In addition, in this embodiment, although the trained model 101 is shown as being stored in the memory 12, the trained model 101 may be stored in an external device. A configuration may be adopted in which the generation device 10 transmits the template and the generation category information, the trained model 101 is used in the external device, and the generation device 10 receives silhouette data, which is the output from the trained model 101.

[0018] The input interface 13 has a circuit for receiving various instructions and information inputs from the user. The input interface 13 has, for example, a circuit related to a pointing device such as a mouse or an input device such as a keyboard. Note that the circuit included in the input interface 13 is not limited to a circuit related to physical operation components such as a mouse and a keyboard. For example, the input interface 13 may have an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the generation device 10 and outputs the received electric signal to various circuits within the generation device 10.

[0019] The communication interface 14 exchanges data with an external device by wire or wirelessly. Since general communication means may be used for the communication method and the structure of the interface, the description thereof is omitted here.

[0020] Next, an operation example of the generation device 10 according to this embodiment will be described with reference to the flowchart of FIG. 2.

[0021] In step SA1, the processing circuit 11 acquires generation category information from the user by the acquisition function 111. Specifically, the acquisition function 111 may acquire, for example, the name of an object input by the user as the generation category information. Alternatively, the acquisition function 111 may acquire an image file selected by the user as the generation category information.

[0022] In step SA2, the processing circuit 11 determines the template using the decision function 112. For example, multiple templates may be stored in memory 12, and the user may be presented with multiple templates, with the selected template being the determined template. Alternatively, the template may be determined in response to user input, such as by using a hand-drawn shape created by the user with a drawing tool.

[0023] In step SA3, the processing circuit 11 inputs the template and generation category information to the trained model 101 via the generation function 113, and the trained model 101 generates silhouette data corresponding to the shape specified in the template. The trained model 101 can be any machine learning model that performs data generation tasks such as image generation and video generation. For example, a neural network model using GANs (Generative Adversarial Networks) or Diffusion Models that has been trained to perform image generation and video generation tasks can be used.

[0024] In step SA4, the processing circuit 11 outputs the silhouette data to a display, for example, using the output function 114.

[0025] Next, an example of a template according to this embodiment will be described with reference to Figure 3. The template according to this embodiment only needs to contain information that allows control over the shape of the silhouette. For example, Figure 3 shows a rectangle, a circle, and an ellipse as examples of two-dimensional templates, and a cuboid, a cylinder, and an ellipsoid as examples of three-dimensional templates. However, the two-dimensional shape may be a square, a polygon, or any other shape. Similarly, the three-dimensional shape may be a sphere, a cube, a polyhedron, or any other cube.

[0026] Furthermore, the shapes used as templates may be expressed using mathematical formulas. The shape of the shapes used as templates can be defined by mathematical formulas. In addition, templates with adjusted size and shape may be generated by the user providing parameters. Specifically, for example, if the template is a cube, the size of the cube can be adjusted by the user inputting a parameter 'a' related to the length of one side. Furthermore, an ellipsoid that has not undergone rotation or translation can be expressed by the following equation (1).

[0027]

number

[0028] In equation (1), the user can adjust the shape and size of the ellipsoid by setting parameters a, b, and c, thereby generating a template of the user's choice. Furthermore, the placement of the template can be adjusted by applying rotation or translation. Similarly, for a cylinder that has not been rotated or translated, the user-configurable parameters r and z0 can be expressed by the following equation (2).

[0029]

number

[0030] Similar to the case of the ellipsoid, the template's position can be adjusted by applying rotation or translation.

[0031] Next, another example of the template according to this embodiment will be described with reference to Figure 4. In the example shown in Figure 4, the template is set using bitmap information that defines the position and shape of the silhouette data to be placed within the image area. Bitmap 40 represents an elliptical template 41 and a rectangular template 42, and the size and position of the templates on bitmap 40 are reflected in the silhouette data. In other words, the processing circuit 11 generates silhouette data that reflects the shape of the templates at the positions of the elliptical template 41 and the rectangular template 42 on bitmap 40 using the generation function 113. Bitmap 40 can be created by the user using software such as an application for creating presentation slides and a drawing tool.

[0032] Next, an example of silhouette generation by the generation device 10 according to this embodiment will be described with reference to Figure 5. The generation device 10 receives a template 51 and generation category information 52 as input. Using a trained model 101, the generation device 10 generates silhouette data 53 that conforms to the shape of the object template 51 indicated by the generation category information 52.

[0033] In Figure 5(a), a rectangle is input as template 51, and the text information "fish" is input as generated category information 52. The generation device 10 generates a silhouette of a fish with its head facing the long side of the rectangle as silhouette data 53.

[0034] In Figure 5(b), a circle is input as template 51, and an image of a "bird" is input as generation category information 52. The generation device 10 generates a silhouette data 53 of a bird that is not a horizontally elongated silhouette like a bird with its wings spread, but rather a silhouette of a bird with a circle that suggests it is resting its wings.

[0035] In Figure 5(c), a bitmap 40 is input as template 51, and the text information "automobile" is input as generation category information 52. Bitmap 40 is data in which two triangles are arranged. The generation device 10 generates a silhouette of an automobile as silhouette data 53 at the position of template 51 specified on bitmap 40. Note that, as shown in Figure 5(c), template 51 is not limited to two spatially separated templates, and any number of templates may be prepared, and the size and shape of each template 51 may also be set arbitrarily. In this case, the determination function 112 may have a function to set any number of templates, and a function to enlarge, reduce or transform templates.

[0036] The generated silhouette may be an image or a video. The generation function 113 allows the processing circuit 11 to generate a video related to the silhouette data if the template or generation category information includes information related to a video. In this case, the input template 51 may also have additional specifications for movement or rotation direction, or it may be a video that includes changes in shape. On the other hand, the generation category information 52 may also be a video or a descriptive text that explains the changes in shape. For example, if the content "video of a bird taking flight" is input as the generation category information 52, silhouette data 53 of a bird taking flight with a shape corresponding to the template 51 can be generated.

[0037] According to the embodiment described above, by inputting generation category information relating to the object or region for which a silhouette is to be generated, and a template defining the shape of the silhouette, into a trained model for image generation, silhouette data corresponding to the shape of the template can be generated. This makes it possible to generate desired silhouette data that takes into account the shape desired by the user.

[0038] In addition, each function according to the embodiment can also be realized by installing a program that performs the processing on a computer such as a workstation and loading it into memory. In this case, the program that can cause the computer to execute the method can also be stored and distributed on a storage medium such as a magnetic disk (hard disk, etc.), optical disk (CD-ROM, DVD, etc.), or semiconductor memory.

[0039] In the above explanation, the term "processor" refers to circuits such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), Application Specific Integrated Circuits (ASICs), and Programmable Logic Devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)).

[0040] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0041] 10 Generator 11 Processing Circuit 12 memory 13 Input Interfaces 14. Communication Interface 40 bitmap 41, 42, 51 Templates 52. Generated Category Information 53 Silhouette Data 101 Pre-trained models 111 Acquisition function 112 Decision Function 113 Generation function 114 Output Functions

Claims

1. An acquisition unit that acquires generation category information, which is information about the type of silhouette to be generated, A determination unit that determines the template for the shape, A generation unit inputs the template and the generated category information into a trained model and outputs silhouette data from the trained model according to the shape specified in the template. A generating device equipped with the following.

2. The generating apparatus according to claim 1, wherein the generated category information is at least one of text, images, or moving images that describes an object or a region within the object.

3. The generation apparatus according to claim 1, wherein the template is a two-dimensional or three-dimensional figure.

4. The generating apparatus according to claim 1, wherein the template is a mathematical formula that defines the shape.

5. The generation apparatus according to claim 1, wherein the template is bitmap information that defines the position and shape of the silhouette data arranged within the image area.

6. The generation apparatus according to claim 1, wherein the template comprises a plurality of spatially separated templates.

7. The generation apparatus according to claim 1, wherein the generation unit generates a moving image relating to the silhouette data when the template or the generation category information includes specified information relating to a moving image.

8. We obtain generation category information, which is information about the type of silhouette to be generated. Determine the template for the shape, The template and the generated category information are input to the trained model, and silhouette data corresponding to the shape specified in the template is output from the trained model. Generation method.

9. On the computer, A function to obtain generation category information, which is information about the type of silhouette to be generated, A decision function that determines a template related to the shape, A generation program for implementing a generation function that inputs the template and the generated category information into a trained model and outputs silhouette data corresponding to the shape specified in the template from the trained model.

Images