Information processing device, program and information processing method

Abstract

To provide a scheme for obtaining content based on different elements.SOLUTION: An information processing device includes: an obtaining unit that obtains input information; and an outputting unit that outputs, from the input information, content generated by a first element and by a second element different from the first element.SELECTED DRAWING: Figure 1-1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a program, an information processing method, and the like. [Background technology]

[0002] For example, Patent Document 1 discloses a technology for automatically generating advertising text. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-140228 Summary of the Invention [Problem to be solved by the invention]

[0004] With the technology of Patent Document 1, it is difficult to acquire content such as catchphrases, product descriptions, product images, advertisements, flyers, posters, artworks, etc. that are based on a plurality of different elements.

[0005] One of the objects of the present invention is to propose a method for acquiring content based on different elements from input information.

[0006] According to a first aspect of the present invention, an information processing device includes an acquisition unit that acquires input information, and an output unit that outputs content generated from the input information by a first element and a second element different from the first element. According to a second aspect of the present invention, a program executed by an information processing device includes acquiring input information by an acquisition unit of the information processing device, and outputting content generated from the input information by a first element and a second element different from the first element by an output unit of the information processing device. According to a third aspect of the present invention, an information processing method of an information processing device includes acquiring input information by an acquisition unit of the information processing device, and outputting content generated from the input information by a first element and a second element different from the first element by an output unit of the information processing device. [Brief explanation of the drawings]

[0007] [Figure 1-1] FIG. 1 is a diagram showing an example of the configuration of an information processing device according to a first embodiment. [Figure 1-2] 10 is a flowchart showing an example of the flow of a main process for generating artwork according to the first embodiment. [Figure 1-3] FIG. 4 is a diagram showing an example of a screen displayed on a display unit of the information processing apparatus. [Figure 1-4] FIG. 10 is a diagram showing an example of the configuration of an information processing device according to a first modified example. [Figure 2-1] FIG. 10 is a diagram showing an example of the configuration of an information processing device according to a second embodiment. [Figure 2-2] FIG. 10 is a diagram showing an example of the configuration of an information processing device according to a second embodiment. [Figure 2-3] 10 is a flowchart showing an example of the flow of a main process for generating artwork according to the second embodiment. [Figure 3-1] FIG. 10 is a diagram showing an example of the configuration of an information processing device according to a third embodiment. [Figure 3-2] 11 is a flowchart showing an example of the flow of a main process for generating artwork according to the third embodiment. [Figure 4-1] FIG. 10 is a diagram showing an example of the configuration of an information processing device according to a fourth embodiment. [Figure 4-2] 13 is a flowchart showing an example of the flow of a main process for generating artwork according to the fourth embodiment. [Figure 5-1] FIG. 13 is a diagram showing an example of the configuration of an information processing device according to a fifth embodiment. [Figure 5-2] 13 is a flowchart showing an example of the flow of a main process for generating artwork according to the fifth embodiment. [Figure 6-1] 10 is a flowchart showing an example of a flow of processing executed by a terminal and a server according to an application example. DETAILED DESCRIPTION OF THE INVENTION

[0008] <Compliance with legal matters> It should be noted that the disclosures set forth herein are subject to compliance with the laws of the country of implementation, such as communications privacy, as required for the implementation of the disclosures.

[0009] <Embodiment> In this specification, for ease of understanding, there are places where the phrase "by way of example and not limitation" is used, but please note that not only those places but also the entire embodiment described below are not limited to the contents of that description.

[0010] An embodiment for implementing a program etc. according to the present disclosure will be described with reference to the drawings.

[0011] The production of the information processing device of the present invention (the information processing device of the present invention) may include, by way of example and not limitation, the concept that a state in which the functions of the present invention can be realized (a state in which the present invention can be executed) is created in the information processing device by receiving (or receiving and storing in the information processing device) a program (by way of example and not limitation, an interactive program) etc. described in this specification.

[0012] The production of a terminal of the claimed invention (terminal of the claimed invention) may include, by way of example and not limitation, the concept that a state is created in which the functions of the claimed invention can be realized (a state in which the claimed invention can be executed) on a terminal owned (possessed) by a user by receiving (or receiving and storing) a program described in this specification (by way of example and not limitation, an application program) or the like.

[0013] Furthermore, the production of the system of the invention claimed in the present application (the system of the invention of the present application) may include, by way of example and not limitation, the concept that a state in which the functions of the invention of the system claimed in the present application can be realized (a state in which the invention claimed in the present application can be executed) is created by receiving, at a terminal included in the system of the present application, a program described in this specification (by way of example and not limitation, an application program) transmitted from a server included in the system of the present application (or by storing the received program in the terminal). For example, and not by way of limitation, if the terminal is a smartphone or a personal computer (PC), an application received from a server may be installed on the smartphone or PC, and some of the elements (processing / operations) claimed herein may be executed via the application. Also, for example, and not by way of limitation, if the terminal is a smartphone or a PC, some of the elements (processing / operations) claimed herein may be executed via a website accessible from the smartphone or PC, without installing an application on the smartphone or PC.

[0014] Furthermore, in this specification, a system may be, by way of example and not limitation, configured to include a plurality of devices (which may also be called information processing devices). The plurality of devices may be a combination of devices of the same type, a combination of devices of different types, or a combination of devices of the same type and devices of different types. Note that a system can be thought of as, for example and not as a limitation, a plurality of devices working together to perform some kind of processing.

[0015] Furthermore, a system relating to a client (client device) and a server can be considered to be, by way of example and not limitation, at least one of the following: (1) Terminals and servers (2) Server (3) Terminal

[0016] (1) is, by way of example and not limitation, a system that includes at least one terminal and at least one server. One example of this is a client-server system.

[0017] The server is configured by the following devices, by way of example and not limitation, and may be a single device or a combination of multiple devices.

[0018] Specifically, the server is configured to have at least one processor (for example, but not limited to, CPU: Central Processing Unit, GPU: Graphics Processing Unit, APU: Accelerated Processing Unit, DSP: Digital Signal Processor (for example, but not limited to, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array), etc.), computer device (processor + memory), control device, arithmetic device, processing device, etc., and may be configured to have multiple of the same type of any one device (for example, but not limited to, CPU + CPU, homogeneous multi-core processor, etc.), or multiple of different types of any one device (for example, but not limited to, CPU + DSP, heterogeneous multi-core processor, etc.), or may be a combination of multiple devices (for example, but not limited to, processor + computer device, processor + arithmetic device, multiple devices made heterogeneous, etc.). The processor may be a virtual processor.

[0019] Furthermore, when a server performs some processing, if the server is configured with a single device, the processing described in the embodiments is performed by the single device. Furthermore, if the server is configured with multiple devices, some processing may be performed by one device, and other processing may be performed by another device. For example, and not by way of limitation, if the server is configured with a processor and an arithmetic device, the processor may perform a first processing, and the arithmetic device may perform a second processing. Furthermore, when a plurality of devices are used, the devices may be located at positions physically separated from one another.

[0020] Furthermore, the functions of the server may be provided in the form of PaaS, IaaS, or SaaS in cloud computing, for example and without limitation.

[0021] The control unit of the system can be at least one of the control unit of the terminal and the control unit of the server, i.e., by way of example and not limitation, the control unit of the system can be either (1A) only the control unit of the terminal, (1B) only the control unit of the server, or (1C) both the control unit of the terminal and the control unit of the server.

[0022] Furthermore, the control and processing (hereinafter collectively referred to as "control, etc.") performed by the control unit of the system may be performed by (1A) only the control unit of the terminal, (1B) only the control unit of the server, or (1C) both the control unit of the terminal and the control unit of the server. In addition, in (1C), as an example and not a limitation, some of the controls performed by the system's control unit may be performed by the terminal's control unit, and the remaining controls may be performed by the server's control unit. In this case, the allocation of controls may be equal or may be allocated in different proportions.

[0023] Furthermore, when referring to the communication unit of a server, if the server is configured with a single device, it may refer to the communication unit itself that the single device has, or if the server is configured with multiple devices, it may be configured to include each communication unit that each device has. As an example and not by way of limitation, if a server comprises a first device and a second device, and the first device has a first communication unit and the second device has a second communication unit, the communication unit of the server may be conceptualized as including the first communication unit and the second communication unit.

[0024] (2) is not limited to, but may be, for example, a system consisting of multiple servers (hereinafter referred to as a "server system"). In this case, the configuration of each server can be similarly applied to the configuration described above.

[0025] The control etc. performed by the server system may be performed by only one of the multiple servers (2A), by only the other servers (2B), or by both the one server and the other servers (2C). In addition, in (2C), as an example and not a limitation, one server may perform some of the control, etc., performed by the server system, and another server may perform the remaining control, etc. In this case, the allocation (allocation) of the control, etc. may be equal or may be allocated in different proportions.

[0026] (3) By way of example and not limitation, the system may be comprised of multiple terminals. The system may be, by way of example and not limitation, a system such as the following: A system that gives server functions to terminals (distributed system). This can be realized using blockchain technology, for example and not by way of limitation. A system in which terminals communicate wirelessly with each other. This can be realized by, for example and without limitation, communicating using a P2P (peer-to-peer) method using short-range wireless communication technology such as Bluetooth (registered trademark).

[0027] The above is not limited to the control unit, but also applies to each functional unit such as an input / output unit, a communication unit, a storage unit, and a clock unit, which may be components of the system.

[0028] In the following embodiment, a system including a terminal and a server (a client-server system) is illustrated as an example and not as a limitation. It is also possible to apply the server system described in (2) above as the server.

[0029] Furthermore, instead of a system including a terminal and a server, a system not including a server, such as the system in (3) above, can also be applied, but is not limited to this. In this case, the embodiment can be configured based on the above-mentioned blockchain technology, etc. Specifically, by way of example and not limitation, data stored and managed in a server described in the following embodiment is stored on the blockchain. Then, a terminal can generate a transaction to the blockchain, and when the transaction is approved on the blockchain, the data stored on the blockchain can be updated.

[0030] It should be noted that even when the term "terminal" is used, this is not limited to the meaning of a terminal as a client device in a client server. That is, a terminal may include the concept of a device that is not in a client-server context.

[0031] Furthermore, in this specification, when the terms "related to" and "related to" are used, "B related to A" or "B related to A" may mean, by way of example and not limitation, "B" that has some relationship with "A."

[0032] Furthermore, when a device processes two or more objects, such as "sending A and B" or "receiving A and B," this may include performing "A" and "B" at the same time (hereinafter referred to as "simultaneous"), and performing "A" and "B" at different times (hereinafter referred to as "non-simultaneous"). As an example and not a limitation, when referring to transmitting first information and second information, this may include both concepts of transmitting the first information and the second information at the same time, and transmitting the first information and the second information at different times. In addition, taking into account the lag (time lag), "simultaneous" may include "almost simultaneously."

[0033] Note that even though "A" and "B" are performed at different times, this only needs to be done with "A" and "B" as the processing targets, and the purposes do not necessarily have to be the same. By way of example and not limitation, when the first information and the second information are transmitted as described above, it is sufficient to transmit the first information and the second information, and this may include cases where the first information and the second information are transmitted for the same purpose, as well as cases where the first information and the second information are transmitted for different purposes.

[0034] <Example> An example of an embodiment to which the present invention is applied will be described below. In the following embodiment, by way of example and not limitation, a method for generating content such as catchphrases, product descriptions, and product images, and a method for generating content such as advertisements using these will be illustrated. In the following embodiments, for convenience, the content of the final product, such as an advertisement, will be referred to as "artwork." Note that catchphrases, product descriptions, product images, and the like, which are intermediate products, may also be types of content, and these may be outputtable. Keywords may also be included in the content.

[0035] <First Example> The first embodiment is an embodiment relating to a basic configuration for realizing content generation.

[0036] The contents described in the first embodiment can be applied to any of the other embodiments and other modified examples. Furthermore, the same components as those already mentioned are given the same reference numerals and will not be described again.

[0037] <Functional configuration of information processing device> FIG. 1-1 is a block diagram showing an example of a functional configuration of an information processing device 1 according to an embodiment of the present disclosure. The information processing device 1 includes, for example and not by way of limitation, a parameter generation unit 101 and an artwork generation unit 103 as functional units included in the control unit 100. These functional units may be configured by, for example and not by way of limitation, a central processing unit (CPU), a microprocessor, a processor core, a multiprocessor, an ASIC, an FPGA, or other circuitry.

[0038] For example, and not by way of limitation, the parameter generation unit 101 generates parameters (parameters for generating artwork) that the artwork generation unit 103 uses to generate artwork, based on input information input via the input unit 50. The parameters generated by the parameter generation unit 101 may be referred to as "generation parameters." In this specification, for convenience, we will refer to these as "parameters," but they may also be thought of as elements contained in the artwork (information contained in the artwork) or elements for generating the artwork (information for generating the artwork).

[0039] The input unit 50 may be provided outside the information processing device 1.

[0040] The input unit 50 may be realized by, for example and without limitation, any one of all types of devices that can accept input from a user and transmit information related to the input to the information processing device 1, or a combination thereof. The input unit 50 may include, but is not limited to, a touch panel, a touch display, hardware keys such as a keyboard, a pointing device such as a mouse, a camera (operation input via moving images), and a microphone (operation input by voice).

[0041] In this embodiment, the input unit 50 may be configured to receive, as an example and not a limitation, text data as input information input by a user.

[0042] The text data may be, for example and without limitation, text data (non-imaged text data) such as a character string input (or output) in a text format. Images of characters based on the text data may also be considered "image information." The text data may be, for example and without limitation, the name of a product or service for which artwork is desired to be generated, and any text data may be input. The text data input to the input unit 50 may be referred to as "input text data."

[0043] It should be noted that the information input to the input unit 50 may be, by way of example and not limitation, composed only of data other than text data (by way of example and not limitation, image data such as still images and moving images, and sound data such as audio), or may include data other than text data in addition to text data. The image data input to the input unit 50 may be referred to as "input image data."

[0044] For example and not by way of limitation, the parameter generating unit 101 may be capable of outputting (A) text as a parameter based on at least input information (for example and not by way of limitation, input text data) input by the input unit 50. (A) Text here may mean, by way of example and not limitation, a string of textual characters (non-imaged text).

[0045] (A) When text is output as a parameter, this may include, by way of example and not limitation, outputting text that is different from the input text data entered as input information. Here, "different text" may include, by way of example and not limitation, that part or all of the input text data is included in the (A) text output by parameter generation unit 101. When all of the input text data is included in the (A) text output by parameter generation unit 101, the (A) text is generated by adding newly generated text to the input text data. As an example and not limitation, when the input text data is "okra," the output (A) text may be "okra grown in the sun." Another example of "different text" may include a case where part or all of the input text data is not included in the (A) text output by the parameter generating unit 101. For example, and not by way of limitation, if the input text data is "okra," the output (A) text may be "It is a stamina vegetable grown under the sun." (A) Text may include, by way of example and not limitation, text that serves as a tagline, product description, keywords, etc.

[0046] Furthermore, the parameter generating unit 101 may be capable of outputting (B) an image as a parameter in addition to or instead of (A) text, for example and not by way of limitation. (B) Images include, by way of example and not limitation, - Images such as illustrations and photographs that do not contain text - Images such as illustrations and photographs that contain text · Imaged text (text strings converted into images) etc. may be included. The image is not limited to a still image, but may be a moving image.

[0047] Other parameters may also be included, by way of example and not limitation: (C) Layout pattern (D) Artwork style (may include pop style, warm style, cool style, etc.) (E) The background color of the artwork (which may include the image color of the artwork).

[0048] (C) The layout pattern may be, by way of example and not limitation, a pattern in which the position or area in which text is placed within the artwork is set, a pattern in which the position or area in which an image is placed within the artwork is set, a pattern in which the position or area in which text and images are placed within the artwork is set, etc.

[0049] The (A) text and (B) images generated by the parameter generating unit 101 may be generated based on at least one of the following elements, for example and without limitation: (a) Text color (b) Font style of the text characters (c) Image color

[0050] (a) Text character color may be, by way of example and not limitation, a factor that defines the character color of non-imaged text and / or the character color of imaged text.

[0051] (b) The font style of the text characters may be, by way of example and not limitation, a factor that defines the font style of the characters of the text that is not an image and / or the font style of the characters of the text that is an image, and may include, by way of example and not limitation, a font type, a font size, a font color, a font family, etc.

[0052] (c) The color of the image may be, by way of example and not limitation, a factor that defines the color of (B) the image. This may include the color of the imaged text, which may be considered as the color of the characters of the imaged text in (a) above.

[0053] The parameter generating unit 101 may generate at least two or more parameters from the above parameters (A) to (E) or (a) to (c).

[0054] The parameter generation unit 101 may be configured by a model such as a language model unit or an image generation model unit, for example and without limitation.

[0055] The language model of the language model unit may include, by way of example and not limitation, various language models such as a sequence to sequence model such as Transformer, an autoregressive model such as GPT, or a large-scale general-purpose language model. Note that other models may be included.

[0056] The language model unit may, for example and without limitation, generate (A) text based on input text data.

[0057] The image generation model of the image generation model unit may include, for example and without limitation, a text-to-image model, a diffusion model, a GAN (generative adversarial network) (for example and without limitation, DCGAN (deep convolutional GAN) etc.), etc. Furthermore, the image generation model of the image generation model unit may include, for example and without limitation, an image-to-image model (for example and without limitation, conditionalGAN (cGAN) etc.). Note that other models may be included.

[0058] For example, and not by way of limitation, the image generation model unit may generate (infer) an image (B) based on text (A) generated by the language model unit. It should be noted that the image generation model unit may, for example and without limitation, infer (B) an image based on input text data. Furthermore, the image generation model unit may, for example and without limitation, infer an image (B) based on input image data. Also, the (A) text generated by the language model unit may be output as a parameter as is.

[0059] Furthermore, the parameter generating unit 101 may include, for example and without limitation, a model (for example and without limitation, a boosting decision tree) that selects and outputs a font style corresponding to input text data or the like. The same can be done for (a) the color of text characters, (c) the color of images, etc.

[0060] By way of example and not limitation, the font style may be selected based on (A) text, (B) images, etc. Similarly, by way of example and not limitation, the background color of (E) artwork may be selected based on input text data, (A) text, (B) images, etc.

[0061] Furthermore, the parameter generating unit 101 may, for example and without limitation, determine the font style based on the style of the artwork (D). For example and not by way of limitation, if the artwork style (D) is a "pop style," a gothic bold font may be selected as the font style, which makes the letters pop on the artwork. This is because, for example and not by way of limitation, if the font style is a hard type, it may give a hard impression as the artwork style, and if the font style is a soft type, it may give a soft impression as the artwork style. The same may be applied to the background color of the artwork (E).

[0062] The artwork generation unit 103 generates artwork based on the generation parameters. The artwork generated by the artwork generation unit 103 is referred to as "generated artwork" as appropriate. The artwork generating unit 103 generates artwork based on the first element and a second element different from the first element. Specifically, as an example and not by way of limitation, artwork is generated by arranging and rendering (A) text (an example of a first element, not by way of limitation) or (B) image (an example of a first element, not by way of limitation) generated by the parameter generation unit 101 based on the aforementioned (C) layout pattern (an example of a second element, not by way of limitation), or by automatically arranging and rendering it. In this case, if (D) the artwork style or (E) the artwork background color is included in the generation parameters, the artwork generation unit 103 can arrange text and images in a layout pattern based on that style and background color. In this example, the artwork generator 103 may be considered an artwork composition unit, a rendering unit, or the like, by way of example and not limitation.

[0063] Also, by way of example and not limitation, when the first element is (A) text and the second element is the font style or size of the characters in the (A) text, this may be included in the case where the artwork generation unit 103 generates artwork based on the first element and the second element different from the first element. Also, by way of example and not limitation, when the first element is (B) image and the second element is the color of the (B) image, this may be included. That is, by way of example and not limitation, the second element may be an element related to the output form of the first element.

[0064] The artwork generation unit 103 may include, for example and not by way of limitation, a model such as a neural network, such as a DNN (for example and not by way of limitation, a GAN). In this case, a trained model that has been trained to learn a layout pattern may be configured as a model that receives an input vector composed of at least elements of the generation parameters.

[0065] Here, the layout pattern (C) is as follows: - Items created in advance by the user The parameter generated by the parameter generating unit 101 Either of the above may be used. When a layout pattern (C) created in advance by the user is used, the user may specify it through the input unit 50. This will also be explained in the later-described Pattern A and First Modification (2).

[0066] When the parameter generating unit 101 generates a layout pattern, the layout pattern may be generated by a layout generation model realized by LayoutGAN or the like, for example and not by way of limitation. Also, the layout pattern may be generated by a model proposed by Ueno et al. (Michihiko Ueno, Shin'ichi Satoh, Continuous and Gradual Style Changes of Graphic Designs with Generative Model, IUI '21: 26th International Conference on Intelligent User Interfaces April 2021 Pages 280-289), for example and not by way of limitation.

[0067] More specifically, as an implementation pattern, by way of example and not limitation, any of the following may be applied. <Pattern A> A pattern that generates artwork using a layout pattern (Pattern A1) A pattern in which the user creates and inputs a layout pattern in advance. When various generation parameters (not including layout patterns) are input from the parameter generation unit 101, the artwork generation unit 103 generates artwork by arranging text and images based on the layout pattern created in advance by the user. Note that, as a non-limiting example, the artwork generation unit 103 may generate artwork by arranging text and images based on a template layout pattern previously set by the system user. In the case of pattern A1, as described above, the user may specify the layout pattern through the input unit 50. This will also be explained in the first modified example (2) below. (Pattern A2) A pattern in which the parameter generating unit 101 generates a layout pattern When various generation parameters (including layout patterns) are input from the parameter generation unit 101, the artwork generation unit 103 generates artwork by arranging text and images based on the input layout pattern. (Pattern A3) A pattern in which the artwork generation unit 103 generates a layout pattern When various generation parameters (not including layout patterns) are input from the parameter generation unit 101, the artwork generation unit 103 generates, by way of example and not limitation, a layout pattern, and generates artwork by arranging text and images based on the generated layout pattern.

[0068] <Pattern B> A pattern that generates artwork without using a layout pattern By way of example and not limitation, when various generation parameters (not including layout patterns) are input from the parameter generation unit 101, the artwork generation unit 103 generates artwork by, for example and not limitation, randomly arranging text and images.

[0069] The artwork generated by the artwork generation unit 103 can be output from the output unit 150 . The output unit 150 may be realized by, for example and without limitation, any of all types of devices capable of outputting the processing results processed by the information processing device 1, or a combination thereof. The output unit 150 may include, for example and without limitation, a touch panel, a touch display, a lens (for example and without limitation, 3D (three dimensions) output or hologram output), a printer, etc.

[0070] The term "output" here may include at least one of the following: - Output of internal information of the device itself (output of information from one functional unit to another, etc.) Display on a display device (display device of an external device, display device of the information processing device 1) Transmission to an external device (transmission by a communication device of the information processing device 1) The external device may include, by way of example and not limitation, a user's terminal, etc., as described below.

[0071] <Processing> FIG. 1-2 is a flowchart showing an example of the flow of the main process of artwork generation (hereinafter referred to as "main artwork generation process") executed by the control unit 100 of the information processing device 1 in this embodiment. First, the control unit 100 acquires input information (for example, but not limited to, text data) to the input unit 50 (E120).

[0072] Next, the parameter generating unit 101 performs a parameter generating process based on the acquired input information (E130).

[0073] Here, as an example of one method for generating parameters, a method using a question sentence will be exemplified. In the following explanation, the sentence (character string) input to the model will be referred to as a "prompt sentence" (prompt character string).

[0074] As a non-limiting example, a template of a question sentence such as the one shown below is stored in advance in the parameter generation unit 101. Then, as a non-limiting example, the parameter generation unit 101 generates a prompt sentence to be input to a language model unit configured by a large-scale general-purpose language model or the like, based on the text data and the question sentence.

[0075] More specifically, as a non-limiting example of a question, the following question may be stored: (Q1) <x>What color do you imagine when you hear this? (Q2) <x>What is your impression? (Q3) <x>What is the tagline for? (Q4) <x>Is that cool? Or cute? However, X is assumed to be substituted with the text data such as the product name mentioned above.

[0076] As an example, and not by way of limitation, if the input text data is "okra (okra)", a prompt sentence is generated that is input to the language model unit and that enables the language model unit to infer answer sentences for each of the above questions. (Q1) What color do you imagine when you hear the word "okra"? (Q2) What is your impression of okra? (Q3) What is the catchphrase for Okra? (Q4) Is Okra cool? Or cute?

[0077] When these prompt sentences are input to the language model unit, the language model unit infers and outputs answer sentences to the respective question sentences. (A1) Green (A2) It is very nutritious. (A3) Not only is it packed with nutrition, but it also regulates the intestines and suppresses blood sugar levels, making it perfect for preventing summer fatigue. (A4) It's rather cute.

[0078] Note that, as a non-limiting example, the parameter generation unit 101 may generate a prompt sentence to be input to the language model unit in accordance with the input text data and the above-mentioned answer sentences (A1) to (A4), so that the language model unit infers a product description based on the input text data. In this case, as a non-limiting example, the prompt sentence may be as follows: " <x>Generate a product description for . <x>is (A1). Also, <x>is (A2). <x>is (A3). <x>is (A4).” The language model unit then infers and outputs a product description in accordance with the generated prompt sentence.

[0079] The questions are not limited to the above questions. Furthermore, the prompt text for generating the product description may include any of the answers to the question text, all of the answers, or no answers.

[0080] Then, based on this response sentence, the image generation model unit generates an image such as the following, by way of example and not limitation: <x>is "okra", then image generation may be as follows: (I1) Image of <Okra> (I2) An image of the text "Okra" in a "green" (rather cute) font style. (I3) An image of the text, "Not only is it packed with nutrition, it also regulates the intestines and suppresses blood sugar levels, making it perfect for preventing summer fatigue," in a rather cute font style.

[0081] In this case, by way of example and not limitation, at least one of the above-mentioned answer sentences may be used to determine the font style of the characters (text). For example, and not by way of limitation, the answer (A4) "It's rather cute" may be input into a model that selects a font style, and based on this input, a font style (for example, and not by way of limitation, POP) that matches this text may be selected.

[0082] Thereafter, the artwork generation unit 103 performs an artwork generation process to generate artwork based on the generation parameters (E140). Specifically, by way of example and not limitation, the above text and images are arranged based on a layout pattern and rendered to generate an image that serves as artwork. The generated artwork is output from the output section 150.

[0083] Next, the control section 100 determines whether or not to end the processing (E190), and if it determines that the processing should be continued (E190: NO), the processing returns to step E120. If it is determined that the process should be ended (E190: YES), the control unit 100 ends the process.

[0084] As mentioned above, the artwork generation unit 103 may, for example and not by way of limitation, arrange and render text generated by the language model unit and images generated by the image generation model unit based on a layout pattern to generate artwork.

[0085] As described above, image data such as an original image may be input to the input unit 50 instead of or in addition to text data (input image data). The image data may be, by way of example and not limitation, image data input (or output) in an image format such as JPEG.

[0086] The input image data may be, by way of example and not limitation, an image of the entire artwork (which may include an image of the entire artwork, etc.), or an image to be part of the artwork (which may include an image that you want to include in the artwork as is, an image of an image that you want to include in the artwork, etc.).

[0087] In this case, the parameter generation unit 101 may be configured to have an image recognition model including, for example and not limitation, an image classification model, an image classification / object localization model, an object detection model, a segmentation model (for example and not limitation, a semantic segmentation model, an instant segmentation model, a panoptic segmentation model), etc. Then, the parameter generating unit 101 may generate parameters based on the results of image recognition by the image recognition model.

[0088] <Display screen example> Below, some techniques for generating advertisements as artwork will be described in more detail with reference to example display screens.

[0089] As shown in FIG. 1-3, an advertisement generation screen ADG is displayed on a display unit (not shown) of the information processing device 1. Such a display may be realized by an application executed on the information processing device 1 (for example, but not limited to, an advertisement generation application).

[0090] <Keyword generation from product name> The advertisement generation screen ADG of this embodiment includes, by way of example and not limitation, a product name area (a), a keyword automatic extraction button Btb, and a keyword area (b).

[0091] A user who wishes to generate an advertisement inputs the name of a product for which an advertisement is to be generated (as an example, not a limitation, of information related to content (including information related to advertisements)) into the product name area (a) into the input unit 50 and taps the automatic keyword extraction button Btb, whereby, as an example, not a limitation, in step E120 of the processing in FIG. 1-2, the control unit 100 (as an example, not a limitation, of an acquisition unit) acquires the input information. In this example, text data is acquired as the input information. In addition to the control unit 100, or instead of the control unit 100, the input unit 50 may be regarded as an acquisition unit that acquires input information. The same may be true below. Then, in step E130 of the processing of Figure 1-2, for example and not by way of limitation, the parameter generation unit 101 obtains a distributed representation corresponding to the product name from the product name based on a model capable of acquiring distributed representations (vector representations) of words, such as Word2Vec.

[0092] Next, the parameter generation unit 101 generates words corresponding to each of the distributed representations as keywords based on the distributed representations distributed within a predetermined distance from the distributed representation of the product name in step E130 of the process in FIG. 1-2, for example and not by way of limitation. That is, in this example, text as a keyword different from the input text data is generated as an element (first element) for generating artwork. Note that a model capable of acquiring distributed representations of words may be an inference-based method such as Word2Vec, or a count-based method such as N-gram. The one or more keywords thus generated are output to the keyword area (b).

[0093] In this example, the user enters the product name "pesticide-free okra" in the product name area (a) and taps the automatic keyword extraction button Btb. Based on this, keywords generated based on the product name, such as okra from Kagoshima Prefecture, direct from the producer, healthy, sticky, etc., are output in the keyword area (b).

[0094] <Product image generation from [product name + keywords]> The advertisement generation screen ADG of this embodiment includes an image generation result area (e).

[0095] For example, but not by way of limitation, in step E130 of the processing in FIG. 1-2, the parameter generation unit 101 generates a predetermined number (four in this example) of product images (not by way of limitation, but an example of content images (including advertising images)) using a Text-to-Image model based on a prompt string consisting of the product name in the product name area (a) and one or more keywords in the keyword area (b). Note that the predetermined number may be "1." That is, in this example, an image is generated as an element (first element) for generating artwork. The generated product images may be photographic images or illustration-style images. The generated predetermined number of product images are displayed in the image generation result area (e).

[0096] In the Text-to-Image model, the prompt string is first converted (encoded) into an embedded representation using an encoding model such as an RNN or Transformer. The Text-to-Image model then generates product images based on the embedded representations, by way of example and not limitation, a diffusion model.

[0097] The Text-to-Image model may generate product images based on the embedded representation using a VAE (Variational Autoencoder) or GAN.

[0098] <Product image generation from original image (user-defined image)> The advertisement generation screen ADG of this embodiment includes an original image area (d) and an image generation button BTd for obtaining an image generation result based on the original image in the original image area (d).

[0099] Instead of image generation results based solely on the prompt string input to the Text-to-Image model (for example, but not limited to, the product name displayed in the product name area (a) and / or one or more keywords displayed in the keyword area (b)), the user can also obtain image generation results based on an image selected by the user, an image obtained as an image search result from an Internet search based on the product name and / or one or more keywords, etc. The image search results may be either automated search results from an application, such as an advertisement generation application, or manual search results from a user.

[0100] By way of example and not limitation, when a user displays an image to be used as an original image in the original image area (d) via the input unit 50 (sets it as an original image) and taps the image generation button BTd, input information is acquired by the control unit 100 in step E120 of the processing in FIG. 1-2, for example and not limitation. In this example, image data is acquired as the input information. Then, by the parameter generation unit 101, in step E130 of the process in FIG. 1-2, for example and not by way of limitation, a predetermined number ("4" in this example) of product images are generated based on at least the image in the original image area (d), and are displayed in the image generation result area (e). Note that the predetermined number may be "1." That is, in this example, an image is generated as an element (first element) for generating artwork.

[0101] For example and without limitation, a predetermined number of product images may be generated using an image-to-image model without using a prompt string and using only the set raw images.

[0102] In addition, if the original image is a photographic image, the image-to-image model may generate a product image by mapping the original image into an illustration-style image using a conditional GAN ​​(cGAN) or the like.

[0103] On the other hand, the parameter generation unit 101 may generate a product image by operating a decoder based on an embedded representation generated using a Text-to-Image model encoder and an Image-to-Image model encoder using a prompt string and a set original image.

[0104] In addition, the original image can be converted into an embedded representation using VAE, Transformer, etc., and then combined with the embedded representation of the product name + keywords to generate an image using a diffusion model or GAN, thereby generating an illustrated-style product image based on the original image and the product name + keywords.

[0105] Furthermore, various filters such as contour extraction may be applied to the original image to generate an illustration-style product image (or the product image may be used as the original image for input).

[0106] <Generate product description from [product name + keywords]> The advertisement generation screen ADG of this embodiment includes a product description area (c) where a product description is displayed, and a description generation button BTc for generating a product description. By way of example and not limitation, when the user taps the description generation button BTc, the parameter generation unit 101 creates a prompt statement based on a set template instructing the parameter generation unit 101 to generate a product description including the product name in the product name area (a) and one or more of the keywords in the keyword area (b) in step E130 of the processing in Figure 1-2.

[0107] For example, and not by way of limitation, a product description is generated based on a prompt sentence using a text-to-text model such as a bidirectional auto-regressive transformer (BART), a generative pre-trained transformer (GPT), or a text-to-text transfer transformer (T5). The prompt sentence may allow for product description settings that can specify the length of the product description, the style of the product description (pop style, formal style, etc.), etc. In other words, in this example, text is generated as an element (first element) for generating artwork.

[0108] Alternatively, instead of using a prompt to generate a product description, a text-to-text model fine-tuned for advertising copywriting can be used to generate the product description based on the product name and keywords.

[0109] The prompt sentence may also include words based on the original image or the generated image. In this case, the words corresponding to the image may be inferred using a CLIP (Contrastive Language-Image Pre-Training) model or the like. The product description thus generated is displayed in the product description area (c).

[0110] <Generate ads from [product name + product description + product image]> The advertisement generation screen ADG of this embodiment includes an advertisement generation result area (g) in which the generated advertisement image is displayed, and an advertisement generation button BTg for generating the advertisement image.

[0111] By way of example and not limitation, when a user selects an image to be used in an advertisement from the product images in the image generation result area (e) (in this example, the product image in the lower right corner is selected) and taps the advertisement generation button BTg, the parameter generation unit 101 creates, based on a set template, a prompt message instructing the generation of a product description including the product name in the product name area (a), the product description in the product description area (c), and the selected product image (hereinafter referred to as the "selected image") in step E130 of Figure 1-2.

[0112] 1-2, the parameter generation unit 101 determines a product name region for rendering the product name, a product description region for rendering the product description, and a selected image region for arranging the selected image using LayoutGAN or the like. That is, in this example, a layout pattern is generated as an example of a second element related to the output form of the first element, which is an element for generating artwork. As mentioned above, the layout pattern may be provided by the user.

[0113] Once the layout of each area is determined, the font point count (not limited to a specific font size) for rendering the product name is calculated based on the size of the product name area and the number of product name characters. If the number of product name characters is large relative to the width of the product name area, the product name character string may be automatically wrapped. The font and style (font color, bold, italics, etc.) for rendering the product name may be selected by the parameter generating unit 101 as described above, or may be a preset font and style, or a font and style specified by the user. In other words, in this example, the color and font style of the text characters are generated as an example of a second element related to the output form of the first element, which is an element for generating artwork.

[0114] Alternatively, the font and style may be automatically determined based on the product name and keywords. In this case, the product name and keywords may be subjected to text classification using a Transformer model or the like, and the font and style associated with the classified category results may be applied. Similarly, for the product description, the font point size, style, and character string wrapping settings for rendering the product description area may be executed.

[0115] Next, the artwork generation unit 103 generates a predetermined number ("2" in this example) of advertisement images (advertisements as output results, image data) as advertisement generation results based on the layout of each area and the rendering settings for each area, for example but not by way of limitation, in step E140 of FIG. 1-2. These advertisement images are an example of image data and an example of generated artwork. The predetermined number of advertisement images thus generated are displayed in the advertisement generation result area (g). This is an example of artwork being output from the output unit 150.

[0116] In addition, the product name may be rendered based on the information in the product name area (a), and the product description may be rendered based on the information in the product description area (c).Then, based on the rendered product name image, product description image, and selected image, the arrangement of each image element may be inferred using LayoutGAN or the like, and an advertising image may be generated that combines the product name image, product description image, and selected image based on the inference results.

[0117] In the above example, an example was described in which a product image is generated based on the input product name (the product name displayed in the product name area (a)) and a keyword automatically generated based on the product name (the keyword output in the keyword area (b)). However, the present invention is not limited to this form, and a product image may be generated based on the input product name and the input keyword (as an example, and not a limitation, a keyword displayed in the keyword area (b) as a result of the user entering the keyword in the same area).

[0118] In addition, a product image may be generated based on the product name, keywords (for example, but not limited to, automatically generated keywords or keywords entered by the user), and a product description that is automatically generated based on the product name and keywords. That is, the prompt string to be input to the Text-to-Image model may include a product description.

[0119] Furthermore, in the above example, an example was described in which a product description is generated based on the input product name (the product name displayed in the product name area (a)) and keywords automatically generated based on the product name (keywords output in the keyword area (b)). However, the present invention is not limited to this form, and the product description may be generated based on the input product name and the input keywords (as an example, and not a limitation, keywords that are displayed in the keyword area (b) as a result of the user entering the keyword in the same area).

[0120] Furthermore, in the above example, based on the user inputting a product name in the product name area (a), keywords may then be automatically generated based on the product name and output to the keyword area (b) without the need for user operation, a predetermined number of product images may be automatically generated based on the product name and keywords, etc. and output to the image generation result area (e), a product description may be automatically generated based on the product name and keywords, etc. and output to the product description area (c), and a predetermined number of advertisements may be automatically generated based on the product name and product description and any product image (for example, and not by way of limitation, an automatically selected product image) and output to the advertisement generation result area (g). In other words, after inputting the product name, the user may be able to have an automatically generated advertising image displayed without having to perform any operations to obtain the output of keywords, product images, product descriptions, and advertising images.

[0121] Also, by way of example and not limitation, if the user does not select an image from the image generation results, a product description (which may or may not be an image) may be output as the generation result.

[0122] <Regarding information processing devices> The information processing device 1 may be regarded as a type of content generation device or output device. In this case, the information processing device 1 may be capable of outputting as content artwork such as an advertisement (advertising image) as a final product generated by the artwork generation unit 103, as well as catchphrases (which may be images, for example and not limitation, such as (a) a catchphrase with the color of the text characters as the second element, or (b) a catchphrase with the font style of the text characters as the second element), product descriptions (which may be images, for example and not limitation, such as (a) a product description with the color of the text characters as the second element, or (b) a product description with the font style of the text characters as the second element), and product images (which may be images, for example and not limitation, such as (c) a product image with the color of the image as the second element) as intermediate products generated by the parameter generation unit 101. The same may be true below.

[0123] In this embodiment, the artwork generation unit 103 generates artwork based on the generation parameters output from the parameter generation unit 101 . In this case, the generation parameters may be considered to correspond to "input information," and the artwork generation unit 103 may be considered to generate artwork based on a first element and a second element different from the first element based on the input information.

[0124] Furthermore, in the above embodiment, the parameter generation unit 101 and the artwork generation unit 103 are separated as functional units, but the parameter generation unit 101 and the artwork generation unit 103 may be configured to form an "artwork generation unit." In other words, input information may be input to the artwork generation unit to generate artwork.

[0125] In the above embodiment, the parameter generating unit 101 generates parameters based on information input by the user, but this may be understood as the parameter generating unit 101 generating artwork elements based on the input information. In other words, the parameter generating unit may be understood as a functional unit that generates content elements based on the input information.

[0126] Furthermore, an information processing device may be configured to include the parameter generation unit 101, and the information processing device may output the parameters generated by the parameter generation unit 101. In this case, the information processing device may be, by way of example and not limitation, a device including a generation unit that generates content elements such as artwork (which may be considered as elements or information used to generate content) based on input information. Such an information processing device may be regarded as a type of generating device or output device that generates elements or parts used to generate content such as artwork.

[0127] <Effects of the First Embodiment> This embodiment shows a configuration in which an information processing device 1 acquires input information such as text data, and outputs content such as a catchphrase, product description, product image, and advertisement from the acquired input information. As an example of an effect of the embodiment obtained by such a configuration, it is possible to output content generated by a first element and a second element different from the first element from acquired input information.

[0128] The information processing device 1 may be considered as a device that generates content based on the first element and the second element based on the acquired input information. The content may also be generated by at least a first element and a second element.

[0129] In this embodiment, the input information includes text data, and the content includes at least one of text data and image data. As an example of the effect of the embodiment obtained by such a configuration, it is possible to generate and output content including at least one of text data and image data based on input information including text data.

[0130] In this embodiment, the input information includes image data, and the content includes at least one of text data and image data. As an example of the effect of the embodiment obtained by such a configuration, content including at least one of text data and image data can be generated and output based on input information including image data.

[0131] Furthermore, this embodiment shows a configuration in which the second element is an element related to the output form of the first element generated based on input information. As an example of an effect of an embodiment obtained by such a configuration, content generated by a first element and a second element related to the output form of the first element generated based on the input information can be output from the acquired input information.

[0132] In this case, the first element may be text, and the second element may include at least one of the color of the characters in the text and the font style of the characters in the text. As an example of an effect of an embodiment obtained by such a configuration, it is possible to output text and content generated by at least one of the color of the characters of the text and the font style of the characters of the text.

[0133] In this case, the input information may be text data, and the first element may be text that is different from the text of the input information. As an example of the effect of the embodiment obtained by such a configuration, it is possible to output content generated by text different from the text of the text data of the input information.

[0134] In addition, the content may be image data such as an advertising image, the first element may be text, and the second element may include at least one of the layout of the text in the image of the image data such as the advertising image (for example, but not limited to, the layout pattern) and the color of the image. As an example of the effect of this embodiment obtained by such a configuration, it is possible to output content generated by at least one of the layout of text in an image of image data that is the content and the color of the image.

[0135] <First Modification Example (1)> Instead of acquiring text data or image data entered by a user, the information processing device 1 may acquire text data or image data stored in a database (not shown), and perform processing similar to that described above, for example and without limitation.

[0136] As a non-limiting example, when a business (company, etc.) uses information processing device 1 to generate artwork for an advertisement, it may obtain some of the text data of the advertisement stored in the database and image data of that advertisement from the database and perform processing similar to that described above.

[0137] More specifically, as an example and not by way of limitation, in a messaging application (an example of a chat application, not by way of limitation), if advertisements that have already been distributed to general users, etc. using a so-called official account (a business's account) are stored in a database, the text data and image data of the advertisement for which the version is to be updated, selected by the user via the input unit 50, may be obtained from the database, and processing similar to that described above may be performed.

[0138] <First Modification (2)> By way of example and not limitation, a user may be able to specify the conditions used to generate content such as artwork.

[0139] FIG. 1-4 is a block diagram showing an example of a functional configuration of the information processing device 1 according to this modification. The configuration of the information processing device 1 is the same as that of FIG. 1-1, for example and not by way of limitation, but the input unit 50 has a condition input unit 51.

[0140] Parameters or some elements thereof are input as conditions to the condition input unit 51. The conditions input to the condition input unit 51 are output to at least one of the functional units of the parameter generation unit 101 and the artwork generation unit 103.

[0141] By way of example and not limitation, the user may be able to specify at least one of the following conditions: (a) text character color, (b) text character font style, and (c) image color. Also, by way of example and not limitation, the user may be able to specify at least one of the aforementioned (C) layout pattern, (D) artwork style, and (E) artwork background color as a condition.

[0142] As a non-limiting example, when the color of the characters of the text is input as a condition, the parameter generating unit 101 may generate and output the text colored in the input color.

[0143] For example, and not by way of limitation, if the background color of the artwork is input as a condition, the artwork generation unit 103 may generate the artwork by arranging and rendering text and images in a layout pattern based on the background color of the input artwork. In the above-mentioned example of <okra>, if the condition "background color of artwork = green" is input, the artwork generation unit 103 may generate artwork by placing text and images in a layout pattern with a green background color.

[0144] In this modification, the input information includes condition information (not limited to this, but an example of setting information relating to at least one of the first element and the second element) input by the user. As an example of the effect of the modified example obtained by such a configuration, content generated by the first element and the second element can be output based on input information entered by a user, the input information including setting information regarding at least one of the first element and the second element.

[0145] In this case, the input information may include at least input text data and condition information regarding the second element input by the user, the content may be image data such as an advertisement image, the first element may be text based on the input text data, and the second element may include one or more of the color of the text characters in the image of the image data such as an advertisement image, the font style of the text characters in this image, the layout of the text in this image, and the color of the image, based on the condition information regarding the input second element. As an example of the effect of a modified example obtained by such a configuration, content can be output based on input information entered by a user, which includes at least text data and setting information regarding a second element, generated by a first element which is text based on the text data, and a second element which includes one or more of the color of the text characters in an image of image data which is content, the font style of the text characters in this image, the layout of the text in this image, and the color of this image, based on the setting information.

[0146] <First Modification (3)> As shown in some examples of display screens, the parameter generation unit 101 may generate multiple product descriptions and product images. Also, the artwork generation unit 103 may generate multiple artworks. In other words, the information processing device 1 may generate multiple contents. In this case, by way of example and not limitation, the artwork generator 103 may generate artwork multiple times with different combinations of elements of the generation parameters.

[0147] In this case, the information processing device 1 may display the generated plurality of contents on a display unit of the device itself, or may transmit the generated plurality of contents to an external device (for example, but not limited to, a user's terminal), and the external device may display the received plurality of contents on a display unit. The information processing device 1 may then determine the content selected based on an input by the user of the device itself or the user of the external device to select the content as the final content.

[0148] In this modification, the information processing device 1 outputs a plurality of generated contents and selects a content based on a user's input. As an example of the effect of this modified example obtained by such a configuration, it is possible to enable the user to select content that meets his / her wishes from among a plurality of generated contents.

[0149] <Second Example> The second embodiment relates to repeating the cycle of content generation using the generated content.

[0150] The contents described in the second embodiment can be applied to any of the other embodiments and other modified examples. Furthermore, the same components as those already mentioned are given the same reference numerals and will not be described again.

[0151] FIG. 2-1 is a block diagram showing an example of the functional configuration of the information processing device 1 according to this embodiment. The functional blocks of this information processing device 1 are the same as those in FIG. 1-1, but in this example, the latest generated artwork (previously generated artwork) is configured to be fed back and input to the parameter generating unit 101. The parameter generating unit 101 is then configured to generate parameters based on the generated artwork that has been fed back and input. In this case, the user only needs to input text data or image data the first time, and does not need to input it from the second time onwards.

[0152] The parameter generating unit 101 obtains information that can be used as input information from the generated artwork that is feedback input, for example and not by way of limitation, and generates parameters based on the obtained information. The parameter generation unit 101 may be configured to include, by way of example and not limitation, the image recognition model described above, and may perform image recognition on the artwork that has been fed back and obtain text data or the like that can be used as input information.

[0153] In this case, the configuration shown in Figure 1-4 may be applied, and the parameter generation unit 101 may generate parameters based on the generated artwork that is feedback-input and the conditions input to the condition input unit 51.

[0154] As another example of the configuration of FIG. 2-1, the generated artwork may be fed back to the artwork generator 103 as shown in FIG. 2-2, by way of example and not limitation. In this case, by way of example and not limitation, the configuration shown in Fig. 1-4 may be applied, and the artwork generation unit 103 may generate artwork based at least on the feedback-input generated artwork and the conditions input to the condition input unit 51. Specifically, by way of example and not limitation, the artwork may be generated by performing a process of correcting (adjusting) the feedback-input generated artwork based on the additional conditions. In the above-mentioned example of <okra>, by way of example and not limitation, if the background color of the artwork is "green" as a condition, the artwork generation unit 103 can generate artwork by arranging text and images in a layout pattern with a green background color, while leaving all elements other than the feedback-input artwork background color as they are.

[0155] Note that the generation parameters generated by parameter generation unit 101 may be fed back and input to parameter generation unit 101. In this case, by way of example and not limitation, the configuration shown in FIG. 1-4 may be applied, and parameter generation unit 101 may generate parameters based on at least the feedback-input generation parameters and the conditions input to condition input unit 51. Specifically, by way of example and not limitation, the parameters may be generated by performing a process of correcting (adjusting) the feedback-input generation parameters based on additional conditions. In the above-mentioned example of <okra>, by way of example and not limitation, if the text character color "green" is input as a condition, the artwork generation unit 103 may generate and output an imaged product description in which the text character color is corrected to green, which is the product description as a generation parameter that is feedback-input as an example and not limitation.

[0156] <Processing> FIG. 2-3 is a flowchart showing an example of the flow of processing executed by the control unit 100 of the information processing device 1 in this embodiment. After step E140, the control unit 100 temporarily stores the generated artwork in the storage unit (E250).

[0157] Next, the control unit 100 determines whether or not an additional condition has been input via the input unit 50 (E260). If no input has been made (E260: NO), the control unit 100 proceeds to step E190, for example and not by way of limitation.

[0158] If the input has been made (E260: YES), the control unit 100 returns the process to step E130, for example and not by way of limitation. Then, the parameter generation unit 101 generates parameters based on the additional conditions and the generated artwork temporarily stored in the storage unit. Note that, for example and not by way of limitation, the control unit 100 returns the process to step E140, and the artwork generation unit 103 may then generate artwork based on the generation parameters obtained in the previous step E130, the temporarily saved generated artwork, and the additional conditions.

[0159] <Effects of the second embodiment> In this embodiment, the information processing device 1 acquires input information based on the generated content (not limited to this, but is an example of input information based on the content output by the output unit), and outputs the content generated by the first element and the second element based on the acquired input information. As an example of the effect of the embodiment obtained by such a configuration, input information based on the content output by the output unit can be acquired, and the content can be regenerated and output.

[0160] <Third Example> The third embodiment is an embodiment that makes it possible to use the artwork generated by the artwork generation unit 103 and the parameters generated by the parameter generation unit 101 for generating content from the next time onwards.

[0161] The contents described in the third embodiment can be applied to any of the other embodiments and other modified examples. Furthermore, the same components as those already mentioned are given the same reference numerals and will not be described again.

[0162] FIG. 3-1 is a block diagram showing an example of a functional configuration of the information processing device 1 according to this embodiment. 1-1, the functional blocks of the information processing device 1 include, by way of example and not limitation, a material database 300. The control unit 100 is configured to associate the generated artwork with the generation parameters that are the basis for generating the generated artwork, and cumulatively store (accumulatively store) the associated artwork in the material database 300.

[0163] For example, but not by way of limitation, the artwork generation unit 103 may generate artwork using at least one of the generation parameters stored in the material database 300. In this way, some parameters can be reused to generate the artwork.

[0164] In this case, by way of example and not limitation, artwork may be generated using at least one of the generation parameters stored in the material database 300, including (a) the color of the text characters, (b) the font style of the text characters, (c) the color of the image, etc. Additionally, artwork may be generated using at least one of (C) layout pattern, (D) artwork style, and (E) artwork background color among the generation parameters stored in material database 300. Elements that a user wishes to reuse, such as text character color and font style, image color, layout pattern, and artwork style, can be obtained from material database 300 based on the user's input to select parameters, and artwork can be generated based on the input information and the obtained elements. In this way, it becomes possible to create a template of an element (not limited to this, but an example of a second element) that the user wants to reuse, and use it to generate content.

[0165] It is not necessary for the material database 300 to store all generation parameters. As a non-limiting example, elements that a user may wish to reuse, such as text color, font style, image color, layout pattern, and artwork style, may be stored in the material database 300 based on the user's input to select parameters.

[0166] Furthermore, the artwork generating unit 103 may, for example and without limitation, read out generated artwork stored in the material database 300 and output it as generated artwork. In this way, the artwork can be reproduced.

[0167] Furthermore, the artwork generation unit 103 generates artwork based on a generated artwork (for example, but not limited to, the previously generated artwork) selected from the generated artwork stored in the material database 300 and the generation parameters used to generate the generated artwork. In other words, a combination of the generation parameters and generated artwork from the same previous time may be selected and used to generate the artwork. Furthermore, the artwork generation unit 103 may generate artwork by selecting two or more combinations of generation parameters and generated artwork (for example, the past three combinations, but not limited to these) from the combinations of generation parameters and generated artwork stored in the material database 300. All stored combinations may also be used.

[0168] Similarly, the parameter generating unit 101 may generate and reproduce parameters based on the generating parameters and generated artwork stored in the material database 300.

[0169] FIG. 3-2 is a flowchart showing an example of the flow of processing executed by the control unit 100 of the information processing device 1 in this embodiment. After step E140, the control unit 100 associates the generated artwork with the generation parameters and stores them in the material database 300 in the storage unit (E350). Then, the control unit 100 proceeds to step E190. The generated artwork and generation parameters stored in the material database 300 are used in the next and subsequent steps E130 and E140, as described above.

[0170] <Effects of the third embodiment> In this embodiment, the information processing device 1 includes a material database 300 that stores generation parameters (not limiting, but an example of a storage unit that stores second elements). The information processing device 1 outputs content that is generated based on input information and the generation parameters stored in the material database 300. As an example of an effect of an embodiment obtained by such a configuration, the second element can be stored in a storage unit, and content generated based on input information and the second element stored in the storage unit can be output. This makes it possible, for example and not by way of limitation, to make the second element a template and use it to generate content.

[0171] In this embodiment, the information processing device 1 includes a material database 300 (not limiting, but an example of a storage unit that stores a second element) that stores generated artwork (not limiting, but an example of content). The information processing device 1 generates content based on at least the artwork stored in the material database 300. As an example of an effect of an embodiment obtained by such a configuration, content including a first element and a second element can be stored in a storage unit, and content can be generated based on at least the content stored in the storage unit, which makes it possible, for example and not by way of limitation, to reproduce the generated content.

[0172] <Third Modification (1)> In the above embodiment, the information processing device 1 generates an artwork (first artwork) based on input information entered by one user (first user) and the generation parameters stored in the material database 300. Thereafter, the information processing device 1 can generate an artwork (second artwork) based on input information entered by another user (second user) and the generation parameters stored in the material database 300.

[0173] Also, by way of example and not limitation, in addition to or instead of the creation parameter elements, the material database 300 may store parameter elements that serve as templates created by the user. Similarly, in addition to or instead of the generated artwork, template artwork created by the user may be stored in the material database 300 .

[0174] In this modified example, the information processing device 1 outputs content generated based on first input information based on input from a first user and second elements stored in the material database 300, and outputs content generated based on second input information based on input from a second user different from the first user and parameters stored in the material database 300. As an example of the effect of the modified example obtained by such a configuration, and by way of example and not limitation, even if different users input different input information, a first content corresponding to the first input information and a second content corresponding to the second input information can be appropriately generated and output based on the second element stored in the memory unit.

[0175] <Fourth Example> The fourth embodiment relates to evaluating the generated artwork generated by the artwork generating unit 103.

[0176] The contents described in the fourth embodiment can be applied to any of the other embodiments and other modified examples. Furthermore, the same components as those already mentioned are given the same reference numerals and will not be described again.

[0177] FIG. 4-1 is a block diagram showing an example of a functional configuration of the information processing device 1 according to this embodiment. The functional blocks of this information processing device 1 include a control unit 100 having an artwork evaluation unit 105 in addition to the configuration of FIG. 1-1.

[0178] The artwork evaluation unit 105 has a function of evaluating the generated artwork. The artwork evaluator 105 may be implemented by, for example and not by way of limitation, a model capable of calculating (inferring) an evaluation score (evaluation score value) of the generated artwork. For example, and not by way of limitation, an evaluation score function that is manually set by a human being from the perspective of the aesthetics of the artwork may be used as the evaluation score function for calculating the evaluation score. In this case, the evaluation score function may be defined as a function that uses, as variables, the arrangement of text and images, color scheme, etc. in the artwork, for example and without limitation. Also, the evaluation score function may be defined as a function that increases the evaluation score as the aesthetic quality of the artwork increases, for example and without limitation. However, the present invention is not limited to this.

[0179] In this case, the artwork evaluation unit 105 may, for example and without limitation, generate an evaluation score model for each artwork included in a set of artworks prepared in advance by machine learning using a combination of the artwork and the evaluation score calculated for that artwork using an evaluation score function as a training data set. A model generated in this manner is referred to as a "trained evaluation score model." The artwork evaluator 105 may then, for example and without limitation, use the generated artwork as input to a trained rating score model to infer a rating score.

[0180] The evaluation score model is not limited to a machine learning model, and may be, for example, a mathematical statistical model such as a linear or nonlinear regression model, or a neural network model such as a DNN.

[0181] Furthermore, as a non-limiting example, as shown in the example display screen, the artwork may be an advertisement, and an evaluation score for the advertisement may be calculated. In this case, the artwork evaluation unit 105 may, by way of example and not limitation, generate a learned evaluation score model for each advertisement included in a set of pre-prepared advertisements by machine learning using a combination of the advertisement and an evaluation score calculated for that advertisement using an evaluation score function as a learning dataset. In this case, the inferred evaluation score can be used as information for evaluating and analyzing the effectiveness of the advertisement, such as the promotional effect of the advertisement or the sales promotion effect of the advertised product (this can also be called effectiveness information regarding the effectiveness of the advertisement).

[0182] Also, unlike this embodiment, instead of the information processing device 1 calculating the evaluation score, an external device with which the information processing device 1 can communicate may calculate the evaluation score, and the information processing device 1 may obtain the evaluation score by receiving it from the external device. In this case, the information processing device 1 may transmit evaluation score request information including the generated artwork to the external device, and the external device may calculate the evaluation score and transmit it to the information processing device 1.

[0183] <Processing> FIG. 4-2 is a flowchart showing an example of the flow of processing executed by the control unit 100 of the information processing device 1 in this embodiment. After step E140, the artwork evaluation unit 105 performs artwork evaluation processing (E450). Specifically, by way of example and not limitation, an evaluation score for the generated artwork is calculated based on the above-described method, and the calculated evaluation score is stored in a storage unit (not shown).

[0184] Next, the control unit 100 performs an evaluation score output process to output the calculated evaluation score (E460). In this case, the calculated evaluation score may be displayed on a display unit (not shown), or may be transmitted to an external device by a communication unit. Then, the control unit 100 proceeds to step E190.

[0185] <Effects of the Fourth Embodiment> In this embodiment, the information processing device 1 is configured to acquire an evaluation score of content such as artwork. As an example of an effect of the embodiment obtained by such a configuration, it becomes possible to obtain an index value for evaluating the generated content (content including the first element and the second element).

[0186] In this case, the artwork is an advertisement, and the information processing device 1 may acquire an evaluation score (not limiting, but an example of effectiveness information regarding the effectiveness of the advertisement) regarding the generated advertisement. As an example of an effect of the embodiment obtained by such a configuration, it becomes possible to acquire an index value for analyzing the effect of the generated advertisement (advertisement including the first element and the second element).

[0187] <Fourth Modification (1)> In the above embodiment, by way of example and not limitation, a business may generate multiple advertisements using the information processing device 1, and post the generated advertisements on its own homepage or distribute them to terminals using an application such as a messaging application. In this case, the information processing device 1 may calculate and tally the conversion rate for each advertisement. Then, the information processing device 1 may be configured to estimate an expected conversion rate for a newly generated advertisement. Furthermore, as the evaluation score model described above, a model capable of estimating the relationship between the advertisement to be generated and the expected conversion rate may be used. In this way, it is possible to estimate (predict) the conversion rate of the generated advertisement.

[0188] <Fifth Example> The fifth embodiment is an embodiment related to optimizing the parameter generating unit 101 and the artwork generating unit 103.

[0189] The contents described in the fifth embodiment can be applied to any of the other embodiments and other modified examples. Furthermore, the same components as those already mentioned are given the same reference numerals and will not be described again.

[0190] FIG. 5-1 is a block diagram showing an example of a functional configuration of the information processing device 1 according to this embodiment. The functional blocks of the information processing device 1 include the control unit 100 having an optimization unit 107 in addition to the configuration of FIG.

[0191] The optimization unit 107 optimizes the functional unit to be optimized (for example, but not limited to, at least one of the parameter generation unit 101 and the artwork generation unit 103) based on the evaluation results such as the evaluation score calculated by the artwork evaluation unit 105.

[0192] In this case, the optimization unit 107 may, by way of example and not limitation, use the artwork generation history (including the history of the evaluation score) to optimize parameters related to the functional unit to be optimized in a direction that increases the evaluation score (where the larger the evaluation score, the higher the evaluation of the artwork).

[0193] As one method, the optimization unit 107 may adjust hyperparameters of the model of the functional unit to be optimized (for example, but not limited to, parameters related to the neural network model of the functional unit to be optimized). In this case, as a non-limiting example, a set of candidate hyperparameter values ​​may be prepared, and a combination of hyperparameter values ​​that changes in the direction of increasing the evaluation score may be determined using techniques such as random search, grid search, or genetic algorithms.

[0194] Note that, as an example and not a limitation, the technique of the second embodiment can be applied so that the information processing device 1 acquires input information based on artwork generated by the artwork generation unit 103, and generates artwork based on the acquired input information. In this case, the information processing device 1 may, for example and not limitation, generate artwork based on input information obtained from the previously generated artwork using a functional unit optimized based on evaluation information of artwork generated in the past. Since optimization is performed based on evaluation information of the artwork, in this example, the information processing device 1 can be considered to acquire input information based on the first artwork and generate the second artwork based on the acquired input information and the evaluation information of the artwork.

[0195] Also, similar to the fourth embodiment, artwork may be an advertisement, and the artwork evaluation unit 105 may infer an evaluation score for the advertisement. Then, the optimization unit 107 may adjust hyperparameters of a model of a functional unit to be optimized, for example and not by way of limitation.

[0196] In this case, too, the method of the second embodiment may be applied as an example and not a limitation, and the information processing device 1 may acquire input information based on the advertisement generated by the artwork generation unit 103, and generate the advertisement based on the acquired input information. Since optimization is performed based on evaluation information of the advertisement (an example of effectiveness information regarding the effectiveness of the advertisement), in this example, the information processing device 1 can be considered to acquire input information based on the first advertisement and generate the second advertisement based on the acquired input information and the effectiveness information regarding the effectiveness of the advertisement.

[0197] Also, unlike this embodiment, instead of the information processing device 1 calculating the evaluation score, an external device with which the information processing device 1 can communicate may calculate the evaluation score, and the information processing device 1 may obtain the evaluation score by receiving it from the external device. In this case, the information processing device 1 may transmit evaluation score request information including the generated artwork to the external device, and the external device may calculate the evaluation score and transmit it to the information processing device 1.

[0198] <Processing> FIG. 5-2 is a flowchart showing an example of the flow of processing executed by the control unit 100 of the information processing device 1 in this embodiment. After step E460, the control unit 100 determines whether or not optimization is to be performed (E570). In this case, by way of example and not limitation, - There was user input instructing optimization to be performed - The conditions regarding the number of times artwork is generated (for example, but not limited to, the number of times artwork has been generated since the last optimization was performed has reached the set number of times) have been met. - The time-related conditions (for example, but not limited to, the time or date for optimization has arrived, or a set amount of time has passed since the last optimization) have been met. Optimization may be performed when the above conditions are met.

[0199] If it is determined that optimization is not to be performed (E570: NO), the control unit 100 advances the process to step E190.

[0200] If it is determined that optimization is to be performed (E570: YES), the control unit 100 performs optimization processing (E580). Specifically, as an example and not by way of limitation, based on generation history data (not shown) that associates artwork stored in a memory unit with an evaluation score, and based on the above-mentioned method, at least one of the functional units of the parameter generation unit 101 and the artwork generation unit 103 is optimized. Then, the control unit 100 proceeds to step E190.

[0201] <Effects of the Fifth Embodiment> This embodiment shows a configuration in which the information processing device 1 optimizes the parameter generating unit 101 based on the generated artwork. As an example of the effect of the embodiment obtained by such a configuration, it is possible to optimize the functional unit that generates elements included in the content based on the generated content.

[0202] Furthermore, this embodiment shows a configuration in which the information processing device 1 optimizes the artwork generating unit 103 based on the generated artwork. As an example of the effect of the embodiment obtained by such a configuration, it is possible to optimize the functional unit that generates the content based on the generated content.

[0203] In addition, this embodiment shows a configuration in which the information processing device 1 acquires input information based on a first artwork (not limited to, but an example of the first content) generated by the artwork generation unit 103, and outputs a second artwork (not limited to, but an example of the second content) generated by the first element and the second element based on the acquired input information and the evaluation score. As an example of an effect of an embodiment obtained by such a configuration, it is possible to output second content generated based on acquired input information based on the first content and evaluation information of the first content.

[0204] In this case, the artwork is an advertisement, and the information processing device 1 is configured to acquire input information based on the first advertisement generated by the artwork generation unit 103 and an evaluation score of the first advertisement, and output a second advertisement generated based on the acquired input information and the acquired evaluation score. As an example of an effect of an embodiment obtained by such a configuration, a second advertisement generated based on acquired input information based on the first advertisement and effect information regarding the effect of the first advertisement can be output.

[0205] <Fifth Modification (1)> The contents of the fourth modified example (1) may be applied to the above embodiment, so that the information processing device 1 can estimate an expected conversion rate for a newly generated advertisement. Furthermore, as the evaluation score model described above, a model capable of estimating the relationship between the advertisement to be generated and the expected conversion rate may be used. Then, parameters related to the functional units to be optimized may be optimized in a direction that increases the expected conversion rate. In this way, it is possible to optimize the functionality associated with generating advertisements based on the conversion rate.

[0206] <Application example> As a specific configuration to which the present invention and the information processing device 1 described in the above embodiment are applied, any of the following may be applied, by way of example and not limitation. (1) Standalone (2) Client-server system

[0207] (1) In a stand-alone configuration, the information processing device 1 may be, for example, a terminal or a server, but is not limited thereto.

[0208] Examples of terminals include, but are not limited to, smartphones, mobile terminals (feature phones), computers (such as, but not limited to, desktops, laptops, and tablets), media computing platforms (such as, but not limited to, cable and satellite set-top boxes and digital video recorders), handheld computing devices (such as, but not limited to, personal digital assistants (PDAs) and email clients), wearable terminals (such as, but not limited to, eyeglasses and watch devices), virtual reality (VR) terminals, smart speakers (voice recognition devices), or other types of computers or communication platforms. Terminals may also be referred to as information processing terminals.

[0209] A server may include, by way of example and not limitation, a server appliance, a computer (by way of example and not limitation, a desktop, laptop, tablet, etc.), a media computing platform (by way of example and not limitation, a cable, satellite set-top box, digital video recorder), a handheld computing device (by way of example and not limitation, a PDA (Personal Digital Assistant), email client, etc.), or any other type of computing or communications platform.

[0210] Furthermore, the server and the terminal may or may not be expressed as information processing devices.

[0211] As a non-limiting example, when a user performs the various inputs described above via the operation unit of the information processing device 1, artwork may be generated and the generated artwork may be displayed on the display unit of the information processing device 1.

[0212] In this configuration, as an example and not a limitation, at least one of the generation parameters and the generated artwork generated by the information processing device 1 may be transmitted to a terminal owned by the user.

[0213] (2) In a client-server system, for example and not by way of limitation, the information processing device 1 may be a server that communicates with a user's terminal, thereby forming a system that realizes the above content. The terminal and the server may be the same devices as those described above. An example of processing in this case will be described below.

[0214] FIG. 6A is a flowchart showing an example of the flow of processing executed by each device in this embodiment. In this figure, the left side shows the processing executed by the control unit (not shown) of the user's terminal, and the right side shows the processing executed by the control unit (not shown) of the server.

[0215] The control unit of the terminal transmits artwork generation request information to the server via a communication unit (not shown) based on a user input to an input unit (not shown) (A110). The artwork generation request information may include information that can identify the user or the terminal, as well as text (text data) and images (image data) input by the user. As mentioned above, the information may also include information on conditions entered by the user.

[0216] When the server's control unit receives artwork generation request information from the terminal via a communication unit (not shown), it performs artwork generation main processing (S110). As the artwork generation main process, various types of processes exemplified as processes performed by the information processing device 1 in each of the above-described embodiments and modifications can be applied.

[0217] Next, the control unit of the server transmits the generated artwork to the terminal via the communication unit (S130). Then, the control unit of the server ends the process.

[0218] After A110, when the communication unit receives the generated artwork from the server, the control unit of the terminal displays the received generated artwork on the display unit (not shown) (A130). Then, the control unit of the terminal ends the process.

[0219] Note that the server may perform the parameter generation process in step S100 and then transmit the generated parameters to the terminal, and the terminal may then display the received generated parameters on a display unit. In this case, the server may transmit keywords, etc. included in the parameters to the terminal, and the terminal may display the received keywords, etc. on a display unit. The terminal may then transmit keyword selection information or keyword editing information to the server based on input by the terminal user to select or edit a keyword. The server may then perform artwork generation processing based on the received information.

[0220] Alternatively, in step S110, the server may generate a plurality of artworks, and in step S130, the generated plurality of artworks may be transmitted to the terminal, and the terminal may then display the received plurality of artworks on the display unit. In this case, the terminal may determine a final artwork based on an input by the terminal user selecting at least one artwork from the plurality of artworks and store the final artwork in the storage unit. In this case, the terminal may transmit artwork selection information to the server, and the server may determine a final artwork based on the received information and store the final artwork in the storage unit.

[0221] These processes may be implemented by an application (for example, but not limited to, an artwork generation application (such as the advertisement generation application described above)). Furthermore, the application may be a native application, a web application, or a hybrid application.

[0222] Furthermore, at least a part of the processing described as being performed by the control unit of the server may be performed by the control unit of the terminal.

[0223] <Other> When using an application such as an advertisement generation application, the server for distributing the application (the server from which the terminal downloads the application) may be configured as a server different from the server that provides the corresponding service (application). In other words, the server for distributing the application and the server that performs application management processing may be configured as physically separated servers, or may be configured as a single server.

[0224] Furthermore, the application is not limited to various application programs, but may include, for example and without limitation, a program that provides a function of another service as one function of the original application, a program for updating the original application, etc. Also, data used in the application program (which may include data for updating the application, etc.) may be included.

[0225] The present invention may also be realized by a system such as the distributed system described above, in which the functions of a server or server system are provided in a terminal. [Explanation of symbols]

[0226] 1. Information processing equipment 50 Input section 100 control section 101 Parameter generation unit 103 Artwork Generation Unit 150 Output section 300 Material Database< / x> < / x> < / x> < / x> < / x> < / x> < / x> < / x> < / x> < / x>

Claims

1. An information processing device, an acquisition unit that acquires input information; an output unit that outputs image content generated by a first element and a second element different from the first element using a trained model that has trained to generate content based on the input information; the input information includes at least one of first text and a first image; the first element is a second text generated by the trained model based on the input information; The second element is a second image generated by the trained model based on the input information, the output unit outputs the image content including the first element and the second element by determining a layout of at least the second text and the second image. Information processing device.

2. 2. The information processing device according to claim 1, an evaluation unit for evaluating the image content; the output unit outputs the image content modified based on the evaluation result of the evaluation unit. Information processing device.

3. 3. The information processing device according to claim 2, the image content is an advertisement; The evaluation result includes effectiveness information regarding the effectiveness of the advertisement. Information processing device.

4. 2. The information processing device according to claim 1, the acquisition unit acquires the input information based on the image content output by the output unit. Information processing device.

5. 2. The information processing device according to claim 1, the output unit outputs the generated image content; a selection unit that selects the image content based on a user input; Information processing device.

6. 2. The information processing device according to claim 1, the layout is determined based on the input information and at least one of the first element and the second element; Information processing device.

7. 2. The information processing device according to claim 1, the layout is determined based on at least one of a style of the image content and a color of the image content; Information processing device.

8. 2. The information processing device according to claim 1, the output unit outputs the image content including the first element and the second element by further determining a color of the second text. Information processing device.

9. 2. The information processing device according to claim 1, the output unit outputs the image content including the first element and the second element by further determining a font style of the second text. Information processing device.

10. 2. The information processing device according to claim 1, the output unit outputs the image content including the first element and the second element by further determining a color of the second image. Information processing device.

11. 2. The information processing device according to claim 1, the output unit outputs at least one of the first element and the second element and the generated image content. Information processing device.

12. 12. The information processing device according to claim 1, The trained model includes a model that generates the first element or the second element. Information processing device.

13. 13. The information processing device according to claim 12, generating a prompt sentence for generating the first element or the second element based on the input information; the model generates the first element or the second element based on the prompt sentence; Information processing device.

14. 12. The information processing device according to claim 1, The trained model includes a large-scale language model and an image generation model. Information processing device.

15. An information processing method for an information processing device, acquiring input information by an acquisition unit of the information processing device; outputting, by an output unit of the information processing device, image content generated by a first element and a second element different from the first element using a trained model that has been trained to generate content based on the input information; the input information includes at least one of first text and a first image; the first element is a second text generated by the trained model based on the input information; The second element is a second image generated by the trained model based on the input information, An information processing method comprising: outputting, by the output unit, the image content including the first element and the second element by determining a layout of at least the second text and the second image.

16. A program executed by an information processing device, acquiring input information by an acquisition unit of the information processing device; outputting, by an output unit of the information processing device, image content generated by a first element and a second element different from the first element using a trained model that has been trained to generate content based on the input information; the input information includes at least one of first text and a first image; the first element is a second text generated by the trained model based on the input information; The second element is a second image generated by the trained model based on the input information, A program executed by the information processing device to output the image content including the first element and the second element by the output unit by determining a layout of at least the second text and the second image.

Images