Information processing program, information processing method, and information processing device

Abstract

To improve accuracy of generating images that represent narratives.SOLUTION: An information processing program causes a computer to execute processing to acquire an evaluation perspective for an image that represents a narrative generated by a first machine learning model based on a first prompt including the narrative related to a message, and to acquire a first image that is generated by inputting a third prompt, which is generated based on the evaluation perspective and a second prompt including instructions to generate the image, into a second machine learning model.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to an information processing program, an information processing method, and an information processing device. [Background technology]

[0002] In the field of AI (Artificial Intelligence), there is a technology called generative AI, which generates new data from text and other sources. Generative AI can generate various types of data, including text, images, and audio. Among generative AI, generative AI that generates images is called image generation AI. For example, when image generation AI receives input of text written in natural language, known as a prompt, it generates an image according to the input prompt.

[0003] One method for effectively communicating a message to others is to use a narrative in conjunction with images and videos. Narrative can be translated as "story," "narration," or "storytelling," and an example of a narrative would be a story based on one's own experiences. By conveying a message through a narrative, it is easier for the listener to empathize and there is less psychological reactance (rejection) in the listener. By using a narrative in conjunction with images and videos that represent the narrative, it is possible to communicate a message more effectively. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2023 / 0377226 [Patent Document 2] U.S. Patent No. 1,179,780 [Patent Document 3] Patent No. 7398723 specification [Patent Document 4] Japanese Patent Publication No. 2022-106147 [Non-patent literature]

[0005] [Non-Patent Document 1] Melvin Wong, Yew-Soon Ong, Abhishek Gupta, Kavitesh Kumar Bali, Caishun Chen, “Prompt Evolution for Generative AI: A Classifier-Guided Approach”, 2023 IEEE Conference on Artificial Intelligence (CAI), IEEE.org, 2003, p.226-229 Summary of the Invention [Problem to be solved by the invention]

[0006] When generating an image that represents a narrative using a generation AI, the user must specify the desired image using a prompt. However, in order for the user to specify the desired image, experience and know-how for creating the prompt are required. Therefore, depending on the content of the prompt, the accuracy of image generation may deteriorate, and it may not be possible to generate an image that represents the narrative.

[0007] In the prior art, a technique has been proposed in which image evaluation criteria are defined, images generated by an image generation AI are evaluated based on the criteria, and images with high evaluation scores are used to iteratively improve image generation. However, because the evaluation criteria are defined by the user in the same way as prompts, it is sometimes impossible to generate images that convey a narrative depending on the definition of the evaluation criteria.

[0008] In one aspect, the present invention aims to provide an information processing program, an information processing method, and an information processing device that can improve the accuracy of generating images that represent a narrative. [Means for solving the problem]

[0009] In one aspect, an information processing program is provided that causes a computer to execute a process of obtaining an evaluation perspective of an image representing a narrative generated by a first machine learning model based on a first prompt including a narrative related to a message, and obtaining a first image generated by inputting a third prompt generated based on the evaluation perspective and a second prompt including instructions for image generation into the second machine learning model.

[0010] In one aspect, there is provided an information processing method in which a computer executes a process similar to the process based on the information processing program.

[0011] In one aspect, an information processing device is provided that executes processing similar to the processing based on the information processing program.

[0012] In one aspect, an information processing program is provided that causes a computer to execute a process of receiving a narrative related to a message, and acquiring a first image generated by inputting a third prompt generated based on an evaluation perspective of an image representing the narrative generated by a first machine learning model based on a first prompt including the narrative and a second prompt including instructions for image generation into a second machine learning model.

[0013] In one aspect, there is provided an information processing method in which a computer executes a process similar to the process based on the information processing program.

[0014] In one aspect, an information processing device is provided that executes processing similar to the processing based on the information processing program. [Effects of the Invention]

[0015] The accuracy of generating images that represent a narrative can be improved. [Brief explanation of the drawings]

[0016] [Figure 1]FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a functional block diagram illustrating the functional configuration of each component of the information processing system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a narrative according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of an initial prompt according to the embodiment. [Figure 5-1] FIG. 5-1 is a diagram illustrating an example of a prompt template (processing No. 1 to No. 2) according to the embodiment. [Figure 5-2] FIG. 5-2 is a diagram illustrating an example of a prompt template (processes No. 3 to No. 4) according to the embodiment. [Figure 5-3] FIG. 5-3 is a diagram illustrating an example of a prompt template (process No. 5) according to the embodiment. [Figure 5-4] FIG. 5-4 is a diagram illustrating an example of a prompt template (process No. 6) according to the embodiment. [Figure 5-5] FIG. 5-5 is a diagram illustrating an example of a prompt template (process No. 7) according to the embodiment. [Figure 5-6] FIG. 5-6 is a diagram illustrating an example of a prompt template (process No. 8) according to the embodiment. [Figure 5-7] FIG. 5-7 is a diagram illustrating an example of a prompt template (process No. 9) according to the embodiment. [Figure 5-8] FIG. 5-8 is a diagram illustrating an example of a prompt template (process No. 10) according to the embodiment. [Figure 5-9] FIG. 5-9 is a diagram illustrating an example of a prompt template (process No. 11) according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a subject according to an embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of an emergent emotion according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of an evaluation list according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the evaluation importance according to the embodiment. [Figure 10-1] FIG. 10-1 is a diagram illustrating an example of a generated image (I0) according to the embodiment. [Figure 10-2] FIG. 10-2 is a diagram illustrating an example of a generated image (I1) according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of an evaluation result according to the example. [Figure 12] FIG. 12 is a diagram illustrating an example of a current prompt according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a sequence diagram of the information processing system according to the embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a processing flow of the control device according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a hardware configuration of a user terminal according to the embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of the hardware configuration of a generation AI server according to the embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of a hardware configuration of a control device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, with reference to the drawings, detailed descriptions will be given of embodiments of an information processing program, an information processing method, and an information processing device according to the present invention. Note that these embodiments are merely examples for implementing the present invention and are not intended to limit the present invention. The prompts and the outputs of each machine learning model are not limited to those of the present embodiment, and the order of processing may be changed or parallel processing may be performed within a consistent range. [Example]

[0018] [System Description] An information processing system according to this embodiment will be described with reference to Figure 1. Figure 1 is a diagram illustrating an example of the configuration of an information processing system according to this embodiment. The information processing system 10 is a system in which a user terminal 11, a control device 12, and a generation AI server 13 are connected via a network 14. The information processing system 10 is a system that generates an image representing a narrative using an image generation AI based on the narrative that the user wants to represent in an image and an initial prompt that instructs image generation.

[0019] The network 14 may be a wired or wireless communication network such as the Internet or an intranet. For example, the network 14 may be configured by connecting multiple intranets, or the Internet and an intranet via a gateway or other device.

[0020] The user terminal 11 is an information processing device that receives input from a user who is generating an image, including the narrative the user wants to express in the image and an initial prompt that instructs image generation, and transmits this to the control device 12. Examples of the user terminal 11 include a personal computer, a tablet terminal, and a smartphone.

[0021] The control device 12 is an information processing device that, upon receiving an initial prompt and narrative from the user terminal 11, controls image generation using an image generation AI based on the initial prompt and narrative, and transmits the generated image to the user terminal 11. The control device 12 is, for example, a server or a personal computer.

[0022] The generation AI server 13 is an information processing device that, upon receiving a prompt or image from the control device 12, generates text or an image using a generation AI based on the prompt or image and transmits it to the control device 12. As an example, the generation AI server 13 is a server or a personal computer.

[0023] [Functional configuration of user terminal 11] 2 is a diagram illustrating an example of a functional block diagram showing the functional configuration of each component of an information processing system according to an embodiment. First, the functional configuration of a user terminal 11 according to an embodiment will be described with reference to FIG. 2. The user terminal 11 has a communication unit 20, an input unit 21, an output unit 22, a control unit 23, and a storage unit 24.

[0024] The communication unit 20 is a processing unit that controls communication with other devices, and is realized by a communication interface such as a NIC (Network Interface Card). For example, narratives, initial prompts, and generated images are input and output via the communication unit 20.

[0025] The input unit 21 is a processing unit that controls input from the user to the user terminal 11, and is realized by, for example, a touch panel or a mouse. For example, the control unit 23 accepts input of a narrative or an initial prompt from the user via the input unit 21.

[0026] The output unit 22 is a processing unit that controls the output from the user terminal 11 to the user, and is realized by, for example, a display. For example, the control unit 23 displays the generated image to the user via the output unit 22.

[0027] The storage unit 24 stores various data or various programs executed by the control unit 23. The storage unit 24 is realized by, for example, a main storage device such as a RAM (Random Access Memory) or an auxiliary storage device such as an HDD (Hard Disk Drive). The storage unit 24 stores a generated image DB 240, for example.

[0028] The generated image DB 240 is a database that stores generated images. The generated images stored in the generated image DB 240 are generated images that the user terminal 11 acquires from the control device 12. The generated images are images generated by the second machine learning model.

[0029] Next, the control unit 23 will be described. The control unit 23 controls the overall processing of the user terminal 11. The control unit 23 is realized by a processor such as a CPU (Central Processing Unit), GPU (Graphical Processing Unit), or DSP (Digital Signal Processor) reading a program stored in a storage device, expanding the program in a main storage device such as RAM, and executing the program. The control unit 23 may be realized by including an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0030] Control unit 23 receives input of a narrative and an initial prompt from the user via input unit 21. Control unit 23 transmits the received narrative and initial prompt to control device 12 via communication unit 20. When control unit 23 receives a generated image from control device 12 via communication unit 20, it stores the received generated image in generated image DB 240.

[0031] In this way, the control unit 23 accepts a narrative related to the message and acquires an image generated by inputting into the second machine learning model an evaluation perspective of an image representing the narrative generated by the first machine learning model based on a first prompt including the narrative and a third prompt generated based on a second prompt including instructions for image generation.

[0032] The control unit 23 displays the received generated image to the user via the output unit 22. As an example, the control unit 23 generates materials and videos using the generated image using known technology. The materials and videos may include images other than the generated image. Examples of materials and videos generated by the control unit 23 will be described later.

[0033] [Functional configuration of Generative AI Server 13] The functional configuration of the generation AI server 13 according to the embodiment will be described with reference to Fig. 2. The generation AI server 13 includes a communication unit 40, a control unit 41, and a storage unit .

[0034] The communication unit 40 is a processing unit that controls communication with other devices, and is realized by a communication interface such as a NIC. Prompts and generated images, for example, are input and output via the communication unit 40.

[0035] The storage unit 42 stores various data or various programs executed by the control unit 41. The storage unit 42 is realized by, for example, a main storage device such as a RAM or an auxiliary storage device such as a HDD. The storage unit 42 stores, as an example, a first machine learning model DB 420, a second machine learning model DB 421, and a third machine learning model DB 422.

[0036] The first machine learning model DB 420 is a database that stores a first machine learning model. The first machine learning model is a machine learning model that generates new text when a prompt is input. As an example, the first machine learning model is a generative AI. As an example, the first machine learning model is an LLM (Large Language Model).

[0037] The second machine learning model DB421 is a database that stores a second machine learning model. The second machine learning model is a machine learning model that generates new image data when a prompt is input. As an example, the second machine learning model is a generative AI. As an example, the second machine learning model is a diffusion model, a generative adversarial network (GAN), an autoregressive model, or a variational autoencoder. As an example, the second machine learning model is a text-to-image machine learning model that outputs an image when text is input. As an example, the second machine learning model is a machine learning model in which the number of images to be generated in response to a single input of a prompt can be set by a parameter. In this embodiment, the second machine learning model generates two images in response to a single input of a prompt.

[0038] The third machine learning model DB422 is a database that stores a third machine learning model. The third machine learning model is a machine learning model that generates new text related to an image when an image and text are input. As an example, the third machine learning model is a generative AI. As an example, the third machine learning model is a multimodal LLM or a large multimodal model (LMM).

[0039] Next, the control unit 41 will be described. The control unit 41 controls the overall processing of the generation AI server 13. The control unit 41 is realized by a processor such as a CPU, GPU, or DSP reading a program stored in a storage device, expanding it into a main storage device such as RAM, and executing it. The control unit 41 may also be realized by including an integrated circuit such as an ASIC or FPGA.

[0040] The control unit 41 acquires a prompt from the control device 12. If the prompt is addressed to the first machine learning model, the control unit 41 inputs the prompt to the first machine learning model. For example, the control unit 41 inputs a prompt generated based on templates of processes No. 1 to No. 4 in FIGS. 5-1 to 5-2 and processes No. 6 to No. 11 in FIGS. 5-4 to 5-9 to the first machine learning model. The control unit 41 transmits text output from the first machine learning model to the control device 12. If the prompt is addressed to the second machine learning model, the control unit 41 inputs the prompt to the second machine learning model. The control unit 41 transmits an image output from the second machine learning model to the control device 12.

[0041] The control unit 41 acquires a prompt and an image from the control device 12. If the prompt and the image are intended for the third machine learning model, the control unit 41 inputs the prompt and the image to the third machine learning model. As an example, the control unit 41 inputs a prompt generated based on the template for process No. 5 in FIG. 5-3 to the third machine learning model. The control unit 41 transmits text output from the third machine learning model to the control device 12.

[0042] [Functional configuration of control device 12] The functional configuration of the control device 12 according to the embodiment will be described with reference to Fig. 2. The control device 12 includes a communication unit 30, a control unit 31, and a storage unit 32.

[0043] The communication unit 30 is a processing unit that controls communication with other devices, and is realized by a communication interface such as a NIC. Prompts, narratives, and generated images, for example, are input and output via the communication unit 30.

[0044] The storage unit 32 stores various data or various programs executed by the control unit 31. The storage unit 32 is realized by, for example, a main storage device such as a RAM or an auxiliary storage device such as a HDD. The storage unit 32 stores, for example, a narrative DB 320, an initial prompt DB 321, a prompt template DB 322, a theme DB 323, an evoked emotion DB 324, an evaluation list DB 325, an evaluation importance DB 326, a generated image DB 327, an evaluation result DB 328, and a current prompt DB 329.

[0045] The narrative DB 320 is a database that stores narratives. A narrative is data in a natural language, such as text data, received from the user terminal 11. Fig. 3 is a diagram illustrating an example of a narrative according to the embodiment.

[0046] Here, we will explain what a narrative is. A narrative is a story that follows a timeline and includes emotions and meanings, such as, "Perseverance is power. If you keep at something without giving up, you will surely get results. Each one is a small action, but it is these small efforts that are powerful." Specific types of narratives include self-narratives (stories about oneself, stories of experiences) such as "the experiences of successful people," messages to others (messages to others) such as "messages from company representatives," moral stories such as "The North Wind and the Sun," and fables.

[0047] A narrative contains two elements: a theme and an elicited emotion. The theme is the message the user wants to convey to the listener through the narrative. An elicited emotion is the emotion the listener experiences when the narrative is conveyed to the listener. The theme and the elicited emotion are not necessarily directly stated in the narrative text. Therefore, by conveying a message through a narrative, the user can naturally appeal to the listener's emotions, making it easier for the listener to empathize and self-identify, leading to greater empathy. Furthermore, because the message is conveyed through narrative rather than simply imposed, it is less likely to cause psychological reactance in the listener. Therefore, for example, by using "the experiences of successful people" in a message aimed at improving motivation, or by using "a company representative message" in a message aimed at motivating employees, messages can be effectively conveyed to the listener. This kind of change in attitude and behavior caused by a narrative is called narrative persuasion.

[0048] Returning to FIG. 2, the initial prompt DB 321 is a database that stores initial prompts. The initial prompt is data in a natural language received from the user terminal 11, and is, for example, text data. FIG. 4 is a diagram illustrating an example of an initial prompt according to the embodiment. The initial prompt is one of the prompts that is received by the user terminal 11 as an input from a user. The prompt is text data written in a natural language that indicates instructions for the first machine learning model, the second machine learning model, and the third machine learning model. The initial prompt is a prompt that includes an instruction for image generation for the second machine learning model.

[0049] Returning to Figure 2, the prompt template DB322 is a database that stores prompt templates, which are templates for generating prompts to be sent to the generation AI server 13. Figures 5-1 to 5-9 are diagrams showing examples of prompt templates according to an embodiment. A prompt template includes the items "Process No.", "Process Item," and "Template." "Process No." is an identifier that identifies the process that uses the template. "Process Item" is an item of the process that uses the template. "Template" is a template for generating a prompt to be used in the process indicated by "Process No." and "Process Item."

[0050] 2, the theme DB 323 is a database that stores the themes of the narrative. Fig. 6 is a diagram for explaining an example of the theme according to the embodiment. Fig. 6 shows an example of the theme of the narrative in Fig. 3.

[0051] 2, the arising emotion DB 324 is a database that stores the arising emotions of the narrative. Fig. 7 is a diagram for explaining an example of the arising emotions according to the embodiment. Fig. 7 is an example of the arising emotions of the narrative of Fig. 3.

[0052] Returning to Figure 2, the evaluation list DB325 is a database that stores evaluation lists. The evaluation list is data that lists evaluation perspectives for evaluating whether a generated image represents a narrative. An evaluation perspective is, for example, a feature that is estimated to be included in an image that represents a narrative. An evaluation perspective is, for example, a visual element that depicts a scene or composition of an image that represents a narrative. Figure 8 is a diagram illustrating an example of an evaluation list according to an embodiment. Numbers at the beginning of sentences are identification numbers for identifying evaluation perspectives. Figure 8 is an example of an evaluation list for the narrative of Figure 3. As an example, the evaluation list includes multiple evaluation perspectives.

[0053] Returning to FIG. 2, the evaluation importance DB 326 is a database that stores evaluation importance. The evaluation importance is a value that indicates the degree to which the evaluation perspective is related to the "theme" and "arising emotion" of the narrative, and indicates the importance of each evaluation perspective. FIG. 9 is a diagram explaining an example of evaluation importance according to an embodiment. The numbers at the beginning of each sentence are identification numbers for identifying the evaluation perspective, and the evaluation importance is indicated for each evaluation perspective in the evaluation list. As an example, the evaluation importance is expressed in the range of 0 to 1, and the closer the value is to 1, the higher the importance. FIG. 9 is an example of the evaluation importance in the evaluation list of FIG. 8.

[0054] Returning to FIG. 2, the generated image DB 327 is a database that stores generated images. The generated images are image data generated by the second machine learning model. FIGS. 10-1 and 10-2 are diagrams showing examples of generated images according to the embodiment. FIG. 10-1 is an example of a generated image (I0) generated based on the initial prompt of FIG. 4. FIG. 10-2 is an example of a generated image (I1) generated based on the initial prompt of FIG. 4.

[0055] Returning to FIG. 2, the evaluation result DB 328 is a database that stores evaluation results, which are results of evaluating a generated image based on the evaluation list. The evaluation results are text data including evaluation details and evaluation scores. FIG. 11 is a diagram illustrating an example of evaluation results according to the embodiment. The letter F at the beginning of a sentence indicates an evaluation viewpoint, and the number following the letter F indicates an identification number of the evaluation viewpoint. The letter I at the beginning of a sentence indicates a generated image, and the number following the letter I indicates an identification number of the generated image. As an example, the evaluation results are stored for each generated image with respect to each evaluation viewpoint. A sentence beginning with F indicates an evaluation viewpoint, and a sentence beginning with I indicates an evaluation result for each generated image. The score at the end of the evaluation result for each generated image is the evaluation score for the evaluation viewpoint for each generated image. As an example, the evaluation score for each evaluation viewpoint is expressed in a range from 0 to 1, with values ​​closer to 1 indicating a higher evaluation. The evaluation results for a generated image are an example of a first evaluation result. FIG. 11 is an example of an evaluation result obtained by evaluating the generated images of FIGS. 10-1 and 10-2 based on the evaluation list of FIG. 8.

[0056] Returning to FIG. 2, the current prompt DB 329 is a database that stores a current prompt. The current prompt is a prompt that is generated by a first machine learning model and includes an instruction for image generation for a second machine learning model. FIG. 12 is a diagram illustrating an example of a current prompt according to the embodiment. The current prompt is an example of a prompt.

[0057] Returning to FIG. 2 , the control unit 31 will be described. The control unit 31 controls the overall processing of the control device 12. The control unit 31 is realized by a processor such as a CPU, GPU, or DSP reading a program stored in a storage device, expanding the program in a main storage device such as a RAM, and executing the program. The control unit 31 may be realized by including an integrated circuit such as an ASIC or FPGA. The control unit 31 has an evaluation list generation unit 310 and an image generation unit 311.

[0058] The rating list generation unit 310 is a processing unit that generates a rating list. The rating list generation unit 310 acquires rating perspectives of an image representing a narrative generated by a first machine learning model based on a first prompt including a narrative related to a message. Specifically, the rating list generation unit 310 acquires a narrative and an initial prompt from the user terminal 11 via the communication unit 30. The rating list generation unit 310 identifies a theme and an elicited emotion of the narrative using the first machine learning model. The rating list generation unit 310 generates a prompt including an instruction to generate a rating list for the narrative based on the identified theme and elicited emotion. The prompt including an instruction to generate a rating list for the narrative is an example of a first prompt. The rating list generation unit 310 transmits the generated prompt to the first machine learning model of the generation AI server 13. The rating list generation unit 310 generates a rating list by acquiring the rating list generated by the first machine learning model from the generation AI server 13. The rating list generation unit 310 identifies the rating importance for each rating perspective in the rating list using the first machine learning model. The processing of the evaluation list generating unit 310 will be described in detail later.

[0059] The image generation unit 311 is a processing unit that controls the generation of an image representing a narrative. The image generation unit 311 acquires a first image generated by inputting a third prompt, which is generated based on an evaluation perspective and a second prompt including an instruction for image generation, into a second machine learning model. Specifically, the image generation unit 311 sends an initial prompt to the second machine learning model of the generation AI server 13 and acquires a generated image generated by the second machine learning model. The initial prompt is an example of a second prompt. The image generation unit 311 evaluates the acquired generated image based on the evaluation perspectives in the evaluation list and acquires the evaluation result. The image generation unit 311 calculates an evaluation score for the generated image calculated from the evaluation score and importance for each evaluation perspective. If there are multiple acquired generated images, the image generation unit 311 calculates an evaluation score for each image and identifies the generated image with the highest evaluation score as the best image. The image generation unit 311 determines whether the evaluation score of the best image is equal to or greater than a threshold. If the evaluation score of the best image is less than a threshold, the image generation unit 311 generates a prompt including an instruction to modify the initial prompt based on the evaluation result. The image generation unit 311 sends the generated prompt to the first machine learning model of the generation AI server 13. The image generation unit 311 modifies the prompt by obtaining, from the generation AI server 13, a modified prompt that is the prompt modified by the first machine learning model. The modified prompt is an example of a current prompt and an example of a third prompt. The image generation unit 311 sends the modified prompt to the second machine learning model of the generation AI server 13 and obtains a generated image generated by the second machine learning model. The generated image generated based on the modified prompt is an example of a first image. The generated image that is the subject of the evaluation result used to modify the prompt, in other words, the generated image generated based on the prompt before modification, is an example of a second image.

[0060] Thereafter, the image generation unit 311 repeats the process of evaluating the generated image, correcting the prompt, and generating an image until the evaluation score for the best image satisfies the condition. In the second and subsequent prompt corrections, the image generation unit 311 generates a prompt including an instruction to correct the current prompt used to generate the generated image to be evaluated based on the evaluation result. The image generation unit 311 corrects the prompt by sending the generated prompt to the first machine learning model of the generation AI server 13 and obtaining the corrected prompt. The corrected prompt is an example of a current prompt and an example of a third prompt. The prompt before correction, in other words, the current prompt used to generate the generated image to be evaluated, is an example of a second prompt. The image generation unit 311 may include an instruction to generate multiple corrected prompts in the prompt including an instruction to correct the prompt. The image generation unit 311 may obtain multiple corrected prompts from the generation AI server 13. The image generation unit 311 may generate an image based on each of the multiple corrected prompts and identify a best image from the generated images.

[0061] If the evaluation score for the best image is equal to or greater than the threshold, image generation unit 311 transmits the generated image, which is the best image, to user terminal 11. Details of the processing of image generation unit 311 will be described later.

[0062] [Function details of the evaluation list generation unit 310] The processing of the evaluation list generation unit 310 will be described in detail with reference to Figures 5-1 and 5-2. The evaluation list generation unit 310 references a prompt template for each processing item, generates a prompt that instructs the first machine learning model, and transmits it to the generation AI server 13. In this embodiment, the generation AI server 13 does not store the prompt obtained from the control device 12 or the response transmitted to the control device 12. Therefore, when the control unit 31 uses the response from the generation AI server 13 prior to the processing as context information, it transmits the prompt transmitted prior to the processing and the response transmitted to the transmitted prompt to the generation AI server 13, and then transmits the prompt related to the processing. Specifically, in processing Nos. 2 to 5-4 of the prompt template DB 322, the evaluation list generation unit 310 transmits the prompt transmitted prior to the processing and the response transmitted to the transmitted prompt to the generation AI server 13, and then transmits the prompt related to the processing to the generation AI server 13.

[0063] The rating list generation unit 310 identifies the theme of the narrative and the elicited emotion in order to generate a rating list. First, the rating list generation unit 310 identifies the theme of the narrative. The rating list generation unit 310 obtains a prompt template for process No. 1 from the prompt template. The rating list generation unit 310 references the narrative DB 320, obtains a narrative, and substitutes it for "{narrative}" in the template. As a result, the rating list generation unit 310 generates a prompt that instructs the first machine learning model to identify the theme of the narrative. The rating list generation unit 310 sends the generated prompt to the first machine learning model of the generation AI server 13. The rating list generation unit 310 obtains the theme identified by the first machine learning model from the generation AI server 13 and identifies the theme of the narrative. The rating list generation unit 310 stores the identified theme in the theme DB 323.

[0064] The evaluation list generation unit 310 identifies the emergent emotion of the narrative. The evaluation list generation unit 310 obtains the prompt sent in process No. 1 and the response obtained in response to the prompt sent in process No. 1 from the storage unit 32, and transmits them to the generation AI server 13. The evaluation list generation unit 310 obtains a template for the prompt of process No. 2 from the prompt template. As a result, the evaluation list generation unit 310 generates a prompt that instructs the first machine learning model to identify the emergent emotion of the narrative. The evaluation list generation unit 310 transmits the generated prompt to the first machine learning model of the generation AI server 13. The evaluation list generation unit 310 obtains the emergent emotion identified by the first machine learning model from the generation AI server 13, and identifies the emergent emotion of the narrative. The evaluation list generation unit 310 stores the identified emergent emotion in the emergent emotion DB 324.

[0065] The evaluation list generation unit 310 generates an evaluation list. The evaluation list generation unit 310 obtains the prompts sent in processes No. 1 to No. 2 and the responses received in response to the prompts sent in processes No. 1 to No. 2 from the storage unit 32, and transmits them to the generation AI server 13. The evaluation list generation unit 310 obtains a template for the prompt for process No. 3 from the prompt template. The evaluation list generation unit 310 then generates a prompt that instructs the first machine learning model to generate an evaluation list. The evaluation list generation unit 310 transmits the generated prompt to the first machine learning model of the generation AI server 13. The evaluation list generation unit 310 obtains the evaluation list generated by the first machine learning model from the generation AI server 13, and generates the evaluation list. The evaluation list generation unit 310 stores the generated evaluation list in the evaluation list DB 325.

[0066] The evaluation list generation unit 310 identifies the evaluation importance for each evaluation perspective in the evaluation list. The evaluation list generation unit 310 obtains the prompts sent in processes No. 1 to No. 3 and the responses received in response to the prompts sent in processes No. 1 to No. 3 from the storage unit 32, and transmits them to the generation AI server 13. The evaluation list generation unit 310 obtains a template for the prompt for process No. 4 from the prompt template. The evaluation list generation unit 310 then generates a prompt that instructs the first machine learning model to identify the evaluation importance. The evaluation list generation unit 310 transmits the generated prompt to the first machine learning model in the generation AI server 13. The evaluation list generation unit 310 obtains the evaluation importance identified by the first machine learning model from the generation AI server 13, and identifies the evaluation importance. The evaluation list generation unit 310 stores the identified evaluation importance in the evaluation importance DB 326. In this way, the evaluation list generation unit 310 acquires the evaluation perspectives and the importance of the evaluation perspectives generated by the first machine learning model based on the first prompt.

[0067] [Function details of the image generation unit 311] 5-3 and 5-4, the details of the processing by the image generation unit 311 will be described. The image generation unit 311 references a prompt template for each processing item, generates prompts that give instructions to the first machine learning model, the second machine learning model, and the third machine learning model, and transmits them to the generation AI server 13.

[0068] First, the image generation unit 311 generates an image based on the initial prompt. The image generation unit 311 obtains the initial prompt from the initial prompt DB 321 and transmits it to the second machine learning model of the generation AI server 13. The image generation unit 311 obtains the generated image generated by the second machine learning model from the generation AI server 13 and generates an image. The image generation unit 311 stores the generated image in the generated image DB 327.

[0069] The image generation unit 311 evaluates the image generated based on the initial prompt. The image generation unit 311 acquires a prompt template for process No. 5 from the prompt template. The image generation unit 311 acquires an evaluation list from the evaluation list DB 325 and substitutes it for "{features}" in the template. As a result, the image generation unit 311 generates a prompt that instructs the third machine learning model to evaluate whether the generated image represents a narrative. The prompt that instructs the evaluation of whether the generated image represents a narrative is an example of a fourth prompt. The image generation unit 311 transmits the generated prompt and the generated image to be evaluated to the third machine learning model of the generation AI server 13, acquires the evaluation result generated by the third machine learning model from the generation AI server 13, and evaluates the image. The image generation unit 311 stores the evaluation result in the evaluation result DB 328. In this way, the image generation unit 311 acquires the first evaluation result generated by the third machine learning model based on the fourth prompt including the evaluation perspective and the second image.

[0070] The image generation unit 311 generates an evaluation score s for each evaluation point included in the evaluation result. ij Based on this, the evaluation score s for the generated image is calculated using Eq. (1). i Calculate.

[0071] TIFF2025167938000002.tif16168

[0072] i is a non-negative integer used to identify the generated image. For example, if the second machine learning model generates M images in response to a single prompt input, i = 0, 1, ... M-1. j indicates the importance of each evaluation point. j is a non-negative integer that identifies the evaluation point. For example, if there are N evaluation points, j = 0, 1, ... N-1. ij indicates the evaluation score for the evaluation aspect indicated by j of the image indicated by i. In this way, the image generation unit 311 evaluates the image by weighting each evaluation aspect with the evaluation importance.

[0073] The image generation unit 311 generates an evaluation score s i The image with the largest evaluation score is identified as the best image. If the evaluation score of the best image is equal to or greater than the threshold value u, the image generation unit 311 transmits the best image to the user terminal 11. The threshold value u ranges from 0 to 1, and is 0.8 as an example.

[0074] If the evaluation score of the best image is less than the threshold u, the image generation unit 311 modifies the prompt used to generate the image based on the evaluation result. The image generation unit 311 acquires a template for the prompt of process No. 6 from the prompt template. The image generation unit 311 acquires the prompt used to generate the image to be evaluated from the initial prompt DB 321 and substitutes it for "{current_prompt}" in the template. The image generation unit 311 acquires the evaluation result from the evaluation result DB 328 and substitutes text generated based on the evaluation result for "{feedback}" in the template. An example of the text substituted for "{feedback}" will be described later. As a result, the image generation unit 311 generates a prompt that instructs the first machine learning model to modify the prompt used to generate the image. The image generation unit 311 transmits the generated prompt to the generation AI server 13. The image generation unit 311 acquires text representing the modified prompt generated by the first machine learning model from the generation AI server 13 and modifies the prompt. The image generation unit 311 stores the modified prompt in the current prompt DB 329. In this way, the third prompt is a prompt generated by the first machine learning model based on the first evaluation result and the second prompt.

[0075] In this embodiment, the image generator 311 identifies the best image based on the evaluation score and modifies the prompt based on the evaluation result corresponding to the best image. Therefore, the image generator 311 substitutes the evaluation result for the best image into "{feedback}" in the template for process No. 6. For example, if the evaluation result is as shown in FIG. 11, "{feedback}" is "F0: Person standing at a crossroads, looking towards "Ask" signpost. E0: Person is standing and looking towards "Ask" signpost. Score: 1.0. F1: "Ask" path leading to a bright, sunny horizon. E1. "Ask" path leads to a sunny horizon. Score: 1.0. F2: "Silence" path leading to a stormy dead end. E2. "Silence" path leads to a stormy sky, not a clear dead end. Score: 0.8." A sentence beginning with "E" indicates the evaluation result, and the number following "E" indicates the identification number of the evaluation result for the evaluation perspective. As an example, the image generation unit 311 may determine that the evaluation perspective with the lowest evaluation score among multiple evaluation perspectives is the evaluation perspective that is expected to have the greatest effect through modification, and may substitute only the evaluation perspective with the lowest evaluation score for "{feedback}".

[0076] The image generation unit 311 generates an image based on the corrected prompt. The image generation unit 311 obtains the corrected prompt from the current prompt DB 329 and transmits it to the second machine learning model of the generation AI server 13. The image generation unit 311 obtains the generated image generated by the second machine learning model from the generation AI server 13 and generates an image. The image generation unit 311 stores the obtained generated image in the generated image DB 327.

[0077] The image generation unit 311 evaluates the generated image generated using the current prompt, which is the corrected prompt, in the same way as when evaluating the image generated based on the initial prompt. The image generation unit 311 identifies the best image from the evaluation result. If the evaluation score of the best image is less than the threshold u, the image generation unit 311 corrects the current prompt based on the evaluation result. Specifically, the image generation unit 311 obtains the current prompt used to generate the image to be evaluated from the current prompt DB 329 and substitutes it into "{current_prompt}" of the template for process No. 6. The image generation unit 311 obtains the evaluation result from the evaluation result DB 328 and substitutes text generated based on the evaluation result into "{feedback}" of the template. As a result, the image generation unit 311 generates a prompt that instructs the first machine learning model to correct the prompt used to generate the image. Thereafter, the image generation unit 311 repeats the prompt correction, image generation, and evaluation until the evaluation score of the best image becomes equal to or greater than the threshold u. In this way, the image generating unit 311 acquires the first image when the evaluation score calculated based on the importance and the first evaluation result is less than the threshold value.

[0078] In this way, the control device 12 modifies the prompts containing instructions for image generation based on the evaluation criteria for the images representing the narrative, thereby improving the accuracy of the image generation AI's generation of images representing the narrative. As an example, to generate an image representing the narrative in Figure 3, the prompt in Figure 4 is input into the second machine learning model, resulting in the generation of the images in Figures 10-1 and 10-2. In this case, Figure 10-2 remains unchanged regardless of whether the user proceeds to "ASK" or "SILENCE," and therefore cannot be said to represent the content of the narrative. However, Figure 10-1 differs in state depending on whether the user proceeds to "ASK" or "SILENCE," and therefore can be said to represent the content of the narrative. While prompts that provide instructions to the image generation AI are created by the user, the user's know-how and experience are important for specifying the intended image. Depending on the prompt, an image representing the narrative may not be generated. Furthermore, because narratives indirectly express the intended subject matter, it is difficult to express them in prompts.

[0079] Prior art has proposed a technique for defining image evaluation criteria, evaluating images generated by an image generation AI based on the evaluation criteria, and iteratively improving image generation using highly rated images. However, because the evaluation criteria are defined by the user, similar to prompts, the accuracy of image generation may deteriorate depending on the definition of the evaluation criteria, as in the case of generating prompts. In this embodiment, the control device 12 uses a generation AI to reduce tasks that depend on user ability, such as defining evaluation criteria and creating prompts. The control device 12 defines evaluation criteria using a first machine learning model, modifies the prompt, and generates images. This allows the control device 12 to improve the accuracy of image generation that represents a narrative.

[0080] [Processing flow of information processing system] 13 is a sequence diagram showing an example of processing of the information processing system according to the embodiment. The flow of processing between the components of the user terminal 11, the control device 12, and the generation AI server 13 included in the information processing system 10 will be described with reference to FIG.

[0081] First, the user terminal 11 accepts input of a narrative and an initial prompt from the user and transmits them to the control device 12 (step S10).

[0082] The control device 12 generates a prompt that instructs the user to identify the subject of the narrative and transmits it to the generation AI server 13 (step S11). The generation AI server 13 inputs the prompt received from the control device 12 into a first machine learning model and transmits the output subject to the control device 12 (step S12).

[0083] The control device 12 generates a prompt that instructs identification of the arising emotion of the narrative and transmits it to the generation AI server 13 (step S13). The generation AI server 13 inputs the prompt acquired from the control device 12 into the first machine learning model and transmits the output arising emotion to the control device 12 (step S14).

[0084] The control device 12 generates a prompt instructing the generation of an evaluation list and transmits it to the generation AI server 13 (step S15). The generation AI server 13 inputs the prompt acquired from the control device 12 into the first machine learning model and transmits the output evaluation list to the control device 12 (step S16).

[0085] The control device 12 generates a prompt instructing the user to specify the evaluation importance and transmits it to the generation AI server 13 (step S17). The generation AI server 13 inputs the prompt acquired from the control device 12 into the first machine learning model and transmits the output evaluation importance to the control device 12 (step S18).

[0086] The control device 12 transmits an initial prompt including an instruction for image generation to the generation AI server 13 (step S19). The generation AI server 13 inputs the prompt acquired from the control device 12 into the second machine learning model and transmits the output generated image to the control device 12 (step S20).

[0087] The control device 12 transmits a prompt instructing evaluation of the generated image and the generated image to be evaluated to the generation AI server 13 (step S21). The generation AI server 13 inputs the prompt and the generated image acquired from the control device 12 into a third machine learning model, and transmits the output evaluation result to the control device 12 (step S22).

[0088] The control device 12 calculates an evaluation score for each generated image based on the evaluation result and the importance of each evaluation viewpoint, and identifies the best image (step S23).

[0089] If the evaluation score of the best image is less than threshold u, control device 12 generates a prompt that instructs modification of the prompt used to generate the generated image to be evaluated, and transmits the generated prompt to generation AI server 13 (step S24). Generation AI server 13 inputs the acquired prompt into the first machine learning model, and transmits the output modified prompt to control device 12 (step S25).

[0090] Thereafter, the control device 12 repeats the processes of steps S19 to S25 until the evaluation score of the best image becomes equal to or greater than the threshold value u. In step S19 of the second or subsequent loops, the control device 12 transmits the corrected prompt acquired in step S24 to the generation AI server 13, and generates an image.

[0091] If the evaluation score of the best image is equal to or greater than the threshold value u, the control device 12 transmits the generated image, which is the best image, to the user terminal 11 (step S26), and ends the process.

[0092] [Control device processing flow] 14 is a flowchart illustrating an example of the processing flow of the control device 12 according to the embodiment. The processing flow of the control device 12 according to the embodiment will be described with reference to FIG.

[0093] The evaluation list generating unit 310 acquires a narrative and an initial prompt from the user terminal 11 (step S100).

[0094] The evaluation list generation unit 310 identifies the theme and the elicited emotion of the narrative (step S101). The evaluation list generation unit 310 generates a prompt instructing the user to identify the theme of the narrative and sends it to the generation AI server 13. The evaluation list generation unit 310 identifies the theme of the narrative by obtaining text indicating the theme of the narrative from the generation AI server 13. The evaluation list generation unit 310 generates a prompt instructing the user to identify the elicited emotion of the narrative and sends it to the generation AI server 13. The evaluation list generation unit 310 identifies the elicited emotion of the narrative by obtaining text indicating the elicited emotion of the narrative from the generation AI server 13.

[0095] The evaluation list generation unit 310 generates an evaluation list based on the theme and elicited emotions of the narrative, and identifies evaluation importance for each evaluation perspective in the evaluation list (step S102). The evaluation list generation unit 310 generates a prompt that instructs the generation of an evaluation list, and transmits it to the generation AI server 13. The evaluation list generation unit 310 generates an evaluation list by acquiring text indicating the evaluation list from the generation AI server 13. The evaluation list generation unit 310 generates a prompt that instructs the generation of evaluation importance for each evaluation perspective in the evaluation list, and transmits it to the generation AI server 13. The evaluation list generation unit 310 identifies evaluation importance by acquiring text indicating the evaluation importance from the generation AI server 13.

[0096] The image generation unit 311 sends an initial prompt to the generation AI server 13 and generates an image (step S103). The image generation unit 311 sends the initial prompt to the generation AI server 13. The image generation unit 311 generates an image by obtaining a generated image from the generation AI server 13.

[0097] The image generation unit 311 evaluates the generated image (step S104). The image generation unit 311 generates a prompt instructing the evaluation of the generated image and transmits it to the generation AI server 13. The image generation unit 311 evaluates the generated image by obtaining text indicating the evaluation result from the generation AI server 13.

[0098] The image generation unit 311 identifies the best image from among the generated images (step S105). The image generation unit 311 calculates an evaluation score s for each generated image using equation (1). i Calculate the evaluation score s i The image with the largest value is identified as the best image.

[0099] The image generating unit 311 determines whether the number of times the process is repeated is equal to or greater than a predetermined number (step S106). If the number of times the process is repeated is not equal to or greater than the predetermined number (step S106 NO), the image generating unit 311 proceeds to step S108.

[0100] The image generation unit 311 generates an evaluation score si The image generating unit 311 determines whether the evaluation score s is equal to or greater than the threshold value u (step S108). i If is not equal to or greater than the threshold value u (NO in step S108), the process proceeds to step S109.

[0101] The image generation unit 311 modifies the prompt used to generate the image to be evaluated (step S109). The image generation unit 311 generates a prompt that instructs modification of the prompt based on the evaluation result, and transmits the generated prompt to the generation AI server 13. The image generation unit 311 acquires text indicating the modified prompt from the generation AI server 13 and modifies the prompt. The image generation unit 311 returns to step S103 and repeats the processes of steps S103 to S109 until the evaluation score of the best image becomes equal to or greater than the threshold value u. Note that in step S103 of the second or subsequent loops, the image generation unit 311 transmits the modified prompt acquired in step S109 to the generation AI server 13 to generate an image. When storing a generated image in the generated image DB 327, the image generation unit 311 may erase an existing generated image and store a newly generated generated image. When storing the evaluation result in the evaluation result DB 328, the image generation unit 311 may erase an existing evaluation result and store a newly generated evaluation result. When storing the current prompt in the current prompt DB 329, the image generating unit 311 may delete the existing current prompt and store the newly generated current prompt.

[0102] If the number of times the process is repeated is equal to or greater than a predetermined number (YES in step S106), or if the evaluation score s i If is equal to or greater than the threshold value u (YES in step S108), the generated image, which is the best image, is transmitted to the user terminal 11 (step S107), and the process ends.

[0103] [Variations] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different modes other than the above-described embodiments.

[0104] The information processing system 10 may have a configuration other than that of this embodiment. As an example, the first machine learning model, the second machine learning model, and the third machine learning model may be stored in different servers. As an example, the first machine learning model, the second machine learning model, and the third machine learning model may be stored in the storage unit 32 of the control device 12. In this case, the control unit 31 of the control device 12 inputs prompts to the first machine learning model, the second machine learning model, and the third machine learning model stored in the storage unit 32, thereby acquiring the output of each process.

[0105] The initial prompt may be a simple image generation instruction such as "Realistic digital illustration." This allows the control device 12 to generate an image that represents a narrative without relying on the user's experience or know-how, thereby improving the accuracy of image generation.

[0106] The initial prompt may be stored in advance as a fixed phrase in the initial prompt DB 321, rather than being acquired from the user terminal 11. This allows the user to acquire an image representing the narrative simply by inputting the narrative.

[0107] The evaluation list generation unit 310 may modify the generated evaluation list. Because the subject and elicited emotion of a narrative are abstract, an evaluation list generated from the subject and elicited emotion may be inappropriate from the perspective of image expression. For example, if the evaluation perspectives include "anxious facial expression" and "a back expressing fear," the facial expression and back of the same person cannot be expressed in a single image, making the expression inappropriate. In this way, inconsistencies may arise between the evaluation perspectives, resulting in image breakdown. Furthermore, the image expression may be less specific and ambiguous. Therefore, the evaluation list generation unit 310 evaluates and modifies the generated evaluation list from the perspective of image expression. Furthermore, the evaluation list generation unit 310 also evaluates from the perspective of the subject and elicited emotion so as not to dilute the elements of the subject and elicited emotion.

[0108] Here, with reference to Figures 5-5 to 5-7, an example of a process for modifying the generated evaluation list will be described. First, the evaluation list generation unit 310 evaluates the evaluation list from an image representation perspective. The evaluation list generation unit 310 acquires a template for process No. 7 from the prompt templates. The evaluation list generation unit 310 references the evaluation list DB 325, acquires the evaluation list, and substitutes it for "{feature_list}" in the template. As a result, the evaluation list generation unit 310 generates a prompt that instructs the first machine learning model to evaluate the evaluation list from an image representation perspective. The prompt that instructs the evaluation list to be evaluated from an image representation perspective is an example of a fifth prompt. The evaluation list generation unit 310 sends the generated prompt to the first machine learning model of the generation AI server 13. The evaluation list generation unit 310 acquires the evaluation result from an image representation perspective, which is text generated by the first machine learning model, from the generation AI server 13, and evaluates the evaluation list from an image representation perspective. The evaluation result of the evaluation list from an image representation perspective is an example of a second evaluation result. As an example, when "0. Anxious facial expression. 1. Person's back expressing his / her fear. 2. Raining scene." is input as the evaluation list, the evaluation result is "Problem with F1: A person's back doesn't easily convey facial expressions like anxiety. Solution: Depict the person's posture with hunched shoulders and tense muscles to suggest fear, complementing F0." In this way, the evaluation list generation unit 310 acquires the second evaluation result in terms of image expression of the evaluation perspective generated by the first machine learning model based on the fifth prompt including the evaluation perspective.

[0109] The rating list generation unit 310 evaluates the rating list from the perspective of the theme and the elicited emotion. The rating list generation unit 310 acquires a template for process No. 8 from the prompt template. The rating list generation unit 310 references the theme DB 323, acquires the theme, and substitutes it for "{theme}" in the template. The rating list generation unit 310 references the elicited emotion DB 324, acquires the elicited emotion, and substitutes it for "{emotion}" in the template. The rating list generation unit 310 references the rating list DB 325, acquires the rating list, and substitutes it for "{feature_list}" in the template. As a result, the rating list generation unit 310 generates a prompt that instructs the first machine learning model to evaluate the rating list from the perspective of the theme and the elicited emotion. The rating list generation unit 310 sends the generated prompt to the first machine learning model of the generation AI server 13. The prompt that instructs the evaluation list to be evaluated from the perspective of the theme and the elicited emotion is an example of a sixth prompt. The evaluation list generation unit 310 acquires evaluation results from the perspective of the theme and the elicited emotion, which are text generated by the first machine learning model, from the generation AI server 13, and evaluates the evaluation list from the perspective of the theme and the elicited emotion. The evaluation results of the evaluation list from the perspective of the theme and the elicited emotion are an example of a third evaluation result. For example, if the theme is "The importance of taking a step forward even in the midst of anxiety," the elicited emotion is "Encouragement and hope," and the evaluation list is "0. Anxious facial expression. 1. Person's back expressing his / her fear. 2. Raining scene.", the evaluation result is, for example, "Problem with F2: Rain often symbolizes sadness. Solution: Transform rain into stepping stones rising upwards, symbolizing overcoming obstacles and moving forward with hope."In this way, the evaluation list generation unit 310 obtains a third evaluation result in terms of the subject matter and elicited emotion of the evaluation perspective generated by the first machine learning model based on the sixth prompt including the evaluation perspective.

[0110] The evaluation list generation unit 310 modifies the evaluation list based on the evaluation results from the image representation perspective and the main theme and elicitation emotion perspective. The evaluation list generation unit 310 acquires a template for process No. 9 from the prompt template. The evaluation list generation unit 310 references the evaluation list DB 325, acquires the evaluation list, and assigns it to "{feature_list}" of the template. The evaluation list generation unit 310 assigns the evaluation results from the image representation perspective to "{feedback_1}" of the template. The evaluation list generation unit 310 assigns the evaluation results from the main theme and elicitation emotion perspective to "{feedback_2}" of the template. As a result, the evaluation list generation unit 310 generates a prompt that instructs the user to modify the evaluation list based on the evaluation results from the image representation perspective and the main theme and elicitation emotion perspective. The prompt that instructs the user to modify the evaluation list is an example of the seventh prompt. The evaluation list generation unit 310 sends the generated prompt to the first machine learning model of the generation AI server 13. The evaluation list generation unit 310 obtains the text generated by the first machine learning model from the generation AI server 13 and modifies the evaluation list. As an example, the modified evaluation list is "0. Anxious facial expression with wide eyes and furrowed brow. 1. Hunched shoulders and tense muscles to indicate fear. 2. Stepping stones rising in the air, symbolizing hope and progress." In this way, the evaluation list generation unit 310 generates a seventh prompt including an instruction to modify the evaluation perspective based on the second evaluation result and the third evaluation result, and obtains the evaluation perspective modified by the first machine learning model based on the seventh prompt. The prompt generated based on the modified evaluation perspective and the second prompt is an example of a third prompt.

[0111] The image generation unit 311 may modify the initial prompt based on the evaluation list and evaluation importance before transmitting the initial prompt to the generation AI server 13 and generating an image. Here, an example of the process of modifying the initial prompt will be described with reference to Figures 5-8. The image generation unit 311 references the prompt template DB 322 to obtain a template for the prompt of process No. 10. The image generation unit 311 obtains the initial prompt from the initial prompt DB 321 and assigns it to "{initial_prompt}" of the template. The image generation unit 311 obtains an evaluation list from the evaluation list DB 325 and assigns it to "{feature_list}" of the template. As a result, the image generation unit 311 generates a prompt that instructs modification of the initial prompt based on the evaluation list and evaluation importance. The image generation unit 311 transmits the generated prompt to the first machine learning model of the generation AI server 13. The image generation unit 311 obtains the corrected initial prompt, which is text generated by the first machine learning model, from the generation AI server 13 and corrects the initial prompt. The image generation unit 311 stores the corrected initial prompt in the initial prompt DB 321. By using the corrected initial prompt, the image generation unit 311 can improve the processing speed and generation accuracy of image generation representing a narrative. The corrected initial prompt is an example of a third prompt. An image generated based on the corrected initial prompt is an example of a first image.

[0112] In this embodiment, the rating list generation unit 310 generates a rating list based on a narrative. However, the rating list generation unit 310 may also generate a rating list based on a narrative and an initial prompt. Here, with reference to FIGS. 5-9, the process of generating a rating list based on a narrative and an initial prompt will be described in detail. The image generation unit 311 references the prompt template DB 322 to obtain a prompt template for process No. 11. The image generation unit 311 obtains an initial prompt from the initial prompt DB 321 and substitutes it for "{initial_prompt}" in the template. This allows the rating list generation unit 310 to generate a prompt that instructs the generation of a rating list based on a narrative and an initial prompt. The rating list generation unit 310 generates a rating list using the prompt generated in process No. 11 instead of the prompt generated in process No. 3. This allows the rating list generation unit 310 to improve the accuracy of generating a rating list and the accuracy of generating images representing narratives.

[0113] In this embodiment, the evaluation list generation unit 310 determines the evaluation importance using the first machine learning model. However, the evaluation importance may also be determined using a text encoder. The text encoder is a processing unit that converts text into features represented by vectors and is a component of the LLM. As an example, the evaluation list generation unit 310 acquires the theme from the theme DB 323, the arising emotion from the arising emotion DB 324, and the evaluation list from the evaluation list DB 325. The evaluation list generation unit 310 inputs the theme, the arising emotion, and the evaluation list into the text encoder and converts them into features. The evaluation list generation unit 310 calculates the dot product of the theme feature and each evaluation aspect in the evaluation list. The evaluation list generation unit 310 calculates the dot product of the arising emotion and each evaluation aspect in the evaluation list. The evaluation list generation unit 310 identifies the dot product with the largest value among the dot products of the theme and the arising emotion calculated for each evaluation aspect, and sets this as the evaluation importance value for each evaluation aspect. As an example, the evaluation list generating unit 310 may linearly convert the value of the inner product to obtain the evaluation importance.

[0114] When generating an image using a corrected prompt, which is a prompt corrected based on the evaluation result of the generated image, the image generation unit 311 may send the best image along with the corrected prompt to the second machine learning model of the generation AI server 13 to generate the image. In this case, the second machine learning model may be an image-to-image image generation AI that accepts text and images as input. If the received prompt and best image are intended for the second machine learning model, the control unit 41 of the generation AI server 13 inputs the prompt and the best image to the second machine learning model. The control unit 41 transmits the image output from the second machine learning model to the control device 12.

[0115] In this embodiment, the second machine learning model generates two images in response to a single prompt input, but the number of images generated may be one or any other number.

[0116] In this embodiment, in processes No. 2 to No. 4, the evaluation list generation unit 310 transmits from the memory unit 32 to the generation AI server 13 the prompts sent before the process and the responses received for the sent prompts, and then transmits a prompt related to the process. The evaluation list generation unit 310 may transmit the prompts sent before the process obtained from the memory unit 32 to the generation AI server 13, and transmit the prompt related to the process after receiving a response from the generation AI server 13. If there are multiple prompts sent before the process, the evaluation list generation unit 310 may send the prompts sent for each process individually and send the next prompt after receiving a response. The evaluation list generation unit 310 may also concatenate the prompts sent for each process and transmit them, and receive the responses for each concatenated process.

[0117] If the input languages ​​of the first, second, and third machine learning models are different from the languages ​​of the narrative and initial prompt acquired from the user terminal, the control unit 31 may generate a prompt by translating the narrative and initial prompt into the input language of each machine learning model. In this embodiment, the narrative is in Japanese, and the input language of each machine learning model is English. The control unit 31 translates the narrative into English and substitutes it for the prompt.

[0118] [Examples of using generated images] As an example, the user terminal 11 generates materials and videos using the generated images. The materials and videos can be used as feedback for crime prevention training. One example is specialized fraud prevention training, which involves participants experiencing simulated specialized frauds to prevent them from being deceived by specialized frauds. In specialized fraud prevention training, participants' psychological state is analyzed based on vital data such as their heart rate to assess their risk of being deceived by specialized frauds. Objective feedback based on numerical data from the analysis results is difficult for participants to understand and remember. Therefore, providing feedback in the form of videos using generated images that express narratives makes the information more memorable and increases participants' vigilance against crime. Other applications include effective feedback for accident prevention and evacuation drills. Using narratives in internal company messages can also improve engagement. Other potential applications include behavioral improvement applications for promoting healthy behavior. Narratives are thus useful for providing feedback on results and increasing motivation toward goals. Using images and videos in conjunction with a narrative makes it easier to convey the message and helps the image stick in the listener's memory.

[0119] [effect] As described above, the control device 12 acquires an evaluation perspective of an image representing a narrative generated by a first machine learning model based on a first prompt including a narrative related to a message, and acquires a first image generated by inputting a third prompt generated based on the evaluation perspective and a second prompt including instructions for image generation into a second machine learning model. In this way, the control device 12 acquires the evaluation perspective generated by the first machine learning model, thereby reducing work that depends on the user's ability and know-how. The control device 12 can instruct image generation using the third prompt generated based on the evaluation perspective, thereby reducing work that depends on the user's ability and know-how. This allows the control device 12 to improve the accuracy of generating an image representing a narrative.

[0120] The control device 12 may acquire a first evaluation result generated by a third machine learning model based on a fourth prompt including an evaluation perspective and the second image. The third prompt may be a prompt generated by the first machine learning model based on the first evaluation result and the second prompt. In this manner, the control device 12 can modify the prompt that instructs image generation based on the evaluation result for the image. By acquiring the evaluation result generated by the third machine learning model, the control device 12 can reduce work that depends on the user's ability and know-how. This allows the control device 12 to improve the accuracy of generating images that represent narratives.

[0121] The control device 12 may acquire the evaluation aspects and the importance of the evaluation aspects generated by the first machine learning model based on the first prompt. The control device 12 may acquire the first image when the evaluation score calculated based on the importance of the evaluation aspects and the first evaluation result is less than a threshold. In this way, the control device 12 can improve the accuracy of generating an image representing a narrative by weighting the evaluation aspects by the importance.

[0122] The control device 12 may acquire a second evaluation result in terms of image representation of the evaluation perspective generated by the first machine learning model based on a fifth prompt including the evaluation perspective, acquire a third evaluation result in terms of the theme and emotional emotion of the evaluation perspective generated by the first machine learning model based on a sixth prompt including the evaluation perspective, generate a seventh prompt including an instruction to modify the evaluation perspective based on the second evaluation result and the third evaluation result, and acquire the evaluation perspective modified by the first machine learning model based on the seventh prompt. The third prompt may be a prompt generated based on the modified evaluation perspective and the second prompt. The control device 12 can modify the evaluation perspective in terms of image representation, theme, and emotional emotion. By acquiring the evaluation perspective modified by the first machine learning model, the control device 12 can reduce work that depends on the user's ability and know-how. This allows the control device 12 to improve the accuracy of generating the evaluation list. The control device 12 can improve the accuracy of generating images representing a narrative by using the modified evaluation list.

[0123] The user terminal 11 receives a narrative related to a message, and acquires an image generated by inputting into the second machine learning model an evaluation perspective of an image representing the narrative generated by the first machine learning model based on the first prompt including the narrative and a third prompt generated based on the second prompt including an instruction for image generation. This allows the user terminal 11 to reduce work that depends on the ability and know-how of the user. The user terminal 11 can acquire an image with improved accuracy in generating an image representing the narrative. The user can efficiently acquire an image representing the narrative. The user can effectively convey a message using an image representing the narrative.

[0124] [Hardware configuration of user terminal 11] Fig. 15 is a diagram illustrating an example of the hardware configuration of a user terminal according to an embodiment. An example of the hardware configuration of a user terminal 11 will be described with reference to Fig. 15. As shown in Fig. 15, the user terminal 11 includes, as an example, a CPU 150, a RAM 151, an input / output interface 152, a communication interface 153, and an HDD 154 as components, and these components are connected via a bus 155. The user terminal 11 also includes an input device 156 and an output device 157, which are connected via the input / output interface 152.

[0125] The CPU 150 is a processor that runs a process that executes the functions of the control unit 23. Specifically, the CPU 150 reads a program that executes the same functions as the control unit 23 from the HDD 154 or the like, loads it into the RAM 151 or the like, and executes the process that executes the functions of the control unit 23. The CPU 150 may obtain the program and data used to execute the program from a medium reading device, or may obtain the data via the communication interface 153. The CPU 150 may have one or more processor cores. The user terminal 11 may be provided with a processor other than a CPU, or may be provided with multiple types of processors.

[0126] The RAM 151 operates as the main storage device of the user terminal 11, and stores programs read from auxiliary storage devices such as the HDD 154, data used to execute the programs, etc. The user terminal 11 may be provided with a memory other than the RAM, or may be provided with multiple memories.

[0127] The input / output interface 152 is an interface for connecting an input device 156 and an output device 157. The input / output interface 152 receives signals from the input device 156 and transmits signals to the output device 157. A plurality of input devices and output devices may be connected to the input / output interface 157.

[0128] The communication interface 153 is an interface for connecting the user terminal 11 to a network. As an example, the standard of the communication interface 153 may be a wired LAN communication standard such as Ethernet (registered trademark), a wireless LAN communication standard such as Wi-Fi (registered trademark), or a wireless mobile communication standard such as local 5G.

[0129] The HDD 154 operates as an auxiliary storage device for storing programs for executing the functions of the operating system (OS) of the user terminal 11 and the control unit 23, as well as data used by the programs. The user terminal 11 may be provided with an auxiliary storage device other than an HDD, such as a solid state drive (SSD), or may be provided with multiple auxiliary storage devices.

[0130] The input device 156 is a device for inputting a narrative, an initial prompt, etc. into the user terminal 11. As an example, the input device 156 is a touch panel display, a mouse, or the like.

[0131] The output device 157 is a device for outputting generated images and the like from the user terminal 11. As an example, the output device 157 is a display, a printer, or the like.

[0132] [Hardware configuration of Generative AI Server 13] Fig. 16 is a diagram illustrating an example of the hardware configuration of a generation AI server according to an embodiment. An example of the hardware configuration of the generation AI server 13 will be described with reference to Fig. 16. As shown in Fig. 16, the generation AI server 13 includes, as an example, a CPU 160, a RAM 161, an input / output interface 162, a communication interface 163, and an HDD 164 as components, and these components are connected via a bus 165.

[0133] The CPU 160 is a processor that runs a process that executes the functions of the control unit 41. Specifically, the CPU 160 reads a program that executes the same functions as the control unit 41 from the HDD 164 or the like, loads it into the RAM 161 or the like, and executes the process that executes the functions of the control unit 41. The CPU 160 may obtain the program and data used to execute the program from a medium reading device, or may obtain the data via the communication interface 163. The CPU 160 may have one or more processor cores. The generation AI server 13 may be equipped with a processor other than a CPU, or may be equipped with multiple types of processors.

[0134] RAM 161 operates as the main memory of the generation AI server 13, and stores programs read from auxiliary storage devices such as HDD 164 and data used to execute the programs. The generation AI server 13 may be equipped with a memory other than RAM, or may be equipped with multiple memories.

[0135] The input / output interface 162 is an interface that inputs signals to the generation AI server 13 and outputs signals from the generation AI server 13. As an example, the input / output interface 162 receives signals from input devices such as a keyboard or mouse connected to the generation AI server 13, and transmits signals such as images to output devices such as a display connected to the generation AI server 13. Multiple input devices and output devices may be connected to the generation AI server 13 via the input / output interface 162.

[0136] The communication interface 163 is an interface for connecting the generation AI server 13 to a network. As an example, the standard of the communication interface 163 may be a wired LAN communication standard such as Ethernet, a wireless LAN communication standard such as Wi-Fi, a wireless mobile communication standard such as local 5G, or a telephone line standard.

[0137] The HDD 164 operates as an auxiliary storage device for storing programs for executing the functions of the OS and control unit 41 of the generation AI server 13, as well as data used by the programs. The generation AI server 13 may be equipped with an auxiliary storage device other than an HDD, such as an SSD, or may be equipped with multiple auxiliary storage devices.

[0138] [Hardware configuration of the control device 12] Fig. 17 is a diagram illustrating an example of the hardware configuration of a control device according to an embodiment. An example of the hardware configuration of the control device 12 will be described with reference to Fig. 17. As shown in Fig. 17, the control device 12 includes, as an example, a CPU 170, a RAM 171, an input / output interface 172, a communication interface 173, and an HDD 174 as components, and these components are connected via a bus 175.

[0139] CPU 170 is a processor that runs a process that executes the functions of control unit 31. Specifically, CPU 170 reads a program that executes the same functions as control unit 31 from HDD 174 or the like, loads it into RAM 171 or the like, and executes the process that executes the functions of control unit 31. CPU 170 may obtain the program and data used to execute the program from a medium reading device, or may obtain the data via communication interface 173. CPU 170 may have one or more processor cores. Control device 12 may be provided with a processor other than a CPU, or may be provided with multiple types of processors.

[0140] The RAM 171 operates as the main storage device of the control device 12, and stores programs read from an auxiliary storage device such as the HDD 174, data used to execute the programs, etc. The control device 12 may include a memory other than the RAM, or may include multiple memories.

[0141] The input / output interface 172 is an interface that inputs signals to the control device 12 and outputs signals from the control device 12. As an example, the input / output interface 172 receives signals from input devices such as a keyboard or a mouse connected to the control device 12, and transmits signals such as images to output devices such as a display connected to the control device 12. A plurality of input devices and output devices may be connected to the control device 12 via the input / output interface 172.

[0142] The communication interface 173 is an interface for connecting the control device 12 to a network. For example, the standard of the communication interface 173 may be a wired LAN communication standard such as Ethernet, a wireless LAN communication standard such as Wi-Fi, a wireless mobile communication standard such as local 5G, or a telephone line standard.

[0143] The HDD 174 operates as an auxiliary storage device for storing programs for executing the functions of the OS and the control unit 31 of the control device 12, and data used by the programs. The control device 12 may be provided with an auxiliary storage device other than an HDD, such as an SSD, or may be provided with multiple auxiliary storage devices.

[0144] Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to those described above. The present invention may be embodied in various different forms other than the above-described embodiments.

[0145] The processing procedures, processing methods, names of parts, various data, etc. shown in the embodiments and drawings are merely examples and may be changed arbitrarily unless otherwise specified.

[0146] The functional configurations and hardware configurations shown in the embodiments and drawings are merely examples, and therefore do not necessarily have to be configured or arranged as shown, and may be changed as desired unless otherwise specified. For example, the components of the functional configurations and hardware configurations may be distributed or integrated in any unit, such as a function.

[0147] The information processing program according to the embodiment is not limited to being executed by the control device 12 and the user terminal 11. For example, the information processing program according to the embodiment may be executed across a plurality of devices.

[0148] Furthermore, the information processing program according to the embodiments may be distributed via a network such as the Internet. The information processing program according to the embodiments may be recorded on a computer-readable recording medium and sold. Examples of the computer-readable recording medium include a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, a floppy disk, and an MO (Magneto-Optical Disk). The information processing program according to the embodiments may be read from the computer-readable recording medium by a computer and executed by the computer. [Explanation of symbols]

[0149] 10 Information Processing Systems 11 User terminal 12 Control device 13 Generative AI Server 20 Communications Department 21 Input section 22 Output section 23 Control Unit 24 Memory section 240 Generated Image DB 30 Communications Department 31 Control Unit 32 Storage section 310 Evaluation list generation unit 311 Image Generation Unit 320 Narrative DB 321 Initial Prompt DB 322 Prompt Template DB 323 Subject DB 324 Emotion DB 325 Rating List DB 326 Evaluation Importance DB 327 Generated Image DB 328 Evaluation Results DB 329 Current Prompt DB 40 Communications Department 41 Control Unit 42 Storage section 420 First Machine Learning Model DB 421 Second Machine Learning Model DB 422 The Third Machine Learning Model DB

Claims

1. obtaining an evaluation point of view of an image representing a narrative generated by a first machine learning model based on a first prompt including the narrative related to the message; A third prompt, which is generated based on the evaluation viewpoint and a second prompt including an instruction for generating an image, is input to a second machine learning model to obtain a first image. An information processing program that causes a computer to execute a process.

2. obtaining a first evaluation result generated by a third machine learning model based on a fourth prompt including the evaluation viewpoint and a second image; The third prompt is a prompt generated by the first machine learning model based on the first evaluation result and the second prompt. The information processing program according to claim 1 .

3. The process of acquiring the evaluation perspective is a process of acquiring the evaluation perspective and an importance of the evaluation perspective generated by the first machine learning model based on the first prompt, The process of acquiring the first image is a process of acquiring the first image when an evaluation score calculated based on the importance and the first evaluation result is less than a threshold. The information processing program according to claim 2 .

4. obtaining a second evaluation result in terms of the image representation of the evaluation perspective generated by the first machine learning model based on a fifth prompt including the evaluation perspective; obtaining a third evaluation result in terms of the subject matter and the elicited emotion of the evaluation perspective, generated by the first machine learning model based on a sixth prompt including the evaluation perspective; generating a seventh prompt including an instruction to modify the evaluation viewpoint based on the second evaluation result and the third evaluation result; obtaining the evaluation viewpoint modified by the first machine learning model based on the seventh prompt; The third prompt is a prompt generated based on the modified evaluation perspective and the second prompt. The information processing program according to claim 1 .

5. Accept the narrative of the message, A first image is generated by inputting a third prompt, which is generated based on an evaluation viewpoint of an image representing the narrative generated by a first machine learning model based on a first prompt including the narrative and a second prompt including an instruction for image generation, into a second machine learning model. An information processing program that causes a computer to execute a process.

6. obtaining an evaluation point of view of an image representing a narrative generated by a first machine learning model based on a first prompt including the narrative related to the message; A third prompt, which is generated based on the evaluation viewpoint and a second prompt including an instruction for generating an image, is input to a second machine learning model to obtain a first image. An information processing method in which processing is performed by a computer.

7. Accept the narrative of the message, A first image is generated by inputting a third prompt, which is generated based on an evaluation viewpoint of an image representing the narrative generated by a first machine learning model based on a first prompt including the narrative and a second prompt including an instruction for image generation, into a second machine learning model. An information processing method in which processing is performed by a computer.

8. obtaining an evaluation point of view of an image representing a narrative generated by a first machine learning model based on a first prompt including the narrative related to the message; A third prompt, which is generated based on the evaluation viewpoint and a second prompt including an instruction for generating an image, is input to a second machine learning model to obtain a first image. An information processing device having a control unit.

9. Accept the narrative of the message, A first image is generated by inputting a third prompt, which is generated based on an evaluation viewpoint of an image representing the narrative generated by a first machine learning model based on a first prompt including the narrative and a second prompt including an instruction for image generation, into a second machine learning model. An information processing device having a control unit.

Images