System

Abstract

To provide a system.SOLUTION: A system includes: first means for receiving input of a scenario to be text information describing a content and a structure of a material a user wants to create; second means which generates a first prompt statement for outputting a story of the material on the basis of the scenario; third means which inputs the generated first prompt statement to a generative AI model to generate the story; fourth means generating an advertisement material on the basis of the generated story; and fifth means providing a user with the generated story and advertisement material.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, creating materials has required a great deal of time and effort, and creating stories and diagrams in particular requires specialized knowledge and skills. Furthermore, when utilizing existing materials, it can be difficult to find the appropriate materials. [Means for solving the problem]

[0005] In this invention, when a user inputs the scenario of the document they want to create, the generative AI generates a story and generates diagrams using documents stored in-house as external information. This improves the efficiency of document creation. Furthermore, the generative AI analyzes the scenario using natural language processing technology, and the means for generating diagrams uses image recognition technology to select appropriate diagrams from documents stored in-house. This also improves the quality of document creation. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] This embodiment of the present invention provides a text input interface as a means for the user to input the scenario of the material they wish to create. This interface is designed so that the user can freely write the scenario. For example, if the user inputs a scenario for a "new product presentation,"

[0029] "Example 2"

[0030] Generative AI uses natural language processing technology to analyze the input scenario. This analysis extracts keywords and themes from the scenario and generates a story based on them. For example, from a "new product presentation" scenario, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[0031] "Example 3"

[0032] The means of generating diagrams is to use documents stored within the company as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "features of a new product" or "comparison with competitors" are selected. These diagrams are provided as answers along with the generated story, and users can use this as a basis for creating their own documents.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The user inputs a scenario through a text input interface. For example, the user inputs "New product presentation" as the scenario.

[0036] Step 2: Generative AI uses natural language processing technology to analyze the input scenario. Through this analysis, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted.

[0037] Step 3: Stories are generated based on the extracted keywords.

[0038] "Example 2"

[0039] Step 1: The generative AI analyzes the input scenario using natural language processing technology.

[0040] Step 2: A story is generated based on the keywords and themes extracted through the analysis.

[0041] Step 3: The generated story is provided to the user.

[0042] "Example 3"

[0043] Step 1: Use the data accumulated within the company as external information.

[0044] Step 2: Use image recognition technology to select the appropriate figure from the document.

[0045] Step 3: The selected diagrams are generated and provided as answers along with the generated stories.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Conventional document creation systems required users to manually think about the content of documents and create diagrams, which required a great deal of time and effort. Furthermore, the quality and consistency of the documents depended on the skills of the users, making it difficult to create documents of uniform quality. Furthermore, there was a lack of effective ways to utilize documents stored within the company, and information was not reused sufficiently.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes: a means for inputting a scenario for a document the user wants to create; a means for generating a prompt sentence based on the input scenario; a means for sending the generated prompt sentence to a generative AI model to generate the document; a means for returning the generated document to the user; a means for using documents stored in-house as external information to generate diagrams; and a means for providing the generated document and diagram together as a response. This allows users to easily automatically generate high-quality documents. Furthermore, by effectively utilizing information stored in-house, information reuse is promoted, improving the efficiency of document creation.

[0051] "User" means any person or organization that uses the System to create Materials.

[0052] A "scenario" refers to text information that describes the content and structure of the material that a user wants to create.

[0053] "Prompt sentence" refers to an instruction sentence sent to a generative AI model based on a scenario.

[0054] "Generative AI model" refers to an artificial intelligence model that automatically generates materials based on input prompts.

[0055] "Documents" refers to documents, including text and diagrams, generated by a generative AI model.

[0056] "Diagram" refers to visual content generated based on materials accumulated within the company and external information.

[0057] "Server" refers to the computer system that manages the overall system processing, receives input from users, sends prompts to the generative AI model, and returns the generated materials.

[0058] "External information" refers to data obtained from sources other than materials stored within the company.

[0059] "Natural language processing technology" refers to the technology used to analyze scenarios and generate prompts and materials.

[0060] "Image recognition technology" refers to the technology used to select and generate appropriate images from materials stored within the company.

[0061] MODE FOR CARRYING OUT THE INVENTION

[0062] This invention is a system that automatically generates materials based on a scenario input by the user for the materials they want to create. This system is composed of multiple components, including a server, a terminal, and a generative AI model.

[0063] System configuration

[0064] 1. Server

[0065] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt sentences, and sends them to the generative AI model. It also receives the materials returned from the generative AI model and returns them to the user. The server uses a database to temporarily store scenarios and generated materials.

[0066] 2. Terminal

[0067] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and check generated materials.

[0068] 3. Generative AI Models

[0069] A generative AI model is an artificial intelligence model that automatically generates materials based on prompts sent from a server. For example, OpenAI's GPT-4 (registered trademark) is used as a generative AI model.

[0070] Program processing

[0071] The server receives the scenario entered by the user through the device's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model, and is in the form of a specific instruction for the content of the scenario.

[0072] The generative AI model generates materials based on the received prompts. The materials are formatted according to the scenario content and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[0073] Specific examples

[0074] As a concrete example, consider the case where a user inputs "New product presentation" as a scenario. The user inputs "New product presentation" into the text input interface of the terminal. The server receives this scenario and generates the following prompt sentence:

[0075] Based on the scenario "New Product Presentation," create a presentation that includes the following:

[0076] 1. New Product Features

[0077] 2. Market analysis

[0078] 3. Comparison with competing products

[0079] 4. Sales Strategy

[0080] The generative AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are then sent back to the user via the server. The user can review the generated materials and make corrections or additions as necessary.

[0081] In this way, a system that automatically generates documents through collaboration between the server, terminal, and generative AI model is realized. This system allows users to easily create high-quality documents and effectively utilize information accumulated within the company.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] The user inputs a scenario

[0085] The user uses the text input interface of the terminal to input the scenario of the material they want to create. The input scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and inputs "new product presentation" into the text box. Input: Scenario text (e.g., "new product presentation"). Output: Scenario text sent to the server.

[0086] Step 2:

[0087] The server receives the scenario.

[0088] The server receives the scenario sent by the user and saves it in the database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is saved in the database. Input: Scenario text sent by the user. Output: Scenario text saved in the database.

[0089] Step 3:

[0090] The server generates a prompt

[0091] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates the following prompt: Based on the "New Product Presentation" scenario, please create a presentation containing the following content: 1. New product features 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[0092] Step 4:

[0093] The server sends a prompt to the generative AI model

[0094] The server sends the generated prompt sentence to the generative AI model. Specifically, the server generates an API request and sends the request including the prompt sentence to the generative AI model's endpoint. Input: Generated prompt sentence. Output: Prompt sentence sent to the generative AI model.

[0095] Step 5:

[0096] Generative AI model generates materials

[0097] The generative AI model generates materials based on the received prompt. Specifically, the generative AI model analyzes the prompt and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: The prompt sent to the generative AI model. Output: The generated materials.

[0098] Step 6:

[0099] The server receives the generated material.

[0100] The server receives the materials generated from the generative AI model. Specifically, the server receives the API response and extracts the generated materials from the response body. The extracted materials are temporarily saved. Input: Materials sent from the generative AI model. Output: Generated materials temporarily saved.

[0101] Step 7:

[0102] The server returns the material to the user

[0103] The server returns the generated materials to the user. Specifically, the server generates an HTTP response and sends the response including the generated materials to the user's terminal. The user checks the materials in a web browser. Input: Generated materials temporarily saved. Output: Generated materials sent to the user's terminal.

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Conventional material creation systems lack the functionality to automatically generate advertising materials based on a scenario entered by the user, which reduces the efficiency of advertising production. Furthermore, there was a need for the generated story and illustrations to be provided in a format that could be used as advertising materials, rather than simply being presented together as a response.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated in-house as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story. This makes it possible to automatically generate advertising materials based on the scenario input by the user, thereby improving the efficiency of advertising production.

[0109] "User" means an individual or entity that uses the system to input material scenarios and receive generated stories and advertising materials.

[0110] A "scenario" is text information that describes the content and structure of the material that a user wants to create.

[0111] A "generative AI means" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[0112] "In-house stored materials" refers to a collection of data and information held internally by a company or organization.

[0113] "External information" refers to data and information obtained from outside the company, other than materials stored within the company.

[0114] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using materials accumulated within the company and external information.

[0115] A "generated story" is a narrative or exposition created based on a scenario by generative AI means.

[0116] "Advertising materials" are content such as images, videos, and text for advertising created based on the generated story.

[0117] "Answering means" refers to a system or method for providing generated stories, illustrations, and advertising materials to users.

[0118] A system for implementing this invention includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story.

[0119] System Program

[0120] The system is implemented using Python (registered trademark) and uses OpenAI's GPT-3 (registered trademark) as its generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are also automatically generated.

[0121] Processing Description

[0122] The server operates using the following hardware and software:

[0123] Hardware:

[0124] Smartphone or PC

[0125] software:

[0126] Python

[0127] OpenAI API

[0128] Data processing and calculation:

[0129] 1. Scenario input: The user inputs a scenario through a text input interface. For example, the user inputs a scenario such as "Presentation of a new product."

[0130] 2. Story generation: The input scenario is sent as a prompt to a generative AI model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[0131] 3. Diagram generation: Using internally accumulated data as external information, appropriate diagrams are generated using image recognition technology.

[0132] 4. Generate advertising materials: Generate advertising materials (images, videos, text) based on the generated story.

[0133] 5. Providing answers: Providing the generated stories, diagrams, and advertising materials to the user.

[0134] Specific examples

[0135] If a user enters "new product presentation," the following text is returned as an example of advertising material that might be generated:

[0136] Example prompt sentence:

[0137] Advertising scenario: New product presentation

[0138] Generate advertising materials based on this scenario.

[0139] Examples of generated advertising materials:

[0140] New product presentation

[0141] This new product is developed using the latest technology. It is both high-performance and easy to use, making your daily life more convenient. Buy it now and experience the future of life!

[0142] In this way, advertising materials can be automatically generated based on a scenario input by a user, thereby improving the efficiency of advertising production.

[0143] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0144] Step 1:

[0145] The user inputs a scenario. Using the text input interface, the user inputs the scenario of the material they want to create. For example, they input a scenario such as "Presentation of a new product." The input scenario is sent to the server.

[0146] Step 2:

[0147] The server receives the scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generative AI model. For example, it might be in the format "Advertising scenario: New product presentation. Please generate advertising materials based on this scenario."

[0148] Step 3:

[0149] The server sends a prompt to the generative AI model, which generates a story. The server uses OpenAI's GPT-3 API to send the prompt and receive the story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated that reads, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[0150] Step 4:

[0151] The server generates diagrams using documents stored in the company. The server uses image recognition technology to select and generate appropriate diagrams from documents stored in the company. For example, a diagram showing the technical specifications and usage of a new product can be generated.

[0152] Step 5:

[0153] The server generates advertising materials (images, videos, text) based on the generated story. For example, advertising copy and product images are created based on the generated story.

[0154] Step 6:

[0155] The server provides the generated story, illustrations, and advertising materials to the user. The server transmits the generated story, illustrations, and advertising materials to the user, allowing the user to review them. The user can create an advertisement using the provided materials.

[0156] In this way, advertising materials can be automatically generated based on a scenario input by the user, thereby improving the efficiency of advertising production.

[0157] Example 2

[0158] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] In conventional document creation systems, even if users input a scenario, it was difficult to generate an appropriate story based on that scenario. Furthermore, the system lacked the functionality to automatically generate diagrams related to the generated story, requiring users to create the diagrams manually. This resulted in the problem of requiring a great deal of time and effort to create documents.

[0160] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting a scenario of a document that a user wants to create, a means for analyzing the input scenario and extracting keywords and themes, a generative AI means for generating a story based on the extracted keywords and themes, a means for generating diagrams using documents accumulated in-house as external information, and a means for responding with the generated story and diagrams together. This makes it possible to automatically generate diagrams related to a story based on the scenario input by the user, thereby efficiently creating documents.

[0161] "User" means an individual or entity that uses the System to create materials.

[0162] A "scenario" is a sentence or text that indicates the content and structure of the material that the user wants to create.

[0163] "Input means" refers to the interface or device that allows the user to input a scenario into the system.

[0164] "Means for analysis" is a function for analyzing the input scenario using natural language processing technology and extracting keywords and themes.

[0165] "Keywords" are words or phrases that are particularly important in a scenario.

[0166] The "theme" is the central theme or topic of the scenario.

[0167] "Generative AI methods" are artificial intelligence techniques for generating stories based on extracted keywords and themes.

[0168] "In-house stored materials" refers to data and information held internally by a company or organization.

[0169] "External information" refers to all information available to the system, including materials stored within the company.

[0170] "Means for generating diagrams" refers to a function that automatically creates appropriate diagrams based on materials accumulated within the company and external information.

[0171] The "means of response" is the interface or device that presents the generated story and diagrams to the user.

[0172] This invention relates to a system that inputs a scenario for a document that a user wants to create and automatically generates a story and related diagrams based on that scenario. A specific embodiment of this system will be described below.

[0173] System program generation

[0174] The server generates a program to implement the generative AI model. This program uses natural language processing technology to analyze the input scenario and extract keywords and themes. It also includes a function to generate a story based on the extracted keywords and themes.

[0175] Hardware and software used

[0176] The server implements natural language processing technology using the Python programming language and the TENSORFLOW® library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the BERT® (Bidirectional Encoder Representations from Transformers) model. Based on the extracted keywords and themes, the server generates a story using the GPT-3 (Generative Pre-trained Transformer 3) model.

[0177] Specific examples

[0178] Consider the case where a user inputs, "Please generate a story for a presentation of our new product. Please include the new product's features, comparison with competitors, and market positioning." The scenario input from the device is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, based on these keywords, a story is generated using the GPT-3 model. The generated story might be, for example, "A presentation that highlights the new product's features, compares it with competitors, and explains its market positioning."

[0179] Prompt Sentence Examples

[0180] Examples of prompts that users can enter into a generative AI model include:

[0181] "Generate a story to present your new product. Include the new product's features, comparison with competitors, and market positioning."

[0182] In this way, the processing of the system's programs is explained while clarifying how the server, terminals, and users are involved.

[0183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0184] Step 1:

[0185] The user inputs a scenario. The user inputs the scenario into the input field of the terminal. For example, the user might input, "Please generate a story for a presentation of a new product. Please include the features of the new product, a comparison with competitors, and its position in the market." The input scenario becomes the input data for the terminal.

[0186] Step 2:

[0187] The terminal sends the scenario to the server. The terminal sends the scenario entered by the user to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[0188] Step 3:

[0189] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (for example, NLTK (registered trademark) or SpaCy (registered trademark)). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[0190] Step 4:

[0191] The server extracts keywords and themes. The server inputs the tokenized data into the BERT model to extract important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is the tokenized scenario, and the output data is the extracted keywords and themes.

[0192] Step 5:

[0193] The server generates a story based on the extracted keywords and themes. The server generates a story using the GPT-3 model based on the extracted keywords and themes. The generated story might be, for example, "a presentation that highlights the features of a new product, compares it with competitors, and explains its position in the market." The input data are the extracted keywords and themes, and the output data is the generated story.

[0194] Step 6:

[0195] The server sends the generated story to the terminal. The server packages the generated story in JSON format and sends it to the terminal as an HTTP response. The input data is the generated story, and the output data is the story sent to the terminal.

[0196] Step 7:

[0197] The terminal displays the story to the user. The terminal displays the received story on its user interface. The user can review the generated story and make corrections or additions as needed. The input data is the story sent to the terminal, and the output data is the story displayed to the user.

[0198] (Application example 2)

[0199] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0200] In the traditional advertising campaign creation process, scenario analysis, keyword extraction, and story generation are often done manually, which is time-consuming and labor-intensive. Furthermore, the generated stories can be inconsistent or not suited to the target audience, making it difficult to create effective ads.

[0201] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating illustrations using documents accumulated in-house as external information, a means for responding with the generated story and illustrations together, and a means for analyzing the advertising campaign scenario and extracting keywords and themes to generate an advertising story. This makes it possible to automate the process from analyzing the advertising campaign scenario to generating the story, and to quickly create effective and consistent advertisements.

[0202] "User" means an individual or organization that uses the system to create materials and advertising campaign scripts.

[0203] A "scenario" is text information entered by a user for the purpose of creating an advertising campaign or materials, and serves as the basis for generating a story.

[0204] "Generative AI means" is an artificial intelligence technology that analyzes an input scenario and generates a story using natural language processing technology.

[0205] "In-house accumulated materials" refers to a collection of documents and data created in the past within a company or organization, and are used as external information.

[0206] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company and external information.

[0207] "Answering means" refers to the methods and techniques used to present the generated stories and diagrams to users.

[0208] An "advertising campaign" is a series of marketing activities designed to promote a particular product or service.

[0209] "Keywords" are important words and phrases extracted from a scenario and are the basis for generating a story.

[0210] The "theme" is the central theme or topic of a scenario and determines the direction of the story.

[0211] An "advertising story" is a coherent narrative that is tailored to the objectives of an advertising campaign and appeals to a target audience.

[0212] A system for implementing this invention is configured as follows: First, a terminal is required for inputting the materials and advertising campaign scenario that the user wants to create. This terminal is a device such as a PC or smartphone, and provides an interface for the user to input the scenario.

[0213] Next, the server receives the input scenario and analyzes it using a generative AI tool. This analysis uses natural language processing technology to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. This generative AI uses, for example, the OpenAI API.

[0214] Furthermore, the server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated story and diagrams are then provided to the user's device as a response.

[0215] As a concrete example, consider the case where a user inputs a scenario for a presentation of a new product. This scenario includes information such as "features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates a prompt sentence like the following:

[0216] Example prompt sentence:

[0217] Scenario: A new product presentation highlights the product's features, how it compares to competitors, and its position in the market.

[0218] Keywords: new product, presentation, features, competitors, market

[0219] Generate an advertising story based on this scenario.

[0220] If we feed this prompt into a generative AI model, it might generate an advertising story like this:

[0221] An example of a generated advertising story:

[0222] The new X-100 is equipped with innovative features not found in other products. Compared to its competitors, the X-100 offers better performance and cost-effectiveness. Its market positioning makes it the perfect choice for users seeking cutting-edge technology. Get your X-100 now and experience the future.

[0223] In this way, users can easily create effective advertising stories. This system automates the process from analyzing advertising campaign scenarios to generating stories, enabling users to quickly create effective and consistent advertisements.

[0224] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0225] Step 1:

[0226] The user uses a terminal to input the scenario for the materials or advertising campaign they want to create, and the input scenario is sent to the server in text format.

[0227] Step 2:

[0228] The server analyzes the received scenario. This analysis uses natural language processing technology. Specifically, it extracts keywords and themes from the scenario. The input for this process is the text data of the scenario, and the output is a list of extracted keywords and themes.

[0229] Step 3:

[0230] The server generates a prompt based on the extracted keywords and topics. The generated prompt is then input to a generative AI model. The input to this process is a list of keywords and topics, and the output is a prompt.

[0231] Step 4:

[0232] The server uses a generative AI model to generate a story based on the prompt. Specifically, it uses OpenAI's API to generate the story. The input of this process is the prompt, and the output is the generated story.

[0233] Step 5:

[0234] The server uses internally stored documents as external information to generate diagrams. It uses image recognition technology to select and generate appropriate diagrams. The input for this process is internal document data, and the output is the generated diagram.

[0235] Step 6:

[0236] The server provides the generated story and diagram together to the user's terminal as a response. The input of this process is the generated story and diagram, and the output is the response data sent to the user's terminal.

[0237] Step 7:

[0238] The user checks the story and illustrations provided through the terminal, and makes corrections or additions as necessary to complete the final materials or advertising campaign content. The input to this process is the response data sent from the server, and the output is the completed materials or advertising campaign content.

[0239] Example 3

[0240] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0241] Conventional document creation systems require users to manually search for documents, select appropriate figures, and create stories, which takes time and effort. Selecting appropriate figures also requires specialized knowledge, making it difficult for users to create documents efficiently. Another issue is the inconsistency in the quality of the generated documents.

[0242] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0243] In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for using documents stored in the company as external information and selecting and generating appropriate illustrations using image recognition technology, and a means for responding with the generated story and illustrations together, thereby enabling users to efficiently create high-quality documents.

[0244] "User" means an individual or organization that uses the System to create materials.

[0245] A "scenario" is text information that indicates the content and purpose of the material that a user wants to create.

[0246] "Generative AI means" refers to means that use artificial intelligence technology to generate a story based on an input scenario.

[0247] "In-house stored materials" refers to information resources such as documents, images, and data stored within a company or organization.

[0248] "External information" refers to materials accumulated within the company, which are sources of information for use by the system.

[0249] "Image recognition technology" is a technology for analyzing images and diagrams and understanding their contents.

[0250] "Appropriate illustrations" are diagrams or images selected to visually represent information relevant to the scenario.

[0251] The "means for generating" is a means for creating a new diagram based on a selected diagram.

[0252] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[0253] The term "system" refers to a computer-based device or software for carrying out a series of processes including the above-mentioned means.

[0254] The present invention relates to a system that enables users to efficiently create high-quality materials. A specific embodiment of this system will be described below.

[0255] System Overview

[0256] The system includes the following main means:

[0257] 1. A way for users to input the scenario of the material they want to create

[0258] 2. Generative AI means to generate stories based on input scenarios

[0259] 3. Using internally stored data as external information, and using image recognition technology to select and generate appropriate diagrams.

[0260] 4. A means of answering questions with generated stories and diagrams

[0261] Hardware and software used

[0262] The server runs the system using the following hardware and software:

[0263] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[0264] Software: Google® Cloud Vision API, Amazon® Rekognition, Adobe Illustrator®, Canva®, generative AI models (e.g., OpenAI's GPT-4)

[0265] Data processing and calculation

[0266] The server processes and calculates data in the following steps:

[0267] 1. Collecting materials:

[0268] The server accesses internal databases and file servers to search and retrieve relevant materials, including PDFs, Word documents, presentation files, and more.

[0269] 2. Application of image recognition technology:

[0270] The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and diagrams in the collected materials.

[0271] 3. Selecting the right diagram:

[0272] Based on the analysis results, the server filters and selects appropriate diagrams related to the scenario.

[0273] 4. Figure Generation:

[0274] The server uses Adobe Illustrator or Canva to generate a new diagram based on the selected diagram.

[0275] 5. Story generation:

[0276] The server uses a generative AI model (GPT-4) to generate a story based on the generated diagrams.

[0277] 6. Providing answers:

[0278] The server compiles the generated diagrams and stories into a single document and provides it to the user.

[0279] Specific examples

[0280] For example, if a user wants to create a document explaining the "features of a new product," they can input the following prompt into the generative AI model:

[0281] "Please generate a diagram to explain the features of a new product. Please select and generate an appropriate diagram based on internal company materials."

[0282] When the user enters this prompt, the server generates a diagram according to the procedure described above and provides it along with the story. Users can then create materials based on the answers.

[0283] In this way, the user can efficiently create high-quality materials. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0284] Step 1: Gather your materials

[0285] The server accesses the company's databases and file servers to search for and retrieve relevant materials. As input, the user is given a scenario for the material they want to create. The server extracts keywords based on this scenario and uses these keywords to search for materials. As output, it obtains related materials such as PDFs, Word documents, and presentation files.

[0286] Step 2: Applying image recognition technology

[0287] The server applies image recognition technology to the collected materials. The materials collected in step 1 are provided as input. The server uses Google Cloud Vision API and Amazon Rekognition to analyze the images and diagrams within the materials. Specifically, the server processes each document sequentially and extracts the contents of the images and diagrams as text data. The analysis results are obtained as output.

[0288] Step 3: Select the right diagram

[0289] The server uses image recognition technology to select appropriate diagrams from the documents. The analysis results obtained in step 2 are given as input. The server matches the analysis results with keywords related to the scenario, and filters and selects highly relevant diagrams. Specifically, the server calculates a relevance score and selects diagrams with high scores. The selected diagrams are obtained as output.

[0290] Step 4: Generate the diagram

[0291] The server generates a new diagram based on the selected diagram. As input, it receives the diagram selected in step 3. The server uses Adobe Illustrator or Canva to create a new diagram using the selected diagram as a template and adding the necessary information. Specifically, the server adjusts the diagram layout and adds annotations and data. As output, it receives the generated new diagram.

[0292] Step 5: Generate stories

[0293] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related explanations and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[0294] Step 6: Provide your answers

[0295] The server compiles the generated diagrams and stories into a single document and provides it to the user. As input, the diagram generated in step 4 and the story generated in step 5 are given. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The final document is obtained as output.

[0296] (Application example 3)

[0297] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0298] Maintenance work in factories is complex and diverse, making it difficult for workers to quickly grasp the appropriate procedures. Furthermore, the sheer volume of maintenance records and technical documents makes it difficult to efficiently search and use the necessary information. For this reason, there is a demand for improved efficiency and accuracy in maintenance work.

[0299] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0300] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents stored in-house as external information; means for responding with the generated story and diagrams together; means for selecting and generating appropriate diagrams using image recognition technology using maintenance records and technical documents stored in the factory as external information; and means for supporting maintenance work based on the selected diagrams. This enables maintenance workers to quickly obtain the information they need and perform their work efficiently and accurately.

[0301] "User" refers to a person who uses the system to create materials and perform maintenance work.

[0302] A "scenario" refers to a plan or structure that indicates the content and purpose of the material that the user wants to create.

[0303] "Generative AI means" refers to artificial intelligence technology that generates a story based on an input scenario.

[0304] "Documents accumulated within the company" refers to information such as technical documents and maintenance records that have been created within the company in the past.

[0305] "External information" refers to materials and data accumulated within the company for use by the system.

[0306] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company.

[0307] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[0308] "Maintenance records" refers to information that records the history and details of maintenance work performed within a factory.

[0309] "Technical documents" refers to technical information and procedures related to equipment and systems within a factory.

[0310] The term "means for supporting maintenance work" refers to a support technology that enables a maintenance worker to perform work efficiently and accurately based on the selected diagram.

[0311] A system for carrying out the present invention is configured as follows: First, an interface is provided for a user to input a scenario for a material that the user wishes to create. The user inputs the scenario through this interface.

[0312] The server then generates a story based on the input scenario. This is done using a generative AI means that uses natural language processing technology to analyze the scenario and generate an appropriate story. For example, the Hugging Face® transformers® library can be used as the generative AI means.

[0313] Furthermore, the server uses internally stored documents as external information to generate diagrams. Image recognition technology is used to generate diagrams. Specifically, software such as OpenCV (registered trademark), PIL (Python Imaging Library), and pytesseract (registered trademark) is used to extract and analyze text and features from images.

[0314] Maintenance records and technical documents stored within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[0315] The generated story and diagram are provided to users together, allowing them to quickly obtain the information they need and work efficiently and accurately.

[0316] As a concrete example, the following prompt sentence can be used:

[0317] Example prompt sentence:

[0318] "Generate a story about the part replacement procedure based on the following images: ['maintenance_step1.png', 'maintenance_step2.png']"

[0319] Based on this prompt, the generative AI means generates a story about the part replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[0320] In this way, the efficiency of factory robot maintenance work can be improved and the burden on workers can be reduced.

[0321] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0322] Step 1:

[0323] The user inputs the scenario of the material they want to create.

[0324] Input: A scenario in which the user provides input through an interface.

[0325] Output: The input scenario data.

[0326] Specific operation: The user accesses the system interface and inputs the scenario of the material they want to create in text format.

[0327] Step 2:

[0328] The server generates a story based on the input scenario.

[0329] Input: The entered scenario data.

[0330] Output: The generated stories.

[0331] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generate an appropriate story using natural language processing technology.

[0332] Step 3:

[0333] The server uses materials stored within the company as external information to generate diagrams.

[0334] Input: Materials accumulated within the company.

[0335] Output: The generated diagram.

[0336] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate diagrams from materials stored within the company.

[0337] Step 4:

[0338] The server uses maintenance records and technical documents stored within the factory as external information and uses image recognition technology to select and generate appropriate diagrams.

[0339] Input: Maintenance records and technical documents accumulated within the factory.

[0340] Output: Selected diagrams.

[0341] Specific operation: The server uses image recognition technology to select and generate the necessary diagrams from maintenance records and technical documents.

[0342] Step 5:

[0343] The server responds with the generated story and diagrams.

[0344] Input: The generated story and selected diagrams.

[0345] Output: The story and diagrams provided to the user.

[0346] Specific operation: The server combines the generated story with the selected diagrams and provides it to the user, who can then create materials based on it.

[0347] Step 6:

[0348] The maintenance work is carried out based on the stories and diagrams provided by the user.

[0349] Input: The story and illustrations provided.

[0350] Output: Efficient and accurate maintenance work.

[0351] Specific behavior: The user performs maintenance tasks efficiently and accurately while referring to the provided stories and diagrams.

[0352] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0353] "Example 1"

[0354] As one embodiment of the present invention, a system is provided that includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagram together, and an emotion engine that recognizes the user's emotions.

[0355] "Example 2"

[0356] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a happy expression, the emotion engine captures that information and the generative AI generates a story based on that emotion. Specifically, if the emotion of joy is recognized, the generative AI generates a story that includes positive elements.

[0357] "Example 3"

[0358] The emotion engine also selects and generates appropriate images according to the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures that information, and the image generation means selects and generates an image that conveys surprise and freshness.

[0359] The processing flow of each embodiment will be described below.

[0360] "Example 1"

[0361] Step 1: The user inputs the scenario of the material they want to create.

[0362] Step 2: Based on the input scenario, the generative AI generates a story.

[0363] Step 3: Use the data accumulated within the company as external information to generate the diagram.

[0364] Step 4: Answer the generated story and diagram together.

[0365] Step 5: The emotion engine that recognizes the user's emotions is activated.

[0366] "Example 2"

[0367] Step 1: The emotion engine recognizes the user's emotion from their tone of voice and facial expression.

[0368] Step 2: The generative AI generates a story based on the emotions recognized by the emotion engine.

[0369] "Example 3"

[0370] Step 1: The emotion engine recognizes the user's emotion.

[0371] Step 2: Select and generate an appropriate illustration according to the emotion recognized by the emotion engine.

[0372] Example 1

[0373] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0374] Conventional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which requires a great deal of time and effort. Furthermore, they do not provide feedback that takes into account the user's feelings, which reduces the efficiency and quality of document creation. Furthermore, there is a lack of effective ways to utilize the documents accumulated within the company, which can lead to a lack of consistency and reliability in the documents.

[0375] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0376] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback according to the user's emotions, and enables effective use of internal documents.

[0377] "Means for inputting the scenario of the material that the user wants to create" refers to an interface that allows the user to freely input text, and is a means for inputting the scenario of the material.

[0378] "Generative AI means" refers to artificial intelligence technology used to generate a story based on an input scenario, and is a means of analyzing the scenario and generating a story using natural language processing technology.

[0379] "Means of generating diagrams using materials accumulated within the company as external information" refers to means of generating diagrams by referencing materials stored in an internal database and using data visualization technology.

[0380] "Means for providing answers using the generated story and diagrams together" refers to a means for integrating the generated story and diagrams and providing them to the user as a single document.

[0381] An "emotion engine" refers to technology that analyzes user input and reactions and recognizes emotions. It is a means of analyzing user emotions using natural language processing technology and providing appropriate feedback.

[0382] This invention is a system that inputs a scenario for a document that a user wants to create, generates a story and illustrations based on that scenario, and finally provides the document as a single document. A specific embodiment of this system will be described below.

[0383] System configuration

[0384] The system consists of the following main components:

[0385] 1. Text Input Interface

[0386] 2. Generative AI means

[0387] 3. Database Reference Method

[0388] 4. Data Visualization Methods

[0389] 5. Integration Methods

[0390] 6. Emotion Engine

[0391] Text Input Interface

[0392] The user uses the text input interface of the terminal to input the scenario of the material they want to create. This interface is designed so that the user can input text freely. For example, the user can input "New product presentation."

[0393] Generative AI means

[0394] The server receives the scenario entered by the user and inputs it as a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates an appropriate story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[0395] Database Reference Method

[0396] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib (registered trademark) and Pandas (registered trademark) to format and visualize the data.

[0397] Data visualization tools

[0398] The server uses data visualization techniques to generate diagrams based on the information retrieved from the database. For example, it uses SQL queries to extract the necessary information from the database and generates graphs using Python's Matplotlib. The generated graphs are saved as image files.

[0399] Integration Methods

[0400] The server integrates the generated stories and figures and compiles them into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX (registered trademark) or Microsoft (registered trademark) Word API). The generated document is saved as a PDF file.

[0401] Emotion Engine

[0402] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[0403] Examples of specific examples and prompts

[0404] Specific examples

[0405] The user enters "New Product Presentation" into the text entry interface.

[0406] The server uses a generative AI model to generate a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[0407] The server references past sales data and market research results from an internal database and creates relevant graphs and charts.

[0408] Finally, the generated stories and diagrams are integrated and provided to the user.

[0409] Prompt Sentence Examples

[0410] "Please enter a scenario for a new product presentation. For example, please include the following: new product features, target market, competitive analysis, sales strategy, etc."

[0411] In this way, users can easily create high-quality materials.

[0412] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0413] Step 1:

[0414] The user inputs a scenario.

[0415] The user uses the device's text input interface to input the scenario of the document they want to create, for example, "New product presentation." The input scenario is sent to the server in real time.

[0416] Input: User-entered scenario text

[0417] Output: Scenario text sent to the server

[0418] Step 2:

[0419] The server generates the story using a generative AI model.

[0420] The server inputs the received scenario as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[0421] Input: Scenario text

[0422] Output: Generated story text

[0423] Step 3:

[0424] The server references the company's internal database to generate the diagram.

[0425] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib and Pandas to format and visualize the data.

[0426] Input: Data obtained from our internal database

[0427] Output: Image files of generated graphs and charts

[0428] Step 4:

[0429] The server integrates the generated stories and diagrams.

[0430] The server integrates the generated stories and figures into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX or Microsoft Word API). The generated document is saved as a PDF file.

[0431] Input: Generated story text, image files of graphs and charts

[0432] Output: Flattened PDF document

[0433] Step 5:

[0434] The server recognizes the user's emotions and provides feedback.

[0435] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[0436] Input: User feedback text

[0437] Output: Analysis results by the emotion engine and feedback text

[0438] (Application example 1)

[0439] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0440] The traditional process of creating advertising campaigns required a lot of time and effort, including creating a scenario, story, and visuals. It was also difficult to propose optimal advertising materials that took into account user emotions, making it difficult to quickly create effective advertising campaigns. This meant that advertising agencies and marketers were unable to work efficiently, making it difficult to maximize the effectiveness of their advertising.

[0441] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0442] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents accumulated in-house as external information; means for responding with the generated story and diagrams together; means including an emotion engine for recognizing the user's emotions; and means for generating visuals for advertising based on the generated story. This makes it possible to streamline the advertising campaign creation process and propose optimal advertising materials according to the user's emotions.

[0443] "User" means an individual or organization that uses the system to create a scenario for a material.

[0444] A "scenario" is text information that describes the content and structure of the material that a user wants to create.

[0445] "Generative AI means" refers to means that use artificial intelligence technology to generate a story based on an input scenario.

[0446] "In-house accumulated materials" refers to a collection of documents and data that have been created in the past within a company or organization.

[0447] "Means for generating diagrams" refers to the technology for creating appropriate diagrams using materials accumulated within the company.

[0448] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[0449] An "emotion engine" is a technology that recognizes the user's emotions and makes appropriate suggestions based on them.

[0450] "Advertising Visuals" are images and graphics used in advertising campaigns.

[0451] The system for implementing this invention has the following configuration: First, a text input interface is provided for the user to input the scenario of the material they wish to create. Using this interface, the user can freely write the scenario.

[0452] Next, a generative AI method is used to generate a story based on the input scenario. This generative AI method analyzes the scenario using natural language processing technology and generates an appropriate story. Specifically, it uses OpenAI's API to generate a story from the scenario.

[0453] Based on the generated story, a means of generating diagrams is run using internally stored materials. This means uses image recognition technology to select and generate appropriate diagrams from internal materials. PIL (Python Imaging Library) is used to create visuals that match the generated story.

[0454] Additionally, a means is included to present the generated story and diagrams together to the user, integrating the generated content and providing a consistent answer to the user.

[0455] It also includes a means for including an emotion engine to recognize user emotions. This emotion engine analyzes user emotions and suggests optimal advertising materials based on the analysis. The emotion_recognition library is used to recognize user emotions.

[0456] Finally, a means for generating advertising visuals based on the generated story is included, which generates images and graphic materials for use in an advertising campaign based on the generated story.

[0457] As a concrete example, if a user inputs a scenario such as "an advertising campaign for a new smartphone product," the generative AI will generate a story like the one below.

[0458] Example prompt sentence:

[0459] Advertising Campaign Scenario: Advertising campaign for a new smartphone product

[0460] story:

[0461] By inputting this prompt into a generative AI model, an appropriate story is generated, and based on the generated story, visuals for the ad are created and provided to the user.

[0462] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0463] Step 1:

[0464] The user inputs the scenario of the material they want to create.

[0465] The user uses a text input interface to freely describe the scenario of an advertising campaign, and the input scenario is sent to the system.

[0466] Input: User-entered scenario text

[0467] Output: The scenario text is sent to the system

[0468] Step 2:

[0469] The server generates a story based on the input scenario.

[0470] The server uses generative AI tools to analyze the input scenario and generate an appropriate story. Specifically, it uses OpenAI's API to generate a story from the scenario.

[0471] Input: Scenario text

[0472] Output: Generated story text

[0473] Step 3:

[0474] The server generates diagrams based on the generated stories.

[0475] The server uses the data stored in-house to select and generate appropriate illustrations using image recognition technology, and uses PIL (Python Imaging Library) to create visuals that match the generated story.

[0476] Input: Generated story text

[0477] Output: Generated diagram (visual)

[0478] Step 4:

[0479] The server responds with the generated story and diagrams.

[0480] The server integrates the generated stories and diagrams to provide a consistent answer to the user.

[0481] Input: Generated story text, generated illustrations

[0482] Output: Integrated story and diagrams

[0483] Step 5:

[0484] The server recognizes the user's emotions.

[0485] The server uses the emotion engine to analyze the user's emotions. The emotion_recognition library is used to recognize the user's emotions.

[0486] Input: User's emotional data (e.g., facial expression images and voice data)

[0487] Output: Recognized emotion information

[0488] Step 6:

[0489] The server generates visuals for advertising based on the generated story.

[0490] The server then creates images and graphic materials for use in advertising campaigns based on the generated stories.

[0491] Input: Generated story text, recognized emotion information

[0492] Output: Visuals for advertising

[0493] Step 7:

[0494] The server provides the final advertising campaign materials to the user.

[0495] The server integrates the generated stories, illustrations, and advertising visuals and provides them to the user.

[0496] Input: Generated story text, generated illustrations, advertising visuals

[0497] Output: Final advertising campaign materials

[0498] Example 2

[0499] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0500] Conventional document creation systems can generate stories based on scenarios entered by users, but they have the problem of not being able to adjust the story to take the user's emotions into account. Also, when providing the generated story and illustrations together, they do not make appropriate adjustments based on the user's emotions, making it difficult to create documents that reflect the user's intentions and emotions.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0502] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating diagrams using documents accumulated in-house as external information, means for responding with the generated story and diagrams, means for sensing the user's tone of voice and facial expression, means for recognizing the user's emotions using an emotion engine, and means for adjusting the story based on the recognized emotions. This makes it possible to provide a story and diagrams that reflect the user's emotions.

[0503] "User" refers to a person who uses the system to input a material scenario and receive the generated stories and illustrations.

[0504] "Scenario" refers to input information in text format that indicates the content and structure of the material that the user wants to create.

[0505] "Generative AI means" refers to artificial intelligence technology for generating stories based on input scenarios.

[0506] "Natural language processing technology" refers to technology for analyzing scenarios and extracting keywords and themes.

[0507] "Means for generating diagrams" refers to the technology for generating appropriate diagrams using materials accumulated within the company as external information.

[0508] "Means of sensing" refers to technology that senses the user's tone of voice and facial expressions in real time.

[0509] "Emotion engine" refers to technology that recognizes a user's emotions from detected tone of voice and facial expressions.

[0510] "Adjustment means" refers to techniques for appropriately adjusting the generated story based on the recognized emotions.

[0511] "Story" refers to a series of sentences or content generated by generative AI means based on a scenario.

[0512] "Means of answering" refers to the technology used to present the generated stories and diagrams to users.

[0513] This invention is a system that inputs a scenario for a document that a user wants to create, generates a story based on that scenario, and provides a story that reflects the user's emotions. A specific embodiment of this system will be described below.

[0514] Hardware and software used

[0515] 1. Server:

[0516] Natural language processing techniques: Using generative AI models such as BERT and GPT-3.

[0517] Emotion engine: Software for analyzing the user's tone of voice and facial expressions.

[0518] Database: A database for storing data accumulated within the company and for use as external information.

[0519] 2. Terminal:

[0520] Input interface: A keyboard or touch screen for users to input scenarios.

[0521] Camera and microphone: Hardware that captures your tone of voice and facial expressions.

[0522] Data processing and calculation

[0523] 1. Scenario input and analysis:

[0524] The user uses the input interface of the terminal to input the scenario in text format.

[0525] The server receives the input scenario and analyzes it using natural language processing techniques, specifically using generative AI models such as BERT and GPT-3 to extract keywords and themes from the scenario.

[0526] 2. Story generation:

[0527] The server uses a generative AI model to generate a story based on the extracted keywords and themes. For example, if a scenario for a "new product presentation" is input, the server will extract keywords such as "new product features," "comparison with competitors," and "market positioning," and generate a story based on them.

[0528] 3. Emotional awareness and story adjustment:

[0529] The device uses a camera and microphone to detect the user's tone of voice and facial expressions in real time.

[0530] The device uses an emotion engine to recognize the user's emotions from the detected tone of voice and facial expressions. For example, if the user shows a happy expression, the device sends that information to the emotion engine, which then recognizes the emotion as happy.

[0531] The server adjusts the generated story based on the recognized emotion, for example, adding positive elements to the story if positive emotion is recognized.

[0532] 4. Final Story Delivery:

[0533] The server then provides the final adjusted story to the user, specifically by sending the generated story and diagrams together.

[0534] Examples of specific examples and prompts

[0535] Specific examples

[0536] Scenario: New product presentation

[0537] Extracted keywords: new product features, comparison with competitors, market positioning

[0538] The story generated: A presentation that highlights the new product's features, demonstrates its competitive advantage, and clarifies its position in the market.

[0539] Prompt Sentence Examples

[0540] "Enter a scenario for a new product presentation. Generative AI will analyze the scenario, extract keywords, and generate a story."

[0541] "Detect tone of voice and facial expressions to recognize the user's emotions. Generative AI will generate a story based on the emotions recognized by the emotion engine."

[0542] In this way, the system can generate and deliver the best story based on the user's input and emotions.

[0543] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0544] Step 1:

[0545] The user inputs a scenario.

[0546] Input: The user inputs the scenario in text format using the terminal's input interface.

[0547] What happens: The user types "New Product Presentation."

[0548] Output: The input scenario is saved on the device and sent to the server.

[0549] Step 2:

[0550] The server receives the scenario and analyzes it using natural language processing technology.

[0551] Input: Scenario text sent from the terminal.

[0552] What happens: The server uses a generative AI model such as BERT or GPT-3 to parse the text "New Product Presentation."

[0553] Output: Keywords and themes extracted from the scenario.

[0554] Step 3:

[0555] The server extracts keywords and themes from the scenario.

[0556] Input: Scenario text parsed using natural language processing techniques.

[0557] Specific operation: The server extracts the keywords "new product features," "comparison with competitors," and "market positioning."

[0558] Output: A list of extracted keywords or themes.

[0559] Step 4:

[0560] The server generates stories based on the extracted keywords and themes.

[0561] Input: A list of extracted keywords or themes.

[0562] What it does: The server uses the generative AI model to generate a presentation that highlights the new product's features, demonstrates its advantages over competitors, and clarifies its position in the market.

[0563] Output: The generated stories.

[0564] Step 5:

[0565] The device detects the user's tone of voice and facial expressions.

[0566] Input: The user's real-time tone of voice and facial expressions.

[0567] What it does: The device uses the camera and microphone to detect whether the user is smiling or has a positive tone of voice.

[0568] Output: Detected voice tone and facial expression data.

[0569] Step 6:

[0570] The terminal recognizes the user's emotions using an emotion engine.

[0571] Input: Detected voice tone and facial expression data.

[0572] Specific operation: The device uses the emotion engine to recognize the user's emotion of joy.

[0573] Output: Recognized emotion data.

[0574] Step 7:

[0575] The server adjusts the story based on the perceived sentiment.

[0576] Input: Recognized emotion data and generated stories.

[0577] What it does: The server adds positivity to the story, creating a more upbeat presentation.

[0578] Output: A final, adjusted story.

[0579] Step 8:

[0580] The server provides the final story to the user.

[0581] Input: The final, adjusted story.

[0582] What it does: The server displays to the user "a positive-toned presentation that highlights the new product's features, demonstrates its advantages over competitors, and clarifies its position in the market."

[0583] Output: The final story delivered to the user.

[0584] (Application example 2)

[0585] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0586] Conventional document creation systems can generate stories based on scenarios entered by users, but they cannot generate stories that take the user's emotions into account, making it difficult to create documents that reflect the user's intentions and emotions.In addition, when responding with the generated story and diagrams, it is not possible to reflect the user's emotions, so more effective document creation is needed.

[0587] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0588] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents stored in the company as external information; means for responding with the generated story and diagrams together; emotion engine means for recognizing emotions from the user's tone of voice and facial expression; and means for generating a story based on the recognized emotions. This makes it possible to generate a story that reflects the user's emotions, enabling more effective document creation.

[0589] "User" refers to a person who uses the system to create materials.

[0590] A "scenario" refers to a text or plot that indicates the content and structure of the material that the user wants to create.

[0591] "Generative AI" refers to artificial intelligence technology that generates a story based on an input scenario.

[0592] "In-house stored materials" refers to data and information collected and stored within a company or organization.

[0593] "External information" refers to data obtained from sources other than materials stored within the company.

[0594] "Figures" refers to visual elements such as graphs, charts, and illustrations included in the document.

[0595] "Emotion engine" refers to technology that recognizes emotions from the user's tone of voice and facial expressions.

[0596] "Voice tone" refers to the audio characteristics of the user's voice, such as pitch, strength, and rhythm.

[0597] "Facial expressions" refer to the user's facial movements and expressions.

[0598] A "story" refers to a series of sentences or content generated based on a scenario.

[0599] "Answer" refers to the act of providing the generated story and diagram to the user.

[0600] A system for implementing this invention includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents accumulated within the company as external information; means for responding with the generated story and diagrams together; emotion engine means for recognizing emotions from the user's tone of voice and facial expression; and means for generating a story based on the recognized emotions.

[0601] System Program

[0602] The server runs the following program: First, the user inputs a scenario using a device such as a smartphone or PC. Next, the generative AI analyzes the scenario using natural language processing technology and extracts keywords and themes. It then generates a story based on the extracted information.

[0603] Furthermore, the emotion engine recognizes the user's tone of voice and facial expressions, and provides this emotional information to the generative AI. The generative AI then adjusts the story taking the emotional information into account, generating content that is more in line with the user's intentions. The generated story is complemented by a means of generating appropriate diagrams using internally accumulated materials as external information. Finally, the generated story and diagrams are provided to the user together.

[0604] Hardware and software used

[0605] Hardware: smartphones, PCs, servers

[0606] Software: OpenCV (image processing), Transformers (generative AI and emotion recognition), TextBlob (registered trademark) (text analysis)

[0607] Data processing and calculation

[0608] 1. Image processing: Images taken with a smartphone or computer camera are used to perform facial recognition and analyze facial expressions using OpenCV.

[0609] 2. Speech processing: Speech recorded using a smartphone or computer microphone is converted into text and emotions are recognized using an emotion engine.

[0610] 3. Story generation: Use generative AI (such as GPT-3) to generate stories based on recognized emotions and scenarios.

[0611] 4. Diagram generation: Select and generate appropriate diagrams from the internal data.

[0612] Specific examples

[0613] For example, if a user wants to create an ad about the camera features of a new smartphone, they might enter the following prompt:

[0614] Prompt Sentence Examples

[0615] Create an advertisement story for a product. The user is feeling happy and the sentiment is positive. The product information is: A new smartphone with advanced camera features.

[0616] Based on this prompt, the generative AI generates a positive advertising story that highlights the new smartphone's camera features. The emotion engine recognizes the user's happy facial expression and provides that information to the generative AI, which then generates a more emotionally relevant advertising story. The generated story then uses internally stored data to generate appropriate illustrations, which are then ultimately provided to the user.

[0617] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0618] Step 1:

[0619] The user inputs a scenario using a terminal.

[0620] Input: Scenario of the material the user wants to create (text format)

[0621] Output: Input scenario (text data)

[0622] Specific operation: The user inputs a scenario into the input screen of the terminal and presses the send button. The terminal then sends the input scenario to the server.

[0623] Step 2:

[0624] The server uses generative AI to analyze the scenario and extract keywords and themes.

[0625] Input: Input scenario (text data)

[0626] Output: Extracted keywords and themes (text data)

[0627] How it works: The server uses generative AI (natural language processing) to analyze the scenario and extract key keywords and themes. The extracted information is used in the next step.

[0628] Step 3:

[0629] The server uses an emotion engine to recognize the user's tone of voice and facial expressions.

[0630] Input: User's voice and image data

[0631] Output: Recognized emotion (text data)

[0632] Specific operation: The user records their voice and facial expressions using the device's camera and microphone. The server uses OpenCV to analyze facial expressions from the image data and the emotion engine to analyze the tone of voice from the audio data. The recognized emotion is generated as the analysis result.

[0633] Step 4:

[0634] The server generates a story based on the recognized emotions.

[0635] Input: extracted keywords, themes, and recognized sentiment

[0636] Output: Generated story (text data)

[0637] How it works: The server uses generative AI to generate a story based on the extracted keywords, themes, and recognized emotions. The generated story is used in the next step.

[0638] Step 5:

[0639] The server uses data stored within the company to generate appropriate diagrams.

[0640] Input: Generated story (text data)

[0641] Output: Generated diagram (image data)

[0642] Specific operation: The server uses image recognition technology to select and generate appropriate diagrams from internally stored data. The generated diagrams are used in the next step.

[0643] Step 6:

[0644] The server provides the generated story and diagrams together to the user.

[0645] Input: Generated story (text data), generated diagram (image data)

[0646] Output: Materials provided to users (text data and image data)

[0647] Specific operation: The server compiles the generated story and diagrams into a single document and sends it to the user's device, where the user can view the document.

[0648] Example 3

[0649] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0650] Conventional document creation systems required users to create documents manually, which required a great deal of time and effort. Furthermore, there was no function to select appropriate images based on the user's feelings, which meant that the content of the document sometimes did not match the user's intentions. Furthermore, there was a lack of a way to effectively utilize the information accumulated within the company, which could lead to a decline in the quality of the documents.

[0651] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0652] In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using internally stored information as external information, a means for responding with the generated story and diagrams together, a means for analyzing the user's emotions, and a means for selecting appropriate diagrams based on the analyzed emotions, thereby enabling users to efficiently create high-quality documents.

[0653] "User" means any individual or organization that uses the System to create materials.

[0654] A "scenario" refers to a plan or story that indicates the content and structure of the material that the user wants to create.

[0655] "Generative AI means" refers to artificial intelligence technology for generating stories based on input scenarios.

[0656] "Internally stored information" refers to data and materials collected and stored within a company or organization.

[0657] "External information" refers to diagrams and data generated using information stored internally.

[0658] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on information stored internally.

[0659] "Image recognition technology" refers to technology for analyzing image data and understanding and classifying its contents.

[0660] "Means for analyzing emotions" refers to technology for analyzing the emotions of users.

[0661] "Generated story" refers to a narrative or narrative created by generative AI means.

[0662] "Means of answering" refers to the technology used to present the generated stories and diagrams to users.

[0663] The present invention is a system for enabling users to efficiently create high-quality materials. A specific embodiment of this system will be described below.

[0664] System configuration

[0665] This system consists of three main components: a server, a terminal, and a user. The server acquires materials, performs image recognition, emotion analysis, generates diagrams and stories, and provides them to users. The terminal is responsible for capturing user input and emotions. Users create materials using the system.

[0666] Hardware and software used

[0667] Server: A server with a powerful processor and large memory capacity is required. The server also needs storage to access the database and retrieve materials.

[0668] Device: A computer or smart device with a camera and microphone is required, allowing for facial and audio capture of the user.

[0669] software:

[0670] Image recognition technology: Use image recognition services such as Google Cloud Vision API and Amazon Rekognition.

[0671] Emotion analysis technology: Uses Microsoft Azure's (registered trademark) Emotion API, etc.

[0672] Generative AI models: Use natural language processing models such as OpenAI's GPT-4.

[0673] Program processing

[0674] The server first accesses the company's internal database and retrieves stored documents, which can include PDFs, Word documents, presentation slides, etc. The server then uses image recognition technology to analyze the images in the documents and select appropriate figures based on specific keywords or context.

[0675] The device captures the user's facial expression with a camera and sends it to the emotion engine. The emotion engine analyzes the user's emotion and sends the results to the server. The server selects an appropriate illustration based on the analysis results.

[0676] The server sends the selected diagram along with a prompt to the generative AI model, which then generates a story based on the prompt and provides it to the user along with the selected diagram.

[0677] Examples and prompts

[0678] For example, if a user wants to create a document related to the "features of a new product," the server selects a diagram related to the "features of a new product" from the document. Then, it inputs the following prompt sentence to the generative AI model:

[0679] Prompt: "Select a picture to illustrate the features of your new product and generate a story."

[0680] The generative AI model generates a story based on this prompt and provides it to the user along with the selected illustrations.

[0681] If the user shows a surprised expression, the server will input the following prompt into the generative AI model:

[0682] Prompt: "Choose the appropriate illustration to generate a story when the user is surprised."

[0683] The generative AI model generates a story based on this prompt and presents it to the user along with illustrations that convey surprise and freshness.

[0684] In this way, the user can efficiently create high-quality materials. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0685] Step 1: Obtaining materials

[0686] The server accesses the company's internal database and retrieves the accumulated documents. A document search query is required as input. The server searches the database for the relevant documents and retrieves them in the form of PDF, Word document, presentation slides, etc. The retrieved documents are then output. Specifically, the server searches for documents related to "new product features" and downloads the relevant files.

[0687] Step 2: Selecting images using image recognition

[0688] The server uses image recognition technology to extract images from the acquired documents. The acquired documents are required as input. The server analyzes the images in the documents using image recognition services such as Google Cloud Vision API or Amazon Rekognition. The output is images selected based on specific keywords or context. Specifically, the server analyzes the images in the documents and selects images related to "new product features" or "comparison with competitors."

[0689] Step 3: Sentiment Analysis

[0690] The device captures the user's facial expression with a camera and sends it to the emotion engine. An image of the user's facial expression is required as input. The device analyzes the user's emotions using Microsoft Azure's Emotion API, etc. The output is the analyzed emotion data. Specifically, the device uses a camera to capture the user's facial expression and sends it to the emotion engine.

[0691] Step 4: Generate diagrams and stories

[0692] The server sends the selected diagram along with a prompt to be input to the generative AI model. The selected diagram and the prompt are required as input. The server uses a generative AI model, such as OpenAI's GPT-4, to generate a story based on the input prompt. The output is the generated story. Specifically, the server sends the following prompt to the generative AI model:

[0693] Prompt: "Select a picture to illustrate the features of your new product and generate a story."

[0694] Step 5: Provide to users

[0695] The server provides the generated story and selected diagrams to the user. The generated story and diagrams are required as input. The server sends these to the user's terminal. The output is the story and diagrams received by the user. In concrete terms, the server sends the generated story and diagrams to the user's terminal, and the user uses them to create materials.

[0696] (Application example 3)

[0697] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0698] Conventional document creation systems can generate stories and diagrams based on the scenario of the document the user wants to create, but they cannot generate appropriate diagrams according to the user's emotions. This makes it difficult to create effective documents that are in line with the user's emotions. In addition, there was a lack of technology to analyze user emotions in real time and generate diagrams based on that, which limited their application in fields such as advertising and presentations.

[0699] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0700] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated in-house as external information, a means for responding with the generated story and diagrams together, an emotion recognition means for analyzing the user's emotions, and a means for selecting and generating appropriate diagrams based on the analyzed emotions. This makes it possible to create effective documents according to the user's emotions.

[0701] "User" means an individual or entity that uses the System to create materials.

[0702] A "scenario" is an outline of a plan or story that indicates the content and structure of the material that the user wants to create.

[0703] "Generative AI methods" are methods that use artificial intelligence technology to generate stories based on input scenarios.

[0704] "In-house stored materials" refers to information such as documents, images, and data that have been created and stored within a company or organization in the past.

[0705] "External information" refers to information sources for use by the system, including materials stored within the company.

[0706] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company and external information.

[0707] A "generated story" is a narrative or exposition created based on a scenario by generative AI means.

[0708] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[0709] "Emotion recognition means" refers to technology or devices for analyzing a user's emotions.

[0710] "Analyzed emotion" means the user's emotional state detected and analyzed by the emotion recognition means.

[0711] An "appropriate illustration" is an illustration that best suits the material and is selected based on the analyzed emotions and scenarios.

[0712] A system for implementing this invention is configured as follows: First, an interface is provided for users to input a scenario for the material they wish to create. This interface allows users to input the scenario via a terminal such as a PC or smartphone.

[0713] Next, a generative AI means is activated to generate a story based on the input scenario. This generative AI means analyzes the scenario using natural language processing technology and generates an appropriate story. The generated story is provided to the user in a format that is easy to understand.

[0714] Furthermore, a means for generating diagrams is operated using documents stored within the company as external information. This means uses image recognition technology to select and generate appropriate diagrams from internal documents. The generated diagrams are provided to users along with the story.

[0715] The system also includes an emotion recognition means for analyzing the user's emotions. This emotion recognition means uses smart glasses or a camera-equipped device to analyze the user's facial expressions in real time to detect their emotions. Based on the analyzed emotions, a means for selecting and generating appropriate diagrams is activated. This enables the creation of effective materials that are in line with the user's emotions.

[0716] Specific hardware used includes smart glasses, camera-equipped devices, and PCs, while software such as OpenCV (image processing library), Keras (deep learning library), and PIL (Python Imaging Library) are used.

[0717] For example, if a user shows a surprised expression, the emotion recognition means detects the expression and generates a prompt sentence saying, "Please generate an advertisement that is optimal for a user who shows a surprised expression." Based on this prompt sentence, the generative AI means generates an advertisement for a new product or a special offer that corresponds to the surprise and provides it to the user.

[0718] In addition, if a user shows a happy expression, a prompt message is generated saying, "Please generate an advertisement that is best suited to a user who is showing a happy expression," and discount coupons and campaign information corresponding to the user's happiness are provided.

[0719] In this way, effective materials and advertisements can be created in response to the user's emotions.

[0720] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0721] Step 1:

[0722] The user inputs the scenario of the material using the terminal.

[0723] Input: The scenario text of the material the user wants to create.

[0724] Output: The scenario text is sent to the server.

[0725] Specific operations: The user enters the scenario of the material in text format through the terminal interface and presses the send button.

[0726] Step 2:

[0727] The server receives the scenario text, and the generative AI means analyzes the scenario.

[0728] Input: Scenario text.

[0729] Output: Analyzed scenario data.

[0730] Specific operation: The server analyzes the scenario text using natural language processing technology and extracts data for story generation.

[0731] Step 3:

[0732] A generative AI means generates a story based on the analyzed scenario data.

[0733] Input: Parsed scenario data.

[0734] Output: The generated story text.

[0735] Specific operation: The server uses a generative AI model to generate story text from the analyzed scenario data.

[0736] Step 4:

[0737] The server uses the data stored within the company as external information and operates the means for generating diagrams.

[0738] Input: The generated story text.

[0739] Output: A suitable diagram.

[0740] Specific operation: The server uses image recognition technology to select and generate illustrations related to the story text from internal documents.

[0741] Step 5:

[0742] The server provides the generated story and diagrams together to the user.

[0743] Input: Generated story text and figures.

[0744] Output: The material provided to the user.

[0745] Specific operation: The server integrates the generated story text and illustrations, creates materials to display to the user, and sends them to the terminal.

[0746] Step 6:

[0747] The terminal operates an emotion recognition means to analyze the user's emotions in real time.

[0748] Input: Video of the user's facial expression.

[0749] Output: Parsed emotion data.

[0750] Specific operation: The device's camera captures the user's facial expression and analyzes the emotion using an emotion recognition model.

[0751] Step 7:

[0752] The server operates a means for selecting and generating an appropriate illustration based on the analyzed emotion data.

[0753] Input: Parsed emotion data.

[0754] Output: Emotion-aware diagrams.

[0755] Specific operation: The server generates a prompt sentence based on the emotion data and generates an appropriate diagram using a generative AI model.

[0756] Step 8:

[0757] The server provides the user with a diagram corresponding to the emotion.

[0758] Input: Emotion-aware diagrams.

[0759] Output: The diagram provided to the user.

[0760] Specific operation: The server sends the generated diagram to the user's terminal and displays it.

[0761] As a specific example, if a user shows a surprised expression, the emotion recognition means detects the expression and generates a prompt sentence saying, "Please generate an advertisement that is optimal for a user who shows a surprised expression." Based on this prompt sentence, the generative AI means generates an advertisement for a new product or a special offer that corresponds to the surprise and provides it to the user.

[0762] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0763] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0765] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0766] [Second embodiment]

[0767] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0768] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0769] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32.

[0770] The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0771] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0772] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0773] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0774] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0775] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0776] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0777] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0778] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0779] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0780] "Example 1"

[0781] This embodiment of the present invention provides a text input interface as a means for the user to input the scenario of the material they wish to create. This interface is designed so that the user can freely write the scenario. For example, if the user inputs a scenario for a "new product presentation,"

[0782] "Example 2"

[0783] Generative AI uses natural language processing technology to analyze the input scenario. This analysis extracts keywords and themes from the scenario and generates a story based on them. For example, from a "new product presentation" scenario, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[0784] "Example 3"

[0785] The means of generating diagrams is to use documents stored within the company as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "features of a new product" or "comparison with competitors" are selected. These diagrams are provided as answers along with the generated story, and users can use this as a basis for creating their own documents.

[0786] The processing flow of each embodiment will be described below.

[0787] "Example 1"

[0788] Step 1: The user inputs a scenario through a text input interface. For example, the user inputs "New product presentation" as the scenario.

[0789] Step 2: Generative AI uses natural language processing technology to analyze the input scenario. Through this analysis, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted.

[0790] Step 3: Stories are generated based on the extracted keywords.

[0791] "Example 2"

[0792] Step 1: The generative AI analyzes the input scenario using natural language processing technology.

[0793] Step 2: A story is generated based on the keywords and themes extracted through the analysis.

[0794] Step 3: The generated story is provided to the user.

[0795] "Example 3"

[0796] Step 1: Use the data accumulated within the company as external information.

[0797] Step 2: Use image recognition technology to select the appropriate figure from the document.

[0798] Step 3: The selected diagrams are generated and provided as answers along with the generated stories.

[0799] Example 1

[0800] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0801] Conventional document creation systems required users to manually think about the content of documents and create diagrams, which required a great deal of time and effort. Furthermore, the quality and consistency of the documents depended on the skills of the users, making it difficult to create documents of uniform quality. Furthermore, there was a lack of effective ways to utilize documents stored within the company, and information was not reused sufficiently.

[0802] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0803] In this invention, the server includes: a means for inputting a scenario for a document the user wants to create; a means for generating a prompt sentence based on the input scenario; a means for sending the generated prompt sentence to a generative AI model to generate the document; a means for returning the generated document to the user; a means for using documents stored in-house as external information to generate diagrams; and a means for providing the generated document and diagram together as a response. This allows users to easily automatically generate high-quality documents. Furthermore, by effectively utilizing information stored in-house, information reuse is promoted, improving the efficiency of document creation.

[0804] "User" means any person or organization that uses the System to create Materials.

[0805] A "scenario" refers to text information that describes the content and structure of the material that a user wants to create.

[0806] "Prompt sentence" refers to an instruction sentence sent to a generative AI model based on a scenario.

[0807] "Generative AI model" refers to an artificial intelligence model that automatically generates materials based on input prompts.

[0808] "Documents" refers to documents, including text and diagrams, generated by a generative AI model.

[0809] "Diagram" refers to visual content generated based on materials accumulated within the company and external information.

[0810] "Server" refers to the computer system that manages the overall system processing, receives input from users, sends prompts to the generative AI model, and returns the generated materials.

[0811] "External information" refers to data obtained from sources other than materials stored within the company.

[0812] "Natural language processing technology" refers to the technology used to analyze scenarios and generate prompts and materials.

[0813] "Image recognition technology" refers to the technology used to select and generate appropriate images from materials stored within the company.

[0814] MODE FOR CARRYING OUT THE INVENTION

[0815] This invention is a system that automatically generates materials based on a scenario input by the user for the materials they want to create. This system is composed of multiple components, including a server, a terminal, and a generative AI model.

[0816] System configuration

[0817] 1. Server

[0818] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt sentences, and sends them to the generative AI model. It also receives the materials returned from the generative AI model and returns them to the user. The server uses a database to temporarily store scenarios and generated materials.

[0819] 2. Terminal

[0820] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and check generated materials.

[0821] 3. Generative AI Models

[0822] A generative AI model is an artificial intelligence model that automatically generates materials based on prompts sent from the server. For example, OpenAI's GPT-4 is used as a generative AI model.

[0823] Program processing

[0824] The server receives the scenario entered by the user through the device's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model, and is in the form of a specific instruction for the content of the scenario.

[0825] The generative AI model generates materials based on the received prompts. The materials are formatted according to the scenario content and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[0826] Specific examples

[0827] As a concrete example, consider the case where a user inputs "New product presentation" as a scenario. The user inputs "New product presentation" into the text input interface of the terminal. The server receives this scenario and generates the following prompt sentence:

[0828] Based on the scenario "New Product Presentation," create a presentation that includes the following:

[0829] 1. New Product Features

[0830] 2. Market analysis

[0831] 3. Comparison with competing products

[0832] 4. Sales Strategy

[0833] The generative AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are then sent back to the user via the server. The user can review the generated materials and make corrections or additions as necessary.

[0834] In this way, a system that automatically generates documents through collaboration between the server, terminals, and generative AI models is realized. This system allows users to easily create high-quality documents and effectively utilize information accumulated within the company.

[0835] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0836] Step 1:

[0837] The user inputs a scenario

[0838] The user uses the text input interface of the terminal to input the scenario of the material they want to create. The input scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and inputs "new product presentation" into the text box. Input: Scenario text (e.g., "new product presentation"). Output: Scenario text sent to the server.

[0839] Step 2:

[0840] The server receives the scenario.

[0841] The server receives the scenario sent by the user and saves it in the database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is saved in the database. Input: Scenario text sent by the user. Output: Scenario text saved in the database.

[0842] Step 3:

[0843] The server generates a prompt

[0844] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates the following prompt: Based on the "New Product Presentation" scenario, please create a presentation containing the following content: 1. New product features 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[0845] Step 4:

[0846] The server sends a prompt to the generative AI model

[0847] The server sends the generated prompt sentence to the generative AI model. Specifically, the server generates an API request and sends the request including the prompt sentence to the generative AI model's endpoint. Input: Generated prompt sentence. Output: Prompt sentence sent to the generative AI model.

[0848] Step 5:

[0849] Generative AI model generates materials

[0850] The generative AI model generates materials based on the received prompt. Specifically, the generative AI model analyzes the prompt and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: The prompt sent to the generative AI model. Output: The generated materials.

[0851] Step 6:

[0852] The server receives the generated material.

[0853] The server receives the materials generated from the generative AI model. Specifically, the server receives the API response and extracts the generated materials from the response body. The extracted materials are temporarily saved. Input: Materials sent from the generative AI model. Output: Generated materials temporarily saved.

[0854] Step 7:

[0855] The server returns the material to the user

[0856] The server returns the generated materials to the user. Specifically, the server generates an HTTP response and sends the response including the generated materials to the user's terminal. The user checks the materials in a web browser. Input: Generated materials temporarily saved. Output: Generated materials sent to the user's terminal.

[0857] (Application example 1)

[0858] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0859] Conventional material creation systems lack the functionality to automatically generate advertising materials based on a scenario entered by the user, which reduces the efficiency of advertising production. Furthermore, there was a need for the generated story and illustrations to be provided in a format that could be used as advertising materials, rather than simply being presented together as a response.

[0860] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0861] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated in-house as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story. This makes it possible to automatically generate advertising materials based on the scenario input by the user, thereby improving the efficiency of advertising production.

[0862] "User" means an individual or entity that uses the system to input material scenarios and receive generated stories and advertising materials.

[0863] A "scenario" is text information that describes the content and structure of the material that a user wants to create.

[0864] A "generative AI means" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[0865] "In-house stored materials" refers to a collection of data and information held internally by a company or organization.

[0866] "External information" refers to data and information obtained from outside the company, other than materials stored within the company.

[0867] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using materials accumulated within the company and external information.

[0868] A "generated story" is a narrative or exposition created based on a scenario by generative AI means.

[0869] "Advertising materials" are content such as images, videos, and text for advertising created based on the generated story.

[0870] "Answering means" refers to a system or method for providing generated stories, illustrations, and advertising materials to users.

[0871] A system for implementing this invention includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story.

[0872] System Program

[0873] The system is implemented using Python and uses OpenAI's GPT-3 as its generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are also automatically generated.

[0874] Processing Description

[0875] The server operates using the following hardware and software:

[0876] Hardware:

[0877] Smartphone or PC

[0878] software:

[0879] Python

[0880] OpenAI API

[0881] Data processing and calculation:

[0882] 1. Scenario input: The user inputs a scenario through a text input interface. For example, the user inputs a scenario such as "Presentation of a new product."

[0883] 2. Story generation: The input scenario is sent as a prompt to a generative AI model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[0884] 3. Diagram generation: Using internally accumulated data as external information, appropriate diagrams are generated using image recognition technology.

[0885] 4. Generate advertising materials: Generate advertising materials (images, videos, text) based on the generated story.

[0886] 5. Providing answers: Providing the generated stories, diagrams, and advertising materials to the user.

[0887] Specific examples

[0888] If a user enters "new product presentation," the following text is returned as an example of advertising material that might be generated:

[0889] Example prompt sentence:

[0890] Advertising scenario: New product presentation

[0891] Generate advertising materials based on this scenario.

[0892] Examples of generated advertising materials:

[0893] New product presentation

[0894] This new product is developed using the latest technology. It is both high-performance and easy to use, making your daily life more convenient. Buy it now and experience the future of life!

[0895] In this way, advertising materials can be automatically generated based on a scenario input by a user, thereby improving the efficiency of advertising production.

[0896] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0897] Step 1:

[0898] The user inputs a scenario. Using a text input interface, the user inputs the scenario of the material they want to create. For example, they input a scenario such as "Presentation of a new product." The input scenario is sent to the server.

[0899] Step 2:

[0900] The server receives the scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generative AI model. For example, it might be in the format "Advertising scenario: New product presentation. Please generate advertising materials based on this scenario."

[0901] Step 3:

[0902] The server sends a prompt to the generative AI model, which generates a story. The server uses OpenAI's GPT-3 API to send the prompt and receive the story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated that reads, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[0903] Step 4:

[0904] The server generates diagrams using documents stored in the company. The server uses image recognition technology to select and generate appropriate diagrams from documents stored in the company. For example, a diagram showing the technical specifications and usage of a new product is generated.

[0905] Step 5:

[0906] The server generates advertising materials (images, videos, text) based on the generated story. For example, advertising copy and product images are created based on the generated story.

[0907] Step 6:

[0908] The server provides the generated story, illustrations, and advertising materials to the user. The server transmits the generated story, illustrations, and advertising materials to the user, allowing the user to review them. The user can create an advertisement using the provided materials.

[0909] In this way, advertising materials can be automatically generated based on a scenario input by the user, thereby improving the efficiency of advertising production.

[0910] Example 2

[0911] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] In conventional document creation systems, even if users input a scenario, it was difficult to generate an appropriate story based on that scenario. Furthermore, the system lacked the functionality to automatically generate diagrams related to the generated story, requiring users to create the diagrams manually. This resulted in the problem of requiring a great deal of time and effort to create documents.

[0913] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting a scenario of a document that a user wants to create, a means for analyzing the input scenario and extracting keywords and themes, a generative AI means for generating a story based on the extracted keywords and themes, a means for generating diagrams using documents accumulated in-house as external information, and a means for responding with the generated story and diagrams together. This makes it possible to automatically generate diagrams related to a story based on the scenario input by the user, thereby efficiently creating documents.

[0914] "User" means an individual or organization that uses the System to create materials.

[0915] A "scenario" is a sentence or text that indicates the content and structure of the material that the user wants to create.

[0916] "Input means" refers to the interface or device that allows the user to input a scenario into the system.

[0917] "Means for analysis" is a function for analyzing the input scenario using natural language processing technology and extracting keywords and themes.

[0918] "Keywords" are words or phrases that are particularly important in a scenario.

[0919] The "theme" is the central theme or topic of the scenario.

[0920] "Generative AI methods" are artificial intelligence techniques for generating stories based on extracted keywords and themes.

[0921] "In-house stored materials" refers to data and information held internally by a company or organization.

[0922] "External information" refers to all information available to the system, including materials stored within the company.

[0923] "Means for generating diagrams" refers to a function that automatically creates appropriate diagrams based on materials accumulated within the company and external information.

[0924] The "means of response" is the interface or device that presents the generated story and diagrams to the user.

[0925] This invention relates to a system that inputs a scenario for a document that a user wants to create and automatically generates a story and related diagrams based on that scenario. A specific embodiment of this system will be described below.

[0926] System program generation

[0927] The server generates a program to implement the generative AI model. This program uses natural language processing technology to analyze the input scenario and extract keywords and themes. It also includes a function to generate a story based on the extracted keywords and themes.

[0928] Hardware and software used

[0929] The server implements natural language processing techniques using the Python programming language and the TensorFlow library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the Bidirectional Encoder Representations from Transformers (BERT) model. Based on the extracted keywords and themes, the server generates a story using the Generative Pre-trained Transformer 3 (GPT-3) model.

[0930] Specific examples

[0931] Consider the case where a user inputs, "Please generate a story for a presentation of our new product. Please include the new product's features, comparison with competitors, and market positioning." The scenario input from the device is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, based on these keywords, a story is generated using the GPT-3 model. The generated story might be, for example, "A presentation that highlights the new product's features, compares it with competitors, and explains its market positioning."

[0932] Prompt Sentence Examples

[0933] Examples of prompts that users can enter into a generative AI model include:

[0934] "Generate a story to present your new product. Include the new product's features, comparison with competitors, and market positioning."

[0935] In this way, the processing of the system's programs is explained while clarifying how the server, terminals, and users are involved.

[0936] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0937] Step 1:

[0938] The user inputs a scenario. The user inputs the scenario into the input field of the terminal. For example, the user might input, "Please generate a story for a presentation of a new product. Please include the features of the new product, a comparison with competitors, and its position in the market." The input scenario becomes the input data for the terminal.

[0939] Step 2:

[0940] The terminal sends the scenario to the server. The terminal sends the scenario entered by the user to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[0941] Step 3:

[0942] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (e.g., NLTK or SpaCy). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[0943] Step 4:

[0944] The server extracts keywords and themes. The server inputs the tokenized data into the BERT model to extract important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is the tokenized scenario, and the output data is the extracted keywords and themes.

[0945] Step 5:

[0946] The server generates a story based on the extracted keywords and themes. The server generates a story using the GPT-3 model based on the extracted keywords and themes. The generated story might be, for example, "a presentation that highlights the features of a new product, compares it with competitors, and explains its position in the market." The input data are the extracted keywords and themes, and the output data is the generated story.

[0947] Step 6:

[0948] The server sends the generated story to the terminal. The server packages the generated story in JSON format and sends it to the terminal as an HTTP response. The input data is the generated story, and the output data is the story sent to the terminal.

[0949] Step 7:

[0950] The terminal displays the story to the user. The terminal displays the received story on its user interface. The user can review the generated story and make corrections or additions as needed. The input data is the story sent to the terminal, and the output data is the story displayed to the user.

[0951] (Application example 2)

[0952] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0953] In the traditional advertising campaign creation process, scenario analysis, keyword extraction, and story generation are often done manually, which is time-consuming and labor-intensive. Furthermore, the generated stories can be inconsistent or not suited to the target audience, making it difficult to create effective ads.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating illustrations using documents accumulated in-house as external information, a means for responding with the generated story and illustrations together, and a means for analyzing the advertising campaign scenario and extracting keywords and themes to generate an advertising story. This makes it possible to automate the process from analyzing the advertising campaign scenario to generating the story, and to quickly create effective and consistent advertisements.

[0955] "User" means an individual or organization that uses the system to create materials and advertising campaign scripts.

[0956] A "scenario" is text information entered by a user for the purpose of creating an advertising campaign or materials, and serves as the basis for generating a story.

[0957] "Generative AI means" is an artificial intelligence technology that analyzes an input scenario and generates a story using natural language processing technology.

[0958] "In-house accumulated materials" refers to a collection of documents and data created in the past within a company or organization, and are used as external information.

[0959] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company and external information.

[0960] "Answering means" refers to the methods and techniques used to present the generated stories and diagrams to users.

[0961] An "advertising campaign" is a series of marketing activities designed to promote a particular product or service.

[0962] "Keywords" are important words and phrases extracted from a scenario and are the basis for generating a story.

[0963] The "theme" is the central theme or topic of a scenario and determines the direction of the story.

[0964] An "advertising story" is a coherent narrative that is tailored to the objectives of an advertising campaign and appeals to a target audience.

[0965] A system for implementing this invention is configured as follows: First, a terminal is required for inputting the materials and advertising campaign scenario that the user wants to create. This terminal is a device such as a PC or smartphone, and provides an interface for the user to input the scenario.

[0966] Next, the server receives the input scenario and analyzes it using a generative AI tool. This analysis uses natural language processing technology to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. This generative AI uses, for example, the OpenAI API.

[0967] Furthermore, the server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated story and diagrams are then provided to the user's device as a response.

[0968] As a concrete example, consider the case where a user inputs a scenario for a presentation of a new product. This scenario includes information such as "features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates a prompt sentence like the following:

[0969] Example prompt sentence:

[0970] Scenario: A new product presentation highlights the product's features, how it compares to competitors, and its position in the market.

[0971] Keywords: new product, presentation, features, competitors, market

[0972] Generate an advertising story based on this scenario.

[0973] If we feed this prompt into a generative AI model, it might generate an advertising story like this:

[0974] An example of a generated advertising story:

[0975] The new X-100 is equipped with innovative features not found in other products. Compared to its competitors, the X-100 offers better performance and cost-effectiveness. Its market positioning makes it the perfect choice for users seeking cutting-edge technology. Get your X-100 now and experience the future.

[0976] In this way, users can easily create effective advertising stories. This system automates the process from analyzing advertising campaign scenarios to generating stories, enabling users to quickly create effective and consistent advertisements.

[0977] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0978] Step 1:

[0979] The user uses a terminal to input the scenario for the materials or advertising campaign they want to create, and the input scenario is sent to the server in text format.

[0980] Step 2:

[0981] The server analyzes the received scenario. This analysis uses natural language processing technology. Specifically, it extracts keywords and themes from the scenario. The input for this process is the text data of the scenario, and the output is a list of extracted keywords and themes.

[0982] Step 3:

[0983] The server generates a prompt based on the extracted keywords and topics. The generated prompt is then input to a generative AI model. The input to this process is a list of keywords and topics, and the output is a prompt.

[0984] Step 4:

[0985] The server uses a generative AI model to generate a story based on the prompt. Specifically, it uses OpenAI's API to generate the story. The input of this process is the prompt, and the output is the generated story.

[0986] Step 5:

[0987] The server uses internally stored documents as external information to generate diagrams. It uses image recognition technology to select and generate appropriate diagrams. The input for this process is internal document data, and the output is the generated diagram.

[0988] Step 6:

[0989] The server provides the generated story and diagram together to the user's terminal as a response. The input of this process is the generated story and diagram, and the output is the response data sent to the user's terminal.

[0990] Step 7:

[0991] The user checks the story and illustrations provided through the terminal, and makes corrections or additions as necessary to complete the final materials or advertising campaign content. The input to this process is the response data sent from the server, and the output is the completed materials or advertising campaign content.

[0992] Example 3

[0993] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0994] Conventional document creation systems require users to manually search for documents, select appropriate figures, and create stories, which takes time and effort. Selecting appropriate figures also requires specialized knowledge, making it difficult for users to create documents efficiently. Another issue is the inconsistency in the quality of the generated documents.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0996] In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for using documents stored in the company as external information and selecting and generating appropriate illustrations using image recognition technology, and a means for responding with the generated story and illustrations together, thereby enabling users to efficiently create high-quality documents.

[0997] "User" means an individual or organization that uses the System to create materials.

[0998] A "scenario" is text information that indicates the content and purpose of the material that a user wants to create.

[0999] "Generative AI means" refers to means that use artificial intelligence technology to generate a story based on an input scenario.

[1000] "In-house stored materials" refers to information resources such as documents, images, and data stored within a company or organization.

[1001] "External information" refers to materials accumulated within the company, which are sources of information for use by the system.

[1002] "Image recognition technology" is a technology for analyzing images and diagrams and understanding their contents.

[1003] "Appropriate illustrations" are diagrams or images selected to visually represent information relevant to the scenario.

[1004] The "means for generating" is a means for creating a new diagram based on a selected diagram.

[1005] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[1006] The term "system" refers to a computer-based device or software for carrying out a series of processes including the above-mentioned means.

[1007] The present invention relates to a system that enables users to efficiently create high-quality materials. A specific embodiment of this system will be described below.

[1008] System Overview

[1009] The system includes the following main means:

[1010] 1. A way for users to input the scenario of the material they want to create

[1011] 2. Generative AI means to generate stories based on input scenarios

[1012] 3. Using internally stored data as external information, and using image recognition technology to select and generate appropriate diagrams.

[1013] 4. A means of answering questions with generated stories and diagrams

[1014] Hardware and software used

[1015] The server runs the system using the following hardware and software:

[1016] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[1017] Software: Google Cloud Vision API, Amazon Rekognition, Adobe Illustrator, Canva, generative AI models (e.g., OpenAI's GPT-4)

[1018] Data processing and calculation

[1019] The server processes and calculates data in the following steps:

[1020] 1. Collecting materials:

[1021] The server accesses internal databases and file servers to search and retrieve relevant materials, including PDFs, Word documents, presentation files, and more.

[1022] 2. Application of image recognition technology:

[1023] The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and diagrams in the collected materials.

[1024] 3. Selecting the right diagram:

[1025] Based on the analysis results, the server filters and selects appropriate diagrams related to the scenario.

[1026] 4. Figure Generation:

[1027] The server uses Adobe Illustrator or Canva to generate a new diagram based on the selected diagram.

[1028] 5. Story generation:

[1029] The server uses a generative AI model (GPT-4) to generate a story based on the generated diagrams.

[1030] 6. Providing answers:

[1031] The server compiles the generated diagrams and stories into a single document and provides it to the user.

[1032] Specific examples

[1033] For example, if a user wants to create a document explaining the "features of a new product," they can input the following prompt into the generative AI model:

[1034] "Please generate a diagram to explain the features of a new product. Please select and generate an appropriate diagram based on internal company materials."

[1035] When this prompt is entered, the server generates a diagram according to the procedure described above and provides it along with the story. Users can then create materials based on the answers.

[1036] In this way, the user can efficiently create high-quality materials. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1037] Step 1: Gather your materials

[1038] The server accesses the company's databases and file servers to search for and retrieve relevant materials. As input, the user is given a scenario for the material they want to create. The server extracts keywords based on this scenario and uses these keywords to search for materials. As output, it obtains related materials such as PDFs, Word documents, and presentation files.

[1039] Step 2: Applying image recognition technology

[1040] The server applies image recognition technology to the collected materials. The materials collected in step 1 are provided as input. The server uses Google Cloud Vision API and Amazon Rekognition to analyze the images and diagrams within the materials. Specifically, the server processes each document sequentially and extracts the contents of the images and diagrams as text data. The analysis results are obtained as output.

[1041] Step 3: Select the right diagram

[1042] The server uses image recognition technology to select appropriate diagrams from the documents. The analysis results obtained in step 2 are given as input. The server matches the analysis results with keywords related to the scenario, and filters and selects highly relevant diagrams. Specifically, the server calculates a relevance score and selects diagrams with high scores. The selected diagrams are obtained as output.

[1043] Step 4: Generate the diagram

[1044] The server generates a new diagram based on the selected diagram. As input, it receives the diagram selected in step 3. The server uses Adobe Illustrator or Canva to create a new diagram using the selected diagram as a template and adding the necessary information. Specifically, the server adjusts the diagram layout and adds annotations and data. As output, it receives the generated new diagram.

[1045] Step 5: Generate stories

[1046] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related explanations and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[1047] Step 6: Provide your answers

[1048] The server compiles the generated diagrams and stories into a single document and provides it to the user. As input, the diagram generated in step 4 and the story generated in step 5 are given. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The final document is obtained as output.

[1049] (Application example 3)

[1050] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1051] Maintenance work in factories is complex and diverse, making it difficult for workers to quickly grasp the appropriate procedures. Furthermore, the sheer volume of maintenance records and technical documents makes it difficult to efficiently search and use the necessary information. For this reason, there is a demand for improved efficiency and accuracy in maintenance work.

[1052] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1053] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents stored in-house as external information; means for responding with the generated story and diagrams together; means for selecting and generating appropriate diagrams using image recognition technology using maintenance records and technical documents stored in the factory as external information; and means for supporting maintenance work based on the selected diagrams. This enables maintenance workers to quickly obtain the information they need and perform their work efficiently and accurately.

[1054] "User" refers to a person who uses the system to create materials and perform maintenance work.

[1055] A "scenario" refers to a plan or structure that indicates the content and purpose of the material that the user wants to create.

[1056] "Generative AI means" refers to artificial intelligence technology that generates a story based on an input scenario.

[1057] "Documents accumulated within the company" refers to information such as technical documents and maintenance records that have been created within the company in the past.

[1058] "External information" refers to materials and data accumulated within the company for use by the system.

[1059] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company.

[1060] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[1061] "Maintenance records" refers to information that records the history and details of maintenance work performed within a factory.

[1062] "Technical documents" refers to technical information and procedures related to equipment and systems within a factory.

[1063] The term "means for supporting maintenance work" refers to a support technology that enables a maintenance worker to perform work efficiently and accurately based on the selected diagram.

[1064] A system for carrying out the present invention is configured as follows: First, an interface is provided for a user to input a scenario for a material that the user wishes to create. The user inputs the scenario through this interface.

[1065] The server then generates a story based on the input scenario using a generative AI mechanism that uses natural language processing to analyze the scenario and generate an appropriate story. For example, the Hugging Face transformers library can be used as a generative AI mechanism.

[1066] Furthermore, the server uses internally stored documents as external information to generate diagrams. Image recognition technology is used to generate diagrams. Specifically, software such as OpenCV, PIL (Python Imaging Library), and pytesseract are used to extract and analyze text and features from images.

[1067] Maintenance records and technical documents stored within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[1068] The generated story and diagram are provided to users together, allowing them to quickly obtain the information they need and work efficiently and accurately.

[1069] As a concrete example, the following prompt sentence can be used:

[1070] Example prompt sentence:

[1071] "Generate a story about the part replacement procedure based on the following images: ['maintenance_step1.png', 'maintenance_step2.png']"

[1072] Based on this prompt, the generative AI means generates a story about the part replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[1073] In this way, the efficiency of maintenance work for factory robots can be improved and the burden on workers can be reduced.

[1074] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1075] Step 1:

[1076] The user inputs the scenario of the material they want to create.

[1077] Input: A scenario in which the user provides input through an interface.

[1078] Output: The input scenario data.

[1079] Specific operation: The user accesses the system interface and inputs the scenario of the material they want to create in text format.

[1080] Step 2:

[1081] The server generates a story based on the input scenario.

[1082] Input: The entered scenario data.

[1083] Output: The generated stories.

[1084] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generate an appropriate story using natural language processing technology.

[1085] Step 3:

[1086] The server uses materials stored within the company as external information to generate diagrams.

[1087] Input: Materials accumulated within the company.

[1088] Output: The generated diagram.

[1089] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate diagrams from materials stored within the company.

[1090] Step 4:

[1091] The server uses maintenance records and technical documents stored within the factory as external information and uses image recognition technology to select and generate appropriate diagrams.

[1092] Input: Maintenance records and technical documents accumulated within the factory.

[1093] Output: Selected diagrams.

[1094] Specific operation: The server uses image recognition technology to select and generate the necessary diagrams from maintenance records and technical documents.

[1095] Step 5:

[1096] The server responds with the generated story and diagrams.

[1097] Input: The generated story and selected diagrams.

[1098] Output: The story and diagrams provided to the user.

[1099] Specific operation: The server combines the generated story with the selected diagrams and provides it to the user, who can then create materials based on it.

[1100] Step 6:

[1101] The maintenance work is carried out based on the stories and diagrams provided by the user.

[1102] Input: The story and illustrations provided.

[1103] Output: Efficient and accurate maintenance work.

[1104] Specific behavior: The user performs maintenance tasks efficiently and accurately while referring to the provided stories and diagrams.

[1105] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1106] "Example 1"

[1107] As one embodiment of the present invention, a system is provided that includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagram together, and an emotion engine that recognizes the user's emotions.

[1108] "Example 2"

[1109] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a happy expression, the emotion engine captures that information and the generative AI generates a story based on that emotion. Specifically, if the emotion of joy is recognized, the generative AI generates a story that includes positive elements.

[1110] "Example 3"

[1111] The emotion engine also selects and generates appropriate images according to the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures that information, and the image generation means selects and generates an image that conveys surprise and freshness.

[1112] The processing flow of each embodiment will be described below.

[1113] "Example 1"

[1114] Step 1: The user inputs the scenario of the material they want to create.

[1115] Step 2: Based on the input scenario, the generative AI generates a story.

[1116] Step 3: Use the data accumulated within the company as external information to generate the diagram.

[1117] Step 4: Answer the generated story and diagram together.

[1118] Step 5: The emotion engine that recognizes the user's emotions is activated.

[1119] "Example 2"

[1120] Step 1: The emotion engine recognizes the user's emotion from their tone of voice and facial expression.

[1121] Step 2: The generative AI generates a story based on the emotions recognized by the emotion engine.

[1122] "Example 3"

[1123] Step 1: The emotion engine recognizes the user's emotion.

[1124] Step 2: Select and generate an appropriate illustration according to the emotion recognized by the emotion engine.

[1125] Example 1

[1126] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1127] Conventional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which requires a great deal of time and effort. Furthermore, they do not provide feedback that takes into account the user's feelings, which reduces the efficiency and quality of document creation. Furthermore, there is a lack of effective ways to utilize the documents accumulated within the company, which can lead to a lack of consistency and reliability in the documents.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1129] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback according to the user's emotions, and enables effective use of internal documents.

[1130] "Means for inputting the scenario of the material that the user wants to create" refers to an interface that allows the user to freely input text, and is a means for inputting the scenario of the material.

[1131] "Generative AI means" refers to artificial intelligence technology used to generate a story based on an input scenario, and is a means of analyzing the scenario and generating a story using natural language processing technology.

[1132] "Means of generating diagrams using materials accumulated within the company as external information" refers to means of generating diagrams by referencing materials stored in an internal database and using data visualization technology.

[1133] "Means for providing answers using the generated story and diagrams together" refers to a means for integrating the generated story and diagrams and providing them to the user as a single document.

[1134] An "emotion engine" refers to technology that analyzes user input and reactions and recognizes emotions. It is a means of analyzing user emotions using natural language processing technology and providing appropriate feedback.

[1135] This invention is a system that inputs a scenario for a document that a user wants to create, generates a story and illustrations based on that scenario, and finally provides the document as a single document. A specific embodiment of this system will be described below.

[1136] System configuration

[1137] The system consists of the following main components:

[1138] 1. Text Input Interface

[1139] 2. Generative AI means

[1140] 3. Database Reference Method

[1141] 4. Data Visualization Methods

[1142] 5. Integration Methods

[1143] 6. Emotion Engine

[1144] Text Input Interface

[1145] The user uses the text input interface of the terminal to input the scenario of the material they want to create. This interface is designed so that the user can input text freely. For example, the user can input "New product presentation."

[1146] Generative AI means

[1147] The server receives the scenario entered by the user and inputs it as a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates an appropriate story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1148] Database Reference Method

[1149] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib and Pandas to format and visualize the data.

[1150] Data visualization tools

[1151] The server uses data visualization techniques to generate diagrams based on the information retrieved from the database. For example, it uses SQL queries to extract the necessary information from the database and generates graphs using Python's Matplotlib. The generated graphs are saved as image files.

[1152] Integration Methods

[1153] The server integrates the generated stories and figures into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX or Microsoft Word API). The generated document is saved as a PDF file.

[1154] Emotion Engine

[1155] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1156] Examples of specific examples and prompts

[1157] Specific examples

[1158] The user enters "New Product Presentation" into the text entry interface.

[1159] The server uses a generative AI model to generate a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1160] The server references past sales data and market research results from an internal database and creates relevant graphs and charts.

[1161] Finally, the generated stories and diagrams are integrated and provided to the user.

[1162] Prompt Sentence Examples

[1163] "Please enter a scenario for a new product presentation. For example, please include the following: new product features, target market, competitive analysis, sales strategy, etc."

[1164] In this way, users can easily create high-quality materials.

[1165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1166] Step 1:

[1167] The user inputs a scenario.

[1168] The user uses the device's text input interface to input the scenario of the document they want to create, for example, "New product presentation." The input scenario is sent to the server in real time.

[1169] Input: User-entered scenario text

[1170] Output: Scenario text sent to the server

[1171] Step 2:

[1172] The server generates the story using a generative AI model.

[1173] The server inputs the received scenario as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1174] Input: Scenario text

[1175] Output: Generated story text

[1176] Step 3:

[1177] The server references the company's internal database to generate the diagram.

[1178] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib and Pandas to format and visualize the data.

[1179] Input: Data obtained from our internal database

[1180] Output: Image files of generated graphs and charts

[1181] Step 4:

[1182] The server integrates the generated stories and diagrams.

[1183] The server integrates the generated stories and figures into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX or Microsoft Word API). The generated document is saved as a PDF file.

[1184] Input: Generated story text, image files of graphs and charts

[1185] Output: Flattened PDF document

[1186] Step 5:

[1187] The server recognizes the user's emotions and provides feedback.

[1188] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1189] Input: User feedback text

[1190] Output: Analysis results by the emotion engine and feedback text

[1191] (Application example 1)

[1192] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1193] The traditional process of creating advertising campaigns required a lot of time and effort, including creating a scenario, story, and visuals. It was also difficult to propose optimal advertising materials that took into account user emotions, making it difficult to quickly create effective advertising campaigns. This meant that advertising agencies and marketers were unable to work efficiently, making it difficult to maximize the effectiveness of their advertising.

[1194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1195] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents accumulated in-house as external information; means for responding with the generated story and diagrams together; means including an emotion engine for recognizing the user's emotions; and means for generating visuals for advertising based on the generated story. This makes it possible to streamline the advertising campaign creation process and propose optimal advertising materials according to the user's emotions.

[1196] "User" means an individual or organization that uses the system to create a scenario for a material.

[1197] A "scenario" is text information that describes the content and structure of the material that a user wants to create.

[1198] "Generative AI means" refers to means that use artificial intelligence technology to generate a story based on an input scenario.

[1199] "In-house accumulated materials" refers to a collection of documents and data that have been created in the past within a company or organization.

[1200] "Means for generating diagrams" refers to the technology for creating appropriate diagrams using materials accumulated within the company.

[1201] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[1202] An "emotion engine" is a technology that recognizes the user's emotions and makes appropriate suggestions based on them.

[1203] "Advertising Visuals" are images and graphics used in advertising campaigns.

[1204] The system for implementing this invention has the following configuration: First, a text input interface is provided for the user to input the scenario of the material they wish to create. Using this interface, the user can freely write the scenario.

[1205] Next, a generative AI method is used to generate a story based on the input scenario. This generative AI method analyzes the scenario using natural language processing technology and generates an appropriate story. Specifically, it uses OpenAI's API to generate a story from the scenario.

[1206] Based on the generated story, a means of generating diagrams is run using internally stored materials. This means uses image recognition technology to select and generate appropriate diagrams from internal materials. PIL (Python Imaging Library) is used to create visuals that match the generated story.

[1207] Additionally, a means is included to present the generated story and diagrams together to the user, integrating the generated content and providing a consistent answer to the user.

[1208] It also includes a means for including an emotion engine to recognize user emotions. This emotion engine analyzes user emotions and suggests optimal advertising materials based on the analysis. The emotion_recognition library is used to recognize user emotions.

[1209] Finally, a means for generating advertising visuals based on the generated story is included, which generates images and graphic materials for use in an advertising campaign based on the generated story.

[1210] As a concrete example, if a user inputs a scenario such as "an advertising campaign for a new smartphone product," the generative AI will generate a story like the one below.

[1211] Example prompt sentence:

[1212] Advertising Campaign Scenario: Advertising campaign for a new smartphone product

[1213] story:

[1214] By inputting this prompt into a generative AI model, an appropriate story is generated, and based on the generated story, visuals for the ad are created and provided to the user.

[1215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1216] Step 1:

[1217] The user inputs the scenario of the material they want to create.

[1218] The user uses a text input interface to freely describe the scenario of an advertising campaign, and the input scenario is sent to the system.

[1219] Input: User-entered scenario text

[1220] Output: The scenario text is sent to the system

[1221] Step 2:

[1222] The server generates a story based on the input scenario.

[1223] The server uses generative AI tools to analyze the input scenario and generate an appropriate story. Specifically, it uses OpenAI's API to generate a story from the scenario.

[1224] Input: Scenario text

[1225] Output: Generated story text

[1226] Step 3:

[1227] The server generates diagrams based on the generated stories.

[1228] The server uses the data stored in-house to select and generate appropriate illustrations using image recognition technology, and uses PIL (Python Imaging Library) to create visuals that match the generated story.

[1229] Input: Generated story text

[1230] Output: Generated diagram (visual)

[1231] Step 4:

[1232] The server responds with the generated story and diagrams.

[1233] The server integrates the generated stories and diagrams to provide a consistent answer to the user.

[1234] Input: Generated story text, generated illustrations

[1235] Output: Integrated story and diagrams

[1236] Step 5:

[1237] The server recognizes the user's emotions.

[1238] The server uses the emotion engine to analyze the user's emotions. The emotion_recognition library is used to recognize the user's emotions.

[1239] Input: User's emotional data (e.g., facial expression images and voice data)

[1240] Output: Recognized emotion information

[1241] Step 6:

[1242] The server generates visuals for advertising based on the generated story.

[1243] The server then creates images and graphic materials for use in advertising campaigns based on the generated stories.

[1244] Input: Generated story text, recognized emotion information

[1245] Output: Visuals for advertising

[1246] Step 7:

[1247] The server provides the final advertising campaign materials to the user.

[1248] The server integrates the generated stories, illustrations, and advertising visuals and provides them to the user.

[1249] Input: Generated story text, generated illustrations, advertising visuals

[1250] Output: Final advertising campaign materials

[1251] Example 2

[1252] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1253] Conventional document creation systems can generate stories based on scenarios entered by users, but they have the problem of not being able to adjust the story to take the user's emotions into account. Also, when providing the generated story and illustrations together, they do not make appropriate adjustments based on the user's emotions, making it difficult to create documents that reflect the user's intentions and emotions.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1255] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating diagrams using documents accumulated in-house as external information, means for responding with the generated story and diagrams, means for sensing the user's tone of voice and facial expression, means for recognizing the user's emotions using an emotion engine, and means for adjusting the story based on the recognized emotions. This makes it possible to provide a story and diagrams that reflect the user's emotions.

[1256] "User" refers to a person who uses the system to input a material scenario and receive the generated stories and illustrations.

[1257] "Scenario" refers to input information in text format that indicates the content and structure of the material that the user wants to create.

[1258] "Generative AI means" refers to artificial intelligence technology for generating stories based on input scenarios.

[1259] "Natural language processing technology" refers to technology for analyzing scenarios and extracting keywords and themes.

[1260] "Means for generating diagrams" refers to the technology for generating appropriate diagrams using materials accumulated within the company as external information.

[1261] "Means of sensing" refers to technology that senses the user's tone of voice and facial expressions in real time.

[1262] "Emotion engine" refers to technology that recognizes a user's emotions from detected tone of voice and facial expressions.

[1263] "Adjustment means" refers to techniques for appropriately adjusting the generated story based on the recognized emotions.

[1264] "Story" refers to a series of sentences or content generated by generative AI means based on a scenario.

[1265] "Means of answering" refers to the technology used to present the generated stories and diagrams to users.

[1266] This invention is a system that inputs a scenario for a document that a user wants to create, generates a story based on that scenario, and provides a story that reflects the user's emotions. A specific embodiment of this system will be described below.

[1267] Hardware and software used

[1268] 1. Server:

[1269] Natural language processing techniques: Using generative AI models such as BERT and GPT-3.

[1270] Emotion engine: Software for analyzing the user's tone of voice and facial expressions.

[1271] Database: A database for storing data accumulated within the company and for use as external information.

[1272] 2. Terminal:

[1273] Input interface: A keyboard or touch screen for users to input scenarios.

[1274] Camera and microphone: Hardware that captures your tone of voice and facial expressions.

[1275] Data processing and calculation

[1276] 1. Scenario input and analysis:

[1277] The user uses the input interface of the terminal to input the scenario in text format.

[1278] The server receives the input scenario and analyzes it using natural language processing techniques, specifically using generative AI models such as BERT and GPT-3 to extract keywords and themes from the scenario.

[1279] 2. Story generation:

[1280] The server uses a generative AI model to generate a story based on the extracted keywords and themes. For example, if a scenario for a "new product presentation" is input, the server will extract keywords such as "new product features," "comparison with competitors," and "market positioning," and generate a story based on them.

[1281] 3. Emotional awareness and story adjustment:

[1282] The device uses a camera and microphone to detect the user's tone of voice and facial expressions in real time.

[1283] The device uses an emotion engine to recognize the user's emotions from the detected tone of voice and facial expressions. For example, if the user shows a happy expression, the device sends that information to the emotion engine, which then recognizes the emotion as happy.

[1284] The server adjusts the generated story based on the recognized emotion, for example, adding positive elements to the story if positive emotion is recognized.

[1285] 4. Final Story Delivery:

[1286] The server then provides the final adjusted story to the user, specifically by sending the generated story and diagrams together.

[1287] Examples of specific examples and prompts

[1288] Specific examples

[1289] Scenario: New product presentation

[1290] Extracted keywords: new product features, comparison with competitors, market positioning

[1291] The story generated: A presentation that highlights the new product's features, demonstrates its competitive advantage, and clarifies its position in the market.

[1292] Prompt Sentence Examples

[1293] "Enter a scenario for a new product presentation. Generative AI will analyze the scenario, extract keywords, and generate a story."

[1294] "Detect tone of voice and facial expressions to recognize the user's emotions. Generative AI will generate a story based on the emotions recognized by the emotion engine."

[1295] In this way, the system can generate and deliver the best story based on the user's input and emotions.

[1296] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1297] Step 1:

[1298] The user inputs a scenario.

[1299] Input: The user inputs the scenario in text format using the terminal's input interface.

[1300] What happens: The user types "New Product Presentation."

[1301] Output: The input scenario is saved on the device and sent to the server.

[1302] Step 2:

[1303] The server receives the scenario and analyzes it using natural language processing technology.

[1304] Input: Scenario text sent from the terminal.

[1305] What happens: The server uses a generative AI model such as BERT or GPT-3 to parse the text "New Product Presentation."

[1306] Output: Keywords and themes extracted from the scenario.

[1307] Step 3:

[1308] The server extracts keywords and themes from the scenario.

[1309] Input: Scenario text parsed using natural language processing techniques.

[1310] Specific operation: The server extracts the keywords "new product features," "comparison with competitors," and "market positioning."

[1311] Output: A list of extracted keywords or themes.

[1312] Step 4:

[1313] The server generates stories based on the extracted keywords and themes.

[1314] Input: A list of extracted keywords or themes.

[1315] What it does: The server uses the generative AI model to generate a presentation that highlights the new product's features, demonstrates its advantages over competitors, and clarifies its position in the market.

[1316] Output: The generated stories.

[1317] Step 5:

[1318] The device detects the user's tone of voice and facial expressions.

[1319] Input: The user's real-time tone of voice and facial expressions.

[1320] What it does: The device uses the camera and microphone to detect whether the user is smiling or has a positive tone of voice.

[1321] Output: Detected voice tone and facial expression data.

[1322] Step 6:

[1323] The terminal recognizes the user's emotions using an emotion engine.

[1324] Input: Detected voice tone and facial expression data.

[1325] Specific operation: The device uses the emotion engine to recognize the user's emotion of joy.

[1326] Output: Recognized emotion data.

[1327] Step 7:

[1328] The server adjusts the story based on the perceived sentiment.

[1329] Input: Recognized emotion data and generated stories.

[1330] What it does: The server adds positivity to the story, creating a more upbeat presentation.

[1331] Output: A final, adjusted story.

[1332] Step 8:

[1333] The server provides the final story to the user.

[1334] Input: The final, adjusted story.

[1335] What it does: The server displays to the user "a positive-toned presentation that highlights the new product's features, demonstrates its advantages over competitors, and clarifies its position in the market."

[1336] Output: The final story delivered to the user.

[1337] (Application example 2)

[1338] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1339] Conventional document creation systems can generate stories based on scenarios entered by users, but they cannot generate stories that take the user's emotions into account, making it difficult to create documents that reflect the user's intentions and emotions.In addition, when responding with the generated story and diagrams, it is not possible to reflect the user's emotions, so more effective document creation is needed.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1341] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents stored in the company as external information; means for responding with the generated story and diagrams together; emotion engine means for recognizing emotions from the user's tone of voice and facial expression; and means for generating a story based on the recognized emotions. This makes it possible to generate a story that reflects the user's emotions, enabling more effective document creation.

[1342] "User" refers to a person who uses the system to create materials.

[1343] A "scenario" refers to a text or plot that indicates the content and structure of the material that the user wants to create.

[1344] "Generative AI" refers to artificial intelligence technology that generates a story based on an input scenario.

[1345] "In-house stored materials" refers to data and information collected and stored within a company or organization.

[1346] "External information" refers to data obtained from sources other than materials stored within the company.

[1347] "Figures" refers to visual elements such as graphs, charts, and illustrations included in the document.

[1348] "Emotion engine" refers to technology that recognizes emotions from the user's tone of voice and facial expressions.

[1349] "Voice tone" refers to the audio characteristics of the user's voice, such as pitch, strength, and rhythm.

[1350] "Facial expressions" refer to the user's facial movements and expressions.

[1351] A "story" refers to a series of sentences or content generated based on a scenario.

[1352] "Answer" refers to the act of providing the generated story and diagram to the user.

[1353] A system for implementing this invention includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents accumulated within the company as external information; means for responding with the generated story and diagrams together; emotion engine means for recognizing emotions from the user's tone of voice and facial expression; and means for generating a story based on the recognized emotions.

[1354] System Program

[1355] The server runs the following program: First, the user inputs a scenario using a device such as a smartphone or PC. Next, the generative AI analyzes the scenario using natural language processing technology and extracts keywords and themes. It then generates a story based on the extracted information.

[1356] Furthermore, the emotion engine recognizes the user's tone of voice and facial expressions, and provides this emotional information to the generative AI. The generative AI then adjusts the story taking the emotional information into account, generating content that is more in line with the user's intentions. The generated story is complemented by a means of generating appropriate diagrams using internally accumulated materials as external information. Finally, the generated story and diagrams are provided to the user together.

[1357] Hardware and software used

[1358] Hardware: smartphones, PCs, servers

[1359] Software: OpenCV (image processing), Transformers (generative AI and emotion recognition), TextBlob (text analysis)

[1360] Data processing and calculation

[1361] 1. Image processing: Images taken with a smartphone or computer camera are used to perform facial recognition and analyze facial expressions using OpenCV.

[1362] 2. Speech processing: Speech recorded using a smartphone or computer microphone is converted into text and emotions are recognized using an emotion engine.

[1363] 3. Story generation: Use generative AI (such as GPT-3) to generate stories based on recognized emotions and scenarios.

[1364] 4. Diagram generation: Select and generate appropriate diagrams from the internal data.

[1365] Specific examples

[1366] For example, if a user wants to create an ad about the camera features of a new smartphone, they might enter the following prompt:

[1367] Prompt Sentence Examples

[1368] Create an advertisement story for a product. The user is feeling happy and the sentiment is positive. The product information is: A new smartphone with advanced camera features.

[1369] Based on this prompt, the generative AI generates a positive advertising story that highlights the new smartphone's camera features. The emotion engine recognizes the user's happy facial expression and provides that information to the generative AI, which then generates a more emotionally relevant advertising story. The generated story then uses internally stored data to generate appropriate illustrations, which are then ultimately provided to the user.

[1370] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1371] Step 1:

[1372] The user inputs a scenario using a terminal.

[1373] Input: Scenario of the material the user wants to create (text format)

[1374] Output: Input scenario (text data)

[1375] Specific operation: The user inputs a scenario into the input screen of the terminal and presses the send button. The terminal then sends the input scenario to the server.

[1376] Step 2:

[1377] The server uses generative AI to analyze the scenario and extract keywords and themes.

[1378] Input: Input scenario (text data)

[1379] Output: Extracted keywords and themes (text data)

[1380] How it works: The server uses generative AI (natural language processing) to analyze the scenario and extract key keywords and themes. The extracted information is used in the next step.

[1381] Step 3:

[1382] The server uses an emotion engine to recognize the user's tone of voice and facial expressions.

[1383] Input: User's voice and image data

[1384] Output: Recognized emotion (text data)

[1385] Specific operation: The user records their voice and facial expressions using the device's camera and microphone. The server uses OpenCV to analyze facial expressions from the image data and the emotion engine to analyze the tone of voice from the audio data. The recognized emotion is generated as the analysis result.

[1386] Step 4:

[1387] The server generates a story based on the recognized emotions.

[1388] Input: extracted keywords, themes, and recognized sentiment

[1389] Output: Generated story (text data)

[1390] How it works: The server uses generative AI to generate a story based on the extracted keywords, themes, and recognized emotions. The generated story is used in the next step.

[1391] Step 5:

[1392] The server uses data stored within the company to generate appropriate diagrams.

[1393] Input: Generated story (text data)

[1394] Output: Generated diagram (image data)

[1395] Specific operation: The server uses image recognition technology to select and generate appropriate diagrams from internally stored data. The generated diagrams are used in the next step.

[1396] Step 6:

[1397] The server provides the generated story and diagrams together to the user.

[1398] Input: Generated story (text data), generated diagram (image data)

[1399] Output: Materials provided to users (text data and image data)

[1400] Specific operation: The server compiles the generated story and diagrams into a single document and sends it to the user's device, where the user can view the document.

[1401] Example 3

[1402] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1403] Conventional document creation systems required users to create documents manually, which required a great deal of time and effort. Furthermore, there was no function to select appropriate images based on the user's feelings, which meant that the content of the document sometimes did not match the user's intentions. Furthermore, there was a lack of a way to effectively utilize the information accumulated within the company, which could lead to a decline in the quality of the documents.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1405] In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using internally stored information as external information, a means for responding with the generated story and diagrams together, a means for analyzing the user's emotions, and a means for selecting appropriate diagrams based on the analyzed emotions, thereby enabling users to efficiently create high-quality documents.

[1406] "User" means any individual or organization that uses the System to create materials.

[1407] A "scenario" refers to a plan or story that indicates the content and structure of the material that the user wants to create.

[1408] "Generative AI means" refers to artificial intelligence technology for generating stories based on input scenarios.

[1409] "Internally stored information" refers to data and materials collected and stored within a company or organization.

[1410] "External information" refers to diagrams and data generated using information stored internally.

[1411] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on information stored internally.

[1412] "Image recognition technology" refers to technology for analyzing image data and understanding and classifying its contents.

[1413] "Means for analyzing emotions" refers to technology for analyzing the emotions of users.

[1414] "Generated story" refers to a narrative or narrative created by generative AI means.

[1415] "Means of answering" refers to the technology used to present the generated stories and diagrams to users.

[1416] The present invention is a system for enabling users to efficiently create high-quality materials. A specific embodiment of this system will be described below.

[1417] System configuration

[1418] This system consists of three main components: a server, a terminal, and a user. The server acquires materials, performs image recognition, emotion analysis, generates diagrams and stories, and provides them to users. The terminal is responsible for capturing user input and emotions. Users create materials using the system.

[1419] Hardware and software used

[1420] Server: A server with a powerful processor and large memory capacity is required. The server also needs storage to access the database and retrieve materials.

[1421] Device: A computer or smart device with a camera and microphone is required, allowing for facial and audio capture of the user.

[1422] software:

[1423] Image recognition technology: Use image recognition services such as Google Cloud Vision API and Amazon Rekognition.

[1424] Sentiment analysis technology: Uses Microsoft Azure's Emotion API, etc.

[1425] Generative AI models: Use natural language processing models such as OpenAI's GPT-4.

[1426] Program processing

[1427] The server first accesses the company's internal database and retrieves stored documents, which can include PDFs, Word documents, presentation slides, etc. The server then uses image recognition technology to analyze the images in the documents and select appropriate figures based on specific keywords or context.

[1428] The device captures the user's facial expression with a camera and sends it to the emotion engine. The emotion engine analyzes the user's emotion and sends the results to the server. The server selects an appropriate illustration based on the analysis results.

[1429] The server sends the selected diagram along with a prompt to the generative AI model, which then generates a story based on the prompt and provides it to the user along with the selected diagram.

[1430] Examples and prompts

[1431] For example, if a user wants to create a document related to the "features of a new product," the server selects a diagram related to the "features of a new product" from the document. Then, it inputs the following prompt sentence to the generative AI model:

[1432] Prompt: "Select a picture to illustrate the features of your new product and generate a story."

[1433] The generative AI model generates a story based on this prompt and provides it to the user along with the selected illustrations.

[1434] If the user shows a surprised expression, the server will input the following prompt into the generative AI model:

[1435] Prompt: "Choose the appropriate illustration to generate a story when the user is surprised."

[1436] The generative AI model generates a story based on this prompt and presents it to the user along with illustrations that convey surprise and freshness.

[1437] In this way, the user can efficiently create high-quality materials. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1438] Step 1: Obtaining materials

[1439] The server accesses the company's internal database and retrieves the accumulated documents. A document search query is required as input. The server searches the database for the relevant documents and retrieves them in the form of PDF, Word document, presentation slides, etc. The retrieved documents are then output. Specifically, the server searches for documents related to "new product features" and downloads the relevant files.

[1440] Step 2: Selecting images using image recognition

[1441] The server uses image recognition technology to extract images from the acquired documents. The acquired documents are required as input. The server analyzes the images in the documents using image recognition services such as Google Cloud Vision API or Amazon Rekognition. The output is images selected based on specific keywords or context. Specifically, the server analyzes the images in the documents and selects images related to "new product features" or "comparison with competitors."

[1442] Step 3: Sentiment Analysis

[1443] The device captures the user's facial expression with a camera and sends it to the emotion engine. An image of the user's facial expression is required as input. The device analyzes the user's emotions using Microsoft Azure's Emotion API, etc. The output is the analyzed emotion data. Specifically, the device uses a camera to capture the user's facial expression and sends it to the emotion engine.

[1444] Step 4: Generate diagrams and stories

[1445] The server sends the selected diagram along with a prompt to be input to the generative AI model. The selected diagram and the prompt are required as input. The server uses a generative AI model, such as OpenAI's GPT-4, to generate a story based on the input prompt. The output is the generated story. Specifically, the server sends the following prompt to the generative AI model:

[1446] Prompt: "Select a picture to illustrate the features of your new product and generate a story."

[1447] Step 5: Provide to users

[1448] The server provides the generated story and selected diagrams to the user. The generated story and diagrams are required as input. The server sends these to the user's terminal. The output is the story and diagrams received by the user. In concrete terms, the server sends the generated story and diagrams to the user's terminal, and the user uses them to create materials.

[1449] (Application example 3)

[1450] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1451] Conventional document creation systems can generate stories and diagrams based on the scenario of the document the user wants to create, but they cannot generate appropriate diagrams according to the user's emotions. This makes it difficult to create effective documents that are in line with the user's emotions. In addition, there was a lack of technology to analyze user emotions in real time and generate diagrams based on that, which limited their application in fields such as advertising and presentations.

[1452] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1453] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated in-house as external information, a means for responding with the generated story and diagrams together, an emotion recognition means for analyzing the user's emotions, and a means for selecting and generating appropriate diagrams based on the analyzed emotions. This makes it possible to create effective documents according to the user's emotions.

[1454] "User" means an individual or entity that uses the System to create materials.

[1455] A "scenario" is an outline of a plan or story that indicates the content and structure of the material that the user wants to create.

[1456] "Generative AI methods" are methods that use artificial intelligence technology to generate stories based on input scenarios.

[1457] "In-house stored materials" refers to information such as documents, images, and data that have been created and stored within a company or organization in the past.

[1458] "External information" refers to information sources for use by the system, including materials stored within the company.

[1459] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company and external information.

[1460] A "generated story" is a narrative or exposition created based on a scenario by generative AI means.

[1461] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[1462] "Emotion recognition means" refers to technology or devices for analyzing a user's emotions.

[1463] "Analyzed emotion" means the user's emotional state detected and analyzed by the emotion recognition means.

[1464] An "appropriate illustration" is an illustration that best suits the material and is selected based on the analyzed emotions and scenarios.

[1465] A system for implementing this invention is configured as follows: First, an interface is provided for users to input a scenario for the material they wish to create. This interface allows users to input the scenario via a terminal such as a PC or smartphone.

[1466] Next, a generative AI means is activated to generate a story based on the input scenario. This generative AI means analyzes the scenario using natural language processing technology and generates an appropriate story. The generated story is provided to the user in a format that is easy to understand.

[1467] Furthermore, a means for generating diagrams is operated using documents stored within the company as external information. This means uses image recognition technology to select and generate appropriate diagrams from internal documents. The generated diagrams are provided to users along with the story.

[1468] The system also includes an emotion recognition means for analyzing the user's emotions. This emotion recognition means uses smart glasses or a camera-equipped device to analyze the user's facial expressions in real time to detect their emotions. Based on the analyzed emotions, a means for selecting and generating appropriate diagrams is activated. This enables the creation of effective materials that are in line with the user's emotions.

[1469] Specific hardware used includes smart glasses, camera-equipped devices, and PCs, while software such as OpenCV (image processing library), Keras (deep learning library), and PIL (Python Imaging Library) are used.

[1470] For example, if a user shows a surprised expression, the emotion recognition means detects the expression and generates a prompt sentence saying, "Please generate an advertisement that is optimal for a user who shows a surprised expression." Based on this prompt sentence, the generative AI means generates an advertisement for a new product or a special offer that corresponds to the surprise and provides it to the user.

[1471] In addition, if a user shows a happy expression, a prompt message is generated saying, "Please generate an advertisement that is best suited to a user who is showing a happy expression," and discount coupons and campaign information corresponding to the user's happiness are provided.

[1472] In this way, effective materials and advertisements can be created in response to the user's emotions.

[1473] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1474] Step 1:

[1475] The user inputs the scenario of the material using the terminal.

[1476] Input: The scenario text of the material the user wants to create.

[1477] Output: The scenario text is sent to the server.

[1478] Specific operations: The user enters the scenario of the material in text format through the terminal interface and presses the send button.

[1479] Step 2:

[1480] The server receives the scenario text, and the generative AI means analyzes the scenario.

[1481] Input: Scenario text.

[1482] Output: Analyzed scenario data.

[1483] Specific operation: The server analyzes the scenario text using natural language processing technology and extracts data for story generation.

[1484] Step 3:

[1485] A generative AI means generates a story based on the analyzed scenario data.

[1486] Input: Parsed scenario data.

[1487] Output: The generated story text.

[1488] Specific operation: The server uses a generative AI model to generate story text from the analyzed scenario data.

[1489] Step 4:

[1490] The server uses the data stored within the company as external information and operates the means for generating diagrams.

[1491] Input: The generated story text.

[1492] Output: A suitable diagram.

[1493] Specific operation: The server uses image recognition technology to select and generate illustrations related to the story text from internal documents.

[1494] Step 5:

[1495] The server provides the generated story and diagrams together to the user.

[1496] Input: Generated story text and figures.

[1497] Output: The material provided to the user.

[1498] Specific operation: The server integrates the generated story text and illustrations, creates materials to display to the user, and sends them to the terminal.

[1499] Step 6:

[1500] The terminal operates an emotion recognition means to analyze the user's emotions in real time.

[1501] Input: Video of the user's facial expression.

[1502] Output: Parsed emotion data.

[1503] Specific operation: The device's camera captures the user's facial expression and analyzes the emotion using an emotion recognition model.

[1504] Step 7:

[1505] The server operates a means for selecting and generating an appropriate illustration based on the analyzed emotion data.

[1506] Input: Parsed emotion data.

[1507] Output: Emotion-aware diagrams.

[1508] Specific operation: The server generates a prompt sentence based on the emotion data and generates an appropriate diagram using a generative AI model.

[1509] Step 8:

[1510] The server provides the user with a diagram corresponding to the emotion.

[1511] Input: Emotion-aware diagrams.

[1512] Output: The diagram provided to the user.

[1513] Specific operation: The server sends the generated diagram to the user's terminal and displays it.

[1514] As a specific example, if a user shows a surprised expression, the emotion recognition means detects the expression and generates a prompt sentence saying, "Please generate an advertisement that is optimal for a user who shows a surprised expression." Based on this prompt sentence, the generative AI means generates an advertisement for a new product or a special offer that corresponds to the surprise and provides it to the user.

[1515] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1516] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1517] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1518] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1519] [Third embodiment]

[1520] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1521] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1522] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1523] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1524] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1526] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1527] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1528] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1529] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1530] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1531] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1532] "Example 1"

[1533] This embodiment of the present invention provides a text input interface as a means for the user to input the scenario of the material they wish to create. This interface is designed so that the user can freely write the scenario. For example, if the user inputs a scenario for a "new product presentation,"

[1534] "Example 2"

[1535] Generative AI uses natural language processing technology to analyze the input scenario. This analysis extracts keywords and themes from the scenario and generates a story based on them. For example, from a "new product presentation" scenario, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[1536] "Example 3"

[1537] The means of generating diagrams is to use documents stored within the company as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "features of a new product" or "comparison with competitors" are selected. These diagrams are provided as answers along with the generated story, and users can use this as a basis for creating their own documents.

[1538] The processing flow of each embodiment will be described below.

[1539] "Example 1"

[1540] Step 1: The user inputs a scenario through a text input interface. For example, the user inputs "New product presentation" as the scenario.

[1541] Step 2: Generative AI uses natural language processing technology to analyze the input scenario. Through this analysis, keywords such as "new product features," "comparison with competitors," and "market positioning" are extracted.

[1542] Step 3: Stories are generated based on the extracted keywords.

[1543] "Example 2"

[1544] Step 1: The generative AI analyzes the input scenario using natural language processing technology.

[1545] Step 2: A story is generated based on the keywords and themes extracted through the analysis.

[1546] Step 3: The generated story is provided to the user.

[1547] "Example 3"

[1548] Step 1: Use the data accumulated within the company as external information.

[1549] Step 2: Use image recognition technology to select the appropriate figure from the document.

[1550] Step 3: The selected diagrams are generated and provided as answers along with the generated stories.

[1551] Example 1

[1552] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1553] Conventional document creation systems required users to manually think about the content of documents and create diagrams, which required a great deal of time and effort. Furthermore, the quality and consistency of the documents depended on the skills of the users, making it difficult to create documents of uniform quality. Furthermore, there was a lack of effective ways to utilize documents stored within the company, and information was not reused sufficiently.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1555] In this invention, the server includes: a means for inputting a scenario for a document the user wants to create; a means for generating a prompt sentence based on the input scenario; a means for sending the generated prompt sentence to a generative AI model to generate the document; a means for returning the generated document to the user; a means for using documents stored in-house as external information to generate diagrams; and a means for providing the generated document and diagram together as a response. This allows users to easily automatically generate high-quality documents. Furthermore, by effectively utilizing information stored in-house, information reuse is promoted, improving the efficiency of document creation.

[1556] "User" means any person or organization that uses the System to create Materials.

[1557] A "scenario" refers to text information that describes the content and structure of the material that a user wants to create.

[1558] "Prompt sentence" refers to an instruction sentence sent to a generative AI model based on a scenario.

[1559] "Generative AI model" refers to an artificial intelligence model that automatically generates materials based on input prompts.

[1560] "Documents" refers to documents, including text and diagrams, generated by a generative AI model.

[1561] "Diagram" refers to visual content generated based on materials accumulated within the company and external information.

[1562] "Server" refers to the computer system that manages the overall system processing, receives input from users, sends prompts to the generative AI model, and returns the generated materials.

[1563] "External information" refers to data obtained from sources other than materials stored within the company.

[1564] "Natural language processing technology" refers to the technology used to analyze scenarios and generate prompts and materials.

[1565] "Image recognition technology" refers to the technology used to select and generate appropriate images from materials stored within the company.

[1566] MODE FOR CARRYING OUT THE INVENTION

[1567] This invention is a system that automatically generates materials based on a scenario input by the user for the materials they want to create. This system is composed of multiple components, including a server, a terminal, and a generative AI model.

[1568] System configuration

[1569] 1. Server

[1570] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt sentences, and sends them to the generative AI model. It also receives the materials returned from the generative AI model and returns them to the user. The server uses a database to temporarily store scenarios and generated materials.

[1571] 2. Terminal

[1572] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and check generated materials.

[1573] 3. Generative AI Models

[1574] A generative AI model is an artificial intelligence model that automatically generates materials based on prompts sent from the server. For example, OpenAI's GPT-4 is used as a generative AI model.

[1575] Program processing

[1576] The server receives the scenario entered by the user through the device's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model, and is in the form of a specific instruction for the content of the scenario.

[1577] The generative AI model generates materials based on the received prompts. The materials are formatted according to the scenario content and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[1578] Specific examples

[1579] As a concrete example, consider the case where a user inputs "New product presentation" as a scenario. The user inputs "New product presentation" into the text input interface of the terminal. The server receives this scenario and generates the following prompt sentence:

[1580] Based on the scenario "New Product Presentation," create a presentation that includes the following:

[1581] 1. New Product Features

[1582] 2. Market analysis

[1583] 3. Comparison with competing products

[1584] 4. Sales Strategy

[1585] The generative AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are then sent back to the user via the server. The user can review the generated materials and make corrections or additions as necessary.

[1586] In this way, a system that automatically generates documents through collaboration between the server, terminals, and generative AI models is realized. This system allows users to easily create high-quality documents and effectively utilize information accumulated within the company.

[1587] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1588] Step 1:

[1589] The user inputs a scenario

[1590] The user uses the text input interface of the terminal to input the scenario of the material they want to create. The input scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and inputs "new product presentation" into the text box. Input: Scenario text (e.g., "new product presentation"). Output: Scenario text sent to the server.

[1591] Step 2:

[1592] The server receives the scenario.

[1593] The server receives the scenario sent by the user and saves it in the database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is saved in the database. Input: Scenario text sent by the user. Output: Scenario text saved in the database.

[1594] Step 3:

[1595] The server generates a prompt

[1596] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generative AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates the following prompt: Based on the "New Product Presentation" scenario, please create a presentation containing the following content: 1. New product features 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[1597] Step 4:

[1598] The server sends a prompt to the generative AI model

[1599] The server sends the generated prompt sentence to the generative AI model. Specifically, the server generates an API request and sends the request including the prompt sentence to the generative AI model's endpoint. Input: Generated prompt sentence. Output: Prompt sentence sent to the generative AI model.

[1600] Step 5:

[1601] Generative AI model generates materials

[1602] The generative AI model generates materials based on the received prompt. Specifically, the generative AI model analyzes the prompt and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: The prompt sent to the generative AI model. Output: The generated materials.

[1603] Step 6:

[1604] The server receives the generated material.

[1605] The server receives the materials generated from the generative AI model. Specifically, the server receives the API response and extracts the generated materials from the response body. The extracted materials are temporarily saved. Input: Materials sent from the generative AI model. Output: Generated materials temporarily saved.

[1606] Step 7:

[1607] The server returns the material to the user

[1608] The server returns the generated materials to the user. Specifically, the server generates an HTTP response and sends the response including the generated materials to the user's terminal. The user checks the materials in a web browser. Input: Generated materials temporarily saved. Output: Generated materials sent to the user's terminal.

[1609] (Application example 1)

[1610] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1611] Conventional material creation systems lack the functionality to automatically generate advertising materials based on a scenario entered by the user, which reduces the efficiency of advertising production. Furthermore, there was a need for the generated story and illustrations to be provided in a format that could be used as advertising materials, rather than simply being presented together as a response.

[1612] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1613] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated in-house as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story. This makes it possible to automatically generate advertising materials based on the scenario input by the user, thereby improving the efficiency of advertising production.

[1614] "User" means an individual or entity that uses the system to input material scenarios and receive generated stories and advertising materials.

[1615] A "scenario" is text information that describes the content and structure of the material that a user wants to create.

[1616] A "generative AI means" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[1617] "In-house stored materials" refers to a collection of data and information held internally by a company or organization.

[1618] "External information" refers to data and information obtained from outside the company, other than materials stored within the company.

[1619] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using materials accumulated within the company and external information.

[1620] A "generated story" is a narrative or exposition created based on a scenario by generative AI means.

[1621] "Advertising materials" are content such as images, videos, and text for advertising created based on the generated story.

[1622] "Answering means" refers to a system or method for providing generated stories, illustrations, and advertising materials to users.

[1623] A system for implementing this invention includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and a means for generating advertising materials based on the generated story.

[1624] System Program

[1625] The system is implemented using Python and uses OpenAI's GPT-3 as its generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are also automatically generated.

[1626] Processing Description

[1627] The server operates using the following hardware and software:

[1628] Hardware:

[1629] Smartphone or PC

[1630] software:

[1631] Python

[1632] OpenAI API

[1633] Data processing and calculation:

[1634] 1. Scenario input: The user inputs a scenario through a text input interface. For example, the user inputs a scenario such as "Presentation of a new product."

[1635] 2. Story generation: The input scenario is sent as a prompt to a generative AI model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[1636] 3. Diagram generation: Using internally accumulated data as external information, appropriate diagrams are generated using image recognition technology.

[1637] 4. Generate advertising materials: Generate advertising materials (images, videos, text) based on the generated story.

[1638] 5. Providing answers: Providing the generated stories, diagrams, and advertising materials to the user.

[1639] Specific examples

[1640] If a user enters "new product presentation," the following text is returned as an example of advertising material that might be generated:

[1641] Example prompt sentence:

[1642] Advertising scenario: New product presentation

[1643] Generate advertising materials based on this scenario.

[1644] Examples of generated advertising materials:

[1645] New product presentation

[1646] This new product is developed using the latest technology. It is both high-performance and easy to use, making your daily life more convenient. Buy it now and experience the future of life!

[1647] In this way, advertising materials can be automatically generated based on a scenario input by a user, thereby improving the efficiency of advertising production.

[1648] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1649] Step 1:

[1650] The user inputs a scenario. Using a text input interface, the user inputs the scenario of the material they want to create. For example, they input a scenario such as "Presentation of a new product." The input scenario is sent to the server.

[1651] Step 2:

[1652] The server receives the scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generative AI model. For example, it might be in the format "Advertising scenario: New product presentation. Please generate advertising materials based on this scenario."

[1653] Step 3:

[1654] The server sends a prompt to the generative AI model, which generates a story. The server uses OpenAI's GPT-3 API to send the prompt and receive the story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated that reads, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[1655] Step 4:

[1656] The server generates diagrams using documents stored in the company. The server uses image recognition technology to select and generate appropriate diagrams from documents stored in the company. For example, a diagram showing the technical specifications and usage of a new product is generated.

[1657] Step 5:

[1658] The server generates advertising materials (images, videos, text) based on the generated story. For example, advertising copy and product images are created based on the generated story.

[1659] Step 6:

[1660] The server provides the generated story, illustrations, and advertising materials to the user. The server transmits the generated story, illustrations, and advertising materials to the user, allowing the user to review them. The user can create an advertisement using the provided materials.

[1661] In this way, advertising materials can be automatically generated based on a scenario input by the user, thereby improving the efficiency of advertising production.

[1662] Example 2

[1663] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1664] In conventional document creation systems, even if users input a scenario, it was difficult to generate an appropriate story based on that scenario. Furthermore, the system lacked the functionality to automatically generate diagrams related to the generated story, requiring users to create the diagrams manually. This resulted in the problem of requiring a great deal of time and effort to create documents.

[1665] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting a scenario of a document that a user wants to create, a means for analyzing the input scenario and extracting keywords and themes, a generative AI means for generating a story based on the extracted keywords and themes, a means for generating diagrams using documents accumulated in-house as external information, and a means for responding with the generated story and diagrams together. This makes it possible to automatically generate diagrams related to a story based on the scenario input by the user, thereby efficiently creating documents.

[1666] "User" means an individual or organization that uses the System to create materials.

[1667] A "scenario" is a sentence or text that indicates the content and structure of the material that the user wants to create.

[1668] "Input means" refers to the interface or device that allows the user to input a scenario into the system.

[1669] "Means for analysis" is a function for analyzing the input scenario using natural language processing technology and extracting keywords and themes.

[1670] "Keywords" are words or phrases that are particularly important in a scenario.

[1671] The "theme" is the central theme or topic of the scenario.

[1672] "Generative AI methods" are artificial intelligence techniques for generating stories based on extracted keywords and themes.

[1673] "In-house stored materials" refers to data and information held internally by a company or organization.

[1674] "External information" refers to all information available to the system, including materials stored within the company.

[1675] "Means for generating diagrams" refers to a function that automatically creates appropriate diagrams based on materials accumulated within the company and external information.

[1676] The "means of response" is the interface or device that presents the generated story and diagrams to the user.

[1677] This invention relates to a system that inputs a scenario for a document that a user wants to create and automatically generates a story and related diagrams based on that scenario. A specific embodiment of this system will be described below.

[1678] System program generation

[1679] The server generates a program to implement the generative AI model. This program uses natural language processing technology to analyze the input scenario and extract keywords and themes. It also includes a function to generate a story based on the extracted keywords and themes.

[1680] Hardware and software used

[1681] The server implements natural language processing techniques using the Python programming language and the TensorFlow library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the Bidirectional Encoder Representations from Transformers (BERT) model. Based on the extracted keywords and themes, the server generates a story using the Generative Pre-trained Transformer 3 (GPT-3) model.

[1682] Specific examples

[1683] Consider the case where a user inputs, "Please generate a story for a presentation of our new product. Please include the new product's features, comparison with competitors, and market positioning." The scenario input from the device is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, based on these keywords, a story is generated using the GPT-3 model. The generated story might be, for example, "A presentation that highlights the new product's features, compares it with competitors, and explains its market positioning."

[1684] Prompt Sentence Examples

[1685] Examples of prompts that users can enter into a generative AI model include:

[1686] "Generate a story to present your new product. Include the new product's features, comparison with competitors, and market positioning."

[1687] In this way, the processing of the system's programs is explained while clarifying how the server, terminals, and users are involved.

[1688] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1689] Step 1:

[1690] The user inputs a scenario. The user inputs the scenario into the input field of the terminal. For example, the user might input, "Please generate a story for a presentation of a new product. Please include the features of the new product, a comparison with competitors, and its position in the market." The input scenario becomes the input data for the terminal.

[1691] Step 2:

[1692] The terminal sends the scenario to the server. The terminal sends the scenario entered by the user to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[1693] Step 3:

[1694] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (e.g., NLTK or SpaCy). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[1695] Step 4:

[1696] The server extracts keywords and themes. The server inputs the tokenized data into the BERT model to extract important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is the tokenized scenario, and the output data is the extracted keywords and themes.

[1697] Step 5:

[1698] The server generates a story based on the extracted keywords and themes. The server generates a story using the GPT-3 model based on the extracted keywords and themes. The generated story might be, for example, "a presentation that highlights the features of a new product, compares it with competitors, and explains its position in the market." The input data are the extracted keywords and themes, and the output data is the generated story.

[1699] Step 6:

[1700] The server sends the generated story to the terminal. The server packages the generated story in JSON format and sends it to the terminal as an HTTP response. The input data is the generated story, and the output data is the story sent to the terminal.

[1701] Step 7:

[1702] The terminal displays the story to the user. The terminal displays the received story on its user interface. The user can review the generated story and make corrections or additions as needed. The input data is the story sent to the terminal, and the output data is the story displayed to the user.

[1703] (Application example 2)

[1704] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1705] In the traditional advertising campaign creation process, scenario analysis, keyword extraction, and story generation are often done manually, which is time-consuming and labor-intensive. Furthermore, the generated stories can be inconsistent or not suited to the target audience, making it difficult to create effective ads.

[1706] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating illustrations using documents accumulated in-house as external information, a means for responding with the generated story and illustrations together, and a means for analyzing the advertising campaign scenario and extracting keywords and themes to generate an advertising story. This makes it possible to automate the process from analyzing the advertising campaign scenario to generating the story, and to quickly create effective and consistent advertisements.

[1707] "User" means an individual or organization that uses the system to create materials and advertising campaign scripts.

[1708] A "scenario" is text information entered by a user for the purpose of creating an advertising campaign or materials, and serves as the basis for generating a story.

[1709] "Generative AI means" is an artificial intelligence technology that analyzes an input scenario and generates a story using natural language processing technology.

[1710] "In-house accumulated materials" refers to a collection of documents and data created in the past within a company or organization, and are used as external information.

[1711] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company and external information.

[1712] "Answering means" refers to the methods and techniques used to present the generated stories and diagrams to users.

[1713] An "advertising campaign" is a series of marketing activities designed to promote a particular product or service.

[1714] "Keywords" are important words and phrases extracted from a scenario and are the basis for generating a story.

[1715] The "theme" is the central theme or topic of a scenario and determines the direction of the story.

[1716] An "advertising story" is a coherent narrative that is tailored to the objectives of an advertising campaign and appeals to a target audience.

[1717] A system for implementing this invention is configured as follows: First, a terminal is required for inputting the materials and advertising campaign scenario that the user wants to create. This terminal is a device such as a PC or smartphone, and provides an interface for the user to input the scenario.

[1718] Next, the server receives the input scenario and analyzes it using a generative AI tool. This analysis uses natural language processing technology to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. This generative AI uses, for example, the OpenAI API.

[1719] Furthermore, the server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated story and diagrams are then provided to the user's device as a response.

[1720] As a concrete example, consider the case where a user inputs a scenario for a presentation of a new product. This scenario includes information such as "features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates a prompt sentence like the following:

[1721] Example prompt sentence:

[1722] Scenario: A new product presentation highlights the product's features, how it compares to competitors, and its position in the market.

[1723] Keywords: new product, presentation, features, competitors, market

[1724] Generate an advertising story based on this scenario.

[1725] If we feed this prompt into a generative AI model, it might generate an advertising story like this:

[1726] An example of a generated advertising story:

[1727] The new X-100 is equipped with innovative features not found in other products. Compared to its competitors, the X-100 offers better performance and cost-effectiveness. Its market positioning makes it the perfect choice for users seeking cutting-edge technology. Get your X-100 now and experience the future.

[1728] In this way, users can easily create effective advertising stories. This system automates the process from analyzing advertising campaign scenarios to generating stories, enabling users to quickly create effective and consistent advertisements.

[1729] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1730] Step 1:

[1731] The user uses a terminal to input the scenario for the materials or advertising campaign they want to create, and the input scenario is sent to the server in text format.

[1732] Step 2:

[1733] The server analyzes the received scenario. This analysis uses natural language processing technology. Specifically, it extracts keywords and themes from the scenario. The input for this process is the text data of the scenario, and the output is a list of extracted keywords and themes.

[1734] Step 3:

[1735] The server generates a prompt based on the extracted keywords and topics. The generated prompt is then input to a generative AI model. The input to this process is a list of keywords and topics, and the output is a prompt.

[1736] Step 4:

[1737] The server uses a generative AI model to generate a story based on the prompt. Specifically, it uses OpenAI's API to generate the story. The input of this process is the prompt, and the output is the generated story.

[1738] Step 5:

[1739] The server uses internally stored documents as external information to generate diagrams. It uses image recognition technology to select and generate appropriate diagrams. The input for this process is internal document data, and the output is the generated diagram.

[1740] Step 6:

[1741] The server provides the generated story and diagram together to the user's terminal as a response. The input of this process is the generated story and diagram, and the output is the response data sent to the user's terminal.

[1742] Step 7:

[1743] The user checks the story and illustrations provided through the terminal, and makes corrections or additions as necessary to complete the final materials or advertising campaign content. The input to this process is the response data sent from the server, and the output is the completed materials or advertising campaign content.

[1744] Example 3

[1745] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1746] Conventional document creation systems require users to manually search for documents, select appropriate figures, and create stories, which takes time and effort. Selecting appropriate figures also requires specialized knowledge, making it difficult for users to create documents efficiently. Another issue is the inconsistency in the quality of the generated documents.

[1747] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1748] In this invention, the server includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for using documents stored in the company as external information and selecting and generating appropriate illustrations using image recognition technology, and a means for responding with the generated story and illustrations together, thereby enabling users to efficiently create high-quality documents.

[1749] "User" means an individual or organization that uses the System to create materials.

[1750] A "scenario" is text information that indicates the content and purpose of the material that a user wants to create.

[1751] "Generative AI means" refers to means that use artificial intelligence technology to generate a story based on an input scenario.

[1752] "In-house stored materials" refers to information resources such as documents, images, and data stored within a company or organization.

[1753] "External information" refers to materials accumulated within the company, which are sources of information for use by the system.

[1754] "Image recognition technology" is a technology for analyzing images and diagrams and understanding their contents.

[1755] "Appropriate illustrations" are diagrams or images selected to visually represent information relevant to the scenario.

[1756] The "means for generating" is a means for creating a new diagram based on a selected diagram.

[1757] "Means of answering" refers to the means for providing the generated story and diagram to the user.

[1758] The term "system" refers to a computer-based device or software for carrying out a series of processes including the above-mentioned means.

[1759] The present invention relates to a system that enables users to efficiently create high-quality materials. A specific embodiment of this system will be described below.

[1760] System Overview

[1761] The system includes the following main means:

[1762] 1. A way for users to input the scenario of the material they want to create

[1763] 2. Generative AI means to generate stories based on input scenarios

[1764] 3. Using internally stored data as external information, and using image recognition technology to select and generate appropriate diagrams.

[1765] 4. A means of answering questions with generated stories and diagrams

[1766] Hardware and software used

[1767] The server runs the system using the following hardware and software:

[1768] Hardware: powerful processor, sufficient memory, storage devices, and network interfaces

[1769] Software: Google Cloud Vision API, Amazon Rekognition, Adobe Illustrator, Canva, generative AI models (e.g., OpenAI's GPT-4)

[1770] Data processing and calculation

[1771] The server processes and calculates data in the following steps:

[1772] 1. Collecting materials:

[1773] The server accesses internal databases and file servers to search and retrieve relevant materials, including PDFs, Word documents, presentation files, and more.

[1774] 2. Application of image recognition technology:

[1775] The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and diagrams in the collected materials.

[1776] 3. Selecting the right diagram:

[1777] Based on the analysis results, the server filters and selects appropriate diagrams related to the scenario.

[1778] 4. Figure Generation:

[1779] The server uses Adobe Illustrator or Canva to generate a new diagram based on the selected diagram.

[1780] 5. Story generation:

[1781] The server uses a generative AI model (GPT-4) to generate a story based on the generated diagrams.

[1782] 6. Providing answers:

[1783] The server compiles the generated diagrams and stories into a single document and provides it to the user.

[1784] Specific examples

[1785] For example, if a user wants to create a document explaining the "features of a new product," they can input the following prompt into the generative AI model:

[1786] "Please generate a diagram to explain the features of a new product. Please select and generate an appropriate diagram based on internal company materials."

[1787] When this prompt is entered, the server generates a diagram according to the procedure described above and provides it along with the story. Users can then create materials based on the answers.

[1788] In this way, the user can efficiently create high-quality materials. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1789] Step 1: Gather your materials

[1790] The server accesses the company's databases and file servers to search for and retrieve relevant materials. As input, the user is given a scenario for the material they want to create. The server extracts keywords based on this scenario and uses these keywords to search for materials. As output, it obtains related materials such as PDFs, Word documents, and presentation files.

[1791] Step 2: Applying image recognition technology

[1792] The server applies image recognition technology to the collected materials. The materials collected in step 1 are provided as input. The server uses Google Cloud Vision API and Amazon Rekognition to analyze the images and diagrams within the materials. Specifically, the server processes each document sequentially and extracts the contents of the images and diagrams as text data. The analysis results are obtained as output.

[1793] Step 3: Select the right diagram

[1794] The server uses image recognition technology to select appropriate diagrams from the documents. The analysis results obtained in step 2 are given as input. The server matches the analysis results with keywords related to the scenario, and filters and selects highly relevant diagrams. Specifically, the server calculates a relevance score and selects diagrams with high scores. The selected diagrams are obtained as output.

[1795] Step 4: Generate the diagram

[1796] The server generates a new diagram based on the selected diagram. As input, it receives the diagram selected in step 3. The server uses Adobe Illustrator or Canva to create a new diagram using the selected diagram as a template and adding the necessary information. Specifically, the server adjusts the diagram layout and adds annotations and data. As output, it receives the generated new diagram.

[1797] Step 5: Generate stories

[1798] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related explanations and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[1799] Step 6: Provide your answers

[1800] The server compiles the generated diagrams and stories into a single document and provides it to the user. As input, the diagram generated in step 4 and the story generated in step 5 are given. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The final document is obtained as output.

[1801] (Application example 3)

[1802] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1803] Maintenance work in factories is complex and diverse, making it difficult for workers to quickly understand the appropriate procedures. Furthermore, the sheer volume of maintenance records and technical documents makes it difficult to efficiently search and use the necessary information. For this reason, there is a demand for improved efficiency and accuracy in maintenance work.

[1804] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1805] In this invention, the server includes: means for inputting a scenario for a document that a user wants to create; generative AI means for generating a story based on the input scenario; means for generating diagrams using documents stored in-house as external information; means for responding with the generated story and diagrams together; means for selecting and generating appropriate diagrams using image recognition technology using maintenance records and technical documents stored in the factory as external information; and means for supporting maintenance work based on the selected diagrams. This enables maintenance workers to quickly obtain the information they need and perform their work efficiently and accurately.

[1806] "User" refers to a person who uses the system to create materials and perform maintenance work.

[1807] A "scenario" refers to a plan or structure that indicates the content and purpose of the material that the user wants to create.

[1808] "Generative AI means" refers to artificial intelligence technology that generates a story based on an input scenario.

[1809] "Documents accumulated within the company" refers to information such as technical documents and maintenance records that have been created within the company in the past.

[1810] "External information" refers to materials and data accumulated within the company for use by the system.

[1811] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on materials accumulated within the company.

[1812] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[1813] "Maintenance records" refers to information that records the history and details of maintenance work performed within a factory.

[1814] "Technical documents" refers to technical information and procedures related to equipment and systems within a factory.

[1815] The term "means for supporting maintenance work" refers to a support technology that enables a maintenance worker to perform work efficiently and accurately based on the selected diagram.

[1816] A system for carrying out the present invention is configured as follows: First, an interface is provided for a user to input a scenario for a material that the user wishes to create. The user inputs the scenario through this interface.

[1817] The server then generates a story based on the input scenario using a generative AI mechanism that uses natural language processing to analyze the scenario and generate an appropriate story. For example, the Hugging Face transformers library can be used as a generative AI mechanism.

[1818] Furthermore, the server uses internally stored documents as external information to generate diagrams. Image recognition technology is used to generate diagrams. Specifically, software such as OpenCV, PIL (Python Imaging Library), and pytesseract are used to extract and analyze text and features from images.

[1819] Maintenance records and technical documents stored within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[1820] The generated story and diagram are provided to users together, allowing them to quickly obtain the information they need and work efficiently and accurately.

[1821] As a concrete example, the following prompt sentence can be used:

[1822] Example prompt sentence:

[1823] "Generate a story about the part replacement procedure based on the following images: ['maintenance_step1.png', 'maintenance_step2.png']"

[1824] Based on this prompt, the generative AI means generates a story about the part replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[1825] In this way, the efficiency of maintenance work for factory robots can be improved and the burden on workers can be reduced.

[1826] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1827] Step 1:

[1828] The user inputs the scenario of the material they want to create.

[1829] Input: A scenario in which the user provides input through an interface.

[1830] Output: The input scenario data.

[1831] Specific operation: The user accesses the system interface and inputs the scenario of the material they want to create in text format.

[1832] Step 2:

[1833] The server generates a story based on the input scenario.

[1834] Input: The entered scenario data.

[1835] Output: The generated stories.

[1836] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generate an appropriate story using natural language processing technology.

[1837] Step 3:

[1838] The server uses materials stored within the company as external information to generate diagrams.

[1839] Input: Materials accumulated within the company.

[1840] Output: The generated diagram.

[1841] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate diagrams from materials stored within the company.

[1842] Step 4:

[1843] The server uses maintenance records and technical documents stored within the factory as external information and uses image recognition technology to select and generate appropriate diagrams.

[1844] Input: Maintenance records and technical documents accumulated within the factory.

[1845] Output: Selected figures.

[1846] Specific operation: The server uses image recognition technology to select and generate the necessary diagrams from maintenance records and technical documents.

[1847] Step 5:

[1848] The server responds with the generated story and diagrams.

[1849] Input: The generated story and selected diagrams.

[1850] Output: The story and diagrams provided to the user.

[1851] Specific operation: The server combines the generated story with the selected diagrams and provides it to the user, who can then create materials based on it.

[1852] Step 6:

[1853] The maintenance work is carried out based on the stories and diagrams provided by the user.

[1854] Input: The story and illustrations provided.

[1855] Output: Efficient and accurate maintenance work.

[1856] Specific behavior: The user performs maintenance tasks efficiently and accurately while referring to the provided stories and diagrams.

[1857] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1858] "Example 1"

[1859] As one embodiment of the present invention, a system is provided that includes a means for inputting a scenario for a document that a user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagram together, and an emotion engine that recognizes the user's emotions.

[1860] "Example 2"

[1861] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a happy expression, the emotion engine captures that information and the generative AI generates a story based on that emotion. Specifically, if the emotion of joy is recognized, the generative AI generates a story that includes positive elements.

[1862] "Example 3"

[1863] The emotion engine also selects and generates appropriate images according to the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures that information, and the image generation means selects and generates an image that conveys surprise and freshness.

[1864] The processing flow of each embodiment will be described below.

[1865] "Example 1"

[1866] Step 1: The user inputs the scenario of the material they want to create.

[1867] Step 2: Based on the input scenario, the generative AI generates a story.

[1868] Step 3: Use the data accumulated within the company as external information to generate the diagram.

[1869] Step 4: Answer the generated story and diagram together.

[1870] Step 5: The emotion engine that recognizes the user's emotions is activated.

[1871] "Example 2"

[1872] Step 1: The emotion engine recognizes the user's emotion from their tone of voice and facial expression.

[1873] Step 2: The generative AI generates a story based on the emotions recognized by the emotion engine.

[1874] "Example 3"

[1875] Step 1: The emotion engine recognizes the user's emotion.

[1876] Step 2: Select and generate an appropriate illustration according to the emotion recognized by the emotion engine.

[1877] Example 1

[1878] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1879] Conventional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which requires a great deal of time and effort. Furthermore, they do not provide feedback that takes into account the user's feelings, which reduces the efficiency and quality of document creation. Furthermore, there is a lack of effective ways to utilize the documents accumulated within the company, which can lead to a lack of consistency and reliability in the documents.

[1880] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1881] In this invention, the server includes a means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, a means for generating diagrams using documents accumulated within the company as external information, a means for responding with the generated story and diagrams together, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback according to the user's emotions, and enables effective use of internal documents.

[1882] "Means for inputting the scenario of the material that the user wants to create" refers to an interface that allows the user to freely input text, and is a means for inputting the scenario of the material.

[1883] "Generative AI means" refers to artificial intelligence technology used to generate a story based on an input scenario, and is a means of analyzing the scenario and generating a story using natural language processing technology.

[1884] "Means of generating diagrams using materials accumulated within the company as external information" refers to means of generating diagrams by referencing materials stored in an internal database and using data visualization technology.

[1885] "Means for providing answers using the generated story and diagrams together" refers to a means for integrating the generated story and diagrams and providing them to users as a single document.

[1886] An "emotion engine" refers to technology that analyzes user input and reactions and recognizes emotions. It is a means of analyzing user emotions using natural language processing technology and providing appropriate feedback.

[1887] This invention is a system that allows a user to input a scenario for a document they wish to create, generate a story and illustrations based on that scenario, and ultimately provide the document as a single document. A specific embodiment of this system will be described below.

[1888] System configuration

[1889] The system consists of the following main components:

[1890] 1. Text Input Interface

[1891] 2. Generative AI means

[1892] 3. Database Reference Method

[1893] 4. Data Visualization Methods

[1894] 5. Integration Methods

[1895] 6. Emotion Engine

[1896] Text Input Interface

[1897] The user uses the text input interface of the terminal to input the scenario of the material they want to create. This interface is designed so that the user can input text freely. For example, the user can input "New product presentation."

[1898] Generative AI means

[1899] The server receives the scenario entered by the user and inputs it as a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates an appropriate story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1900] Database Reference Method

[1901] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib and Pandas to format and visualize the data.

[1902] Data visualization tools

[1903] The server uses data visualization techniques to generate diagrams based on the information retrieved from the database. For example, it uses SQL queries to extract the necessary information from the database and generates graphs using Python's Matplotlib. The generated graphs are saved as image files.

[1904] Integration Methods

[1905] The server integrates the generated stories and figures into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX or Microsoft Word API). The generated document is saved as a PDF file.

[1906] Emotion Engine

[1907] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1908] Examples of concrete examples and prompts

[1909] Specific examples

[1910] The user enters "New Product Presentation" into the text entry interface.

[1911] The server uses a generative AI model to generate a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1912] The server references past sales data and market research results from an internal database and creates relevant graphs and charts.

[1913] Finally, the generated stories and diagrams are integrated and provided to the user.

[1914] Prompt Sentence Examples

[1915] "Please enter a scenario for a new product presentation. For example, please describe the contents, including the new product's features, target market, competitive analysis, and sales strategy."

[1916] In this way, users can easily create high-quality materials.

[1917] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1918] Step 1:

[1919] The user inputs a scenario.

[1920] The user uses the text input interface of the device to input the scenario of the document they want to create, for example, "New product presentation." The input scenario is sent to the server in real time.

[1921] Input: User-entered scenario text

[1922] Output: Scenario text sent to the server

[1923] Step 2:

[1924] The server generates the story using a generative AI model.

[1925] The server inputs the received scenario as a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it creates a story that includes content such as "new product features, target market, competitive analysis, and sales strategy."

[1926] Input: Scenario text

[1927] Output: Generated story text

[1928] Step 3:

[1929] The server references the company's internal database to generate the diagram.

[1930] The server accesses the company's internal database to search for relevant documents and data. For example, it references past sales data and market research results, and generates graphs and charts based on this data. The server uses Python libraries such as Matplotlib and Pandas to format and visualize the data.

[1931] Input: Data obtained from our internal database

[1932] Output: Image files of generated graphs and charts

[1933] Step 4:

[1934] The server integrates the generated stories and diagrams.

[1935] The server integrates the generated stories and figures into a single document. For example, it places figures corresponding to each section of the story in the appropriate positions to create a coherent document as a whole. This process uses a document generation tool (for example, LaTeX or Microsoft Word API). The generated document is saved as a PDF file.

[1936] Input: Generated story text, image files of graphs and charts

[1937] Output: Flattened PDF document

[1938] Step 5:

[1939] The server recognizes the user's emotions and provides feedback.

[1940] The server uses an emotion engine to analyze user input and reactions. For example, if a user inputs "I don't understand this part," the server detects this and provides feedback such as "Please tell me specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1941] Input: User feedback text

[1942] Output: Analysis results by the emotion engine and feedback text

[1943] (Application example 1)

[1944] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1945] The traditional process of creating advertising campaigns required a lot of time and effort, including creating a scenario, story, and visuals. It was also difficult to propose optimal advertising materials that took into account user emotions, making it difficult to quickly create effective advertising campaigns. This meant that advertising agencies and marketers were unable to work efficiently, making it difficult to maximize the effectiveness of their advertising. ...

Claims

1. A first means for accepting input of a scenario which is text information describing the content and structure of a material that a user wishes to create; A second means for generating a first prompt sentence for outputting a story of the material based on the scenario; a third means for inputting the generated first prompt sentence into a generative AI model to generate the story; A fourth means for generating advertising material based on the generated story; A fifth means for providing the generated story and the advertising material to a user; A system including:

2. A sixth means for recognizing an emotion of the user from a tone of voice and a facial expression of the user using an emotion engine; a seventh means for adjusting the generated story based on the user's emotion; The system of claim 1 , comprising:

3. The seventh means adds a positive element to the story when a positive emotion is recognized as the emotion of the user. The system of claim 2.

Images