system

Abstract

To provide a system.SOLUTION: A system includes: first means for receiving input of a scenario to be text information describing a content and a structure of a material a user wants to create; second means which generates a first prompt statement for outputting a story of the material on the basis of the scenario; third means which inputs the generated first prompt statement to a generative AI model to generate the story; fourth means generating an advertisement material on the basis of the generated story; and fifth means providing a user with the generated story and advertisement material.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, creating materials requires a great deal of time and effort. In particular, creating a story and creating diagrams require specialized knowledge and techniques. Also, when utilizing existing materials, it is difficult to find appropriate materials.

Means for Solving the Problems

[0005] In the present invention, when a user inputs a scenario of a material to be created, a generative AI generates a story and uses the materials accumulated within the company as external information to generate diagrams. Thereby, the efficiency of creating materials can be improved. Furthermore, the generative AI analyzes the scenario using natural language processing technology, and the means for generating diagrams selects appropriate diagrams from the materials accumulated within the company using image recognition technology. Thereby, the quality of creating materials is also improved. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16] It is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Embodiment Example 3. [Figure 17] It is a sequence diagram showing the processing flow of the data processing system in Embodiment Example 1 of Embodiment Example 1 when combined with an emotion engine. [Figure 18] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Embodiment Example 1 when combined with an emotion engine. [Figure 19] It is a sequence diagram showing the processing flow of the data processing system in Embodiment Example 2 of Embodiment Example 2 when combined with an emotion engine. [Figure 20] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Embodiment Example 2 when combined with an emotion engine. [Figure 21] It is a sequence diagram showing the processing flow of the data processing system in Embodiment Example 3 of Embodiment Example 3 when combined with an emotion engine. [Figure 22] It is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Embodiment Example 3 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0007] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the language used in the following description will be explained.

[0009] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)), etc.

[0010] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0011] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0012] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0014] [First Embodiment]

[0015] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0016] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention provides a text input interface as a means for the user to input a scenario for a document they wish to create. This interface is designed to allow the user to freely write a scenario. For example, if the user inputs "New Product Presentation" as the scenario,

[0029] "Example of form 2"

[0030] Generative AI uses natural language processing techniques to analyze input scenarios. This analysis extracts keywords and themes from the scenario, and then generates a story based on them. For example, from a scenario about a "new product presentation," keywords such as "features of the new product," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[0031] "Example of form 3"

[0032] The method for generating the diagrams utilizes internally stored documents as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "new product features" or "comparisons with competitors" might be selected. These diagrams, along with the generated story, are provided as answers, and users create their documents based on them.

[0033] The following describes the processing flow for each example of the form.

[0034] "Example of form 1"

[0035] Step 1: The user enters the scenario through a text input interface. For example, they might enter "New Product Presentation" as the scenario.

[0036] Step 2: The generative AI analyzes the input scenario using natural language processing techniques. This analysis extracts keywords such as "features of the new product," "comparison with competitors," and "market positioning."

[0037] Step 3: A story is generated based on the extracted keywords.

[0038] "Example of form 2"

[0039] Step 1: The generative AI analyzes the input scenario using natural language processing techniques.

[0040] Step 2: A story is generated based on the keywords and themes extracted through analysis.

[0041] Step 3: The generated story is provided to the user.

[0042] "Example of form 3"

[0043] Step 1: Use internally accumulated data as external information.

[0044] Step 2: Use image recognition technology to select the appropriate figure from the document.

[0045] Step 3: The selected figure is generated and provided as the answer along with the generated story.

[0046] (Example 1)

[0047] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0048] Traditional document creation systems required users to manually conceive of document content and create diagrams, which was time-consuming and labor-intensive. Furthermore, the quality and consistency of documents depended on the user's skills, making it difficult to produce documents of uniform quality. Additionally, there was a lack of effective means to utilize internally accumulated documents, resulting in insufficient information reuse.

[0049] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0050] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating prompt sentences based on the input scenario, means for sending the generated prompt sentences to a generation AI model to generate the document, means for returning the generated document to the user, means for using documents accumulated within the company as external information to generate a diagram, and means for providing the generated document and diagram together as a response. This makes it possible for users to easily and automatically generate high-quality documents. Furthermore, by effectively utilizing information accumulated within the company, information reuse is promoted and the efficiency of document creation is improved.

[0051] "User" refers to an individual or organization that uses the system to create documents.

[0052] A "scenario" refers to text information that describes the content and structure of the document that the user wants to create.

[0053] A "prompt message" refers to an instruction message sent to the AI ​​model based on a scenario.

[0054] A "generative AI model" refers to an artificial intelligence model that automatically generates documents based on input prompt text.

[0055] "Documents" refers to documents containing text and diagrams generated by a generative AI model.

[0056] "Diagrams" refer to visual content generated based on internal company documents and external information.

[0057] A "server" refers to a computer system that manages the processing of the entire system, receives input from users, sends prompt messages to the generating AI model, and returns the generated materials.

[0058] "External information" refers to data obtained from sources other than internal company documents.

[0059] "Natural language processing technology" refers to the techniques used to analyze scenarios and generate prompts and other information.

[0060] "Image recognition technology" refers to the technology used to select and generate appropriate diagrams from documents accumulated within a company.

[0061] Modes for carrying out the invention

[0062] This invention is a system that allows users to input a scenario for the material they wish to create, and then automatically generates the material based on that scenario. This system consists of multiple components, including a server, a terminal, and a generation AI model.

[0063] System Configuration

[0064] 1. Server

[0065] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt statements, and sends them to the generating AI model. It also receives the data returned by the generating AI model and sends it back to the user. The server uses a database to temporarily store scenarios and generated data.

[0066] 2. Terminal

[0067] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and review generated materials.

[0068] 3. Generative AI Models

[0069] A generative AI model is an artificial intelligence model that automatically generates documents based on prompt messages sent from a server. Examples of generative AI models include OpenAI's GPT-4®.

[0070] Program processing

[0071] The server receives a scenario entered by the user through the terminal's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model, specifically instructing it on the content of the scenario.

[0072] The generative AI model generates materials based on the received prompt text. These materials are in a format consistent with the scenario and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[0073] Specific example

[0074] As a concrete example, consider a scenario where a user inputs "New Product Presentation" as the scenario. The user enters "New Product Presentation" into the terminal's text input interface. The server receives this scenario and generates a prompt message like the following:

[0075] Based on the "New Product Presentation" scenario, please create presentation materials that include the following:

[0076] 1. Features of the new product

[0077] 2. Market analysis

[0078] 3. Comparison with competing products

[0079] 4. Sales Strategy

[0080] The generation AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are sent back to the user via the server. The user can review the generated materials and make corrections or additions as needed.

[0081] In this way, a system is realized in which servers, terminals, and generation AI models work together to automatically generate documents. This system allows users to easily create high-quality documents and effectively utilize the information accumulated within the company.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The user enters the scenario.

[0085] The user uses the terminal's text input interface to enter the scenario for the document they want to create. The entered scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and enters "New Product Presentation" into the text box. Input: Scenario text (e.g., "New Product Presentation"). Output: Scenario text sent to the server.

[0086] Step 2:

[0087] The server receives the scenario

[0088] The server receives scenarios submitted by users and stores them in a database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is then stored in the database. Input: Scenario text submitted by the user. Output: Scenario text stored in the database.

[0089] Step 3:

[0090] The server generates a prompt message.

[0091] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates a prompt like this: Based on the "New Product Presentation" scenario, please create a presentation document that includes the following: 1. Features of the new product 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[0092] Step 4:

[0093] The server sends a prompt message to the generated AI model.

[0094] The server sends the generated prompt to the AI ​​model. Specifically, the server generates an API request and sends the request, including the prompt, to the AI ​​model's endpoint. Input: The generated prompt. Output: The prompt sent to the AI ​​model.

[0095] Step 5:

[0096] The generative AI model generates the data.

[0097] The generative AI model generates materials based on the received prompt text. Specifically, the generative AI model analyzes the prompt text and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: Prompt text sent to the generative AI model. Output: Generated materials.

[0098] Step 6:

[0099] The server receives the generated data.

[0100] The server receives the data generated from the generative AI model. Specifically, the server receives the API response and extracts the generated data from the response body. The extracted data is then temporarily stored. Input: Data sent from the generative AI model. Output: Temporarily stored generated data.

[0101] Step 7:

[0102] The server returns the document to the user.

[0103] The server returns the generated document to the user. Specifically, the server generates an HTTP response and sends the response containing the generated document to the user's terminal. The user then views the document in a web browser. Input: Temporarily saved generated document. Output: Generated document sent to the user's terminal.

[0104] (Application Example 1)

[0105] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0106] Conventional document creation systems lacked the functionality to automatically generate advertising materials based on user-input scenarios, resulting in reduced advertising production efficiency. Furthermore, there was a need to provide the generated story and diagrams not just as answers, but in a format suitable for use as advertising materials.

[0107] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0108] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story. This makes it possible to automatically generate advertising material based on a scenario input by the user and improve the efficiency of advertising production.

[0109] A "user" is an individual or organization that uses the system to input scenarios for materials and receives the generated stories and advertising materials.

[0110] A "scenario" is text information that describes the content and structure of the document that the user wants to create.

[0111] A "generative AI system" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[0112] "Documents accumulated within the company" refers to a collection of data and information that a company or organization possesses internally.

[0113] "External information" refers to data and information obtained from external sources, other than documents accumulated within the company.

[0114] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using internally accumulated data and external information.

[0115] A "generated story" is a narrative or explanatory text created based on a scenario using generative AI methods.

[0116] "Advertising materials" refer to content such as images, videos, and text created for advertising purposes based on a generated story.

[0117] "Means of providing responses" refers to systems and methods for providing users with generated stories, diagrams, and advertising materials.

[0118] The system for implementing this invention includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents accumulated within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story.

[0119] System program

[0120] This system is implemented using Python (registered trademark) and utilizes OpenAI's GPT-3 (registered trademark) as a generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are automatically generated as well.

[0121] Explanation of the process

[0122] The server operates using the following hardware and software:

[0123] Hardware:

[0124] Smartphone or PC

[0125] software:

[0126] Python

[0127] OpenAI API

[0128] Data processing and data computation:

[0129] 1. Scenario Input: The user inputs the scenario through a text input interface. For example, they might input a scenario such as "New Product Presentation".

[0130] 2. Story Generation: The input scenario is sent as a prompt to the AI ​​model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[0131] 3. Diagram generation: Use internally accumulated data as external information and generate appropriate diagrams using image recognition technology.

[0132] 4. Generating ad materials: Generate ad materials (images, videos, text) based on the generated story.

[0133] 5. Providing the response: Provide users with the generated story, diagrams, and advertising materials.

[0134] Specific example

[0135] If a user enters "New Product Presentation," the following text will be returned as an example of the generated advertising material:

[0136] Example of a prompt:

[0137] Advertising Scenario: New Product Presentation

[0138] Please generate ad materials based on this scenario.

[0139] Examples of generated ad materials:

[0140] "New Product Presentation"

[0141] This new product was developed using the latest technology. It's high-performance yet easy to use, making everyday life more convenient. Buy it now and experience the future of living!

[0142] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[0143] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0144] Step 1:

[0145] The user enters the scenario. The user uses a text input interface to enter the scenario for the document they want to create. For example, they might enter a scenario such as "New Product Presentation." The entered scenario is then sent to the server.

[0146] Step 2:

[0147] The server receives a scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generation AI model. For example, it might take the form of "Advertising Scenario: Presentation of a new product. Generate advertising material based on this scenario."

[0148] Step 3:

[0149] The server sends a prompt to the generative AI model, which then generates a story. The server uses the OpenAI GPT-3 API to send the prompt and receives a story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated such as, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[0150] Step 4:

[0151] The server generates diagrams using documents stored within the company. The server uses image recognition technology to select and generate appropriate diagrams from the company's internal records. For example, diagrams illustrating the technical specifications and usage of a new product can be generated.

[0152] Step 5:

[0153] The server generates advertising materials based on the generated story. The server creates advertising materials (images, videos, text) based on the generated story. For example, advertising taglines and product images are created based on the generated story.

[0154] Step 6:

[0155] The server provides the user with generated stories, diagrams, and advertising materials. The server sends the generated stories, diagrams, and advertising materials to the user so that the user can review them. The user can then create advertisements using the provided materials.

[0156] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[0157] (Example 2)

[0158] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] Traditional document creation systems struggled to generate appropriate stories based on user-input scenarios. Furthermore, they lacked the functionality to automatically generate diagrams related to the generated stories, requiring users to create them manually. This resulted in a significant amount of time and effort being required for document creation.

[0160] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for analyzing the input scenario and extracting keywords and themes, means for generating a story based on the extracted keywords and themes, means for using documents stored within the company as external information and generating a diagram, and means for providing the generated story and diagram together as a response. This makes it possible to automatically generate a story and related diagrams based on the scenario input by the user, and to efficiently create documents.

[0161] A "user" is an individual or organization that uses the system to create materials.

[0162] A "scenario" is a document or text that outlines the content and structure of the material that the user wants to create.

[0163] "Means of input" refers to the interface or device that allows users to input scenarios into the system.

[0164] "Means of analysis" refers to a function that uses natural language processing technology to analyze the input scenario and extract keywords and themes.

[0165] "Keywords" are particularly important words or phrases within a scenario.

[0166] The "theme" is the central theme or topic of the scenario.

[0167] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on extracted keywords and themes.

[0168] "Documents accumulated within the company" refers to data and information that a company or organization possesses internally.

[0169] "External information" refers to all information available to the system, including documents stored within the company.

[0170] "Methods for generating diagrams" refers to a function that automatically creates appropriate diagrams based on internally accumulated documents and external information.

[0171] "Means of providing answers" refer to interfaces and devices that provide users with the generated stories and diagrams.

[0172] This invention relates to a system that automatically generates a story and related diagrams based on a scenario input by the user for a document they wish to create. A specific embodiment of this system is described below.

[0173] System program generation

[0174] The server generates a program for implementing a generative AI model. This program has the functionality to analyze input scenarios using natural language processing techniques and extract keywords and themes. It also includes the functionality to generate a story based on the extracted keywords and themes.

[0175] Hardware and software to be used

[0176] The server implements natural language processing techniques using the Python programming language and the TENSORFLOW® library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the BERT® (Bidirectional Encoder Representations from Transformers) model. Based on the extracted keywords and themes, the server generates a story using the GPT-3 (Generative Pre-trained Transformer 3) model.

[0177] Specific example

[0178] Consider a scenario where a user inputs the following: "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning." The scenario entered from the terminal is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, it uses these keywords to generate a story using the GPT-3 model. The generated story might be, for example, "a presentation that highlights the new product's features, compares it to competitors, and explains its market positioning."

[0179] Example of a prompt

[0180] Examples of prompts that users might input into the generated AI model include the following:

[0181] "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning."

[0182] In this way, the system's program processing is explained while clearly defining how the server, terminal, and user are involved.

[0183] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0184] Step 1:

[0185] The user inputs a scenario. The user enters the scenario into the input field on the terminal. For example, they might enter, "Generate a story for a new product presentation. Include the product's features, comparison with competitors, and market positioning." The entered scenario becomes the input data on the terminal.

[0186] Step 2:

[0187] The terminal sends the scenario to the server. The terminal sends the user-entered scenario to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[0188] Step 3:

[0189] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (e.g., NLTK® or SpaCy®). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[0190] Step 4:

[0191] The server extracts keywords and themes. The server inputs tokenized data into a BERT model and extracts important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is a tokenized scenario, and the output data is the extracted keywords and themes.

[0192] Step 5:

[0193] The server generates a story based on the extracted keywords and themes. The server uses a GPT-3 model to generate the story based on the extracted keywords and themes. The generated story might be, for example, "a presentation highlighting the features of a new product, comparing it to competitors, and explaining its market positioning." The input data consists of the extracted keywords and themes, while the output data is the generated story.

[0194] Step 6:

[0195] The server sends the generated story to the device. The server packages the generated story in JSON format and sends it to the device as an HTTP response. The input data is the generated story, and the output data is the story sent to the device.

[0196] Step 7:

[0197] The device displays the story to the user. The device displays the received story in the user interface. The user can review the generated story and modify or add to it as needed. The input data is the story sent to the device, and the output data is the story displayed to the user.

[0198] (Application Example 2)

[0199] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0200] Traditional advertising campaign creation processes often involved manual tasks such as scenario analysis, keyword extraction, and story generation, which were time-consuming and labor-intensive. Furthermore, the generated stories could lack consistency or be unsuitable for the target audience, making it difficult to create effective advertisements.

[0201] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for a generative AI that generates a story based on the input scenario, means for using internally stored documents as external information to generate a diagram, means for providing the generated story and diagram together as a response, and means for analyzing an advertising campaign scenario, extracting keywords and themes, and generating an advertising story. This automates the process from scenario analysis of an advertising campaign to story generation, making it possible to quickly create effective and consistent advertisements.

[0202] A "user" is an individual or organization that uses the system to create materials or advertising campaign scenarios.

[0203] A "scenario" is text information entered by users for advertising campaigns or document creation, and it forms the basis for story generation.

[0204] "Generative AI methods" refer to artificial intelligence technologies that analyze input scenarios and generate stories using natural language processing techniques.

[0205] "Documents accumulated within the company" refers to a collection of documents and data created in the past within a company or organization, which are used as external information.

[0206] "Methods for generating diagrams" refers to technologies that select and generate appropriate diagrams based on internally accumulated documents and external information.

[0207] "Means of providing answers" refers to the methods and technologies for providing users with the generated stories and diagrams.

[0208] An "advertising campaign" is a series of marketing activities aimed at promoting a specific product or service.

[0209] "Keywords" are important words or phrases extracted from the scenario, and they form the basis for generating the story.

[0210] The "theme" is the central theme or topic of the scenario, and it determines the direction of the story.

[0211] An "advertising story" is a consistent narrative created in line with the objectives of an advertising campaign, designed to appeal to the target audience.

[0212] The system for implementing this invention is configured as follows: First, a terminal is required for the user to input the materials or advertising campaign scenarios they wish to create. This terminal is a device such as a personal computer or smartphone, and provides an interface for the user to input the scenarios.

[0213] Next, the server receives the input scenario and analyzes it using generative AI tools. This analysis utilizes natural language processing techniques to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. For example, the OpenAI API might be used for this generative AI.

[0214] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated stories and diagrams are then provided from the server to the user's terminal as answers.

[0215] As a concrete example, consider a scenario where a user inputs a presentation scenario for a new product. This scenario includes information such as "the features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates prompt messages like the following.

[0216] Example of a prompt:

[0217] Scenario: In a new product presentation, emphasize the product's features, comparison with competitors, and market positioning.

[0218] Keywords: New product, presentation, features, competitors, market

[0219] Please generate an advertising story based on this scenario.

[0220] If this prompt is input into a generative AI model, it may generate an advertising story like the following.

[0221] Example of a generated ad story:

[0222] The new X-100 boasts innovative features not found in other products. Compared to its competitors, the X-100 offers superior performance and cost-effectiveness. Positioned in the market, the X-100 is the perfect choice for users seeking cutting-edge technology. Get your X-100 today and experience the future.

[0223] In this way, users can easily generate effective advertising stories. This system automates the process from scenario analysis to story generation for advertising campaigns, enabling the rapid creation of effective and consistent advertisements.

[0224] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0225] Step 1:

[0226] The user uses their device to input the document or advertising campaign scenario they want to create. The entered scenario is sent to the server in text format.

[0227] Step 2:

[0228] The server analyzes the received scenario. This analysis uses natural language processing techniques. Specifically, it extracts keywords and themes from the scenario. The input to this process is the text data of the scenario, and the output is a list of the extracted keywords and themes.

[0229] Step 3:

[0230] The server generates prompt sentences based on the extracted keywords and themes. The generated prompt sentences are then input to a generative AI model. The input to this process is a list of keywords and themes, and the output is prompt sentences.

[0231] Step 4:

[0232] The server uses a generative AI model to generate a story based on the prompt text. Specifically, it uses the OpenAI API to generate the story. The input to this process is the prompt text, and the output is the generated story.

[0233] Step 5:

[0234] The server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The input for this process is internal data, and the output is the generated diagram.

[0235] Step 6:

[0236] The server provides the generated story and diagram together as the answer to the user's terminal. The input to this process is the generated story and diagram, and the output is the answer data sent to the user's terminal.

[0237] Step 7:

[0238] Users review the stories and diagrams provided through their devices and make modifications or additions as needed. They then finalize the materials and advertising campaign content. The input for this process is the response data sent from the server, and the output is the completed materials and advertising campaign content.

[0239] (Example 3)

[0240] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0241] Traditional document creation systems required users to manually search for materials, select appropriate figures, and create stories, which was time-consuming and labor-intensive. Furthermore, specialized knowledge was required to select appropriate figures, making it difficult for users to create documents efficiently. Inconsistent quality of generated documents was also a challenge.

[0242] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0243] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating a story based on the input scenario using a generative AI, means for using documents stored within the company as external information, selecting and generating appropriate figures using image recognition technology, and means for providing the generated story and figures together as a response. This enables the user to efficiently create high-quality documents.

[0244] A "user" is an individual or organization that uses the system to create materials.

[0245] A "scenario" is text information that indicates the content and purpose of the document that the user wants to create.

[0246] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[0247] "Materials accumulated within the company" refers to information resources such as documents, images, and data stored within a company or organization.

[0248] "External information" refers to documents accumulated within the company, which are the source of information for the system to use.

[0249] "Image recognition technology" is a technology that analyzes images and diagrams to understand their content.

[0250] An "appropriate diagram" is a chart or image selected to visually represent information related to the scenario.

[0251] "Means of generation" refers to the means of creating a new diagram based on the selected diagram.

[0252] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[0253] "System" refers to a computer-based device or software for performing a series of processes, including the means described above.

[0254] This invention relates to a system for enabling users to efficiently create high-quality materials. Specific embodiments of this system are described below.

[0255] System Overview

[0256] This system includes the following main means:

[0257] 1. A means for users to input the scenario of the document they want to create.

[0258] 2. Generative AI means for generating a story based on an input scenario

[0259] 3. A means of using internally accumulated data as external information, and selecting and generating appropriate figures using image recognition technology.

[0260] 4. A means of answering with the generated story and diagram together.

[0261] Hardware and software to be used

[0262] The server runs the system using the following hardware and software:

[0263] Hardware: High-performance processor, sufficient memory, storage devices, network interface

[0264] Software: Google® Cloud Vision API, Amazon® Rekognition, Adobe Illustrator®, Canva®, Generative AI models (e.g., OpenAI's GPT-4)

[0265] Data processing and data calculation

[0266] The server performs data processing and calculations using the following steps:

[0267] 1. Data collection:

[0268] The server accesses internal databases and file servers to search for and retrieve relevant documents, including PDFs, Word documents, and presentation files.

[0269] 2. Application of image recognition technology:

[0270] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the collected materials.

[0271] 3. Selection of Appropriate Figures:

[0272] Based on the analysis results, the server filters and selects appropriate figures related to the scenario.

[0273] 4. Generation of Figures:

[0274] The server uses Adobe Illustrator or Canva to generate new figures based on the selected figures.

[0275] 5. Generation of Stories:

[0276] The server uses a generative AI model (GPT-4) to generate a story based on the generated figures.

[0277] 6. Provision of Answers:

[0278] The server combines the generated figures and story into a single document and provides it to the user.

[0279] Specific Example

[0280] For example, when a user wants to create a document explaining the features of a new product, the following prompt sentence is input into the generative AI model:

[0281] "Please generate a figure to explain the features of the new product. Select and generate appropriate figures based on the in-house materials."[[ID=4;2]]

[0282] When this prompt sentence is input, the server generates figures according to the above procedure and provides them together with the story. The user can create a document based on this answer.

[0283] In this way, the user can efficiently create high-quality documents. The flow of the specific process in Example 3 will be described using FIG. 15.

[0284] Step 1: Gathering materials

[0285] The server accesses internal databases and file servers to search for and retrieve relevant documents. The user provides a scenario for the document they wish to create as input. Based on this scenario, the server extracts keywords and uses these keywords to search for documents. The output includes relevant documents such as PDFs, Word documents, and presentation files.

[0286] Step 2: Application of image recognition technology

[0287] The server applies image recognition technology to the collected data. The data collected in Step 1 is provided as input. The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the data. Specifically, the server processes each data item sequentially and extracts the content of images and charts as text data. The analysis results are obtained as output.

[0288] Step 3: Select the appropriate figure

[0289] The server uses image recognition technology to select appropriate figures from the document. The analysis results obtained in step 2 are given as input. The server matches keywords related to the scenario with the analysis results and filters and selects the most relevant figures. Specifically, the server calculates a relevance score and selects the figures with the highest scores. The selected figures are obtained as output.

[0290] Step 4: Generate the diagram

[0291] The server generates a new diagram based on the selected diagram. The diagram selected in step 3 is given as input. Using Adobe Illustrator or Canva, the server uses the selected diagram as a template, adding the necessary information to create a new diagram. Specifically, the server adjusts the diagram's layout and adds annotations and data. The output is the generated new diagram.

[0292] Step 5: Generating the Story

[0293] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related descriptive text and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[0294] Step 6: Provide your response

[0295] The server combines the generated diagrams and stories into a single document and provides it to the user. The input consists of the diagrams generated in step 4 and the stories generated in step 5. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The output is the final document.

[0296] (Application Example 3)

[0297] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0298] Maintenance work in a factory is complex and diverse, making it difficult for workers to quickly grasp the appropriate procedures. Also, due to the vast amount of maintenance records and technical materials, it is difficult to efficiently search for and utilize the necessary information. Therefore, there is a need to improve the efficiency and accuracy of maintenance work.

[0299] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0300] In this invention, the server includes means for inputting a scenario of the material that the user wants to create, generation-based AI means for generating a story based on the input scenario, means for using the materials stored in the company as external information to generate a diagram, means for answering the generated story and diagram together, means for using the maintenance records and technical materials stored in the factory as external information and selecting / generating an appropriate diagram using image recognition technology, and means for assisting the maintenance work based on the selected diagram. As a result, it becomes possible for maintenance workers to quickly obtain the necessary information and perform their work efficiently and accurately.

[0301] "User" refers to a person who creates materials or performs maintenance work using the system.

[0302] "Scenario" refers to a plan or configuration indicating the content and purpose of the material that the user wants to create.

[0303] "Generation-based AI means" refers to artificial intelligence technology that generates a story based on the input scenario.

[0304] "Materials stored in the company" refers to information such as technical materials and maintenance records created in the past within the company.

[0305] "External information" refers to materials and data stored in the company for the system to use.

[0306] "Methods for generating diagrams" refers to the technology of selecting and generating appropriate diagrams based on materials accumulated within the company.

[0307] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[0308] "Maintenance records" refer to information that records the history and details of maintenance work performed within the factory.

[0309] "Technical documents" refer to technical information and procedures related to equipment and systems within a factory.

[0310] "Means of supporting maintenance work" refers to support technologies that enable maintenance workers to perform tasks efficiently and accurately based on selected diagrams.

[0311] The system for implementing this invention is configured as follows: First, an interface is provided for the user to input a scenario for the document they wish to create. The user inputs the scenario through this interface.

[0312] Next, the server generates a story based on the input scenario. This uses generative AI tools, leveraging natural language processing techniques to analyze the scenario and generate an appropriate story. For example, the transformers® library from Hugging Face® can be used as a generative AI tool.

[0313] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to generate these diagrams. Specifically, software such as OpenCV®, PIL (Python Imaging Library), and pytesseract® are used to extract and analyze text and features from images.

[0314] Maintenance records and technical documents accumulated within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[0315] The generated story and diagrams are provided to the user together. This allows the user to quickly obtain the necessary information and work efficiently and accurately.

[0316] As a concrete example, the following prompt statement can be used.

[0317] Example of a prompt:

[0318] "Based on the following images, please generate a story about the parts replacement procedure: ['maintenance_step1.png', 'maintenance_step2.png']"

[0319] Based on this prompt, the generative AI system generates a story about the parts replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[0320] In this way, maintenance work on factory robots can be made more efficient, and the burden on workers can be reduced.

[0321] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0322] Step 1:

[0323] Enter the scenario for the document you want to create.

[0324] Input: A scenario in which the user enters information through an interface.

[0325] Output: Input scenario data.

[0326] Specific operation: The user accesses the system interface and enters the scenario for the document they want to create in text format.

[0327] Step 2:

[0328] The server generates a story based on the input scenario.

[0329] Input: The entered scenario data.

[0330] Output: The generated story.

[0331] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generates an appropriate story using natural language processing techniques.

[0332] Step 3:

[0333] The server uses internally stored data as external information to generate diagrams.

[0334] Input: Documents accumulated within the company.

[0335] Output: The generated figure.

[0336] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate figures from documents stored within the company.

[0337] Step 4:

[0338] The server utilizes maintenance records and technical documents accumulated within the factory as external information, and uses image recognition technology to select and generate appropriate diagrams.

[0339] Input: Maintenance records and technical documents accumulated within the factory.

[0340] Output: Selected figure.

[0341] Specific operation: The server uses image recognition technology to select and generate necessary diagrams from maintenance records and technical documents.

[0342] Step 5:

[0343] The server will provide the generated story and diagram together as the answer.

[0344] Input: Generated story and selected diagram.

[0345] Output: The story and diagrams provided to the user.

[0346] Specific operation: The server combines the generated story and selected diagrams and provides them to the user. The user can then create materials based on this.

[0347] Step 6:

[0348] Maintenance work is performed based on the stories and diagrams provided by the user.

[0349] Input: Provided story and diagram.

[0350] Output: Efficient and accurate maintenance work.

[0351] Specific operation: Users perform maintenance tasks efficiently and accurately by referring to the provided stories and diagrams.

[0352] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0353] "Example of form 1"

[0354] One embodiment of the present invention provides a system that includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for using documents accumulated within the company as external information to generate a diagram, means for providing the generated story and diagram together as a response, and an emotion engine that recognizes the user's emotions.

[0355] "Example of form 2"

[0356] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a joyful expression, the emotion engine captures that information, and the generative AI generates a story based on that emotion. Specifically, if joy is recognized, the generative AI generates a story that includes positive elements.

[0357] "Example of form 3"

[0358] Furthermore, the emotion engine selects and generates appropriate images based on the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures this information, and the image generation mechanism selects and generates an image that conveys surprise or novelty.

[0359] The following describes the processing flow for each example of the form.

[0360] "Example of form 1"

[0361] Step 1: Enter the scenario for the document you want to create.

[0362] Step 2: Based on the input scenario, the generative AI generates a story.

[0363] Step 3: Use internally accumulated data as external information to generate diagrams.

[0364] Step 4: Answer by combining the generated story and diagram.

[0365] Step 5: The emotion engine, which recognizes the user's emotions, starts operating.

[0366] "Example of form 2"

[0367] Step 1: The emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[0368] Step 2: Based on the emotions recognized by the emotion engine, the generative AI generates a story.

[0369] "Example of form 3"

[0370] Step 1: The emotion engine recognizes the user's emotions.

[0371] Step 2: The emotion engine selects and generates an appropriate diagram based on the emotions it recognizes.

[0372] (Example 1)

[0373] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0374] Traditional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which is time-consuming and labor-intensive. Furthermore, the lack of feedback that considers user emotions leads to decreased efficiency and quality in document creation. Additionally, the lack of effective means to utilize internally accumulated documents can result in a lack of consistency and reliability in the materials.

[0375] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0376] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored documents as external information, means for providing the generated story and diagram together as a response, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback that responds to the user's emotions, and enables the effective use of internal company documents.

[0377] "A means for users to input the scenario of the document they want to create" refers to an interface that allows users to freely input text, and is a means for inputting the scenario of the document.

[0378] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios, and are methods that analyze scenarios using natural language processing technology to generate stories.

[0379] "A means of generating diagrams by using internally accumulated data as external information" refers to a method of generating diagrams by referring to data stored in an internal database and using data visualization technology.

[0380] "A means of providing a combined answer using generated stories and diagrams" refers to a method of integrating generated stories and diagrams and providing them to the user as a single document.

[0381] An "emotion engine" refers to a technology that analyzes user input and responses to recognize emotions. It uses natural language processing technology to analyze user emotions and provide appropriate feedback.

[0382] This invention is a system that allows users to input a scenario for a document they wish to create, generates a story and diagrams based on that scenario, and ultimately provides it as a single document. A specific embodiment of this system is described below.

[0383] System Configuration

[0384] This system consists of the following main components:

[0385] 1. Text input interface

[0386] 2. Generative AI means

[0387] 3. Database Reference Methods

[0388] 4. Data Visualization Methods

[0389] 5. Integration means

[0390] 6. Emotional Engine

[0391] Text input interface

[0392] The user uses the terminal's text input interface to enter the scenario for the document they want to create. This interface is designed to allow the user to freely enter text. For example, the user might type "New Product Presentation".

[0393] Generative AI means

[0394] The server receives the user-entered scenario and inputs it as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model then generates an appropriate story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[0395] Database Reference Means

[0396] The server accesses the company's internal database and searches for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib® and Pandas® to format and visualize the data.

[0397] Data visualization means

[0398] The server generates graphs using data visualization techniques based on information retrieved from the database. For example, it extracts necessary information from the database using SQL queries and generates graphs using Python's Matplotlib library. The generated graphs are saved as image files.

[0399] Integration means

[0400] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX® or Microsoft Word API). The generated document is saved as a PDF file.

[0401] Emotional Engine

[0402] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[0403] Examples of specific cases and prompt statements

[0404] Specific example

[0405] The user enters "New product presentation" into the text input interface.

[0406] The server uses a generation AI model to generate stories that include content such as "new product features, target market, competitive analysis, and sales strategy."

[0407] The server accesses past sales data and market research results from the company's internal database and creates relevant graphs and charts.

[0408] Finally, the generated stories and diagrams are integrated and presented to the user.

[0409] Example of a prompt

[0410] Please enter a presentation scenario for your new product. For example, include details such as "new product features, target market, competitive analysis, and sales strategy."

[0411] In this way, users can easily create high-quality documents.

[0412] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0413] Step 1:

[0414] The user enters the scenario.

[0415] The user uses the terminal's text input interface to enter the scenario for the document they want to create. For example, they might enter "New Product Presentation." The entered scenario is sent to the server in real time.

[0416] Input: Scenario text entered by the user

[0417] Output: Scenario text sent to the server

[0418] Step 2:

[0419] The server generates stories using an AI model.

[0420] The server inputs the received scenario as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[0421] Input: Scenario text

[0422] Output: Generated story text

[0423] Step 3:

[0424] The server generates the diagram by referencing the company's internal database.

[0425] The server accesses the company's internal database to search for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib and Pandas to format and visualize the data.

[0426] Input: Data retrieved from the company's internal database

[0427] Output: Image files of the generated graphs and charts

[0428] Step 4:

[0429] The server integrates the generated stories and diagrams.

[0430] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX or Microsoft Word APIs). The generated document is saved as a PDF file.

[0431] Input: Generated story text, image files of graphs and charts

[0432] Output: Integrated PDF document

[0433] Step 5:

[0434] The server recognizes the user's emotions and provides feedback.

[0435] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[0436] Input: User feedback text

[0437] Output: Analysis results and feedback text from the emotion engine.

[0438] (Application Example 1)

[0439] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0440] The traditional advertising campaign creation process involved a great deal of effort and time, including scenario writing, story generation, and visual creation. Furthermore, it was difficult to propose optimal advertising materials that considered user emotions, making it challenging to quickly create effective advertising campaigns. This resulted in advertising agencies and marketing personnel being unable to work efficiently, making it difficult to maximize the effectiveness of their advertising.

[0441] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0442] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored documents as external information, means for providing the generated story and diagram together as a response, means including an emotion engine for recognizing the user's emotions, and means for generating visuals for advertising based on the generated story. This streamlines the advertising campaign creation process and enables the suggestion of optimal advertising materials that respond to the user's emotions.

[0443] A "user" is an individual or organization that uses the system to create a scenario for a document.

[0444] A "scenario" is text information that describes the content and structure of the document that the user wants to create.

[0445] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[0446] "Documents accumulated within the company" refers to a collection of documents and data created in the past within a company or organization.

[0447] "Methods for generating diagrams" refers to the technology for creating appropriate diagrams using materials accumulated within the company.

[0448] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[0449] An "emotion engine" is a technology that recognizes the user's emotions and makes appropriate suggestions based on those emotions.

[0450] "Advertising visuals" refer to images and graphic materials used in advertising campaigns.

[0451] The system for implementing this invention has the following configuration. First, it provides a text input interface for the user to input the scenario of the document they wish to create. The user can use this interface to freely write the scenario.

[0452] Next, a generative AI system is used to generate a story based on the input scenario. This generative AI system analyzes the scenario using natural language processing techniques and generates an appropriate story. Specifically, it uses the OpenAI API to generate a story from the scenario.

[0453] Based on the generated story, a system for generating diagrams using internally stored materials is activated. This system uses image recognition technology to select and generate appropriate diagrams from internal documents. PIL (Python Imaging Library) is used to create visuals that match the generated story.

[0454] Furthermore, the system includes a means of providing the user with the generated story and diagrams together. This means integrates the generated content and provides the user with a consistent answer.

[0455] It also includes a mechanism that incorporates an emotion engine to recognize the user's emotions. This emotion engine analyzes the user's emotions and suggests the most suitable advertising material based on that analysis. It uses the emotion_recognition library to recognize the user's emotions.

[0456] Finally, it includes a means of generating visuals for advertising based on the generated story. This means creates images and graphic materials to be used in the advertising campaign based on the generated story.

[0457] As a concrete example, if a user inputs "an advertising campaign for a new smartphone" as a scenario, the generating AI will produce a story like the following:

[0458] Example of a prompt:

[0459] Advertising campaign scenario: Advertising campaign for a new smartphone product

[0460] story:

[0461] By inputting this prompt into the AI ​​generation model, an appropriate story is generated. Based on the generated story, visuals for advertising are created and provided to the user.

[0462] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0463] Step 1:

[0464] Enter the scenario for the document you want to create.

[0465] Users freely write advertising campaign scenarios using a text input interface. The entered scenarios are then sent to the system.

[0466] Input: Scenario text entered by the user

[0467] Output: Scenario text is sent to the system.

[0468] Step 2:

[0469] The server generates a story based on the input scenario.

[0470] The server uses generative AI tools to analyze the input scenario and generate an appropriate story. Specifically, it uses the OpenAI API to generate a story from the scenario.

[0471] Input: Scenario text

[0472] Output: Generated story text

[0473] Step 3:

[0474] The server generates a diagram based on the generated story.

[0475] The server utilizes internally stored data, uses image recognition technology to select and generate appropriate diagrams, and then uses PIL (Python Imaging Library) to create visuals that match the generated story.

[0476] Input: Generated story text

[0477] Output: Generated diagram (visual)

[0478] Step 4:

[0479] The server will provide the generated story and diagram together as the answer.

[0480] The server integrates the generated stories and diagrams to provide users with a consistent answer.

[0481] Input: Generated story text, generated diagrams

[0482] Output: Integrated story and diagram

[0483] Step 5:

[0484] The server recognizes the user's emotions.

[0485] The server uses an emotion engine to analyze the user's emotions. It recognizes the user's emotions using the emotion_recognition library.

[0486] Input: User's emotional data (e.g., facial expression images or audio data)

[0487] Output: Recognized emotion information

[0488] Step 6:

[0489] The server generates visuals for advertising based on the generated story.

[0490] The server creates images and graphic materials to be used in the advertising campaign based on the generated story.

[0491] Input: Generated story text, recognized emotion information

[0492] Output: Visuals for advertising

[0493] Step 7:

[0494] The server provides the final advertising campaign materials to the user.

[0495] The server integrates the generated stories, diagrams, and advertising visuals and delivers them to the user.

[0496] Input: Generated story text, generated diagrams, advertising visuals

[0497] Output: Final advertising campaign materials

[0498] (Example 2)

[0499] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0500] Conventional document creation systems can generate stories based on user-inputted scenarios, but they have the problem of not being able to adjust the stories to take into account the user's emotions. Furthermore, when the generated stories and diagrams are provided together, appropriate adjustments based on the user's emotions are not made, making it difficult to create documents that reflect the user's intentions and feelings.

[0501] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0502] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating a story based on the input scenario using a generative AI, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, means for sensing the tone of the user's voice and facial expressions, means for recognizing the user's emotions using an emotion engine, and means for adjusting the story based on the recognized emotions. This makes it possible to provide a story and diagram that reflect the user's emotions.

[0503] "User" refers to a person who uses the system to input a scenario for a document and receives the generated story and diagrams.

[0504] A "scenario" refers to text-based input information that indicates the content and structure of the document the user wants to create.

[0505] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios.

[0506] "Natural language processing technology" refers to techniques for analyzing scenarios and extracting keywords and themes.

[0507] "Methods for generating diagrams" refers to technologies that utilize internally accumulated data as external information to generate appropriate diagrams.

[0508] "Means of sensing" refers to technology for sensing the tone of the user's voice and facial expressions in real time.

[0509] An "emotion engine" refers to technology that recognizes a user's emotions from the perceived tone of voice and facial expressions.

[0510] "Means of adjustment" refers to techniques for appropriately adjusting stories that have been generated based on perceived emotions.

[0511] "Story" refers to a series of sentences or contents generated by generative AI based on a scenario.

[0512] "Means of providing answers" refers to the technology used to deliver generated stories and diagrams to users.

[0513] This invention is a system that allows users to input a scenario for a document they wish to create, generates a story based on that scenario, and further provides a story that reflects the user's emotions. A specific embodiment of this system is described below.

[0514] Hardware and software to be used

[0515] 1. Server:

[0516] Natural language processing techniques: Generative AI models such as BERT and GPT-3 are used.

[0517] Emotion engine: Software used to analyze the tone of the user's voice and facial expressions.

[0518] Database: A database used to store internal company data and make it available as external information.

[0519] 2. Terminal:

[0520] Input interface: A keyboard or touchscreen for the user to input scenarios.

[0521] Camera and microphone: Hardware used to detect the user's voice tone and facial expressions.

[0522] Data processing and data calculation

[0523] 1. Scenario Input and Analysis:

[0524] The user enters the scenario in text format using the terminal's input interface.

[0525] The server receives the input scenario and performs analysis using natural language processing techniques. Specifically, it uses generative AI models such as BERT and GPT-3 to extract keywords and themes from the scenario.

[0526] 2. Story generation:

[0527] The server generates a story using a generative AI model based on extracted keywords and themes. For example, if the scenario is "New Product Presentation," the server extracts keywords such as "New Product Features," "Comparison with Competitors," and "Market Positioning," and generates a story based on them.

[0528] 3. Recognizing emotions and adjusting the story:

[0529] The device uses a camera and microphone to sense the user's voice tone and facial expressions in real time.

[0530] The device uses an emotion engine to recognize the user's emotions from the tone of voice and facial expressions it perceives. For example, if the user shows a joyful expression, the device sends that information to the emotion engine, which recognizes the emotion of joy.

[0531] The server adjusts the generated story based on the recognized emotions. For example, if positive emotions are recognized, positive elements will be added to the story.

[0532] 4. Providing the final story:

[0533] The server then provides the user with the finalized story. Specifically, it provides the generated story and diagram together as the answer.

[0534] Examples of specific cases and prompt statements

[0535] Specific example

[0536] Scenario: New product presentation

[0537] Keywords to be extracted: New product features, comparison with competitors, market positioning

[0538] The generated story: A presentation that highlights the new product's features, demonstrates its competitive advantages, and clearly defines its market positioning.

[0539] Example of a prompt

[0540] "Please enter a scenario for your new product presentation. Our generative AI will analyze the scenario, extract keywords, and generate a story."

[0541] "To recognize the user's emotions, the system detects their tone of voice and facial expressions. Based on the emotions recognized by the emotion engine, the generative AI generates a story."

[0542] In this way, the system can generate and deliver the most suitable story based on the user's input and emotions.

[0543] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0544] Step 1:

[0545] The user enters the scenario.

[0546] Input: The user enters the scenario in text format using the terminal's input interface.

[0547] Specific action: The user types "New product presentation".

[0548] Output: The entered scenario is saved on the terminal and sent to the server.

[0549] Step 2:

[0550] The server receives the scenario and analyzes it using natural language processing techniques.

[0551] Input: Scenario text sent from the terminal.

[0552] Specific operation: The server uses generative AI models such as BERT or GPT-3 to analyze the text "New product presentation".

[0553] Output: Keywords and themes extracted from the scenario.

[0554] Step 3:

[0555] The server extracts keywords and themes from the scenario.

[0556] Input: Scenario text analyzed using natural language processing techniques.

[0557] Specific operation: The server extracts keywords such as "features of the new product," "comparison with competitors," and "market positioning."

[0558] Output: A list of extracted keywords and topics.

[0559] Step 4:

[0560] The server generates a story based on extracted keywords and themes.

[0561] Input: A list of extracted keywords or topics.

[0562] Specific operation: The server uses the generated AI model to create a presentation that "emphasizes the features of the new product, demonstrates its competitive advantages, and clarifies its market positioning."

[0563] Output: The generated story.

[0564] Step 5:

[0565] The device detects the user's voice tone and facial expressions.

[0566] Input: User's real-time voice tone and facial expressions.

[0567] Specific operation: The device uses its camera and microphone to detect the user's smile and cheerful tone of voice.

[0568] Output: Data on the perceived tone of voice and facial expressions.

[0569] Step 6:

[0570] The device uses an emotion engine to recognize the user's emotions.

[0571] Input: Data on the perceived tone of voice and facial expressions.

[0572] Specific operation: The device uses an emotion engine to recognize the user's emotions of joy.

[0573] Output: Recognized emotion data.

[0574] Step 7:

[0575] The server adjusts the story based on the emotions it recognizes.

[0576] Input: Recognized emotion data and generated story.

[0577] Specific action: The server adds positive elements to the story and generates a presentation with a brighter tone.

[0578] Output: The refined, final story.

[0579] Step 8:

[0580] The server provides the final story to the user.

[0581] Input: The final, adjusted storyline.

[0582] Specific operation: The server displays a "positive-toned presentation that highlights the features of the new product, demonstrates its competitive advantages, and clarifies its market positioning" to the user.

[0583] Output: The final story provided to the user.

[0584] (Application Example 2)

[0585] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0586] Conventional document creation systems can generate stories based on user-inputted scenarios, but they cannot create stories that take into account the user's emotions. This makes it difficult to create documents that align with the user's intentions and feelings. Furthermore, when users submit answers using the generated stories and diagrams together, it is not possible to reflect their emotions, creating a need for more effective document creation.

[0587] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0588] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, an emotion engine means for recognizing emotions from the user's voice tone and facial expressions, and means for generating a story based on the recognized emotions. This makes it possible to generate a story that reflects the user's emotions, enabling more effective document creation.

[0589] "User" refers to a person who uses the system to create documents.

[0590] A "scenario" refers to a document or plot that outlines the content and structure of the material that the user wants to create.

[0591] "Generative AI" refers to artificial intelligence technology that generates stories based on input scenarios.

[0592] "Documents accumulated within the company" refers to data and information collected and stored within a company or organization.

[0593] "External information" refers to data obtained from sources other than internally stored documents within the company.

[0594] "Figures" refer to visual elements such as graphs, charts, and illustrations included in a document.

[0595] An "emotion engine" refers to technology that recognizes emotions from the tone of a user's voice and facial expressions.

[0596] "Voice tone" refers to the vocal characteristics of a user's voice, such as its pitch, strength, and rhythm.

[0597] "Facial expression" refers to the movements and expressions of the user's face.

[0598] "Story" refers to a series of sentences or pieces of content generated based on a scenario.

[0599] "Answer" refers to the act of providing the generated story and diagram to the user.

[0600] The system for implementing this invention includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents accumulated within the company as external information, means for providing the generated story and diagram together as a response, an emotion engine means for recognizing emotions from the tone of the user's voice and facial expressions, and means for generating a story based on the recognized emotions.

[0601] System program

[0602] The server executes the following program: First, the user inputs a scenario using a device such as a smartphone or PC. Next, a generative AI analyzes the scenario using natural language processing technology and extracts keywords and themes. After that, it generates a story based on the extracted information.

[0603] Furthermore, the emotion engine recognizes the user's tone of voice and facial expressions, and provides this emotional information to the generative AI. The generative AI adjusts the story considering the emotional information to generate content that better aligns with the user's intentions. The generated story is supplemented by a means of generating appropriate diagrams using internally accumulated materials as external information. Finally, the generated story and diagrams are provided to the user together.

[0604] Hardware and software to be used

[0605] Hardware: Smartphones, PCs, servers

[0606] Software: OpenCV (image processing), Transformers (generative AI and emotion recognition), TextBlob (registered trademark) (text analysis)

[0607] Data processing and data calculation

[0608] 1. Image Processing: Images captured with a smartphone or computer camera are used to perform face recognition and analyze facial expressions using OpenCV.

[0609] 2. Voice Processing: Audio recorded with a smartphone or computer microphone is converted into text, and emotions are recognized by an emotion engine.

[0610] 3. Story Generation: Generative AI (such as GPT-3) is used to generate stories based on recognized emotions and scenarios.

[0611] 4. Diagram generation: Select and generate appropriate diagrams from the materials accumulated within the company.

[0612] Specific example

[0613] For example, if a user wants to create an advertisement about the camera features of a new smartphone, they would enter a prompt like this:

[0614] Example of a prompt

[0615] Create an advertisement story for a product. The user is feeling happy and the sentiment is positive. The product information is: A new smartphone with advanced camera features.

[0616] Based on this prompt, the generative AI generates a positive advertising story highlighting the new smartphone's camera features. The emotion engine recognizes the user's expressions of joy and provides this information to the generative AI, resulting in a more emotionally resonant advertising story. The generated story is then presented to the user, with appropriate diagrams generated using internally stored materials.

[0617] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0618] Step 1:

[0619] The user enters the scenario using a terminal.

[0620] Input: Scenario (text format) for the document the user wants to create.

[0621] Output: Input scenario (text data)

[0622] Specific operation: The user enters a scenario into the terminal's input screen and presses the submit button. The terminal sends the entered scenario to the server.

[0623] Step 2:

[0624] The server uses generative AI to analyze the scenario and extract keywords and themes.

[0625] Input: Entered scenario (text data)

[0626] Output: Extracted keywords and themes (text data)

[0627] Specific operation: The server uses generative AI (natural language processing technology) to analyze the scenario and extract important keywords and themes. The extracted information is used in the next step.

[0628] Step 3:

[0629] The server uses an emotion engine to recognize the user's voice tone and facial expressions.

[0630] Input: User's voice data and image data

[0631] Output: Recognized emotions (text data)

[0632] Specific operation: The user records voice and facial expressions using the device's camera and microphone. The server uses OpenCV to analyze facial expressions from the image data and an emotion engine to analyze voice tone from the voice data. The recognized emotion is generated as a result of the analysis.

[0633] Step 4:

[0634] The server generates a story based on the emotions it recognizes.

[0635] Input: Extracted keywords or themes, perceived emotions

[0636] Output: Generated story (text data)

[0637] Specific operation: The server uses generative AI to generate a story based on extracted keywords, themes, and recognized emotions. The generated story is then used in the next step.

[0638] Step 5:

[0639] The server uses internally stored data to generate appropriate diagrams.

[0640] Input: Generated story (text data)

[0641] Output: Generated figure (image data)

[0642] Specific operation: The server uses image recognition technology to select and generate appropriate diagrams from the company's internally stored materials. The generated diagrams are then used in the next step.

[0643] Step 6:

[0644] The server provides the user with the generated story and diagram together.

[0645] Input: Generated story (text data), generated diagram (image data)

[0646] Output: Materials provided to the user (text data and image data)

[0647] Specific operation: The server combines the generated story and diagram into a single document and sends it to the user's terminal. The user can then view the document on their terminal.

[0648] (Example 3)

[0649] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0650] Traditional document creation systems had the problem of requiring users to manually create documents, which was time-consuming and laborious. Furthermore, they lacked a function to select appropriate diagrams based on the user's emotions, sometimes resulting in documents that didn't match the user's intentions. Additionally, there was a lack of effective means to utilize information accumulated within the company, potentially leading to a decline in document quality.

[0651] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0652] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored information as external information, means for providing the generated story and diagram together as a response, means for analyzing the user's emotions, and means for selecting an appropriate diagram based on the analyzed emotions. This enables the user to efficiently create high-quality documents.

[0653] "User" refers to an individual or organization that uses the system to create documents.

[0654] A "scenario" refers to a plan or story that outlines the content and structure of the material that the user wants to create.

[0655] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios.

[0656] "Internal information" refers to data and documents collected and stored within a company or organization.

[0657] "External information" refers to diagrams and data generated using information stored internally.

[0658] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on internally stored information.

[0659] "Image recognition technology" refers to the technology used to analyze image data and understand and classify its content.

[0660] "Means of analyzing emotions" refers to technologies used to analyze the emotions of users.

[0661] "Generated stories" refer to narratives or descriptions created by generative AI methods.

[0662] "Means of providing answers" refers to the technology used to deliver generated stories and diagrams to users.

[0663] This invention is a system for users to efficiently create high-quality materials. Specific embodiments of this system are described below.

[0664] System Configuration

[0665] This system consists of three main elements: a server, a terminal, and a user. The server handles data acquisition, image recognition, sentiment analysis, generation of diagrams and stories, and delivery to the user. The terminal is responsible for capturing user input and emotions. The user uses the system to create data.

[0666] Hardware and software to be used

[0667] Server: A server with a high-performance processor and large memory capacity is required. The server also needs storage to access the database and retrieve data.

[0668] Device: A computer or smart device equipped with a camera and microphone is required. This allows for the capture of the user's facial expressions and voice.

[0669] software:

[0670] Image recognition technology: Uses image recognition services such as Google Cloud Vision API and Amazon Rekognition.

[0671] Emotion analysis technology: Uses Microsoft Azure's Emotion API, etc.

[0672] Generative AI model: Uses natural language processing models such as OpenAI's GPT-4.

[0673] Program processing

[0674] The server first accesses the company's internal database and retrieves stored documents. These documents include PDFs, Word documents, and presentation slides. Next, the server uses image recognition technology to analyze the images within the documents and selects appropriate figures based on specific keywords and context.

[0675] The device captures the user's facial expressions with its camera and sends them to the emotion engine. The emotion engine analyzes the user's emotions and sends the results to the server. The server selects an appropriate image based on the analysis results.

[0676] The server sends the selected figure along with a prompt message to input into the generating AI model. The generating AI model generates a story based on the input prompt message and provides it to the user along with the selected figure.

[0677] Specific examples and prompt statements

[0678] For example, if a user wants to create a document related to "new product features," the server selects a diagram related to "new product features" from within the document. Next, it inputs a prompt message like the following into the generating AI model:

[0679] Prompt: "Select a diagram to illustrate the features of the new product and generate a story."

[0680] The generative AI model generates a story based on this prompt and provides it to the user along with the selected figure.

[0681] Furthermore, if the user shows a surprised expression, the server inputs the following prompt message into the AI ​​model:

[0682] Prompt: "Select an appropriate image to describe the user's surprise and generate a story."

[0683] The generative AI model generates a story based on this prompt and presents it to the user along with illustrations that convey surprise and freshness.

[0684] In this way, users can efficiently create high-quality materials. The flow of the specific processing in Example 3 will be explained using Figure 21.

[0685] Step 1: Obtaining materials

[0686] The server accesses the company's internal database and retrieves stored documents. A search query for the documents is required as input. The server searches the database for the relevant documents and retrieves them in formats such as PDF, Word documents, and presentation slides. The retrieved documents are provided as output. Specifically, the server searches for documents related to "new product features" and downloads the corresponding files.

[0687] Step 2: Selecting a figure using image recognition

[0688] The server extracts diagrams from acquired documents using image recognition technology. Acquired documents are required as input. The server analyzes images within the documents using image recognition services such as Google Cloud Vision API or Amazon Rekognition. The output consists of diagrams selected based on specific keywords and context. Specifically, the server analyzes images within the documents and selects diagrams related to "new product features" or "comparison with competitors."

[0689] Step 3: Emotion Analysis

[0690] The device captures the user's facial expressions with its camera and sends them to the emotion engine. The user's facial image is required as input. The device analyzes the user's emotions using Microsoft Azure's Emotion API, etc. The analyzed emotion data is obtained as output. Specifically, the device uses its camera to capture the user's facial expressions and sends them to the emotion engine.

[0691] Step 4: Generating Diagrams and Stories

[0692] The server sends the selected image along with a prompt to input into the generative AI model. The input requires both the selected image and the prompt. The server uses a generative AI model, such as OpenAI's GPT-4, to generate a story based on the input prompt. The output is the generated story. Specifically, the server sends prompts to the generative AI model such as the following:

[0693] Prompt: "Select a diagram to illustrate the features of the new product and generate a story."

[0694] Step 5: Provision to users

[0695] The server provides the user with the generated story and selected diagrams. The generated story and diagrams are required as input. The server sends these to the user's terminal. The user receives the story and diagrams as output. Specifically, the server sends the generated story and diagrams to the user's terminal, and the user creates materials based on them.

[0696] (Application Example 3)

[0697] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0698] Conventional document creation systems could generate stories and diagrams based on the user's desired document scenario, but they could not generate appropriate diagrams that reflected the user's emotions. Therefore, creating effective documents that resonated with users' feelings was difficult. Furthermore, the lack of technology to analyze user emotions in real time and generate diagrams based on that analysis limited their application in fields such as advertising and presentations.

[0699] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0700] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, emotion recognition means for analyzing the user's emotions, and means for selecting and generating an appropriate diagram based on the analyzed emotions. This enables the creation of effective documents that respond to the user's emotions.

[0701] A "user" is an individual or organization that uses the system to create materials.

[0702] A "scenario" is a plan or outline of a story that shows the content and structure of the material that the user wants to create.

[0703] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[0704] "Documents accumulated within the company" refers to information such as documents, images, and data that have been created and stored within a company or organization in the past.

[0705] "External information" refers to information sources used by the system, including documents accumulated within the company.

[0706] "Methods for generating diagrams" refers to technologies that select and generate appropriate diagrams based on internally accumulated documents and external information.

[0707] A "generated story" is a narrative or explanatory text created based on a scenario using generative AI methods.

[0708] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[0709] "Emotion recognition means" refers to technologies and devices used to analyze a user's emotions.

[0710] "Analyzed emotions" refer to the user's emotional state as detected and analyzed by emotion recognition tools.

[0711] An "appropriate diagram" is a diagram that is best suited to the document, selected based on the analyzed emotions and scenarios.

[0712] The system for implementing this invention is configured as follows: First, an interface is provided for the user to input a scenario for the document they wish to create. This interface allows the user to input the scenario via a terminal such as a personal computer or smartphone.

[0713] Next, a generative AI system operates to generate a story based on the input scenario. This generative AI system analyzes the scenario using natural language processing techniques and generates an appropriate story. The generated story is provided in a format that is easy for the user to understand.

[0714] Furthermore, a mechanism is in place to generate diagrams using internally accumulated data as external information. This mechanism uses image recognition technology to select and generate appropriate diagrams from internal documents. The generated diagrams are then provided to the user along with a story.

[0715] The system also includes an emotion recognition mechanism that analyzes the user's emotions. This mechanism uses smart glasses or a camera-equipped device to analyze the user's facial expressions in real time and detect their emotions. Based on the analyzed emotions, a mechanism is activated to select and generate an appropriate image. This enables the creation of effective materials that are tailored to the user's emotions.

[0716] Specific hardware used will include smart glasses, camera-equipped devices, and personal computers. Software used will include OpenCV (image processing library), Keras (deep learning library), and PIL (Python Imaging Library).

[0717] For example, if a user displays a surprised expression, the emotion recognition system detects this expression and generates a prompt message such as, "Generate the most suitable advertisement for a user who is showing a surprised expression." Based on this prompt message, the generation AI system generates advertisements for new products or special offers that are appropriate for the surprised expression and provides them to the user.

[0718] Furthermore, if a user displays a joyful expression, a prompt message will be generated stating, "Generate the most suitable advertisement for users who are showing joyful expressions," and discount coupons or campaign information corresponding to that joyful expression will be provided.

[0719] In this way, effective material creation and advertising display tailored to the user's emotions can be achieved.

[0720] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0721] Step 1:

[0722] The user inputs the scenario for the document using a terminal.

[0723] Input: Scenario text for the document the user wants to create.

[0724] Output: The scenario text is sent to the server.

[0725] Specific operation: The user enters the document scenario in text format through the terminal interface and presses the submit button.

[0726] Step 2:

[0727] The server receives the scenario text, and the generative AI system analyzes the scenario.

[0728] Input: Scenario text.

[0729] Output: Analyzed scenario data.

[0730] Specific operation: The server analyzes the scenario text using natural language processing techniques and extracts data for story generation.

[0731] Step 3:

[0732] The generative AI system generates a story based on the analyzed scenario data.

[0733] Input: Analyzed scenario data.

[0734] Output: The generated story text.

[0735] Specific operation: The server uses a generative AI model to generate story text from the analyzed scenario data.

[0736] Step 4:

[0737] The server uses internally stored data as external information and activates a mechanism to generate diagrams.

[0738] Input: The generated story text.

[0739] Output: Appropriate diagram.

[0740] Specific operation: The server uses image recognition technology to select and generate diagrams related to the story text from internal company documents.

[0741] Step 5:

[0742] The server provides the user with the generated story and diagram together.

[0743] Input: Generated story text and diagrams.

[0744] Output: Materials provided to the user.

[0745] Specific operation: The server integrates the generated story text and diagrams, creates materials for display to the user, and sends them to the terminal.

[0746] Step 6:

[0747] The device activates an emotion recognition mechanism to analyze the user's emotions in real time.

[0748] Input: Video of the user's facial expressions.

[0749] Output: Analyzed sentiment data.

[0750] Specific operation: The device's camera captures the user's facial expression, and an emotion recognition model is used to analyze the emotion.

[0751] Step 7:

[0752] The server operates a mechanism to select and generate an appropriate image based on the analyzed sentiment data.

[0753] Input: Analyzed sentiment data.

[0754] Output: A diagram corresponding to emotions.

[0755] Specific operation: The server generates prompt sentences based on sentiment data and uses a generative AI model to generate appropriate diagrams.

[0756] Step 8:

[0757] The server provides the user with a diagram that corresponds to their emotions.

[0758] Input: A diagram corresponding to an emotion.

[0759] Output: A diagram provided to the user.

[0760] Specific operation: The server sends the generated diagram to the user's terminal for display.

[0761] As a concrete example, if a user displays a surprised expression, the emotion recognition system detects this expression and generates a prompt message such as, "Generate the most suitable advertisement for a user who is showing a surprised expression." Based on this prompt message, the generation AI system generates advertisements for new products or special offers that are appropriate for the surprised expression and provides them to the user.

[0762] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0763] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0765] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0766] [Second Embodiment]

[0767] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0768] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0769] The data processing device 12 includes a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a “computer” related to the technology of this disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32.

[0770] The processor 28, RAM 30, and storage 32 are connected to the bus 34. The database 24 and communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to the network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0771] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0772] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0773] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0774] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0775] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0776] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0777] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0778] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0779] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0780] "Example of form 1"

[0781] One embodiment of the present invention provides a text input interface as a means for the user to input a scenario for a document they wish to create. This interface is designed to allow the user to freely write a scenario. For example, if the user inputs "New Product Presentation" as the scenario,

[0782] "Example of form 2"

[0783] Generative AI uses natural language processing techniques to analyze input scenarios. This analysis extracts keywords and themes from the scenario, and then generates a story based on them. For example, from a scenario about a "new product presentation," keywords such as "features of the new product," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[0784] "Example of form 3"

[0785] The method for generating the diagrams utilizes internally stored documents as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "new product features" or "comparisons with competitors" might be selected. These diagrams, along with the generated story, are provided as answers, and users create their documents based on them.

[0786] The following describes the processing flow for each example of the form.

[0787] "Example of form 1"

[0788] Step 1: The user enters the scenario through a text input interface. For example, they might enter "New Product Presentation" as the scenario.

[0789] Step 2: The generative AI analyzes the input scenario using natural language processing techniques. This analysis extracts keywords such as "features of the new product," "comparison with competitors," and "market positioning."

[0790] Step 3: A story is generated based on the extracted keywords.

[0791] "Example of form 2"

[0792] Step 1: The generative AI analyzes the input scenario using natural language processing techniques.

[0793] Step 2: A story is generated based on the keywords and themes extracted through analysis.

[0794] Step 3: The generated story is provided to the user.

[0795] "Example of form 3"

[0796] Step 1: Use internally accumulated data as external information.

[0797] Step 2: Use image recognition technology to select the appropriate figure from the document.

[0798] Step 3: The selected figure is generated and provided as the answer along with the generated story.

[0799] (Example 1)

[0800] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0801] Traditional document creation systems required users to manually conceive of document content and create diagrams, which was time-consuming and labor-intensive. Furthermore, the quality and consistency of documents depended on the user's skills, making it difficult to produce documents of uniform quality. Additionally, there was a lack of effective means to utilize internally accumulated documents, resulting in insufficient information reuse.

[0802] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0803] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating prompt sentences based on the input scenario, means for sending the generated prompt sentences to a generation AI model to generate the document, means for returning the generated document to the user, means for using documents accumulated within the company as external information to generate a diagram, and means for providing the generated document and diagram together as a response. This makes it possible for users to easily and automatically generate high-quality documents. Furthermore, by effectively utilizing information accumulated within the company, information reuse is promoted and the efficiency of document creation is improved.

[0804] "User" refers to an individual or organization that uses the system to create documents.

[0805] A "scenario" refers to text information that describes the content and structure of the document that the user wants to create.

[0806] A "prompt message" refers to an instruction message sent to the AI ​​model based on a scenario.

[0807] A "generative AI model" refers to an artificial intelligence model that automatically generates documents based on input prompt text.

[0808] "Documents" refers to documents containing text and diagrams generated by a generative AI model.

[0809] "Diagrams" refer to visual content generated based on internal company documents and external information.

[0810] A "server" refers to a computer system that manages the processing of the entire system, receives input from users, sends prompt messages to the generating AI model, and returns the generated materials.

[0811] "External information" refers to data obtained from sources other than internal company documents.

[0812] "Natural language processing technology" refers to the techniques used to analyze scenarios and generate prompts and other information.

[0813] "Image recognition technology" refers to the technology used to select and generate appropriate diagrams from documents accumulated within a company.

[0814] Modes for carrying out the invention

[0815] This invention is a system that allows users to input a scenario for the material they wish to create, and then automatically generates the material based on that scenario. This system consists of multiple components, including a server, a terminal, and a generation AI model.

[0816] System Configuration

[0817] 1. Server

[0818] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt statements, and sends them to the generating AI model. It also receives the data returned by the generating AI model and sends it back to the user. The server uses a database to temporarily store scenarios and generated data.

[0819] 2. Terminal

[0820] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and review generated materials.

[0821] 3. Generative AI Models

[0822] A generative AI model is an artificial intelligence model that automatically generates documents based on prompt messages sent from a server. Examples of generative AI models include OpenAI's GPT-4.

[0823] Program processing

[0824] The server receives a scenario entered by the user through the terminal's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model, specifically instructing it on the content of the scenario.

[0825] The generative AI model generates materials based on the received prompt text. These materials are in a format consistent with the scenario and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[0826] Specific example

[0827] As a concrete example, consider a scenario where a user inputs "New Product Presentation" as the scenario. The user enters "New Product Presentation" into the terminal's text input interface. The server receives this scenario and generates a prompt message like the following:

[0828] Based on the "New Product Presentation" scenario, please create presentation materials that include the following:

[0829] 1. Features of the new product

[0830] 2. Market analysis

[0831] 3. Comparison with competing products

[0832] 4. Sales Strategy

[0833] The generation AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are sent back to the user via the server. The user can review the generated materials and make corrections or additions as needed.

[0834] In this way, a system is realized in which servers, terminals, and generation AI models work together to automatically generate documents. This system allows users to easily create high-quality documents and effectively utilize the information accumulated within the company.

[0835] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0836] Step 1:

[0837] The user enters the scenario.

[0838] The user uses the terminal's text input interface to enter the scenario for the document they want to create. The entered scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and enters "New Product Presentation" into the text box. Input: Scenario text (e.g., "New Product Presentation"). Output: Scenario text sent to the server.

[0839] Step 2:

[0840] The server receives the scenario

[0841] The server receives scenarios submitted by users and stores them in a database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is then stored in the database. Input: Scenario text submitted by the user. Output: Scenario text stored in the database.

[0842] Step 3:

[0843] The server generates a prompt message.

[0844] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates a prompt like this: Based on the "New Product Presentation" scenario, please create a presentation document that includes the following: 1. Features of the new product 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[0845] Step 4:

[0846] The server sends a prompt message to the generated AI model.

[0847] The server sends the generated prompt to the AI ​​model. Specifically, the server generates an API request and sends the request, including the prompt, to the AI ​​model's endpoint. Input: The generated prompt. Output: The prompt sent to the AI ​​model.

[0848] Step 5:

[0849] The generative AI model generates the data.

[0850] The generative AI model generates materials based on the received prompt text. Specifically, the generative AI model analyzes the prompt text and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: Prompt text sent to the generative AI model. Output: Generated materials.

[0851] Step 6:

[0852] The server receives the generated data.

[0853] The server receives the data generated from the generative AI model. Specifically, the server receives the API response and extracts the generated data from the response body. The extracted data is then temporarily stored. Input: Data sent from the generative AI model. Output: Temporarily stored generated data.

[0854] Step 7:

[0855] The server returns the document to the user.

[0856] The server returns the generated document to the user. Specifically, the server generates an HTTP response and sends the response containing the generated document to the user's terminal. The user then views the document in a web browser. Input: Temporarily saved generated document. Output: Generated document sent to the user's terminal.

[0857] (Application Example 1)

[0858] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0859] Conventional document creation systems lacked the functionality to automatically generate advertising materials based on user-input scenarios, resulting in reduced advertising production efficiency. Furthermore, there was a need to provide the generated story and diagrams not just as answers, but in a format suitable for use as advertising materials.

[0860] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0861] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story. This makes it possible to automatically generate advertising material based on a scenario input by the user and improve the efficiency of advertising production.

[0862] A "user" is an individual or organization that uses the system to input scenarios for materials and receives the generated stories and advertising materials.

[0863] A "scenario" is text information that describes the content and structure of the document that the user wants to create.

[0864] A "generative AI system" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[0865] "Documents accumulated within the company" refers to a collection of data and information that a company or organization possesses internally.

[0866] "External information" refers to data and information obtained from external sources, other than documents accumulated within the company.

[0867] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using internally accumulated data and external information.

[0868] A "generated story" is a narrative or explanatory text created based on a scenario using generative AI methods.

[0869] "Advertising materials" refer to content such as images, videos, and text created for advertising purposes based on a generated story.

[0870] "Means of providing responses" refers to systems and methods for providing users with generated stories, diagrams, and advertising materials.

[0871] The system for implementing this invention includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents accumulated within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story.

[0872] System program

[0873] This system is implemented using Python and utilizes OpenAI's GPT-3 as a generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are automatically generated as well.

[0874] Explanation of the process

[0875] The server operates using the following hardware and software:

[0876] Hardware:

[0877] Smartphone or PC

[0878] software:

[0879] Python

[0880] OpenAI API

[0881] Data processing and data computation:

[0882] 1. Scenario Input: The user inputs the scenario through a text input interface. For example, they might input a scenario such as "New Product Presentation".

[0883] 2. Story Generation: The input scenario is sent as a prompt to the AI ​​model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[0884] 3. Diagram generation: Use internally accumulated data as external information and generate appropriate diagrams using image recognition technology.

[0885] 4. Generating ad materials: Generate ad materials (images, videos, text) based on the generated story.

[0886] 5. Providing the response: Provide users with the generated story, diagrams, and advertising materials.

[0887] Specific example

[0888] If a user enters "New Product Presentation," the following text will be returned as an example of the generated advertising material:

[0889] Example of a prompt:

[0890] Advertising Scenario: New Product Presentation

[0891] Please generate ad materials based on this scenario.

[0892] Examples of generated ad materials:

[0893] "New Product Presentation"

[0894] This new product was developed using the latest technology. It's high-performance yet easy to use, making everyday life more convenient. Buy it now and experience the future of living!

[0895] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[0896] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0897] Step 1:

[0898] The user enters the scenario. The user uses a text input interface to enter the scenario for the document they want to create. For example, they might enter a scenario such as "New Product Presentation." The entered scenario is then sent to the server.

[0899] Step 2:

[0900] The server receives a scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generation AI model. For example, it might take the form of "Advertising Scenario: Presentation of a new product. Generate advertising material based on this scenario."

[0901] Step 3:

[0902] The server sends a prompt to the generative AI model, which then generates a story. The server uses the OpenAI GPT-3 API to send the prompt and receives a story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated such as, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[0903] Step 4:

[0904] The server generates diagrams using documents stored within the company. The server uses image recognition technology to select and generate appropriate diagrams from the company's internal records. For example, diagrams illustrating the technical specifications and usage of a new product can be generated.

[0905] Step 5:

[0906] The server generates advertising materials based on the generated story. The server creates advertising materials (images, videos, text) based on the generated story. For example, advertising taglines and product images are created based on the generated story.

[0907] Step 6:

[0908] The server provides the user with generated stories, diagrams, and advertising materials. The server sends the generated stories, diagrams, and advertising materials to the user so that the user can review them. The user can then create advertisements using the provided materials.

[0909] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[0910] (Example 2)

[0911] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0912] Traditional document creation systems struggled to generate appropriate stories based on user-input scenarios. Furthermore, they lacked the functionality to automatically generate diagrams related to the generated stories, requiring users to create them manually. This resulted in a significant amount of time and effort being required for document creation.

[0913] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for analyzing the input scenario and extracting keywords and themes, means for generating a story based on the extracted keywords and themes, means for using documents stored within the company as external information and generating a diagram, and means for providing the generated story and diagram together as a response. This makes it possible to automatically generate a story and related diagrams based on the scenario input by the user, and to efficiently create documents.

[0914] A "user" is an individual or organization that uses the system to create materials.

[0915] A "scenario" is a document or text that outlines the content and structure of the material that the user wants to create.

[0916] "Means of input" refers to the interface or device that allows users to input scenarios into the system.

[0917] "Means of analysis" refers to a function that uses natural language processing technology to analyze the input scenario and extract keywords and themes.

[0918] "Keywords" are particularly important words or phrases within a scenario.

[0919] The "theme" is the central theme or topic of the scenario.

[0920] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on extracted keywords and themes.

[0921] "Documents accumulated within the company" refers to data and information that a company or organization possesses internally.

[0922] "External information" refers to all information available to the system, including documents stored within the company.

[0923] "Methods for generating diagrams" refers to a function that automatically creates appropriate diagrams based on internally accumulated documents and external information.

[0924] "Means of providing answers" refer to interfaces and devices that provide users with the generated stories and diagrams.

[0925] This invention relates to a system that automatically generates a story and related diagrams based on a scenario input by the user for a document they wish to create. A specific embodiment of this system is described below.

[0926] System program generation

[0927] The server generates a program for implementing a generative AI model. This program has the functionality to analyze input scenarios using natural language processing techniques and extract keywords and themes. It also includes the functionality to generate a story based on the extracted keywords and themes.

[0928] Hardware and software to be used

[0929] The server implements natural language processing techniques using the Python programming language and the TensorFlow library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the BERT (Bidirectional Encoder Representations from Transformers) model. Based on the extracted keywords and themes, the server generates a story using the GPT-3 (Generative Pre-trained Transformer 3) model.

[0930] Specific example

[0931] Consider a scenario where a user inputs the following: "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning." The scenario entered from the terminal is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, it uses these keywords to generate a story using the GPT-3 model. The generated story might be, for example, "a presentation that highlights the new product's features, compares it to competitors, and explains its market positioning."

[0932] Example of a prompt

[0933] Examples of prompts that users might input into the generated AI model include the following:

[0934] "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning."

[0935] In this way, the system's program processing is explained while clearly defining how the server, terminal, and user are involved.

[0936] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0937] Step 1:

[0938] The user inputs a scenario. The user enters the scenario into the input field on the terminal. For example, they might enter, "Generate a story for a new product presentation. Include the product's features, comparison with competitors, and market positioning." The entered scenario becomes the input data on the terminal.

[0939] Step 2:

[0940] The terminal sends the scenario to the server. The terminal sends the user-entered scenario to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[0941] Step 3:

[0942] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (e.g., NLTK or SpaCy). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[0943] Step 4:

[0944] The server extracts keywords and themes. The server inputs tokenized data into a BERT model and extracts important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is a tokenized scenario, and the output data is the extracted keywords and themes.

[0945] Step 5:

[0946] The server generates a story based on the extracted keywords and themes. The server uses a GPT-3 model to generate the story based on the extracted keywords and themes. The generated story might be, for example, "a presentation highlighting the features of a new product, comparing it to competitors, and explaining its market positioning." The input data consists of the extracted keywords and themes, while the output data is the generated story.

[0947] Step 6:

[0948] The server sends the generated story to the device. The server packages the generated story in JSON format and sends it to the device as an HTTP response. The input data is the generated story, and the output data is the story sent to the device.

[0949] Step 7:

[0950] The device displays the story to the user. The device displays the received story in the user interface. The user can review the generated story and modify or add to it as needed. The input data is the story sent to the device, and the output data is the story displayed to the user.

[0951] (Application Example 2)

[0952] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0953] Traditional advertising campaign creation processes often involved manual tasks such as scenario analysis, keyword extraction, and story generation, which were time-consuming and labor-intensive. Furthermore, the generated stories could lack consistency or be unsuitable for the target audience, making it difficult to create effective advertisements.

[0954] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for a generative AI that generates a story based on the input scenario, means for using internally stored documents as external information to generate a diagram, means for providing the generated story and diagram together as a response, and means for analyzing an advertising campaign scenario, extracting keywords and themes, and generating an advertising story. This automates the process from scenario analysis of an advertising campaign to story generation, making it possible to quickly create effective and consistent advertisements.

[0955] A "user" is an individual or organization that uses the system to create materials or advertising campaign scenarios.

[0956] A "scenario" is text information entered by users for advertising campaigns or document creation, and it forms the basis for story generation.

[0957] "Generative AI methods" refer to artificial intelligence technologies that analyze input scenarios and generate stories using natural language processing techniques.

[0958] "Documents accumulated within the company" refers to a collection of documents and data created in the past within a company or organization, which are used as external information.

[0959] "Methods for generating diagrams" refers to technologies that select and generate appropriate diagrams based on internally accumulated documents and external information.

[0960] "Means of providing answers" refers to the methods and technologies for providing users with the generated stories and diagrams.

[0961] An "advertising campaign" is a series of marketing activities aimed at promoting a specific product or service.

[0962] "Keywords" are important words or phrases extracted from the scenario, and they form the basis for generating the story.

[0963] The "theme" is the central theme or topic of the scenario, and it determines the direction of the story.

[0964] An "advertising story" is a consistent narrative created in line with the objectives of an advertising campaign, designed to appeal to the target audience.

[0965] The system for implementing this invention is configured as follows: First, a terminal is required for the user to input the materials or advertising campaign scenarios they wish to create. This terminal is a device such as a personal computer or smartphone, and provides an interface for the user to input the scenarios.

[0966] Next, the server receives the input scenario and analyzes it using generative AI tools. This analysis utilizes natural language processing techniques to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. For example, the OpenAI API might be used for this generative AI.

[0967] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated stories and diagrams are then provided from the server to the user's terminal as answers.

[0968] As a concrete example, consider a scenario where a user inputs a presentation scenario for a new product. This scenario includes information such as "the features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates prompt messages like the following.

[0969] Example of a prompt:

[0970] Scenario: In a new product presentation, emphasize the product's features, comparison with competitors, and market positioning.

[0971] Keywords: New product, presentation, features, competitors, market

[0972] Please generate an advertising story based on this scenario.

[0973] If this prompt is input into a generative AI model, it may generate an advertising story like the following.

[0974] Example of a generated ad story:

[0975] The new X-100 boasts innovative features not found in other products. Compared to its competitors, the X-100 offers superior performance and cost-effectiveness. Positioned in the market, the X-100 is the perfect choice for users seeking cutting-edge technology. Get your X-100 today and experience the future.

[0976] In this way, users can easily generate effective advertising stories. This system automates the process from scenario analysis to story generation for advertising campaigns, enabling the rapid creation of effective and consistent advertisements.

[0977] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0978] Step 1:

[0979] The user uses their device to input the document or advertising campaign scenario they want to create. The entered scenario is sent to the server in text format.

[0980] Step 2:

[0981] The server analyzes the received scenario. This analysis uses natural language processing techniques. Specifically, it extracts keywords and themes from the scenario. The input to this process is the text data of the scenario, and the output is a list of the extracted keywords and themes.

[0982] Step 3:

[0983] The server generates prompt sentences based on the extracted keywords and themes. The generated prompt sentences are then input to a generative AI model. The input to this process is a list of keywords and themes, and the output is prompt sentences.

[0984] Step 4:

[0985] The server uses a generative AI model to generate a story based on the prompt text. Specifically, it uses the OpenAI API to generate the story. The input to this process is the prompt text, and the output is the generated story.

[0986] Step 5:

[0987] The server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The input for this process is internal data, and the output is the generated diagram.

[0988] Step 6:

[0989] The server provides the generated story and diagram together as the answer to the user's terminal. The input to this process is the generated story and diagram, and the output is the answer data sent to the user's terminal.

[0990] Step 7:

[0991] Users review the stories and diagrams provided through their devices and make modifications or additions as needed. They then finalize the materials and advertising campaign content. The input for this process is the response data sent from the server, and the output is the completed materials and advertising campaign content.

[0992] (Example 3)

[0993] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0994] Traditional document creation systems required users to manually search for materials, select appropriate figures, and create stories, which was time-consuming and labor-intensive. Furthermore, specialized knowledge was required to select appropriate figures, making it difficult for users to create documents efficiently. Inconsistent quality of generated documents was also a challenge.

[0995] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0996] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating a story based on the input scenario using a generative AI, means for using documents stored within the company as external information, selecting and generating appropriate figures using image recognition technology, and means for providing the generated story and figures together as a response. This enables the user to efficiently create high-quality documents.

[0997] A "user" is an individual or organization that uses the system to create materials.

[0998] A "scenario" is text information that indicates the content and purpose of the document that the user wants to create.

[0999] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[1000] "Materials accumulated within the company" refers to information resources such as documents, images, and data stored within a company or organization.

[1001] "External information" refers to documents accumulated within the company, which are the source of information for the system to use.

[1002] "Image recognition technology" is a technology that analyzes images and diagrams to understand their content.

[1003] An "appropriate diagram" is a chart or image selected to visually represent information related to the scenario.

[1004] "Means of generation" refers to the means of creating a new diagram based on the selected diagram.

[1005] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[1006] "System" refers to a computer-based device or software for performing a series of processes, including the means described above.

[1007] This invention relates to a system for enabling users to efficiently create high-quality materials. Specific embodiments of this system are described below.

[1008] System Overview

[1009] This system includes the following main means:

[1010] 1. A means for users to input the scenario of the document they want to create.

[1011] 2. Generative AI means for generating a story based on an input scenario

[1012] 3. A means of using internally accumulated data as external information, and selecting and generating appropriate figures using image recognition technology.

[1013] 4. A means of answering with the generated story and diagram together.

[1014] Hardware and software to be used

[1015] The server runs the system using the following hardware and software:

[1016] Hardware: High-performance processor, sufficient memory, storage devices, network interface

[1017] Software: Google Cloud Vision API, Amazon Rekognition, Adobe Illustrator, Canva, Generative AI Models (e.g., OpenAI's GPT-4)

[1018] Data processing and data calculation

[1019] The server performs data processing and calculations using the following steps:

[1020] 1. Data collection:

[1021] The server accesses internal databases and file servers to search for and retrieve relevant documents, including PDFs, Word documents, and presentation files.

[1022] 2. Application of image recognition technology:

[1023] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the collected materials.

[1024] 3. Selecting the appropriate figure:

[1025] Based on the analysis results, the server filters and selects the appropriate diagrams related to the scenario.

[1026] 4. Figure generation:

[1027] The server uses Adobe Illustrator or Canva to generate new diagrams based on the selected diagrams.

[1028] 5. Story generation:

[1029] The server uses a generative AI model (GPT-4) to generate a story based on the generated diagrams.

[1030] 6. Providing an answer:

[1031] The server combines the generated diagrams and stories into a single document and provides it to the user.

[1032] Specific example

[1033] For example, if a user wants to create a document explaining the features of a new product, they would input the following prompt into the AI ​​model:

[1034] "Please generate a diagram to explain the features of the new product. Based on internal company documents, select and generate an appropriate diagram."

[1035] Upon entering this prompt, the server will generate a diagram following the steps outlined above and provide it along with the story. Users can then use this response to create their own materials.

[1036] In this way, users can efficiently create high-quality materials. The flow of the specific processing in Example 3 will be explained using Figure 15.

[1037] Step 1: Gathering materials

[1038] The server accesses internal databases and file servers to search for and retrieve relevant documents. The user provides a scenario for the document they wish to create as input. Based on this scenario, the server extracts keywords and uses these keywords to search for documents. The output includes relevant documents such as PDFs, Word documents, and presentation files.

[1039] Step 2: Application of image recognition technology

[1040] The server applies image recognition technology to the collected data. The data collected in Step 1 is provided as input. The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the data. Specifically, the server processes each data item sequentially and extracts the content of images and charts as text data. The analysis results are obtained as output.

[1041] Step 3: Select the appropriate figure

[1042] The server uses image recognition technology to select appropriate figures from the document. The analysis results obtained in step 2 are given as input. The server matches keywords related to the scenario with the analysis results and filters and selects the most relevant figures. Specifically, the server calculates a relevance score and selects the figures with the highest scores. The selected figures are obtained as output.

[1043] Step 4: Generate the diagram

[1044] The server generates a new diagram based on the selected diagram. The diagram selected in step 3 is given as input. Using Adobe Illustrator or Canva, the server uses the selected diagram as a template, adding the necessary information to create a new diagram. Specifically, the server adjusts the diagram's layout and adds annotations and data. The output is the generated new diagram.

[1045] Step 5: Generating the Story

[1046] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related descriptive text and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[1047] Step 6: Provide your response

[1048] The server combines the generated diagrams and stories into a single document and provides it to the user. The input consists of the diagrams generated in step 4 and the stories generated in step 5. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The output is the final document.

[1049] (Application Example 3)

[1050] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1051] Factory maintenance work is complex and diverse, making it difficult for workers to quickly grasp the correct procedures. Furthermore, the sheer volume of maintenance records and technical documents makes it difficult to efficiently search for and utilize necessary information. Therefore, there is a need to improve the efficiency and accuracy of maintenance work.

[1052] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1053] In this invention, the server includes means for inputting a scenario for a document that the user wants to create; means for generating a story based on the input scenario using a generative AI; means for generating a diagram using documents accumulated within the company as external information; means for providing the generated story and diagram together as a response; means for selecting and generating an appropriate diagram using image recognition technology, using maintenance records and technical documents accumulated within the factory as external information; and means for supporting maintenance work based on the selected diagram. This enables maintenance workers to quickly acquire the necessary information and perform their work efficiently and accurately.

[1054] "Users" refer to individuals who use the system to create or maintain documents.

[1055] A "scenario" refers to a plan or structure that outlines the content and purpose of the material that the user wants to create.

[1056] "Generative AI methods" refer to artificial intelligence technology that generates stories based on input scenarios.

[1057] "Documents accumulated within the company" refers to information such as technical documents and maintenance records created within the company in the past.

[1058] "External information" refers to documents and data accumulated within the company for use by the system.

[1059] "Methods for generating diagrams" refers to the technology of selecting and generating appropriate diagrams based on materials accumulated within the company.

[1060] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[1061] "Maintenance records" refer to information that records the history and details of maintenance work performed within the factory.

[1062] "Technical documents" refer to technical information and procedures related to equipment and systems within a factory.

[1063] "Means of supporting maintenance work" refers to support technologies that enable maintenance workers to perform tasks efficiently and accurately based on selected diagrams.

[1064] The system for implementing this invention is configured as follows: First, an interface is provided for the user to input a scenario for the document they wish to create. The user inputs the scenario through this interface.

[1065] Next, the server generates a story based on the input scenario. This uses generative AI tools, leveraging natural language processing techniques to analyze the scenario and generate an appropriate story. For example, the transformers library from Hugging Face can be used as a generative AI tool.

[1066] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to generate these diagrams. Specifically, software such as OpenCV, PIL (Python Imaging Library), and pytesseract are used to extract and analyze text and features from images.

[1067] Maintenance records and technical documents accumulated within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[1068] The generated story and diagrams are provided to the user together. This allows the user to quickly obtain the necessary information and work efficiently and accurately.

[1069] As a concrete example, the following prompt statement can be used.

[1070] Example of a prompt:

[1071] "Based on the following images, please generate a story about the parts replacement procedure: ['maintenance_step1.png', 'maintenance_step2.png']"

[1072] Based on this prompt, the generative AI system generates a story about the parts replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[1073] In this way, maintenance work on factory robots can be made more efficient, and the burden on workers can be reduced.

[1074] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1075] Step 1:

[1076] Enter the scenario for the document you want to create.

[1077] Input: A scenario in which the user enters information through an interface.

[1078] Output: Input scenario data.

[1079] Specific operation: The user accesses the system interface and enters the scenario for the document they want to create in text format.

[1080] Step 2:

[1081] The server generates a story based on the input scenario.

[1082] Input: The entered scenario data.

[1083] Output: The generated story.

[1084] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generates an appropriate story using natural language processing techniques.

[1085] Step 3:

[1086] The server uses internally stored data as external information to generate diagrams.

[1087] Input: Documents accumulated within the company.

[1088] Output: The generated figure.

[1089] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate figures from documents stored within the company.

[1090] Step 4:

[1091] The server utilizes maintenance records and technical documents accumulated within the factory as external information, and uses image recognition technology to select and generate appropriate diagrams.

[1092] Input: Maintenance records and technical documents accumulated within the factory.

[1093] Output: Selected figure.

[1094] Specific operation: The server uses image recognition technology to select and generate necessary diagrams from maintenance records and technical documents.

[1095] Step 5:

[1096] The server will provide the generated story and diagram together as the answer.

[1097] Input: Generated story and selected diagram.

[1098] Output: The story and diagrams provided to the user.

[1099] Specific operation: The server combines the generated story and selected diagrams and provides them to the user. The user can then create materials based on this.

[1100] Step 6:

[1101] Maintenance work is performed based on the stories and diagrams provided by the user.

[1102] Input: Provided story and diagram.

[1103] Output: Efficient and accurate maintenance work.

[1104] Specific operation: Users perform maintenance tasks efficiently and accurately by referring to the provided stories and diagrams.

[1105] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1106] "Example of form 1"

[1107] One embodiment of the present invention provides a system that includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for using documents accumulated within the company as external information to generate a diagram, means for providing the generated story and diagram together as a response, and an emotion engine that recognizes the user's emotions.

[1108] "Example of form 2"

[1109] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a joyful expression, the emotion engine captures that information, and the generative AI generates a story based on that emotion. Specifically, if joy is recognized, the generative AI generates a story that includes positive elements.

[1110] "Example of form 3"

[1111] Furthermore, the emotion engine selects and generates appropriate images based on the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures this information, and the image generation mechanism selects and generates an image that conveys surprise or novelty.

[1112] The following describes the processing flow for each example of the form.

[1113] "Example of form 1"

[1114] Step 1: Enter the scenario for the document you want to create.

[1115] Step 2: Based on the input scenario, the generative AI generates a story.

[1116] Step 3: Use internally accumulated data as external information to generate diagrams.

[1117] Step 4: Answer by combining the generated story and diagram.

[1118] Step 5: The emotion engine, which recognizes the user's emotions, starts operating.

[1119] "Example of form 2"

[1120] Step 1: The emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[1121] Step 2: Based on the emotions recognized by the emotion engine, the generative AI generates a story.

[1122] "Example of form 3"

[1123] Step 1: The emotion engine recognizes the user's emotions.

[1124] Step 2: The emotion engine selects and generates an appropriate diagram based on the emotions it recognizes.

[1125] (Example 1)

[1126] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1127] Traditional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which is time-consuming and labor-intensive. Furthermore, the lack of feedback that considers user emotions leads to decreased efficiency and quality in document creation. Additionally, the lack of effective means to utilize internally accumulated documents can result in a lack of consistency and reliability in the materials.

[1128] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1129] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored documents as external information, means for providing the generated story and diagram together as a response, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback that responds to the user's emotions, and enables the effective use of internal company documents.

[1130] "A means for users to input the scenario of the document they want to create" refers to an interface that allows users to freely input text, and is a means for inputting the scenario of the document.

[1131] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios, and are methods that analyze scenarios using natural language processing technology to generate stories.

[1132] "A means of generating diagrams by using internally accumulated data as external information" refers to a method of generating diagrams by referring to data stored in an internal database and using data visualization technology.

[1133] "A means of providing a combined answer using generated stories and diagrams" refers to a method of integrating generated stories and diagrams and providing them to the user as a single document.

[1134] An "emotion engine" refers to a technology that analyzes user input and responses to recognize emotions. It uses natural language processing technology to analyze user emotions and provide appropriate feedback.

[1135] This invention is a system that allows users to input a scenario for a document they wish to create, generates a story and diagrams based on that scenario, and ultimately provides it as a single document. A specific embodiment of this system is described below.

[1136] System Configuration

[1137] This system consists of the following main components:

[1138] 1. Text input interface

[1139] 2. Generative AI means

[1140] 3. Database Reference Methods

[1141] 4. Data Visualization Methods

[1142] 5. Integration means

[1143] 6. Emotional Engine

[1144] Text input interface

[1145] The user uses the terminal's text input interface to enter the scenario for the document they want to create. This interface is designed to allow the user to freely enter text. For example, the user might type "New Product Presentation".

[1146] Generative AI means

[1147] The server receives the user-entered scenario and inputs it as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model then generates an appropriate story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[1148] Database Reference Means

[1149] The server accesses the company's internal database to search for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib and Pandas to format and visualize the data.

[1150] Data visualization means

[1151] The server generates graphs using data visualization techniques based on information retrieved from the database. For example, it extracts necessary information from the database using SQL queries and generates graphs using Python's Matplotlib library. The generated graphs are saved as image files.

[1152] Integration means

[1153] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX or Microsoft Word APIs). The generated document is saved as a PDF file.

[1154] Emotional Engine

[1155] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1156] Examples of specific cases and prompt statements

[1157] Specific example

[1158] The user enters "New product presentation" into the text input interface.

[1159] The server uses a generation AI model to generate stories that include content such as "new product features, target market, competitive analysis, and sales strategy."

[1160] The server accesses past sales data and market research results from the company's internal database and creates relevant graphs and charts.

[1161] Finally, the generated stories and diagrams are integrated and presented to the user.

[1162] Example of a prompt

[1163] Please enter a presentation scenario for your new product. For example, include details such as "new product features, target market, competitive analysis, and sales strategy."

[1164] In this way, users can easily create high-quality documents.

[1165] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1166] Step 1:

[1167] The user enters the scenario.

[1168] The user uses the terminal's text input interface to enter the scenario for the document they want to create. For example, they might enter "New Product Presentation." The entered scenario is sent to the server in real time.

[1169] Input: Scenario text entered by the user

[1170] Output: Scenario text sent to the server

[1171] Step 2:

[1172] The server generates stories using an AI model.

[1173] The server inputs the received scenario as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[1174] Input: Scenario text

[1175] Output: Generated story text

[1176] Step 3:

[1177] The server generates the diagram by referencing the company's internal database.

[1178] The server accesses the company's internal database to search for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib and Pandas to format and visualize the data.

[1179] Input: Data retrieved from the company's internal database

[1180] Output: Image files of the generated graphs and charts

[1181] Step 4:

[1182] The server integrates the generated stories and diagrams.

[1183] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX or Microsoft Word APIs). The generated document is saved as a PDF file.

[1184] Input: Generated story text, image files of graphs and charts

[1185] Output: Integrated PDF document

[1186] Step 5:

[1187] The server recognizes the user's emotions and provides feedback.

[1188] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1189] Input: User feedback text

[1190] Output: Analysis results and feedback text from the emotion engine.

[1191] (Application Example 1)

[1192] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1193] The traditional advertising campaign creation process involved a great deal of effort and time, including scenario writing, story generation, and visual creation. Furthermore, it was difficult to propose optimal advertising materials that considered user emotions, making it challenging to quickly create effective advertising campaigns. This resulted in advertising agencies and marketing personnel being unable to work efficiently, making it difficult to maximize the effectiveness of their advertising.

[1194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1195] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored documents as external information, means for providing the generated story and diagram together as a response, means including an emotion engine for recognizing the user's emotions, and means for generating advertising visuals based on the generated story. This streamlines the advertising campaign creation process and enables the suggestion of optimal advertising materials that respond to the user's emotions.

[1196] A "user" is an individual or organization that uses the system to create a scenario for a document.

[1197] A "scenario" is text information that describes the content and structure of the document that the user wants to create.

[1198] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[1199] "Documents accumulated within the company" refers to a collection of documents and data created in the past within a company or organization.

[1200] "Methods for generating diagrams" refers to the technology for creating appropriate diagrams using materials accumulated within the company.

[1201] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[1202] An "emotion engine" is a technology that recognizes the user's emotions and makes appropriate suggestions based on those emotions.

[1203] "Advertising visuals" refer to images and graphic materials used in advertising campaigns.

[1204] The system for implementing this invention has the following configuration. First, it provides a text input interface for the user to input the scenario of the document they wish to create. The user can use this interface to freely write the scenario.

[1205] Next, a generative AI system is used to generate a story based on the input scenario. This generative AI system analyzes the scenario using natural language processing techniques and generates an appropriate story. Specifically, it uses the OpenAI API to generate a story from the scenario.

[1206] Based on the generated story, a system for generating diagrams using internally stored materials is activated. This system uses image recognition technology to select and generate appropriate diagrams from internal documents. PIL (Python Imaging Library) is used to create visuals that match the generated story.

[1207] Furthermore, the system includes a means of providing the user with the generated story and diagrams together. This means integrates the generated content and provides the user with a consistent answer.

[1208] It also includes a mechanism that incorporates an emotion engine to recognize the user's emotions. This emotion engine analyzes the user's emotions and suggests the most suitable advertising material based on that analysis. It uses the emotion_recognition library to recognize the user's emotions.

[1209] Finally, it includes a means of generating visuals for advertising based on the generated story. This means creates images and graphic materials to be used in the advertising campaign based on the generated story.

[1210] As a concrete example, if a user inputs "an advertising campaign for a new smartphone" as a scenario, the generating AI will produce a story like the following:

[1211] Example of a prompt:

[1212] Advertising campaign scenario: Advertising campaign for a new smartphone product

[1213] story:

[1214] By inputting this prompt into the AI ​​generation model, an appropriate story is generated. Based on the generated story, visuals for advertising are created and provided to the user.

[1215] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[1216] Step 1:

[1217] Enter the scenario for the document you want to create.

[1218] Users freely write advertising campaign scenarios using a text input interface. The entered scenarios are then sent to the system.

[1219] Input: Scenario text entered by the user

[1220] Output: Scenario text is sent to the system.

[1221] Step 2:

[1222] The server generates a story based on the input scenario.

[1223] The server uses generative AI tools to analyze the input scenario and generate an appropriate story. Specifically, it uses the OpenAI API to generate a story from the scenario.

[1224] Input: Scenario text

[1225] Output: Generated story text

[1226] Step 3:

[1227] The server generates a diagram based on the generated story.

[1228] The server utilizes internally stored data, uses image recognition technology to select and generate appropriate diagrams, and then uses PIL (Python Imaging Library) to create visuals that match the generated story.

[1229] Input: Generated story text

[1230] Output: Generated diagram (visual)

[1231] Step 4:

[1232] The server will provide the generated story and diagram together as the answer.

[1233] The server integrates the generated stories and diagrams to provide users with a consistent answer.

[1234] Input: Generated story text, generated diagrams

[1235] Output: Integrated story and diagram

[1236] Step 5:

[1237] The server recognizes the user's emotions.

[1238] The server uses an emotion engine to analyze the user's emotions. It recognizes the user's emotions using the emotion_recognition library.

[1239] Input: User's emotional data (e.g., facial expression images or audio data)

[1240] Output: Recognized emotion information

[1241] Step 6:

[1242] The server generates visuals for advertising based on the generated story.

[1243] The server creates images and graphic materials to be used in the advertising campaign based on the generated story.

[1244] Input: Generated story text, recognized emotion information

[1245] Output: Visuals for advertising

[1246] Step 7:

[1247] The server provides the final advertising campaign materials to the user.

[1248] The server integrates the generated stories, diagrams, and advertising visuals and delivers them to the user.

[1249] Input: Generated story text, generated diagrams, advertising visuals

[1250] Output: Final advertising campaign materials

[1251] (Example 2)

[1252] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1253] Conventional document creation systems can generate stories based on user-inputted scenarios, but they have the problem of not being able to adjust the stories to take into account the user's emotions. Furthermore, when the generated stories and diagrams are provided together, appropriate adjustments based on the user's emotions are not made, making it difficult to create documents that reflect the user's intentions and feelings.

[1254] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1255] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating a story based on the input scenario using a generative AI, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, means for sensing the tone of the user's voice and facial expressions, means for recognizing the user's emotions using an emotion engine, and means for adjusting the story based on the recognized emotions. This makes it possible to provide a story and diagram that reflect the user's emotions.

[1256] "User" refers to a person who uses the system to input a scenario for a document and receives the generated story and diagrams.

[1257] A "scenario" refers to text-based input information that indicates the content and structure of the document the user wants to create.

[1258] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios.

[1259] "Natural language processing technology" refers to techniques for analyzing scenarios and extracting keywords and themes.

[1260] "Methods for generating diagrams" refers to technologies that utilize internally accumulated data as external information to generate appropriate diagrams.

[1261] "Means of sensing" refers to technology for sensing the tone of the user's voice and facial expressions in real time.

[1262] An "emotion engine" refers to technology that recognizes a user's emotions from the perceived tone of voice and facial expressions.

[1263] "Means of adjustment" refers to techniques for appropriately adjusting stories that have been generated based on perceived emotions.

[1264] "Story" refers to a series of sentences or contents generated by generative AI based on a scenario.

[1265] "Means of providing answers" refers to the technology used to deliver generated stories and diagrams to users.

[1266] This invention is a system that allows users to input a scenario for a document they wish to create, generates a story based on that scenario, and further provides a story that reflects the user's emotions. A specific embodiment of this system is described below.

[1267] Hardware and software to be used

[1268] 1. Server:

[1269] Natural language processing techniques: Generative AI models such as BERT and GPT-3 are used.

[1270] Emotion engine: Software used to analyze the tone of the user's voice and facial expressions.

[1271] Database: A database used to store internal company data and make it available as external information.

[1272] 2. Terminal:

[1273] Input interface: A keyboard or touchscreen for the user to input scenarios.

[1274] Camera and microphone: Hardware used to detect the user's voice tone and facial expressions.

[1275] Data processing and data calculation

[1276] 1. Scenario Input and Analysis:

[1277] The user enters the scenario in text format using the terminal's input interface.

[1278] The server receives the input scenario and performs analysis using natural language processing techniques. Specifically, it uses generative AI models such as BERT and GPT-3 to extract keywords and themes from the scenario.

[1279] 2. Story generation:

[1280] The server generates a story using a generative AI model based on extracted keywords and themes. For example, if the scenario is "New Product Presentation," the server extracts keywords such as "New Product Features," "Comparison with Competitors," and "Market Positioning," and generates a story based on them.

[1281] 3. Recognizing emotions and adjusting the story:

[1282] The device uses a camera and microphone to sense the user's voice tone and facial expressions in real time.

[1283] The device uses an emotion engine to recognize the user's emotions from the tone of voice and facial expressions it perceives. For example, if the user shows a joyful expression, the device sends that information to the emotion engine, which recognizes the emotion of joy.

[1284] The server adjusts the generated story based on the recognized emotions. For example, if positive emotions are recognized, positive elements will be added to the story.

[1285] 4. Providing the final story:

[1286] The server then provides the user with the finalized story. Specifically, it provides the generated story and diagram together as the answer.

[1287] Examples of specific cases and prompt statements

[1288] Specific example

[1289] Scenario: New product presentation

[1290] Keywords to be extracted: New product features, comparison with competitors, market positioning

[1291] The generated story: A presentation that highlights the new product's features, demonstrates its competitive advantages, and clearly defines its market positioning.

[1292] Example of a prompt

[1293] "Please enter a scenario for your new product presentation. Our generative AI will analyze the scenario, extract keywords, and generate a story."

[1294] "To recognize the user's emotions, the system detects their tone of voice and facial expressions. Based on the emotions recognized by the emotion engine, the generative AI generates a story."

[1295] In this way, the system can generate and deliver the most suitable story based on the user's input and emotions.

[1296] The flow of the specific processing in Example 2 will be explained using Figure 19.

[1297] Step 1:

[1298] The user enters the scenario.

[1299] Input: The user enters the scenario in text format using the terminal's input interface.

[1300] Specific action: The user types "New product presentation".

[1301] Output: The entered scenario is saved on the terminal and sent to the server.

[1302] Step 2:

[1303] The server receives the scenario and analyzes it using natural language processing techniques.

[1304] Input: Scenario text sent from the terminal.

[1305] Specific operation: The server uses generative AI models such as BERT or GPT-3 to analyze the text "New product presentation".

[1306] Output: Keywords and themes extracted from the scenario.

[1307] Step 3:

[1308] The server extracts keywords and themes from the scenario.

[1309] Input: Scenario text analyzed using natural language processing techniques.

[1310] Specific operation: The server extracts keywords such as "features of the new product," "comparison with competitors," and "market positioning."

[1311] Output: A list of extracted keywords and topics.

[1312] Step 4:

[1313] The server generates a story based on extracted keywords and themes.

[1314] Input: A list of extracted keywords or topics.

[1315] Specific operation: The server uses the generated AI model to create a presentation that "emphasizes the features of the new product, demonstrates its competitive advantages, and clarifies its market positioning."

[1316] Output: The generated story.

[1317] Step 5:

[1318] The device detects the user's voice tone and facial expressions.

[1319] Input: User's real-time voice tone and facial expressions.

[1320] Specific operation: The device uses its camera and microphone to detect the user's smile and cheerful tone of voice.

[1321] Output: Data on the perceived tone of voice and facial expressions.

[1322] Step 6:

[1323] The device uses an emotion engine to recognize the user's emotions.

[1324] Input: Data on the perceived tone of voice and facial expressions.

[1325] Specific operation: The device uses an emotion engine to recognize the user's emotions of joy.

[1326] Output: Recognized emotion data.

[1327] Step 7:

[1328] The server adjusts the story based on the emotions it recognizes.

[1329] Input: Recognized emotion data and generated story.

[1330] Specific action: The server adds positive elements to the story and generates a presentation with a brighter tone.

[1331] Output: The refined, final story.

[1332] Step 8:

[1333] The server provides the final story to the user.

[1334] Input: The final, adjusted storyline.

[1335] Specific operation: The server displays a "positive-toned presentation that highlights the features of the new product, demonstrates its competitive advantages, and clarifies its market positioning" to the user.

[1336] Output: The final story provided to the user.

[1337] (Application Example 2)

[1338] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1339] Conventional document creation systems can generate stories based on user-inputted scenarios, but they cannot create stories that take into account the user's emotions. This makes it difficult to create documents that align with the user's intentions and feelings. Furthermore, when users submit answers using the generated stories and diagrams together, it is not possible to reflect their emotions, creating a need for more effective document creation.

[1340] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1341] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, an emotion engine means for recognizing emotions from the user's voice tone and facial expressions, and means for generating a story based on the recognized emotions. This makes it possible to generate a story that reflects the user's emotions, enabling more effective document creation.

[1342] "User" refers to a person who uses the system to create documents.

[1343] A "scenario" refers to a document or plot that outlines the content and structure of the material that the user wants to create.

[1344] "Generative AI" refers to artificial intelligence technology that generates stories based on input scenarios.

[1345] "Documents accumulated within the company" refers to data and information collected and stored within a company or organization.

[1346] "External information" refers to data obtained from sources other than internally stored documents within the company.

[1347] "Figures" refer to visual elements such as graphs, charts, and illustrations included in a document.

[1348] An "emotion engine" refers to technology that recognizes emotions from the tone of a user's voice and facial expressions.

[1349] "Voice tone" refers to the vocal characteristics of a user's voice, such as its pitch, strength, and rhythm.

[1350] "Facial expression" refers to the movements and expressions of the user's face.

[1351] "Story" refers to a series of sentences or pieces of content generated based on a scenario.

[1352] "Answer" refers to the act of providing the generated story and diagram to the user.

[1353] The system for implementing this invention includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents accumulated within the company as external information, means for providing the generated story and diagram together as a response, an emotion engine means for recognizing emotions from the tone of the user's voice and facial expressions, and means for generating a story based on the recognized emotions.

[1354] System program

[1355] The server executes the following program: First, the user inputs a scenario using a device such as a smartphone or PC. Next, a generative AI analyzes the scenario using natural language processing technology and extracts keywords and themes. After that, it generates a story based on the extracted information.

[1356] Furthermore, the emotion engine recognizes the user's tone of voice and facial expressions, and provides this emotional information to the generative AI. The generative AI adjusts the story considering the emotional information to generate content that better aligns with the user's intentions. The generated story is supplemented by a means of generating appropriate diagrams using internally accumulated materials as external information. Finally, the generated story and diagrams are provided to the user together.

[1357] Hardware and software to be used

[1358] Hardware: Smartphones, PCs, servers

[1359] Software: OpenCV (image processing), Transformers (generative AI and emotion recognition), TextBlob (text analysis)

[1360] Data processing and data calculation

[1361] 1. Image Processing: Images captured with a smartphone or computer camera are used to perform face recognition and analyze facial expressions using OpenCV.

[1362] 2. Voice Processing: Audio recorded with a smartphone or computer microphone is converted into text, and emotions are recognized by an emotion engine.

[1363] 3. Story Generation: Generative AI (such as GPT-3) is used to generate stories based on recognized emotions and scenarios.

[1364] 4. Diagram generation: Select and generate appropriate diagrams from the materials accumulated within the company.

[1365] Specific example

[1366] For example, if a user wants to create an advertisement about the camera features of a new smartphone, they would enter a prompt like this:

[1367] Example of a prompt

[1368] Create an advertisement story for a product. The user is feeling happy and the sentiment is positive. The product information is: A new smartphone with advanced camera features.

[1369] Based on this prompt, the generative AI generates a positive advertising story highlighting the new smartphone's camera features. The emotion engine recognizes the user's expressions of joy and provides this information to the generative AI, resulting in a more emotionally resonant advertising story. The generated story is then presented to the user, with appropriate diagrams generated using internally stored materials.

[1370] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[1371] Step 1:

[1372] The user enters the scenario using a terminal.

[1373] Input: Scenario (text format) for the document the user wants to create.

[1374] Output: Input scenario (text data)

[1375] Specific operation: The user enters a scenario into the terminal's input screen and presses the submit button. The terminal sends the entered scenario to the server.

[1376] Step 2:

[1377] The server uses generative AI to analyze the scenario and extract keywords and themes.

[1378] Input: Entered scenario (text data)

[1379] Output: Extracted keywords and themes (text data)

[1380] Specific operation: The server uses generative AI (natural language processing technology) to analyze the scenario and extract important keywords and themes. The extracted information is used in the next step.

[1381] Step 3:

[1382] The server uses an emotion engine to recognize the user's voice tone and facial expressions.

[1383] Input: User's voice data and image data

[1384] Output: Recognized emotions (text data)

[1385] Specific operation: The user records voice and facial expressions using the device's camera and microphone. The server uses OpenCV to analyze facial expressions from the image data and an emotion engine to analyze voice tone from the voice data. The recognized emotion is generated as a result of the analysis.

[1386] Step 4:

[1387] The server generates a story based on the emotions it recognizes.

[1388] Input: Extracted keywords or themes, perceived emotions

[1389] Output: Generated story (text data)

[1390] Specific operation: The server uses generative AI to generate a story based on extracted keywords, themes, and recognized emotions. The generated story is then used in the next step.

[1391] Step 5:

[1392] The server uses internally stored data to generate appropriate diagrams.

[1393] Input: Generated story (text data)

[1394] Output: Generated figure (image data)

[1395] Specific operation: The server uses image recognition technology to select and generate appropriate diagrams from the company's internally stored materials. The generated diagrams are then used in the next step.

[1396] Step 6:

[1397] The server provides the user with the generated story and diagram together.

[1398] Input: Generated story (text data), generated diagram (image data)

[1399] Output: Materials provided to the user (text data and image data)

[1400] Specific operation: The server combines the generated story and diagram into a single document and sends it to the user's terminal. The user can then view the document on their terminal.

[1401] (Example 3)

[1402] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1403] Traditional document creation systems had the problem of requiring users to manually create documents, which was time-consuming and laborious. Furthermore, they lacked a function to select appropriate diagrams based on the user's emotions, sometimes resulting in documents that didn't match the user's intentions. Additionally, there was a lack of effective means to utilize information accumulated within the company, potentially leading to a decline in document quality.

[1404] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1405] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored information as external information, means for providing the generated story and diagram together as a response, means for analyzing the user's emotions, and means for selecting an appropriate diagram based on the analyzed emotions. This enables the user to efficiently create high-quality documents.

[1406] "User" refers to an individual or organization that uses the system to create documents.

[1407] A "scenario" refers to a plan or story that outlines the content and structure of the material that the user wants to create.

[1408] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios.

[1409] "Internal information" refers to data and documents collected and stored within a company or organization.

[1410] "External information" refers to diagrams and data generated using information stored internally.

[1411] "Means for generating diagrams" refers to the technology for selecting and generating appropriate diagrams based on internally stored information.

[1412] "Image recognition technology" refers to the technology used to analyze image data and understand and classify its content.

[1413] "Means of analyzing emotions" refers to technologies used to analyze the emotions of users.

[1414] "Generated stories" refer to narratives or descriptions created by generative AI methods.

[1415] "Means of providing answers" refers to the technology used to deliver generated stories and diagrams to users.

[1416] This invention is a system for users to efficiently create high-quality materials. Specific embodiments of this system are described below.

[1417] System Configuration

[1418] This system consists of three main elements: a server, a terminal, and a user. The server handles data acquisition, image recognition, sentiment analysis, generation of diagrams and stories, and delivery to the user. The terminal is responsible for capturing user input and emotions. The user uses the system to create data.

[1419] Hardware and software to be used

[1420] Server: A server with a high-performance processor and large memory capacity is required. The server also needs storage to access the database and retrieve data.

[1421] Device: A computer or smart device equipped with a camera and microphone is required. This allows for the capture of the user's facial expressions and voice.

[1422] software:

[1423] Image recognition technology: Uses image recognition services such as Google Cloud Vision API and Amazon Rekognition.

[1424] Emotion analysis technology: Uses Microsoft Azure's Emotion API, etc.

[1425] Generative AI model: Uses natural language processing models such as OpenAI's GPT-4.

[1426] Program processing

[1427] The server first accesses the company's internal database and retrieves stored documents. These documents include PDFs, Word documents, and presentation slides. Next, the server uses image recognition technology to analyze the images within the documents and selects appropriate figures based on specific keywords and context.

[1428] The device captures the user's facial expressions with its camera and sends them to the emotion engine. The emotion engine analyzes the user's emotions and sends the results to the server. The server selects an appropriate image based on the analysis results.

[1429] The server sends the selected figure along with a prompt message to input into the generating AI model. The generating AI model generates a story based on the input prompt message and provides it to the user along with the selected figure.

[1430] Specific examples and prompt statements

[1431] For example, if a user wants to create a document related to "new product features," the server selects a diagram related to "new product features" from within the document. Next, it inputs a prompt message like the following into the generating AI model:

[1432] Prompt: "Select a diagram to illustrate the features of the new product and generate a story."

[1433] The generative AI model generates a story based on this prompt and provides it to the user along with the selected figure.

[1434] Furthermore, if the user shows a surprised expression, the server inputs the following prompt message into the AI ​​model:

[1435] Prompt: "Select an appropriate image to describe the user's surprise and generate a story."

[1436] The generative AI model generates a story based on this prompt and presents it to the user along with illustrations that convey surprise and freshness.

[1437] In this way, users can efficiently create high-quality materials. The flow of the specific processing in Example 3 will be explained using Figure 21.

[1438] Step 1: Obtaining materials

[1439] The server accesses the company's internal database and retrieves stored documents. A search query for the documents is required as input. The server searches the database for the relevant documents and retrieves them in formats such as PDF, Word documents, and presentation slides. The retrieved documents are provided as output. Specifically, the server searches for documents related to "new product features" and downloads the corresponding files.

[1440] Step 2: Selecting a figure using image recognition

[1441] The server extracts diagrams from acquired documents using image recognition technology. Acquired documents are required as input. The server analyzes images within the documents using image recognition services such as Google Cloud Vision API or Amazon Rekognition. The output consists of diagrams selected based on specific keywords and context. Specifically, the server analyzes images within the documents and selects diagrams related to "new product features" or "comparison with competitors."

[1442] Step 3: Emotion Analysis

[1443] The device captures the user's facial expressions with its camera and sends them to the emotion engine. The user's facial image is required as input. The device analyzes the user's emotions using Microsoft Azure's Emotion API, etc. The analyzed emotion data is obtained as output. Specifically, the device uses its camera to capture the user's facial expressions and sends them to the emotion engine.

[1444] Step 4: Generating Diagrams and Stories

[1445] The server sends the selected image along with a prompt to input into the generative AI model. The input requires both the selected image and the prompt. The server uses a generative AI model, such as OpenAI's GPT-4, to generate a story based on the input prompt. The output is the generated story. Specifically, the server sends prompts to the generative AI model such as the following:

[1446] Prompt: "Select a diagram to illustrate the features of the new product and generate a story."

[1447] Step 5: Provision to users

[1448] The server provides the user with the generated story and selected diagrams. The generated story and diagrams are required as input. The server sends these to the user's terminal. The user receives the story and diagrams as output. Specifically, the server sends the generated story and diagrams to the user's terminal, and the user creates materials based on them.

[1449] (Application Example 3)

[1450] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1451] Conventional document creation systems could generate stories and diagrams based on the user's desired document scenario, but they could not generate appropriate diagrams that reflected the user's emotions. Therefore, creating effective documents that resonated with users' feelings was difficult. Furthermore, the lack of technology to analyze user emotions in real time and generate diagrams based on that analysis limited their application in fields such as advertising and presentations.

[1452] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1453] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, emotion recognition means for analyzing the user's emotions, and means for selecting and generating an appropriate diagram based on the analyzed emotions. This enables the creation of effective documents that respond to the user's emotions.

[1454] A "user" is an individual or organization that uses the system to create materials.

[1455] A "scenario" is a plan or outline of a story that shows the content and structure of the material that the user wants to create.

[1456] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[1457] "Documents accumulated within the company" refers to information such as documents, images, and data that have been created and stored within a company or organization in the past.

[1458] "External information" refers to information sources used by the system, including documents accumulated within the company.

[1459] "Methods for generating diagrams" refers to technologies that select and generate appropriate diagrams based on internally accumulated documents and external information.

[1460] A "generated story" is a narrative or explanatory text created based on a scenario using generative AI methods.

[1461] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[1462] "Emotion recognition means" refers to technologies and devices used to analyze a user's emotions.

[1463] "Analyzed emotions" refer to the user's emotional state as detected and analyzed by emotion recognition tools.

[1464] An "appropriate diagram" is a diagram that is best suited to the document, selected based on the analyzed emotions and scenarios.

[1465] The system for implementing this invention is configured as follows: First, an interface is provided for the user to input a scenario for the document they wish to create. This interface allows the user to input the scenario via a terminal such as a personal computer or smartphone.

[1466] Next, a generative AI system operates to generate a story based on the input scenario. This generative AI system analyzes the scenario using natural language processing techniques and generates an appropriate story. The generated story is provided in a format that is easy for the user to understand.

[1467] Furthermore, a mechanism is in place to generate diagrams using internally accumulated data as external information. This mechanism uses image recognition technology to select and generate appropriate diagrams from internal documents. The generated diagrams are then provided to the user along with a story.

[1468] The system also includes an emotion recognition mechanism that analyzes the user's emotions. This mechanism uses smart glasses or a camera-equipped device to analyze the user's facial expressions in real time and detect their emotions. Based on the analyzed emotions, a mechanism is activated to select and generate an appropriate image. This enables the creation of effective materials that are tailored to the user's emotions.

[1469] Specific hardware used will include smart glasses, camera-equipped devices, and personal computers. Software used will include OpenCV (image processing library), Keras (deep learning library), and PIL (Python Imaging Library).

[1470] For example, if a user displays a surprised expression, the emotion recognition system detects this expression and generates a prompt message such as, "Generate the most suitable advertisement for a user who is showing a surprised expression." Based on this prompt message, the generation AI system generates advertisements for new products or special offers that are appropriate for the surprised expression and provides them to the user.

[1471] Furthermore, if a user displays a joyful expression, a prompt message will be generated stating, "Generate the most suitable advertisement for users who are showing joyful expressions," and discount coupons or campaign information corresponding to that joyful expression will be provided.

[1472] In this way, effective material creation and advertising display tailored to the user's emotions can be achieved.

[1473] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[1474] Step 1:

[1475] The user inputs the scenario for the document using a terminal.

[1476] Input: Scenario text for the document the user wants to create.

[1477] Output: The scenario text is sent to the server.

[1478] Specific operation: The user enters the document scenario in text format through the terminal interface and presses the submit button.

[1479] Step 2:

[1480] The server receives the scenario text, and the generative AI system analyzes the scenario.

[1481] Input: Scenario text.

[1482] Output: Analyzed scenario data.

[1483] Specific operation: The server analyzes the scenario text using natural language processing techniques and extracts data for story generation.

[1484] Step 3:

[1485] The generative AI system generates a story based on the analyzed scenario data.

[1486] Input: Analyzed scenario data.

[1487] Output: The generated story text.

[1488] Specific operation: The server uses a generative AI model to generate story text from the analyzed scenario data.

[1489] Step 4:

[1490] The server uses internally stored data as external information and activates a mechanism to generate diagrams.

[1491] Input: The generated story text.

[1492] Output: Appropriate diagram.

[1493] Specific operation: The server uses image recognition technology to select and generate diagrams related to the story text from internal company documents.

[1494] Step 5:

[1495] The server provides the user with the generated story and diagram together.

[1496] Input: Generated story text and diagrams.

[1497] Output: Materials provided to the user.

[1498] Specific operation: The server integrates the generated story text and diagrams, creates materials for display to the user, and sends them to the terminal.

[1499] Step 6:

[1500] The device activates an emotion recognition mechanism to analyze the user's emotions in real time.

[1501] Input: Video of the user's facial expressions.

[1502] Output: Analyzed sentiment data.

[1503] Specific operation: The device's camera captures the user's facial expression, and an emotion recognition model is used to analyze the emotion.

[1504] Step 7:

[1505] The server operates a mechanism to select and generate an appropriate image based on the analyzed sentiment data.

[1506] Input: Analyzed sentiment data.

[1507] Output: A diagram corresponding to emotions.

[1508] Specific operation: The server generates prompt sentences based on sentiment data and uses a generative AI model to generate appropriate diagrams.

[1509] Step 8:

[1510] The server provides the user with a diagram that corresponds to their emotions.

[1511] Input: A diagram corresponding to an emotion.

[1512] Output: A diagram provided to the user.

[1513] Specific operation: The server sends the generated diagram to the user's terminal for display.

[1514] As a concrete example, if a user displays a surprised expression, the emotion recognition system detects this expression and generates a prompt message such as, "Generate the most suitable advertisement for a user who is showing a surprised expression." Based on this prompt message, the generation AI system generates advertisements for new products or special offers that are appropriate for the surprised expression and provides them to the user.

[1515] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1516] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1517] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1518] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1519] [Third Embodiment]

[1520] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1521] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1522] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1523] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1524] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1526] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1527] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1528] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1529] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1530] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1531] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1532] "Example of form 1"

[1533] One embodiment of the present invention provides a text input interface as a means for the user to input a scenario for a document they wish to create. This interface is designed to allow the user to freely write a scenario. For example, if the user inputs "New Product Presentation" as the scenario,

[1534] "Example of form 2"

[1535] Generative AI uses natural language processing techniques to analyze input scenarios. This analysis extracts keywords and themes from the scenario, and then generates a story based on them. For example, from a scenario about a "new product presentation," keywords such as "features of the new product," "comparison with competitors," and "market positioning" are extracted, and a story is generated based on these.

[1536] "Example of form 3"

[1537] The method for generating the diagrams utilizes internally stored documents as external information. Image recognition technology is used to select and generate appropriate diagrams from the documents. For example, diagrams related to "new product features" or "comparisons with competitors" might be selected. These diagrams, along with the generated story, are provided as answers, and users create their documents based on them.

[1538] The following describes the processing flow for each example of the form.

[1539] "Example of form 1"

[1540] Step 1: The user enters the scenario through a text input interface. For example, they might enter "New Product Presentation" as the scenario.

[1541] Step 2: The generative AI analyzes the input scenario using natural language processing techniques. This analysis extracts keywords such as "features of the new product," "comparison with competitors," and "market positioning."

[1542] Step 3: A story is generated based on the extracted keywords.

[1543] "Example of form 2"

[1544] Step 1: The generative AI analyzes the input scenario using natural language processing techniques.

[1545] Step 2: A story is generated based on the keywords and themes extracted through analysis.

[1546] Step 3: The generated story is provided to the user.

[1547] "Example of form 3"

[1548] Step 1: Use internally accumulated data as external information.

[1549] Step 2: Use image recognition technology to select the appropriate figure from the document.

[1550] Step 3: The selected figure is generated and provided as the answer along with the generated story.

[1551] (Example 1)

[1552] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1553] Traditional document creation systems required users to manually conceive of document content and create diagrams, which was time-consuming and labor-intensive. Furthermore, the quality and consistency of documents depended on the user's skills, making it difficult to produce documents of uniform quality. Additionally, there was a lack of effective means to utilize internally accumulated documents, resulting in insufficient information reuse.

[1554] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1555] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating prompt sentences based on the input scenario, means for sending the generated prompt sentences to a generation AI model to generate the document, means for returning the generated document to the user, means for using documents accumulated within the company as external information to generate a diagram, and means for providing the generated document and diagram together as a response. This makes it possible for users to easily and automatically generate high-quality documents. Furthermore, by effectively utilizing information accumulated within the company, information reuse is promoted and the efficiency of document creation is improved.

[1556] "User" refers to an individual or organization that uses the system to create documents.

[1557] A "scenario" refers to text information that describes the content and structure of the document that the user wants to create.

[1558] A "prompt message" refers to an instruction message sent to the AI ​​model based on a scenario.

[1559] A "generative AI model" refers to an artificial intelligence model that automatically generates documents based on input prompt text.

[1560] "Documents" refers to documents containing text and diagrams generated by a generative AI model.

[1561] "Diagrams" refer to visual content generated based on internal company documents and external information.

[1562] A "server" refers to a computer system that manages the processing of the entire system, receives input from users, sends prompt messages to the generating AI model, and returns the generated materials.

[1563] "External information" refers to data obtained from sources other than internal company documents.

[1564] "Natural language processing technology" refers to the techniques used to analyze scenarios and generate prompts and other information.

[1565] "Image recognition technology" refers to the technology used to select and generate appropriate diagrams from documents accumulated within a company.

[1566] Modes for carrying out the invention

[1567] This invention is a system that allows users to input a scenario for the material they wish to create, and then automatically generates the material based on that scenario. This system consists of multiple components, including a server, a terminal, and a generation AI model.

[1568] System Configuration

[1569] 1. Server

[1570] The server is the central component that manages the processing of the entire system. The server receives scenario input from the user, generates prompt statements, and sends them to the generating AI model. It also receives the data returned by the generating AI model and sends it back to the user. The server uses a database to temporarily store scenarios and generated data.

[1571] 2. Terminal

[1572] The terminal provides an interface for users to input scenarios. The terminal communicates with the server via a web browser or dedicated application to input scenarios and review generated materials.

[1573] 3. Generative AI Models

[1574] A generative AI model is an artificial intelligence model that automatically generates documents based on prompt messages sent from a server. Examples of generative AI models include OpenAI's GPT-4.

[1575] Program processing

[1576] The server receives a scenario entered by the user through the terminal's text input interface. The server then generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model, specifically instructing it on the content of the scenario.

[1577] The generative AI model generates materials based on the received prompt text. These materials are in a format consistent with the scenario and can include text and diagrams. The generated materials are sent back to the server, which then sends them back to the user.

[1578] Specific example

[1579] As a concrete example, consider a scenario where a user inputs "New Product Presentation" as the scenario. The user enters "New Product Presentation" into the terminal's text input interface. The server receives this scenario and generates a prompt message like the following:

[1580] Based on the "New Product Presentation" scenario, please create presentation materials that include the following:

[1581] 1. Features of the new product

[1582] 2. Market analysis

[1583] 3. Comparison with competing products

[1584] 4. Sales Strategy

[1585] The generation AI model receives this prompt and generates presentation materials based on the specified content. The generated materials are sent back to the user via the server. The user can review the generated materials and make corrections or additions as needed.

[1586] In this way, a system is realized in which servers, terminals, and generation AI models work together to automatically generate documents. This system allows users to easily create high-quality documents and effectively utilize the information accumulated within the company.

[1587] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1588] Step 1:

[1589] The user enters the scenario.

[1590] The user uses the terminal's text input interface to enter the scenario for the document they want to create. The entered scenario is sent from the terminal to the server. Specifically, the user opens a web browser, accesses the system's web page, and enters "New Product Presentation" into the text box. Input: Scenario text (e.g., "New Product Presentation"). Output: Scenario text sent to the server.

[1591] Step 2:

[1592] The server receives the scenario

[1593] The server receives scenarios submitted by users and stores them in a database. Specifically, the server receives an HTTP request and extracts the scenario text from the request body. The extracted text is then stored in the database. Input: Scenario text submitted by the user. Output: Scenario text stored in the database.

[1594] Step 3:

[1595] The server generates a prompt message.

[1596] The server generates a prompt based on the received scenario. This prompt is an instruction sent to the generating AI model. Specifically, the server analyzes the scenario "New Product Presentation" and generates a prompt like this: Based on the "New Product Presentation" scenario, please create a presentation document that includes the following: 1. Features of the new product 2. Market analysis 3. Comparison with competing products 4. Sales strategy. Input: Scenario text stored in the database. Output: Generated prompt.

[1597] Step 4:

[1598] The server sends a prompt message to the generated AI model.

[1599] The server sends the generated prompt to the AI ​​model. Specifically, the server generates an API request and sends the request, including the prompt, to the AI ​​model's endpoint. Input: The generated prompt. Output: The prompt sent to the AI ​​model.

[1600] Step 5:

[1601] The generative AI model generates the data.

[1602] The generative AI model generates materials based on the received prompt text. Specifically, the generative AI model analyzes the prompt text and generates presentation materials based on the specified content. The generated materials are in text or slide format. Input: Prompt text sent to the generative AI model. Output: Generated materials.

[1603] Step 6:

[1604] The server receives the generated data.

[1605] The server receives the data generated from the generative AI model. Specifically, the server receives the API response and extracts the generated data from the response body. The extracted data is then temporarily stored. Input: Data sent from the generative AI model. Output: Temporarily stored generated data.

[1606] Step 7:

[1607] The server returns the document to the user.

[1608] The server returns the generated document to the user. Specifically, the server generates an HTTP response and sends the response containing the generated document to the user's terminal. The user then views the document in a web browser. Input: Temporarily saved generated document. Output: Generated document sent to the user's terminal.

[1609] (Application Example 1)

[1610] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1611] Conventional document creation systems lacked the functionality to automatically generate advertising materials based on user-input scenarios, resulting in reduced advertising production efficiency. Furthermore, there was a need to provide the generated story and diagrams not just as answers, but in a format suitable for use as advertising materials.

[1612] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1613] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents stored within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story. This makes it possible to automatically generate advertising material based on a scenario input by the user and improve the efficiency of advertising production.

[1614] A "user" is an individual or organization that uses the system to input scenarios for materials and receives the generated stories and advertising materials.

[1615] A "scenario" is text information that describes the content and structure of the document that the user wants to create.

[1616] A "generative AI system" is a system that uses artificial intelligence technology to generate a story based on an input scenario.

[1617] "Documents accumulated within the company" refers to a collection of data and information that a company or organization possesses internally.

[1618] "External information" refers to data and information obtained from external sources, other than documents accumulated within the company.

[1619] "Means for generating diagrams" refers to technologies and systems for generating appropriate diagrams using internally accumulated data and external information.

[1620] A "generated story" is a narrative or explanatory text created based on a scenario using generative AI methods.

[1621] "Advertising materials" refer to content such as images, videos, and text created for advertising purposes based on a generated story.

[1622] "Means of providing responses" refers to systems and methods for providing users with generated stories, diagrams, and advertising materials.

[1623] The system for implementing this invention includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using documents accumulated within the company as external information, means for providing the generated story and diagram together as a response, and means for generating advertising material based on the generated story.

[1624] System program

[1625] This system is implemented using Python and utilizes OpenAI's GPT-3 as a generative AI tool. When a user inputs a scenario, a story is generated based on that scenario, and advertising materials are automatically generated as well.

[1626] Explanation of the process

[1627] The server operates using the following hardware and software:

[1628] Hardware:

[1629] Smartphone or PC

[1630] software:

[1631] Python

[1632] OpenAI API

[1633] Data processing and data computation:

[1634] 1. Scenario Input: The user inputs the scenario through a text input interface. For example, they might input a scenario such as "New Product Presentation".

[1635] 2. Story Generation: The input scenario is sent as a prompt to the AI ​​model (OpenAI's GPT-3). GPT-3 generates a story based on the prompt and returns it in text format.

[1636] 3. Diagram generation: Use internally accumulated data as external information and generate appropriate diagrams using image recognition technology.

[1637] 4. Generating ad materials: Generate ad materials (images, videos, text) based on the generated story.

[1638] 5. Providing the response: Provide users with the generated story, diagrams, and advertising materials.

[1639] Specific example

[1640] If a user enters "New Product Presentation," the following text will be returned as an example of the generated advertising material:

[1641] Example of a prompt:

[1642] Advertising Scenario: New Product Presentation

[1643] Please generate ad materials based on this scenario.

[1644] Examples of generated ad materials:

[1645] "New Product Presentation"

[1646] This new product was developed using the latest technology. It's high-performance yet easy to use, making everyday life more convenient. Buy it now and experience the future of living!

[1647] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[1648] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1649] Step 1:

[1650] The user enters the scenario. The user uses a text input interface to enter the scenario for the document they want to create. For example, they might enter a scenario such as "New Product Presentation." The entered scenario is then sent to the server.

[1651] Step 2:

[1652] The server receives a scenario and generates a prompt. The server creates a prompt based on the received scenario. This prompt is formatted to be sent to the generation AI model. For example, it might take the form of "Advertising Scenario: Presentation of a new product. Generate advertising material based on this scenario."

[1653] Step 3:

[1654] The server sends a prompt to the generative AI model, which then generates a story. The server uses the OpenAI GPT-3 API to send the prompt and receives a story from the generative AI model. The generative AI model generates a story based on the prompt and returns it to the server in text format. For example, a story might be generated such as, "This new product was developed using the latest technology. It is high-performance yet easy to use, making everyday life more convenient."

[1655] Step 4:

[1656] The server generates diagrams using documents stored within the company. The server uses image recognition technology to select and generate appropriate diagrams from the company's internal records. For example, diagrams illustrating the technical specifications and usage of a new product can be generated.

[1657] Step 5:

[1658] The server generates advertising materials based on the generated story. The server creates advertising materials (images, videos, text) based on the generated story. For example, advertising taglines and product images are created based on the generated story.

[1659] Step 6:

[1660] The server provides the user with generated stories, diagrams, and advertising materials. The server sends the generated stories, diagrams, and advertising materials to the user so that the user can review them. The user can then create advertisements using the provided materials.

[1661] In this way, it is possible to automatically generate advertising materials based on scenarios entered by users, thereby improving the efficiency of advertising production.

[1662] (Example 2)

[1663] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1664] Traditional document creation systems struggled to generate appropriate stories based on user-input scenarios. Furthermore, they lacked the functionality to automatically generate diagrams related to the generated stories, requiring users to create them manually. This resulted in a significant amount of time and effort being required for document creation.

[1665] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for analyzing the input scenario and extracting keywords and themes, means for generating a story based on the extracted keywords and themes, means for using documents stored within the company as external information and generating a diagram, and means for providing the generated story and diagram together as a response. This makes it possible to automatically generate a story and related diagrams based on the scenario input by the user, and to efficiently create documents.

[1666] A "user" is an individual or organization that uses the system to create materials.

[1667] A "scenario" is a document or text that outlines the content and structure of the material that the user wants to create.

[1668] "Means of input" refers to the interface or device that allows users to input scenarios into the system.

[1669] "Means of analysis" refers to a function that uses natural language processing technology to analyze the input scenario and extract keywords and themes.

[1670] "Keywords" are particularly important words or phrases within a scenario.

[1671] The "theme" is the central theme or topic of the scenario.

[1672] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on extracted keywords and themes.

[1673] "Documents accumulated within the company" refers to data and information that a company or organization possesses internally.

[1674] "External information" refers to all information available to the system, including documents stored within the company.

[1675] "Methods for generating diagrams" refers to a function that automatically creates appropriate diagrams based on internally accumulated documents and external information.

[1676] "Means of providing answers" refer to interfaces and devices that provide users with the generated stories and diagrams.

[1677] This invention relates to a system that automatically generates a story and related diagrams based on a scenario input by the user for a document they wish to create. A specific embodiment of this system is described below.

[1678] System program generation

[1679] The server generates a program for implementing a generative AI model. This program has the functionality to analyze input scenarios using natural language processing techniques and extract keywords and themes. It also includes the functionality to generate a story based on the extracted keywords and themes.

[1680] Hardware and software to be used

[1681] The server implements natural language processing techniques using the Python programming language and the TensorFlow library. When a scenario is input, the server first tokenizes the text and then uses the tokenized data to extract keywords and themes. This is done using the BERT (Bidirectional Encoder Representations from Transformers) model. Based on the extracted keywords and themes, the server generates a story using the GPT-3 (Generative Pre-trained Transformer 3) model.

[1682] Specific example

[1683] Consider a scenario where a user inputs the following: "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning." The scenario entered from the terminal is sent to the server. The server first tokenizes the scenario and extracts keywords such as "new product," "presentation," "features," "competitors," and "market." Next, it uses these keywords to generate a story using the GPT-3 model. The generated story might be, for example, "a presentation that highlights the new product's features, compares it to competitors, and explains its market positioning."

[1684] Example of a prompt

[1685] Examples of prompts that users might input into the generated AI model include the following:

[1686] "Generate a story for a new product presentation. Include the product's features, comparisons with competitors, and market positioning."

[1687] In this way, the system's program processing is explained while clearly defining how the server, terminal, and user are involved.

[1688] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1689] Step 1:

[1690] The user inputs a scenario. The user enters the scenario into the input field on the terminal. For example, they might enter, "Generate a story for a new product presentation. Include the product's features, comparison with competitors, and market positioning." The entered scenario becomes the input data on the terminal.

[1691] Step 2:

[1692] The terminal sends the scenario to the server. The terminal sends the user-entered scenario to the server as an HTTP request. At this time, the scenario is packaged in JSON format. The input data is the scenario entered by the user, and the output data is the scenario sent to the server.

[1693] Step 3:

[1694] The server tokenizes the scenario. The server tokenizes the received scenario using a natural language processing library (e.g., NLTK or SpaCy). Tokenization is the process of dividing the scenario into words and phrases. The input data is the scenario sent to the server, and the output data is the tokenized scenario.

[1695] Step 4:

[1696] The server extracts keywords and themes. The server inputs tokenized data into a BERT model and extracts important keywords and themes. For example, keywords such as "new product," "features," "competitors," and "market" are extracted. The input data is a tokenized scenario, and the output data is the extracted keywords and themes.

[1697] Step 5:

[1698] The server generates a story based on the extracted keywords and themes. The server uses a GPT-3 model to generate the story based on the extracted keywords and themes. The generated story might be, for example, "a presentation highlighting the features of a new product, comparing it to competitors, and explaining its market positioning." The input data consists of the extracted keywords and themes, while the output data is the generated story.

[1699] Step 6:

[1700] The server sends the generated story to the device. The server packages the generated story in JSON format and sends it to the device as an HTTP response. The input data is the generated story, and the output data is the story sent to the device.

[1701] Step 7:

[1702] The device displays the story to the user. The device displays the received story in the user interface. The user can review the generated story and modify or add to it as needed. The input data is the story sent to the device, and the output data is the story displayed to the user.

[1703] (Application Example 2)

[1704] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1705] Traditional advertising campaign creation processes often involved manual tasks such as scenario analysis, keyword extraction, and story generation, which were time-consuming and labor-intensive. Furthermore, the generated stories could lack consistency or be unsuitable for the target audience, making it difficult to create effective advertisements.

[1706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting a scenario of a document that the user wants to create, means for a generative AI that generates a story based on the input scenario, means for using internally stored documents as external information to generate a diagram, means for providing the generated story and diagram together as a response, and means for analyzing an advertising campaign scenario, extracting keywords and themes, and generating an advertising story. This automates the process from scenario analysis of an advertising campaign to story generation, making it possible to quickly create effective and consistent advertisements.

[1707] A "user" is an individual or organization that uses the system to create materials or advertising campaign scenarios.

[1708] A "scenario" is text information entered by users for advertising campaigns or document creation, and it forms the basis for story generation.

[1709] "Generative AI methods" refer to artificial intelligence technologies that analyze input scenarios and generate stories using natural language processing techniques.

[1710] "Documents accumulated within the company" refers to a collection of documents and data created in the past within a company or organization, which are used as external information.

[1711] "Methods for generating diagrams" refers to technologies that select and generate appropriate diagrams based on internally accumulated documents and external information.

[1712] "Means of providing answers" refers to the methods and technologies for providing users with the generated stories and diagrams.

[1713] An "advertising campaign" is a series of marketing activities aimed at promoting a specific product or service.

[1714] "Keywords" are important words or phrases extracted from the scenario, and they form the basis for generating the story.

[1715] The "theme" is the central theme or topic of the scenario, and it determines the direction of the story.

[1716] An "advertising story" is a consistent narrative created in line with the objectives of an advertising campaign, designed to appeal to the target audience.

[1717] The system for implementing this invention is configured as follows: First, a terminal is required for the user to input the materials or advertising campaign scenarios they wish to create. This terminal is a device such as a personal computer or smartphone, and provides an interface for the user to input the scenarios.

[1718] Next, the server receives the input scenario and analyzes it using generative AI tools. This analysis utilizes natural language processing techniques to extract keywords and themes from the scenario. Based on the extracted keywords and themes, the generative AI generates a story. For example, the OpenAI API might be used for this generative AI.

[1719] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The generated stories and diagrams are then provided from the server to the user's terminal as answers.

[1720] As a concrete example, consider a scenario where a user inputs a presentation scenario for a new product. This scenario includes information such as "the features of the new product," "comparison with competitors," and "market positioning." The server analyzes this scenario and generates prompt messages like the following.

[1721] Example of a prompt:

[1722] Scenario: In a new product presentation, emphasize the product's features, comparison with competitors, and market positioning.

[1723] Keywords: New product, presentation, features, competitors, market

[1724] Please generate an advertising story based on this scenario.

[1725] If this prompt is input into a generative AI model, it may generate an advertising story like the following.

[1726] Example of a generated ad story:

[1727] The new X-100 boasts innovative features not found in other products. Compared to its competitors, the X-100 offers superior performance and cost-effectiveness. Positioned in the market, the X-100 is the perfect choice for users seeking cutting-edge technology. Get your X-100 today and experience the future.

[1728] In this way, users can easily generate effective advertising stories. This system automates the process from scenario analysis to story generation for advertising campaigns, enabling the rapid creation of effective and consistent advertisements.

[1729] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1730] Step 1:

[1731] The user uses their device to input the document or advertising campaign scenario they want to create. The entered scenario is sent to the server in text format.

[1732] Step 2:

[1733] The server analyzes the received scenario. This analysis uses natural language processing techniques. Specifically, it extracts keywords and themes from the scenario. The input to this process is the text data of the scenario, and the output is a list of the extracted keywords and themes.

[1734] Step 3:

[1735] The server generates prompt sentences based on the extracted keywords and themes. The generated prompt sentences are then input to a generative AI model. The input to this process is a list of keywords and themes, and the output is prompt sentences.

[1736] Step 4:

[1737] The server uses a generative AI model to generate a story based on the prompt text. Specifically, it uses the OpenAI API to generate the story. The input to this process is the prompt text, and the output is the generated story.

[1738] Step 5:

[1739] The server uses internally stored data as external information to generate diagrams. Image recognition technology is used to select and generate appropriate diagrams. The input for this process is internal data, and the output is the generated diagram.

[1740] Step 6:

[1741] The server provides the generated story and diagram together as the answer to the user's terminal. The input to this process is the generated story and diagram, and the output is the answer data sent to the user's terminal.

[1742] Step 7:

[1743] Users review the stories and diagrams provided through their devices and make modifications or additions as needed. They then finalize the materials and advertising campaign content. The input for this process is the response data sent from the server, and the output is the completed materials and advertising campaign content.

[1744] (Example 3)

[1745] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1746] Traditional document creation systems required users to manually search for materials, select appropriate figures, and create stories, which was time-consuming and labor-intensive. Furthermore, specialized knowledge was required to select appropriate figures, making it difficult for users to create documents efficiently. Inconsistent quality of generated documents was also a challenge.

[1747] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1748] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, means for generating a story based on the input scenario using a generative AI, means for using documents stored within the company as external information, selecting and generating appropriate figures using image recognition technology, and means for providing the generated story and figures together as a response. This enables the user to efficiently create high-quality documents.

[1749] A "user" is an individual or organization that uses the system to create materials.

[1750] A "scenario" is text information that indicates the content and purpose of the document that the user wants to create.

[1751] "Generative AI methods" refer to methods that use artificial intelligence technology to generate stories based on input scenarios.

[1752] "Materials accumulated within the company" refers to information resources such as documents, images, and data stored within a company or organization.

[1753] "External information" refers to documents accumulated within the company, which are the source of information for the system to use.

[1754] "Image recognition technology" is a technology that analyzes images and diagrams to understand their content.

[1755] An "appropriate diagram" is a chart or image selected to visually represent information related to the scenario.

[1756] "Means of generation" refers to the means of creating a new diagram based on the selected diagram.

[1757] "Means of providing answers" refers to the means of providing users with the generated stories and diagrams.

[1758] "System" refers to a computer-based device or software for performing a series of processes, including the means described above.

[1759] This invention relates to a system for enabling users to efficiently create high-quality materials. Specific embodiments of this system are described below.

[1760] System Overview

[1761] This system includes the following main means:

[1762] 1. A means for users to input the scenario of the document they want to create.

[1763] 2. Generative AI means for generating a story based on an input scenario

[1764] 3. A means of using internally accumulated data as external information, and selecting and generating appropriate figures using image recognition technology.

[1765] 4. A means of answering with the generated story and diagram together.

[1766] Hardware and software to be used

[1767] The server runs the system using the following hardware and software:

[1768] Hardware: High-performance processor, sufficient memory, storage devices, network interface

[1769] Software: Google Cloud Vision API, Amazon Rekognition, Adobe Illustrator, Canva, Generative AI Models (e.g., OpenAI's GPT-4)

[1770] Data processing and data calculation

[1771] The server performs data processing and calculations using the following steps:

[1772] 1. Data collection:

[1773] The server accesses internal databases and file servers to search for and retrieve relevant documents, including PDFs, Word documents, and presentation files.

[1774] 2. Application of image recognition technology:

[1775] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the collected materials.

[1776] 3. Selecting the appropriate figure:

[1777] Based on the analysis results, the server filters and selects the appropriate diagrams related to the scenario.

[1778] 4. Figure generation:

[1779] The server uses Adobe Illustrator or Canva to generate new diagrams based on the selected diagrams.

[1780] 5. Story generation:

[1781] The server uses a generative AI model (GPT-4) to generate a story based on the generated diagrams.

[1782] 6. Providing an answer:

[1783] The server combines the generated diagrams and stories into a single document and provides it to the user.

[1784] Specific example

[1785] For example, if a user wants to create a document explaining the features of a new product, they would input the following prompt into the AI ​​model:

[1786] "Please generate a diagram to explain the features of the new product. Based on internal company documents, select and generate an appropriate diagram."

[1787] Upon entering this prompt, the server will generate a diagram following the steps outlined above and provide it along with the story. Users can then use this response to create their own materials.

[1788] In this way, users can efficiently create high-quality materials. The flow of the specific processing in Example 3 will be explained using Figure 15.

[1789] Step 1: Gathering materials

[1790] The server accesses internal databases and file servers to search for and retrieve relevant documents. The user provides a scenario for the document they wish to create as input. Based on this scenario, the server extracts keywords and uses these keywords to search for documents. The output includes relevant documents such as PDFs, Word documents, and presentation files.

[1791] Step 2: Application of image recognition technology

[1792] The server applies image recognition technology to the collected data. The data collected in Step 1 is provided as input. The server uses the Google Cloud Vision API and Amazon Rekognition to analyze images and charts within the data. Specifically, the server processes each data item sequentially and extracts the content of images and charts as text data. The analysis results are obtained as output.

[1793] Step 3: Select the appropriate figure

[1794] The server uses image recognition technology to select appropriate figures from the document. The analysis results obtained in step 2 are given as input. The server matches keywords related to the scenario with the analysis results and filters and selects the most relevant figures. Specifically, the server calculates a relevance score and selects the figures with the highest scores. The selected figures are obtained as output.

[1795] Step 4: Generate the diagram

[1796] The server generates a new diagram based on the selected diagram. The diagram selected in step 3 is given as input. Using Adobe Illustrator or Canva, the server uses the selected diagram as a template, adding the necessary information to create a new diagram. Specifically, the server adjusts the diagram's layout and adds annotations and data. The output is the generated new diagram.

[1797] Step 5: Generating the Story

[1798] The server generates a story based on the generated diagram. The diagram generated in step 4 is given as input. The server uses a generative AI model (GPT-4) to analyze the content of the diagram and generate related descriptive text and narratives. Specifically, the server analyzes each element of the diagram and generates corresponding text. The generated story is obtained as output.

[1799] Step 6: Provide your response

[1800] The server combines the generated diagrams and stories into a single document and provides it to the user. The input consists of the diagrams generated in step 4 and the stories generated in step 5. The server integrates these to create a single document and provides it to the user. Specifically, the server formats the document and adds necessary metadata. The output is the final document.

[1801] (Application Example 3)

[1802] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

[1803] Factory maintenance work is complex and diverse, making it difficult for workers to quickly grasp the correct procedures. Furthermore, the sheer volume of maintenance records and technical documents makes it difficult to efficiently search for and utilize necessary information. Therefore, there is a need to improve the efficiency and accuracy of maintenance work.

[1804] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1805] In this invention, the server includes means for inputting a scenario for a document that the user wants to create; means for generating a story based on the input scenario using a generative AI; means for generating a diagram using documents accumulated within the company as external information; means for providing the generated story and diagram together as a response; means for selecting and generating an appropriate diagram using image recognition technology, using maintenance records and technical documents accumulated within the factory as external information; and means for supporting maintenance work based on the selected diagram. This enables maintenance workers to quickly acquire the necessary information and perform their work efficiently and accurately.

[1806] "Users" refer to individuals who use the system to create or maintain documents.

[1807] A "scenario" refers to a plan or structure that outlines the content and purpose of the material that the user wants to create.

[1808] "Generative AI methods" refer to artificial intelligence technology that generates stories based on input scenarios.

[1809] "Documents accumulated within the company" refers to information such as technical documents and maintenance records created within the company in the past.

[1810] "External information" refers to documents and data accumulated within the company for use by the system.

[1811] "Methods for generating diagrams" refers to the technology of selecting and generating appropriate diagrams based on materials accumulated within the company.

[1812] "Image recognition technology" refers to the technology of extracting and analyzing text and features from images.

[1813] "Maintenance records" refer to information that records the history and details of maintenance work performed within the factory.

[1814] "Technical documents" refer to technical information and procedures related to equipment and systems within a factory.

[1815] "Means of supporting maintenance work" refers to support technologies that enable maintenance workers to perform tasks efficiently and accurately based on selected diagrams.

[1816] The system for implementing this invention is configured as follows: First, an interface is provided for the user to input a scenario for the document they wish to create. The user inputs the scenario through this interface.

[1817] Next, the server generates a story based on the input scenario. This uses generative AI tools, leveraging natural language processing techniques to analyze the scenario and generate an appropriate story. For example, the transformers library from Hugging Face can be used as a generative AI tool.

[1818] Furthermore, the server utilizes internally stored data as external information to generate diagrams. Image recognition technology is used to generate these diagrams. Specifically, software such as OpenCV, PIL (Python Imaging Library), and pytesseract are used to extract and analyze text and features from images.

[1819] Maintenance records and technical documents accumulated within the factory are also used as external information. Based on this information, the server uses image recognition technology to select and generate appropriate diagrams. The selected diagrams are used to support maintenance work.

[1820] The generated story and diagrams are provided to the user together. This allows the user to quickly obtain the necessary information and work efficiently and accurately.

[1821] As a concrete example, the following prompt statement can be used.

[1822] Example of a prompt:

[1823] "Based on the following images, please generate a story about the parts replacement procedure: ['maintenance_step1.png', 'maintenance_step2.png']"

[1824] Based on this prompt, the generative AI system generates a story about the parts replacement procedure and provides it to the user along with a diagram selected using image recognition technology.

[1825] In this way, maintenance work on factory robots can be made more efficient, and the burden on workers can be reduced.

[1826] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1827] Step 1:

[1828] Enter the scenario for the document you want to create.

[1829] Input: A scenario in which the user enters information through an interface.

[1830] Output: Input scenario data.

[1831] Specific operation: The user accesses the system interface and enters the scenario for the document they want to create in text format.

[1832] Step 2:

[1833] The server generates a story based on the input scenario.

[1834] Input: The entered scenario data.

[1835] Output: The generated story.

[1836] Specific operation: The server uses generative AI tools (e.g., Hugging Face's transformers library) to analyze the input scenario and generates an appropriate story using natural language processing techniques.

[1837] Step 3:

[1838] The server uses internally stored data as external information to generate diagrams.

[1839] Input: Documents accumulated within the company.

[1840] Output: The generated figure.

[1841] Specific operation: The server uses image recognition technology (e.g., OpenCV, PIL, pytesseract) to select and generate appropriate figures from documents stored within the company.

[1842] Step 4:

[1843] The server utilizes maintenance records and technical documents accumulated within the factory as external information, and uses image recognition technology to select and generate appropriate diagrams.

[1844] Input: Maintenance records and technical documents accumulated within the factory.

[1845] Output: Selected figure.

[1846] Specific operation: The server uses image recognition technology to select and generate necessary diagrams from maintenance records and technical documents.

[1847] Step 5:

[1848] The server will provide the generated story and diagram together as the answer.

[1849] Input: Generated story and selected diagram.

[1850] Output: The story and diagrams provided to the user.

[1851] Specific operation: The server combines the generated story and selected diagrams and provides them to the user. The user can then create materials based on this.

[1852] Step 6:

[1853] Maintenance work is performed based on the stories and diagrams provided by the user.

[1854] Input: Provided story and diagram.

[1855] Output: Efficient and accurate maintenance work.

[1856] Specific operation: Users perform maintenance tasks efficiently and accurately by referring to the provided stories and diagrams.

[1857] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1858] "Example of form 1"

[1859] One embodiment of the present invention provides a system that includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for using documents accumulated within the company as external information to generate a diagram, means for providing the generated story and diagram together as a response, and an emotion engine that recognizes the user's emotions.

[1860] "Example of form 2"

[1861] The emotion engine recognizes emotions from the user's tone of voice and facial expressions. For example, if the user shows a joyful expression, the emotion engine captures that information, and the generative AI generates a story based on that emotion. Specifically, if joy is recognized, the generative AI generates a story that includes positive elements.

[1862] "Example of form 3"

[1863] Furthermore, the emotion engine selects and generates appropriate images based on the user's emotions. For example, if the user shows a surprised expression, the emotion engine captures this information, and the image generation mechanism selects and generates an image that conveys surprise or novelty.

[1864] The following describes the processing flow for each example of the form.

[1865] "Example of form 1"

[1866] Step 1: Enter the scenario for the document you want to create.

[1867] Step 2: Based on the input scenario, the generative AI generates a story.

[1868] Step 3: Use internally accumulated data as external information to generate diagrams.

[1869] Step 4: Answer by combining the generated story and diagram.

[1870] Step 5: The emotion engine, which recognizes the user's emotions, starts operating.

[1871] "Example of form 2"

[1872] Step 1: The emotion engine recognizes the user's emotions from their tone of voice and facial expressions.

[1873] Step 2: Based on the emotions recognized by the emotion engine, the generative AI generates a story.

[1874] "Example of form 3"

[1875] Step 1: The emotion engine recognizes the user's emotions.

[1876] Step 2: The emotion engine selects and generates an appropriate diagram based on the emotions it recognizes.

[1877] (Example 1)

[1878] Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1879] Traditional document creation systems require users to manually input scenarios, construct stories, and generate diagrams, which is time-consuming and labor-intensive. Furthermore, the lack of feedback that considers user emotions leads to decreased efficiency and quality in document creation. Additionally, the lack of effective means to utilize internally accumulated documents can result in a lack of consistency and reliability in the materials.

[1880] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1881] In this invention, the server includes means for inputting a scenario for a document that the user wants to create, a generative AI means for generating a story based on the input scenario, means for generating a diagram using internally stored documents as external information, means for providing the generated story and diagram together as a response, and an emotion engine for recognizing the user's emotions. This improves the efficiency and quality of document creation, provides feedback that responds to the user's emotions, and enables the effective use of internal company documents.

[1882] "A means for users to input the scenario of the document they want to create" refers to an interface that allows users to freely input text, and is a means for inputting the scenario of the document.

[1883] "Generative AI methods" refer to artificial intelligence technologies used to generate stories based on input scenarios, and are methods that analyze scenarios using natural language processing technology to generate stories.

[1884] "A means of generating diagrams by using internally accumulated data as external information" refers to a method of generating diagrams by referring to data stored in an internal database and using data visualization technology.

[1885] "A means of providing a combined answer using generated stories and diagrams" refers to a method of integrating generated stories and diagrams and providing them to the user as a single document.

[1886] An "emotion engine" refers to a technology that analyzes user input and responses to recognize emotions. It uses natural language processing technology to analyze user emotions and provide appropriate feedback.

[1887] This invention is a system that allows users to input a scenario for a document they wish to create, generates a story and diagrams based on that scenario, and ultimately provides it as a single document. A specific embodiment of this system is described below.

[1888] System Configuration

[1889] This system consists of the following main components:

[1890] 1. Text input interface

[1891] 2. Generative AI means

[1892] 3. Database Reference Methods

[1893] 4. Data Visualization Methods

[1894] 5. Integration means

[1895] 6. Emotional Engine

[1896] Text input interface

[1897] The user uses the terminal's text input interface to enter the scenario for the document they want to create. This interface is designed to allow the user to freely enter text. For example, the user might type "New Product Presentation".

[1898] Generative AI means

[1899] The server receives the user-inputted scenario and inputs it as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model then generates an appropriate story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[1900] Database Reference Means

[1901] The server accesses the company's internal database to search for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib and Pandas to format and visualize the data.

[1902] Data visualization means

[1903] The server generates graphs using data visualization techniques based on information retrieved from the database. For example, it extracts necessary information from the database using SQL queries and generates graphs using Python's Matplotlib library. The generated graphs are saved as image files.

[1904] Integration means

[1905] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX or Microsoft Word APIs). The generated document is saved as a PDF file.

[1906] Emotional Engine

[1907] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1908] Examples of specific cases and prompt statements

[1909] Specific example

[1910] The user enters "New product presentation" into the text input interface.

[1911] The server uses a generation AI model to generate stories that include content such as "new product features, target market, competitive analysis, and sales strategy."

[1912] The server accesses past sales data and market research results from the company's internal database and creates relevant graphs and charts.

[1913] Finally, the generated stories and diagrams are integrated and presented to the user.

[1914] Example of a prompt

[1915] Please enter a presentation scenario for your new product. For example, include details such as "new product features, target market, competitive analysis, and sales strategy."

[1916] In this way, users can easily create high-quality documents.

[1917] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1918] Step 1:

[1919] The user enters the scenario.

[1920] The user uses the terminal's text input interface to enter the scenario for the document they want to create. For example, they might enter "New Product Presentation." The entered scenario is sent to the server in real time.

[1921] Input: Scenario text entered by the user

[1922] Output: Scenario text sent to the server

[1923] Step 2:

[1924] The server generates stories using an AI model.

[1925] The server inputs the received scenario as a prompt into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a story based on the input scenario. For example, it might create a story that includes content such as "new product features, target market, competitor analysis, and sales strategy."

[1926] Input: Scenario text

[1927] Output: Generated story text

[1928] Step 3:

[1929] The server generates the diagram by referencing the company's internal database.

[1930] The server accesses the company's internal database to search for relevant documents and data. For example, it refers to past sales data and market research results, and generates graphs and charts based on this data. The server uses libraries such as Python's Matplotlib and Pandas to format and visualize the data.

[1931] Input: Data retrieved from the company's internal database

[1932] Output: Image files of the generated graphs and charts

[1933] Step 4:

[1934] The server integrates the generated stories and diagrams.

[1935] The server integrates the generated stories and diagrams, compiling them into a single document. For example, it places diagrams corresponding to each section of the story in the appropriate locations, creating a consistent document overall. This process uses document generation tools (e.g., LaTeX or Microsoft Word APIs). The generated document is saved as a PDF file.

[1936] Input: Generated story text, image files of graphs and charts

[1937] Output: Integrated PDF document

[1938] Step 5:

[1939] The server recognizes the user's emotions and provides feedback.

[1940] The server uses an emotion engine to analyze user input and responses. For example, if a user inputs "This part is difficult to understand," the server will detect this and provide feedback such as, "Please tell us specifically which part is difficult to understand." The emotion engine uses natural language processing technology to analyze the user's emotions and respond appropriately.

[1941] Input: User feedback text

[1942] Output: Analysis results and feedback text from the emotion engine.

[1943] (Application Example 1) 【19...

Claims

[Claim 1] A system comprising a server and a terminal, wherein the server is A means for receiving input of a scenario, which is text information describing the content and structure of the document that the user wants to create, via the terminal, A means for analyzing the scenario using natural language processing technology and extracting keywords from the scenario, A means for generating a prompt statement to output the story of the document based on the aforementioned keywords, A means for inputting the generated prompt sentence into a generating AI model to generate the story, A means for selecting a figure related to the keyword from materials accumulated within the company to which the user belongs, using image recognition technology, A means for analyzing the user's voice data and facial image received from the terminal to recognize the user's emotions, A means for adjusting the generated story based on the user's emotions, A means for generating advertising material based on the adjusted story, Means for providing the adjusted story, the figures, and the advertising materials to the user, A system that includes this.

Images