system

Abstract

Provide a system. 【Solution means】 Means for a user to input a request for document creation, Means for analyzing the request and sending a document creation instruction to a generative artificial intelligence, Means for the generative artificial intelligence to generate a plurality of document samples, Means for providing the generated document samples to the user, Means for the user to select a sample and make modifications, Means for generating a final document reflecting the selected document sample and the modification content, Means for providing the final document to the user, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, document creation is frequently required, and the workload can be very high depending on the department or individual. Furthermore, the quality of documents depends on the skills of the creator and often takes a long time. As a result, there is a risk of reduced business efficiency and insufficient transmission of important information. To solve such a situation, a system that automates efficient and high-quality document creation and allows anyone to easily create high-quality documents is required.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system that includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate multiple document samples, means for providing the generated document samples to the user, means for the user to select and modify a sample, means for generating a final document that reflects the selected document samples and modifications, and means for providing the final document to the user. As a result, the user can easily create efficient and high-quality documents, and an improvement in work efficiency can be expected.

[0006] A "user" refers to a person or individual who uses the system to request document creation, select samples, and modify them.

[0007] A "request" refers to the information that a user enters into the system to communicate the content and conditions for creating a document.

[0008] "Analysis" refers to the process by which a server understands the content of a request received from a user and sends instructions for document creation to a generative artificial intelligence system based on that understanding.

[0009] "Generative artificial intelligence" refers to an artificial intelligence system that automatically generates multiple sample documents based on given instructions.

[0010] "Document samples" refer to multiple candidate documents created by a generative artificial intelligence system based on the user's request.

[0011] "Provision" refers to the act of a server sending a generated sample document to a user in a format that allows them to view it.

[0012] "Modification" refers to changes or additional information added to a document sample selected by the user.

[0013] "Reflection" refers to the process of applying user-submitted corrections to a sample document and generating the final document.

[0014] "Final document" refers to the final document that incorporates user requests and revisions.

[0015] A "server" refers to a computer system that receives and analyzes user requests, sends instructions to a generative artificial intelligence system, and provides generated sample materials and final materials.

[0016] A "terminal" refers to a device used by a user to access the system, enter requests, select and modify sample materials, and receive the final materials. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Modes for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system designed to assist in document creation, enabling users to efficiently produce high-quality documents. The following describes the system's program processing in natural language, along with specific examples.

[0039] Program processing

[0040] 1. User input of request

[0041] Users enter requests for new documents through the document creation system's interface. For example, they might request a document titled "New Product Introduction," specifying that it should include key features, an overview of the target market, and a price comparison with competing products.

[0042] 2. Sending a request

[0043] The terminal converts the user's request into an appropriate data format (e.g., JSON) and sends it to the server. The data includes the type of document, specific items, and other requirements.

[0044] 3. Parsing the request

[0045] The server analyzes the request received from the terminal and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[0046] 4. Instructions for preparing documents

[0047] The server sends instructions to the generative AI for document creation based on the analyzed request. These instructions include an overview of the document, key elements, and specific content.

[0048] 5. Generating sample materials

[0049] The generative AI automatically generates multiple document samples based on the received instructions. This means that even with the same request, documents with different layouts and designs will be provided.

[0050] 6. Obtaining sample materials

[0051] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[0052] 7. Provision of sample materials

[0053] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[0054] 8. User-selected and modified samples.

[0055] Users select the most suitable document from the provided sample materials and enter any necessary modifications. These modifications can range from adding information to specific slides to changing the design.

[0056] 9. Submit selected and modified data

[0057] The terminal sends the selected document sample and the corrections to the server.

[0058] 10. Generating the final document

[0059] The server incorporates the user's modifications and generates the final document. Generative AI may be used again if necessary.

[0060] 11. Provision of final documents

[0061] The server sends the final document to the terminal, which then displays it to the user. The user can then view and use the final document.

[0062] Specific example

[0063] 1. User request input:

[0064] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[0065] 2. Submitting a request:

[0066] The device sends data to the server in the following format: {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0067] 3. Parsing the request:

[0068] The server analyzes this request and instructs the generative AI to create a document introducing the new product.

[0069] 4. Instructions for preparing the document:

[0070] The server sends instructions to the generative AI to create documents based on the analysis results.

[0071] 5. Generating sample materials:

[0072] The generative AI generates multiple sample documents that include descriptions of key features, graphics of the target market, and price comparison charts.

[0073] 6. Obtaining sample materials:

[0074] The server receives the generated sample and saves it as a PDF file.

[0075] 7. Provision of sample materials:

[0076] The server sends a PDF file to the terminal, and the terminal displays it to the user.

[0077] 8. User selection and modification of samples:

[0078] The user selects one sample and enters the necessary modifications, such as "add more detail to the description of the main features" or "revise the graphic for the target market."

[0079] 9. Submit selected and corrected data:

[0080] The terminal sends the ID of the selected sample and the modifications made to the server.

[0081] 10. Generating the final document:

[0082] The server generates the final document, incorporating the received revisions.

[0083] 11. Provision of final documents:

[0084] The server sends the final document to the terminal, which then displays it to the user.

[0085] The following describes the processing flow.

[0086] Step 1:

[0087] The user launches the document creation system and enters the type of document and its specific content. For example, they might instruct the system to include key features, target market, and a price comparison with competing products in a document titled "New Product Introduction."

[0088] Step 2:

[0089] The terminal converts the user's input into an appropriate data format such as JSON and sends it to the server as data in the format {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0090] Step 3:

[0091] The server analyzes the received request. The analysis reveals that the document type is "New Product Introduction," and that the required items are three key features, the target market, and a price comparison.

[0092] Step 4:

[0093] The server sends instructions to the generative AI to create materials based on the analysis results. Specifically, it sends instructions in the form of {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0094] Step 5:

[0095] The generative AI receives instructions from the server and generates multiple sample documents based on the request. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[0096] Step 6:

[0097] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[0098] Step 7:

[0099] The server sends the generated sample documents to the terminal. The terminal displays these samples to the user. The user can view multiple samples.

[0100] Step 8:

[0101] The user selects the most suitable document from the provided sample materials and enters the necessary modifications. For example, they might enter details about the caption on a particular slide or modify the graphics.

[0102] Step 9:

[0103] The terminal sends the user-selected document sample and its modifications to the server. The data format is {"sample_id": 1, "modifications": ["Added function description", "Graphic modification"]}.

[0104] Step 10:

[0105] The server generates the final document, incorporating the user's revisions. If necessary, it uses a generation AI to apply the revisions again.

[0106] Step 11:

[0107] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0108] These steps enable the efficient and high-quality creation of documents.

[0109] (Example 1)

[0110] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] In today's business environment, there is a demand for the rapid and efficient creation of high-quality documents. However, document creation typically requires a significant amount of time and effort, and especially when complex content or design is required, professional assistance is often necessary. Therefore, document creation is a major burden for many companies and individuals. Furthermore, if revisions are needed to a document, manually correcting it each time is inefficient and prone to errors.

[0112] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0113] In this invention, the server includes means for the user to input a request for document creation, means for sending the request from the terminal to the server in JSON format, and means for the server to analyze the request and send instructions for document creation to a generative artificial intelligence. This enables the user to create high-quality documents quickly and efficiently.

[0114] A "user" is the entity that uses the document creation system to create documents.

[0115] A "document creation request" is information in which the user enters the content and requirements of the document they wish to have created.

[0116] A "terminal" is an electronic device used by a user to access a document creation system, enter and submit requests, or view and modify generated documents.

[0117] A "server" is a computer system that receives requests sent from a terminal, sends instructions to a generative artificial intelligence to create documents, and provides the terminal with generated document samples and the final document.

[0118] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation, and is a lightweight data exchange format for structuring and representing data.

[0119] "Generative artificial intelligence" refers to artificial intelligence that automatically generates materials based on user requests.

[0120] "Document samples" refer to some or all of the multiple candidate documents created by a generative artificial intelligence.

[0121] "PDF format" is an abbreviation for Portable Document Format, and it is a file format for displaying and distributing electronic documents.

[0122] A "presentation format" is a file format used to visually present information in the form of slides.

[0123] "Modification details" refer to the changes and additional information added to the document sample selected by the user.

[0124] The "final version" refers to the completed document that incorporates the user's revisions.

[0125] The system of this invention is designed to enable users to efficiently create high-quality materials. Specifically, it includes a terminal in which the user inputs a request for material creation, a server that analyzes the received request and sends instructions for material creation to a generative artificial intelligence (AI), means for providing a sample of the generated material, and means for the user to select a sample, make modifications, and generate and provide the final material.

[0126] Hardware and software to be used

[0127] terminal

[0128] The device is used by the user to access the document creation system, enter and submit requests, and view and modify the generated documents. Specific examples of such devices include personal computers, tablets, and smartphones.

[0129] server

[0130] The server receives and analyzes user requests and sends instructions to the generative artificial intelligence to create the document. The server also converts the generated document sample into PDF or presentation format and provides it to the user's device. Technically, a web server, database server, and AI model server are required to perform these operations.

[0131] Generative artificial intelligence (AI)

[0132] Generative AI is used to generate multiple sample documents based on user requests. Specifically, it can utilize natural language processing (NLP) and machine learning models (e.g., GPT-3®).

[0133] Software Tools

[0134] For sending and receiving data in JSON format, program libraries (e.g., fetch API or axios) are used, and for converting to PDF format, dedicated libraries such as PDFKit are used.

[0135] Data processing and data calculation

[0136] Request parsing

[0137] The server analyzes the request sent from the terminal. This analysis clarifies the purpose and necessary items for creating the document. For example, if the request is for a "new product introduction," the analysis results will include details such as key features, target market, and price comparison.

[0138] Generating document samples

[0139] The server sends the analysis results to the generative AI, which then generates multiple document samples based on those results. The generated documents will include different layouts and designs even for the same request.

[0140] Converting document samples

[0141] The server receives the sample documents created by the generative AI and converts them into an appropriate format (such as PDF or presentation format). This conversion process uses a dedicated software library.

[0142] Reflecting user-submitted changes

[0143] The user selects a provided sample document and enters the necessary revisions. The terminal sends the revisions to the server, which then incorporates them to generate the final document. If necessary, the generation AI may be used again. The final document is saved in PDF format or another format and provided from the server to the terminal.

[0144] Specific example

[0145] For example, a user might input a request such as, "I want to create a new product introduction document. It should include three main features, a target market, and a price comparison." The device sends this request to the server in JSON format. The server parses the request and instructs the generative AI to create the "new product introduction" document. The generative AI receives the instructions and generates several sample documents, each containing a description of the main features, a graphic of the target market, and a price comparison chart. The server saves the generated samples as PDF files and sends them to the device. The user selects one of the samples and inputs revisions, such as "add more detail to the description of the main features" or "revise the graphic of the target market." The device sends the ID of the selected sample and the revisions to the server, which then generates the final document incorporating these changes. The final document is sent to the device and displayed to the user.

[0146] Example of a prompt

[0147] Please create a product introduction document for the new product. It should include three key features, the target market, and a price comparison.

[0148] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0149] Step 1:

[0150] The user accesses the interface of the document creation system and enters a request for a new document. This includes the type of document and specific requirements (e.g., "I want to create a document introducing a new product. It should include three key features, the target market, and a price comparison"). The entered information is temporarily stored by the terminal.

[0151] Step 2:

[0152] The terminal converts the user's request into JSON format. For example, based on the input request, it converts it into a data format like {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}. This converted data becomes the input.

[0153] Step 3:

[0154] The terminal sends the request data, converted to JSON format, to the server. The transmission uses the HTTP protocol, delivering the request to the specified endpoint on the server side. This allows the server to begin parsing.

[0155] Step 4:

[0156] The server analyzes the request data it receives. It extracts information such as the type of request and detailed items, and generates the results as analysis data. For example, if the server receives a request for "new product introduction," it analyzes the items of main features, target market, and price comparison, and stores that data in list format.

[0157] Step 5:

[0158] The server sends instructions to the generating AI for document creation based on the analyzed data. The instructions include the type of document and the requested detailed items. For example, the generating AI might receive a prompt message such as, "Please create an introductory document for the new product. It should include three main features, the target market, and a price comparison."

[0159] Step 6:

[0160] The generation AI automatically generates multiple sample documents based on the instructions it receives. These sample documents include a variety of designs and layouts, such as explanations of key features, graphics of the target market, and price comparison charts. The generated sample documents become the output data.

[0161] Step 7:

[0162] The server receives sample documents generated by the generative AI and converts them into PDF or presentation formats. PDFKit or similar libraries are used for this conversion. The converted files become the new output.

[0163] Step 8:

[0164] The server sends the converted document sample to the terminal. HTTP or HTTPS protocols are used for transmission. The terminal loads the received document sample into an interface for displaying it to the user.

[0165] Step 9:

[0166] The user selects the most suitable document from the provided sample materials and enters any necessary modifications. For example, they might add information to a specific slide or change its design. The user's modifications are temporarily saved on the device.

[0167] Step 10:

[0168] The device converts the data containing the corrections into JSON format and sends it to the server along with the sample selection data. For example, it will be in the format {"sampleID": "12345", "corrections": ["Detailed description of key features", "Modified graphics for the target market"]}.

[0169] Step 11:

[0170] The server incorporates the received revisions and generates the final document. If necessary, the generation AI is used again to apply the final revisions. The generated final document becomes the output data.

[0171] Step 12:

[0172] The server sends the final document to the terminal, which then displays it to the user. The user can then review the final document and use it immediately.

[0173] (Application Example 1)

[0174] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0175] Planning production in a factory is extremely complex, requiring consideration of numerous factors. Furthermore, the efficient allocation of limited resources places a heavy burden on managers. Current systems require replanning every time changes or modifications occur, which is time-consuming and labor-intensive. Additionally, these plans are typically created manually, making them prone to human error. Therefore, a system is needed that can automatically generate production plans and manage them efficiently and accurately.

[0176] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0177] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate multiple document samples, means for providing the generated document samples to the user, means for the user to select and modify the samples, means for generating a final document reflecting the selected document samples and modifications, means for providing the final document to the user, means for inputting a request for production planning, means for analyzing the request and generating multiple production schedules, means for providing the generated production schedules to an administrator, means for the administrator to select and modify schedules, means for generating a final production plan reflecting the selected production schedules and modifications, and means for providing the final production plan to an administrator. This enables the automatic generation and efficient modification of production plans.

[0178] A "user request" is a specific request that a user enters into the system regarding document creation or production planning.

[0179] "Generative artificial intelligence" refers to artificial intelligence that has the ability to automatically generate documents and production schedules based on the data and instructions it receives.

[0180] A "document sample" is an initial draft document with multiple variations, created by a generative artificial intelligence.

[0181] A "production plan request" is when a manager enters specific production goals and requirements into the system.

[0182] "Generated production schedule" refers to an initial production schedule proposal with multiple variations, created by a generative artificial intelligence.

[0183] A "manager" is a person who is responsible for overseeing the planning and progress of a factory or production line.

[0184] "Final document" refers to the document that has been finalized to reflect user revisions.

[0185] The "final production plan" refers to the production schedule that has been finalized to reflect any revisions made by the manager.

[0186] This invention is a system for supporting the automatic generation of production plans and document creation in factories. It is designed to enable users to efficiently create high-quality documents and factory managers to formulate optimal production schedules. The following describes the program processing of this system, with specific examples.

[0187] System Configuration

[0188] The system consists of user terminals and a server. The user terminals provide an interface for inputting and modifying requests for document creation and production planning. The server works in conjunction with generative artificial intelligence (AI) to process requests and generate the final documents and production plans.

[0189] Hardware and software to be used

[0190] Generative artificial intelligence: Uses generative AI models such as GPT-3 from OpenAI (registered trademark).

[0191] Server-side framework: Use FastAPI to process requests.

[0192] Data format: JSON format is used for exchanging requests and generated data.

[0193] Data storage: Use a database (e.g., PostgreSQL) as needed.

[0194] Program Processing Description

[0195] 1. Entering a user request

[0196] Users enter requests for new materials and production plans through the terminal interface. For example, they might enter specific requirements such as, "I want to create a new product introduction document, including key features, target market, and price comparison."

[0197] 2. Sending and parsing requests

[0198] The terminal converts the input request into JSON format and sends it to the server. The server analyzes the request and sends instructions to the generative artificial intelligence for creating documents and production plans.

[0199] 3. Generation by generative artificial intelligence

[0200] Generative artificial intelligence automatically generates multiple document samples and production schedules based on transmitted instructions. It utilizes text generation models and schedule optimization algorithms as needed.

[0201] 4. Obtaining and providing samples

[0202] The server saves generated document samples and production schedules in PDF or presentation format and sends them to terminals. Users and administrators can then view these, select appropriate samples, and make modifications.

[0203] 5. Reflecting selections and modifications

[0204] Users and administrators select the best sample from the provided options and enter any necessary modifications. These modifications are sent to the server and reflected in the final documentation and production schedule.

[0205] 6. Provision of final documents and plans

[0206] The server generates the final documents and production plans reflecting the revisions and sends them to the terminal. Users and administrators can then view and use the finalized documents and plans.

[0207] Specific example

[0208] A user enters the following request into the system: "Generate a production schedule for a new smartphone. Production volume is 1000 units, deadline is December 31, 2024. Use a high-performance battery." The system analyzes this request and generates multiple production schedules. The user then selects the best schedule from the generated ones, makes any necessary modifications, and can review the final production plan.

[0209] Example of a prompt

[0210] "Please generate a production schedule for the new smartphone."

[0211] This allows users to easily enter requests, and generative artificial intelligence can generate documents and plans quickly and accurately.

[0212] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0213] Step 1:

[0214] Users enter requests for document creation and production planning.

[0215] Specific actions:

[0216] Users enter requests through the terminal's interface. For example, they might enter "new product information" or "production schedule for the new smartphone."

[0217] Input: Parent document name, requirements, and other conditions.

[0218] Output: Request data entered on the terminal.

[0219] Step 2:

[0220] Convert the request to JSON format and send it to the server.

[0221] Specific actions:

[0222] The terminal converts the user's input request into JSON format and sends that data to the server.

[0223] Input: Request data entered into the terminal by the user.

[0224] Output: Request data converted to JSON format, sent to the server.

[0225] Step 3:

[0226] The server analyzes the request and sends instructions to the generative artificial intelligence system for creating documents or generating production schedules.

[0227] Specific actions:

[0228] The server analyzes the received request data and sends instructions to the generative artificial intelligence based on the analysis results.

[0229] Input: Request data in JSON format.

[0230] Output: Analysis results, instruction data for the generative AI.

[0231] Step 4:

[0232] Generative artificial intelligence generates multiple document samples and production schedules.

[0233] Specific actions:

[0234] Generative artificial intelligence generates document samples and production schedules based on instructions received from a server. During this process, it performs necessary data processing and calculations.

[0235] Input: Instruction data from the server.

[0236] Output: Multiple document samples or production schedule.

[0237] Step 5:

[0238] The server retrieves the generated document samples and production schedules, saves them in PDF or presentation format, and then sends them to the terminal.

[0239] Specific actions:

[0240] The server receives samples and schedules generated by the generative AI, converts them to a standard file format, saves them, and then sends them to the terminal.

[0241] Input: Sample documents or production schedule data from a generative AI.

[0242] Output: Files converted to PDF or presentation format, data sent to the device.

[0243] Step 6:

[0244] Users and administrators select a sample provided and enter any necessary modifications.

[0245] Specific actions:

[0246] Users and administrators select the best option from several samples displayed on the terminal and enter the necessary modifications.

[0247] Input: Samples and modifications selected by the user or administrator.

[0248] Output: Data including correction instructions.

[0249] Step 7:

[0250] The terminal sends the selected document sample or production schedule and revision details to the server.

[0251] Specific actions:

[0252] The terminal converts the data, including the correction instructions, into JSON format and then sends it to the server.

[0253] Input: Data including correction instructions from users or administrators.

[0254] Output: Correction instruction data converted to JSON format, sent to the server.

[0255] Step 8:

[0256] The server incorporates the correction instructions and generates the final documents and production schedule.

[0257] Specific actions:

[0258] Based on the received correction instructions, the server uses generative artificial intelligence to generate the final documents and production schedules. Re-analysis is performed as needed during this process.

[0259] Input: Correction instruction data in JSON format.

[0260] Output: Final documents or production schedule.

[0261] Step 9:

[0262] The server sends the final documents or production plans to the terminal, which users and administrators then view and use.

[0263] Specific actions:

[0264] The server converts and saves final documents and production schedules in PDF or presentation format and sends them to the terminal. Users and administrators then view and use them on their terminals.

[0265] Input: Final documents or production schedule data.

[0266] Output: Final files converted to PDF or presentation format, sent to devices, and viewed and used by users and administrators.

[0267] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0268] This invention is a system that assists in document creation, designed to enable users to efficiently create high-quality documents. Furthermore, by incorporating an emotion engine that recognizes user emotions, it achieves optimal document creation tailored to the user's emotions. The following describes the system's program processing in natural language, along with specific examples.

[0269] Program processing

[0270] 1. User input of request

[0271] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and should include key features, target market, and a price comparison with competing products.

[0272] 2. Emotion recognition by the emotion engine

[0273] The device uses an emotion engine to recognize the user's emotions when they input data. For example, it might analyze facial expressions using a camera or analyze the tone of voice during voice input.

[0274] 3. Sending a request

[0275] The device converts the user's request and recognized emotions into an appropriate data format, such as JSON, and sends it to the server.

[0276] 4. Parsing the request

[0277] The server analyzes the received request and sentiment data, and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[0278] 5. Instructions for creating documents that take emotional data into consideration.

[0279] The server sends instructions to the generative AI for creating materials based on recognized emotion data. For example, if the user is feeling stressed, it recommends designs and tones that promote relaxation.

[0280] 6. Generating sample materials

[0281] The generative AI automatically generates multiple sample materials based on the received instructions. Each sample includes the main functions, graphics of the target market, and a price comparison chart of competing products.

[0282] 7. Acquisition of Sample Materials

[0283] The server receives the sample materials generated by the generative AI and converts them into an appropriate format (e.g., PDF format or presentation format).

[0284] 8. Provision of Sample Materials

[0285] The server sends the generated sample materials to the terminal, and the terminal displays them to the user. The user can view multiple sample materials.

[0286] 9. Sample Selection and Modification by the User

[0287] The user selects the most suitable one from the provided sample materials and enters the necessary modifications. For example, enter modification points such as elaborating on the description text of a specific slide or modifying the graphics.

[0288] 10. Monitoring of Emotions by the Emotion Engine

[0289] The terminal monitors the changes in the user's emotions using the emotion engine while the user selects and modifies the materials. According to the changes in emotions, the system makes appropriate samples and modification suggestions.

[0290] 11. Transmission of Selection and Modification Data

[0291] The terminal sends the sample materials selected by the user, the modification content, and the emotion data to the server. The data format is in the form of {"sample_id": 1, "modifications": ["Addition of function description text", "Graphic modification"], "emotion": "satisfied"}.

[0292] 12. Generating the final document

[0293] The server generates the final document, reflecting the user's revisions and final sentiment data. If necessary, it uses a generative AI to apply revisions again.

[0294] 13. Provision of final documents

[0295] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0296] Specific example

[0297] 1. User request input:

[0298] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[0299] 2. Emotion recognition by an emotion engine:

[0300] The device analyzes the user's facial expressions using its camera and recognizes that the user is feeling somewhat anxious.

[0301] 3. Submitting a request:

[0302] The device sends the request content and emotion data to the server as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0303] 4. Parsing the request:

[0304] The server analyzes the request and sends instructions to the generative AI to create new product introduction materials, taking into account the user's emotional data at this time.

[0305] 5. Instructions considering emotional data:

[0306] The server sends a document creation instruction to the generative AI that recommends color schemes and fonts that make the user feel at ease.

[0307] 6. Generation of document samples:

[0308] The generative AI focuses on multiple document samples with a calming tone.

[0309] 7. Obtaining document samples:

[0310] The server converts the generated samples into PDF format and receives them.

[0311] 8. Providing document samples:

[0312] The server sends the document samples to the terminal and displays them to the user.

[0313] 9. User selection and modification:

[0314] The user selects one of the samples and instructs to elaborate on the description of the main function.

[0315] 10. Emotion monitoring:

[0316] The terminal recognizes that the user is relaxed and makes modification suggestions.

[0317] 11. Sending selection and modification data:

[0318] The terminal sends the selected sample and the modification content to the server as {"sample_id": 1, "modifications": ["Addition of function description text"], "emotion": "relaxed"}.

[0319] 12. Generation of the final document:

[0320] The server generates the final document, reflecting the changes and sentiment data.

[0321] 13. Provision of final documents:

[0322] The server sends the final document to the terminal and displays it to the user.

[0323] The following describes the processing flow.

[0324] Step 1:

[0325] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and include key features, target market, and a price comparison with competing products.

[0326] Step 2:

[0327] The device uses an emotion engine to recognize the user's emotions along with their input. For example, it uses the camera to analyze facial expressions to determine whether the user is feeling safe or anxious.

[0328] Step 3:

[0329] The device converts the user's request and recognized emotions into an appropriate data format such as JSON, and sends data to the server such as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0330] Step 4:

[0331] The server analyzes the received request and sentiment data. Based on the analysis, it understands that the document type is "new product introduction," and that the necessary items are three key features, the target market, and a price comparison. Furthermore, it considers that the user's sentiment is anxiety.

[0332] Step 5:

[0333] The server sends detailed instructions for document creation to the generative AI based on the analyzed content and emotional data. For example, it sends instructions in the format {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"], "desired_tone": "calm"}.

[0334] Step 6:

[0335] The generative AI receives instructions from the server and generates multiple sample documents with designs and tones that take user emotions into consideration. This results in documents that users can use with confidence.

[0336] Step 7:

[0337] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[0338] Step 8:

[0339] The server sends generated sample materials to the terminal. The terminal displays these samples to the user. The user can review multiple samples and select a design that matches their emotional state.

[0340] Step 9:

[0341] The user selects the most suitable document sample from those provided and enters the necessary modifications. For example, they might enter modifications such as "explain the main features in more detail" or "revise the target market graphic."

[0342] Step 10:

[0343] The device uses an emotion engine to monitor the user's emotional changes while they select and modify materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[0344] Step 11:

[0345] The device sends the user's selected sample document, its modifications, and emotion data to the server. For example, it sends data in the format {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[0346] Step 12:

[0347] The server generates the final document based on user feedback and final sentiment data. If necessary, it applies further revisions using a generative AI.

[0348] Step 13:

[0349] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0350] (Example 2)

[0351] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0352] In modern society, there is an increasing demand for rapid and high-quality document creation, but many users spend too much time on it. Furthermore, since users' emotions during document creation often influence the results, it is desirable to create documents that are emotionally resonant.

[0353] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0354] In this invention, the server includes means for recognizing the user's emotions and reflecting them in the document creation instructions, means for monitoring changes in the user's emotions, and means for sending and receiving document creation requests in JSON format. This enables the creation of optimal documents based on the user's emotions. Furthermore, because the document creation process is streamlined, the user can quickly create high-quality documents.

[0355] A "user" is a person or group that uses the document creation system, enters a document creation request, and reviews and modifies the document generated based on that request.

[0356] A "document creation request" is information that indicates the specific requirements regarding the type and content of the document the user wants to create, and it is entered into the system.

[0357] "Means for analyzing requests" refers to a device or program that receives a document creation request entered by a user, understands and interprets it, and sends the necessary instructions to a generative artificial intelligence.

[0358] "Generative artificial intelligence" refers to an algorithm or software system that automatically generates materials based on specified instructions.

[0359] "Document samples" refer to multiple candidate documents generated by a generative artificial intelligence system, serving as initial documents for the user to select and modify.

[0360] "Means of providing to the user" refers to a device or program for transmitting generated sample materials or final materials to the user's terminal and displaying them.

[0361] "Final document" refers to the document that has been finalized after reflecting the user's selections and revisions.

[0362] "Means of recognizing emotions" refers to a device or program that analyzes the user's facial expressions, voice tone, etc., to identify the user's emotional state.

[0363] "Means for monitoring emotional changes" refers to a device or program that monitors changes in a user's emotional state in real time during the document creation process and records and analyzes that data.

[0364] JSON format is a type of data format that describes data attributes and values ​​using key-value pairs, making it easy for humans to read and easy for machines to analyze.

[0365] "PDF format" is an abbreviation for Portable Document Format, and it is a digital file format for saving and viewing documents and images with high accuracy.

[0366] A "presentation format" is a type of material used for presentations and explanations, where information is visually organized in a slide format.

[0367] This invention is a document creation support system designed to enable users to efficiently create high-quality materials. The system automatically generates product introductions, presentation materials, and other documents based on user requests. Furthermore, it utilizes an emotion engine to recognize the user's emotions and create optimal materials accordingly. The following describes the program processing of this system.

[0368] Program processing

[0369] 1. User input of request

[0370] Users input the type and specific content of the required document through the document creation system's interface. If a user wants to create a document introducing a new product, they would specify items such as key features, target market, and price comparisons with competing products.

[0371] 2. Emotion recognition by the emotion engine

[0372] The device activates an emotion engine when the user inputs text, analyzing the user's facial expressions using the camera (using computer vision technology), or analyzing their voice tone when they input voice text. This allows the device to recognize the user's emotional state, such as whether they are relaxed, tense, or anxious.

[0373] 3. Sending a request

[0374] The device converts the user's request content and recognized emotion data into JSON format and sends it to the server. For example, if the request is for a new product introduction and the user is feeling somewhat anxious, the data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0375] 4. Request parsing and instruction generation

[0376] The server analyzes the received request data and sentiment data. This clarifies the purpose and necessary items for document creation, and generates prompts for the AI ​​model. For example, if the user is feeling anxious, instructions such as "Create a new product introduction document using a design and tone that promotes relaxation" will be generated.

[0377] 5. Generating sample materials using a generative AI model

[0378] The generative AI automatically generates multiple sample documents based on instructions sent from the server. For example, documents containing key features, graphics of the target market, and price comparison charts of competing products can be generated.

[0379] 6. Sending and converting sample documents

[0380] The server receives the generated sample materials and converts them into PDF or presentation format. The samples are saved as PDF document or slide files.

[0381] 7. Provision of sample materials

[0382] The server sends the converted document sample to the terminal, which then displays it to the user. The user can view multiple document samples on the screen and select the most suitable one.

[0383] 8. User selection and modification of samples

[0384] The user selects the most suitable sample material from the provided options and enters the necessary modifications. For example, they might enter specific details such as enriching the caption on a particular slide or modifying a graphic.

[0385] 9. Emotion monitoring using an emotion engine

[0386] The device uses an emotion engine to monitor the user's emotional changes in real time while they select and modify sample materials. This allows for the provision of appropriate samples and modification suggestions based on the user's emotional state.

[0387] 10. Generating the final document

[0388] The server generates the final document, reflecting the user's selections and modifications, as well as sentiment data. If necessary, it re-applies modifications using the regenerating AI model.

[0389] 11. Provision of final documents

[0390] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0391] Specific example

[0392] Here's a concrete example: A user enters the following request into the document creation system: "I want to create a document introducing a new product. It should include three main features, the target market, and a price comparison." The device uses its camera to analyze the user's facial expressions and recognizes that the user is feeling somewhat anxious. The device then sends the request and emotion data to the server.

[0393] The server analyzes the request and sends an instruction to the generative AI model to "create a new product introduction document using a design and tone that conveys a sense of relaxation." The generative AI model creates several sample documents and sends them to the server. The server converts the samples to PDF format and sends them to the user's device. The user reviews the sample documents, selects the most suitable one, and enters suggestions for revisions, such as adding more detail to the description of the main features.

[0394] The device continues to monitor the user's emotions while they are making revisions, recognizing that the user is relaxed. The device sends the revisions to the server, which generates the final document and sends it back to the device. The user can then review and use the final document.

[0395] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0396] Step 1:

[0397] Users input the type and specific content of the required documents through the document creation system's interface.

[0398] In terms of specific operations, if a user requests information about a new product, they will specify detailed items such as key features, target market, and price comparison with competing products. The input data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}.

[0399] Step 2:

[0400] The device activates an emotion engine when the user makes a input, and recognizes the user's emotions.

[0401] Specifically, the system uses computer vision technology to analyze the user's facial expressions with a camera, or analyzes their voice tone through a microphone during voice input. For example, if the user is feeling somewhat anxious, the recognition result will be "anxious." The input data is the user's facial expressions and voice tone, and the output data is their emotional state (e.g., "anxious").

[0402] Step 3:

[0403] The device converts the user's request and recognized sentiment data into JSON format and sends it to the server.

[0404] Specifically, the system receives user request details and emotion data (e.g., {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}) as input data and sends this to the server.

[0405] Step 4:

[0406] The server analyzes the received request data and sentiment data.

[0407] Specifically, the process involves parsing JSON data to identify the purpose of the document creation and the necessary items. Through this analysis, a prompt message is generated. The input data is the received JSON request data, and the output data is the prompt message (e.g., "Create a new product introduction document using a design and tone that conveys a sense of relaxation").

[0408] Step 5:

[0409] The server sends instructions to the AI ​​model for creating the document based on the generated prompt message.

[0410] In terms of specific operations, a prompt message is sent to a generating AI model, requesting it to generate sample materials based on the instructions. The input data is the prompt message, and the output data is the instructions sent to the generating AI model.

[0411] Step 6:

[0412] The generation AI model automatically generates multiple document samples based on instructions sent from the server.

[0413] Specifically, the system creates documents that include key features, graphics for the target market, and price comparison charts of competing products. The input data consists of instructions (prompt messages) from the server, and the output data is a sample of the generated document.

[0414] Step 7:

[0415] The server converts the generated sample documents into the appropriate format.

[0416] Specifically, the process involves converting sample documents into PDF or presentation formats. For example, it converts an HTML document sent from a generative AI model into a PDF. The input data is the generated sample document, and the output data is the sample document converted into the appropriate format.

[0417] Step 8:

[0418] The server sends the converted document sample to the terminal, which then displays it to the user.

[0419] Specifically, the system displays samples on the user interface, allowing the user to select from multiple sample documents. The input data consists of converted sample documents, while the output data is the sample document displayed to the user.

[0420] Step 9:

[0421] The user selects the most suitable document from the provided sample materials and enters any necessary modifications.

[0422] In terms of specific actions, users input details of modifications, such as enriching the description on a particular slide or correcting a graphic, through the interface. The input data consists of the selected document sample and the modifications, while the output data is the modified document sample.

[0423] Step 10:

[0424] The device uses an emotion engine to monitor changes in the user's emotions while they select and modify materials.

[0425] Specifically, the system uses cameras and microphones to analyze the user's emotions in real time and monitors their emotional state. Input data includes the user's facial expressions and voice tone, while output data is their real-time emotional state.

[0426] Step 11:

[0427] The device sends the user's selected document sample, its modifications, and sentiment data to the server.

[0428] Specifically, the terminal packages the revised document sample and sentiment data and sends them to the server. The input data consists of the revised document sample and sentiment data, while the output data is the data sent to the server.

[0429] Step 12:

[0430] The server generates the final document, incorporating user modifications and final sentiment data.

[0431] Specifically, the process involves modifying and creating the final document using an AI model. The input data consists of the modifications and sentiment data, while the output data is the final document.

[0432] Step 13:

[0433] The server sends the final generated document to the terminal, which then displays it to the user.

[0434] Specifically, the final document is displayed on the user interface for review. The input data is the final document, and the output data is the final document displayed to the user.

[0435] (Application Example 2)

[0436] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0437] Currently, when users create advertising materials, they need to consider the design and content individually, which is especially time-consuming when emotions are involved. Furthermore, the fluctuating emotions make it difficult to select the optimal design and content. As a result, creating advertising materials takes a long time, increasing user stress.

[0438] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0439] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate document samples, means for providing the generated document samples to the user, means for recognizing the user's emotions, means for the user to select and modify the samples, means for generating a final document that reflects the selected document samples, modifications, and recognized emotions, and means for providing the final document to the user. This enables the automatic generation of optimal advertising materials tailored to the user's emotions, improving work efficiency and reducing stress.

[0440] A "user" refers to an individual or legal entity that uses the system to request the creation of documents.

[0441] A "document creation request" refers to an operation in which a user specifies the type and content of the document they want to create.

[0442] "Methods for analyzing requests" refers to the process of analyzing the request content entered by the user and extracting the information necessary for creating the document.

[0443] "Generative artificial intelligence" refers to an algorithm or system that automatically creates necessary documents in response to instructions regarding document generation.

[0444] "Sample materials" refer to prototype advertising materials created by a generative artificial intelligence.

[0445] "Means of providing to the user" refers to the process of displaying or sending the generated sample materials to the user.

[0446] "Means of recognizing user emotions" refers to processing or devices that analyze and recognize emotions from a user's facial expressions, voice, etc.

[0447] "Method for selecting and modifying samples" refers to the operation of selecting a suitable sample from the provided materials and making necessary modifications.

[0448] "Method for generating final materials" refers to the process of creating final advertising materials based on the material samples selected and modified by the user.

[0449] The means of providing the "final document" to the user refers to the process of displaying or sending the generated final document to the user.

[0450] This invention is a system that allows users to efficiently create advertising materials, and comprises a cloud server, a terminal, and a generation AI model. The system receives user input from the terminal, and the terminal and cloud server work together to generate advertising materials, which are then ultimately provided to the user.

[0451] Program processing

[0452] 1. User input of request

[0453] The user uses a device (such as a smartphone) to input the specific details of the advertising material they want to create. For example, they might enter a request such as, "New product promotional advertisement. Includes key features, target market, and pricing information."

[0454] 2. Emotion recognition by the emotion engine

[0455] The device uses its camera and microphone to analyze the user's facial expressions and voice, recognizing their emotions (e.g., anxiety, excitement) in real time. This emotion data, along with the user's input, is sent to a cloud server.

[0456] 3. Sending a request

[0457] The user's input and recognized emotion data are converted to JSON format and sent to the cloud server.

[0458] 4. Request analysis and data sample generation

[0459] Generative artificial intelligence (AI) on a cloud server analyzes received requests and emotional data to generate sample advertising materials. The generative AI operates on a server with high-performance computing capabilities and incorporates emotionally responsive design elements such as relaxing color schemes and fonts.

[0460] 5. Provision of sample materials

[0461] The cloud server sends multiple generated document samples to the terminal and displays them to the user.

[0462] 6. User-selected and modified samples.

[0463] The user selects the most suitable document sample from those provided and makes revisions to its content. These revisions may include specific changes such as "explaining the main features in more detail" or "adding pricing information." During this process, the device re-evaluates the user's mood and, if the user is relaxed, suggests colorful designs, for example.

[0464] 7. Generation and provision of final materials

[0465] The cloud server generates the final advertising materials based on the selected sample materials and revisions. The generated final materials are sent to the device in PDF or presentation format and provided to the user.

[0466] Hardware and software to be used

[0467] hardware

[0468] Device: Smartphone (camera, microphone, display)

[0469] Cloud server: A computer with high-performance computing capabilities.

[0470] software

[0471] Emotion recognition engine: Examples include Face++ and Microsoft® Azure® Cognitive Services.

[0472] Generative AI models: OpenAI GPT-4(registered trademark) and DALL-E are used.

[0473] Data format: JSON

[0474] API communication: RESTful API

[0475] Specific example

[0476] Example prompt: Create a promotional ad for your new product. Include key features, target market, and pricing information. Since users are feeling anxious, please use a design that emphasizes safety.

[0477] This system allows users to automatically generate optimal advertising materials that match their own emotions, significantly improving the efficiency of advertising material creation and reducing stress.

[0478] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0479] Step 1:

[0480] The user enters a request for document creation.

[0481] Input: Specific content of the advertising material (e.g., "New product promotional advertisement. Includes key features, target market, and pricing information.")

[0482] Output: Input request content

[0483] Specific actions: The user launches the application on their smartphone and enters detailed requirements for the advertising materials.

[0484] Step 2:

[0485] The device recognizes the user's emotions.

[0486] Input: User's facial expression or voice data

[0487] Output: Emotional data (e.g., anxiety, excitement, etc.)

[0488] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and voice in real time, and recognizes the user's emotions using an emotion engine.

[0489] Step 3:

[0490] The device sends the request details and sentiment data to the cloud server.

[0491] Input: Request details, sentiment data

[0492] Output: Data in JSON format (including request details and sentiment data)

[0493] Specific operation: The device converts the request content and recognized emotion data into JSON format and sends it to the cloud server.

[0494] Step 4:

[0495] The server analyzes the request content and emotional data, and sends instructions to the generative artificial intelligence to create the document.

[0496] Input: Request details and sentiment data in JSON format

[0497] Output: Instructions for generating documents to a generative artificial intelligence.

[0498] Specific operation: The server parses the received JSON data and sends instructions to the generative AI for creating materials based on the user's requests and emotions.

[0499] Step 5:

[0500] A generative AI model generates document samples.

[0501] Input: Document creation instructions

[0502] Output: Multiple document samples

[0503] Specific operation: Generative AI models (e.g., OpenAI GPT-4 and DALL-E) generate sample advertising materials based on instructions received from the server. In doing so, they also consider user sentiment data and incorporate designs and color schemes that emphasize safety.

[0504] Step 6:

[0505] The server sends the generated document sample to the terminal.

[0506] Input: Multiple document samples

[0507] Output: Data including sample materials (e.g., PDF or presentation format)

[0508] Specific operation: The server receives the generated document sample, converts it to an appropriate format (PDF or presentation format), and sends it to the terminal.

[0509] Step 7:

[0510] The user selects a sample document and makes modifications.

[0511] Input: Multiple document samples

[0512] Output: Selected sample and modifications

[0513] Specific operation: The user uses their smartphone to select the most suitable document from several provided sample documents and enter specific modification details (e.g., "Expand the description of the main features," "Modify the graphics," etc.).

[0514] Step 8:

[0515] The device then recognizes the user's emotions again and sends the corrections and emotion data to the server.

[0516] Input: User's facial expression or voice data, correction details

[0517] Output: Data in JSON format (including correction details and sentiment data)

[0518] Specific operation: The device uses the emotion engine again to recognize the user's emotions, converts the corrections and emotion data into JSON format, and sends them to the server.

[0519] Step 9:

[0520] The server generates the final document, reflecting the changes and sentiment data.

[0521] Input: Correction details and sentiment data in JSON format

[0522] Output: Final document

[0523] Specific operation: The server uses generative AI to generate the final document based on the received revisions and emotion data. If the user is relaxed, it will incorporate more colorful designs and color schemes.

[0524] Step 10:

[0525] The server sends the final document to the terminal and provides it to the user.

[0526] Input: Final document

[0527] Output: Data including the final document (e.g., PDF or presentation format)

[0528] Specific operation: The server converts the generated final document into the appropriate format and sends it to the terminal. The user can then view, download, or use the final document through the terminal.

[0529] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0530] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0531] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0532] [Second Embodiment]

[0533] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0534] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0535] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0536] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0537] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0538] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0539] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0540] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0541] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0542] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0543] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0544] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0545] This invention is a system designed to assist in document creation, enabling users to efficiently produce high-quality documents. The following describes the system's program processing in natural language, along with specific examples.

[0546] Program processing

[0547] 1. User input of request

[0548] Users enter requests for new documents through the document creation system's interface. For example, they might request a document titled "New Product Introduction," specifying that it should include key features, an overview of the target market, and a price comparison with competing products.

[0549] 2. Sending a request

[0550] The terminal converts the user's request into an appropriate data format (e.g., JSON) and sends it to the server. The data includes the type of document, specific items, and other requirements.

[0551] 3. Parsing the request

[0552] The server analyzes the request received from the terminal and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[0553] 4. Instructions for preparing documents

[0554] The server sends instructions to the generative AI for document creation based on the analyzed request. These instructions include an overview of the document, key elements, and specific content.

[0555] 5. Generating sample materials

[0556] The generative AI automatically generates multiple document samples based on the received instructions. This means that even with the same request, documents with different layouts and designs will be provided.

[0557] 6. Obtaining sample materials

[0558] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[0559] 7. Provision of sample materials

[0560] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[0561] 8. User-selected and modified samples.

[0562] Users select the most suitable document from the provided sample materials and enter any necessary modifications. These modifications can range from adding information to specific slides to changing the design.

[0563] 9. Submit selected and modified data

[0564] The terminal sends the selected document sample and the corrections to the server.

[0565] 10. Generating the final document

[0566] The server incorporates the user's modifications and generates the final document. Generative AI may be used again if necessary.

[0567] 11. Provision of final documents

[0568] The server sends the final document to the terminal, which then displays it to the user. The user can then view and use the final document.

[0569] Specific example

[0570] 1. User request input:

[0571] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[0572] 2. Submitting a request:

[0573] The device sends data to the server in the following format: {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0574] 3. Parsing the request:

[0575] The server analyzes this request and instructs the generative AI to create a document introducing the new product.

[0576] 4. Instructions for preparing the document:

[0577] The server sends instructions to the generative AI to create documents based on the analysis results.

[0578] 5. Generating sample materials:

[0579] The generative AI generates multiple sample documents that include descriptions of key features, graphics of the target market, and price comparison charts.

[0580] 6. Obtaining sample materials:

[0581] The server receives the generated sample and saves it as a PDF file.

[0582] 7. Provision of sample materials:

[0583] The server sends a PDF file to the terminal, and the terminal displays it to the user.

[0584] 8. User selection and modification of samples:

[0585] The user selects one sample and enters the necessary modifications, such as "add more detail to the description of the main features" or "revise the graphic for the target market."

[0586] 9. Submit selected and corrected data:

[0587] The terminal sends the ID of the selected sample and the modifications made to the server.

[0588] 10. Generating the final document:

[0589] The server generates the final document, incorporating the received revisions.

[0590] 11. Provision of final documents:

[0591] The server sends the final document to the terminal, which then displays it to the user.

[0592] The following describes the processing flow.

[0593] Step 1:

[0594] The user launches the document creation system and enters the type of document and its specific content. For example, they might instruct the system to include key features, target market, and a price comparison with competing products in a document titled "New Product Introduction."

[0595] Step 2:

[0596] The terminal converts the user's input into an appropriate data format such as JSON and sends it to the server as data in the format {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0597] Step 3:

[0598] The server analyzes the received request. The analysis reveals that the document type is "New Product Introduction," and that the required items are three key features, the target market, and a price comparison.

[0599] Step 4:

[0600] The server sends instructions to the generative AI to create materials based on the analysis results. Specifically, it sends instructions in the form of {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[0601] Step 5:

[0602] The generative AI receives instructions from the server and generates multiple sample documents based on the request. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[0603] Step 6:

[0604] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[0605] Step 7:

[0606] The server sends the generated sample documents to the terminal. The terminal displays these samples to the user. The user can view multiple samples.

[0607] Step 8:

[0608] The user selects the most suitable document from the provided sample materials and enters the necessary modifications. For example, they might enter details about the caption on a particular slide or modify the graphics.

[0609] Step 9:

[0610] The terminal sends the user-selected document sample and its modifications to the server. The data format is {"sample_id": 1, "modifications": ["Added function description", "Graphic modification"]}.

[0611] Step 10:

[0612] The server generates the final document, incorporating the user's revisions. If necessary, it uses a generation AI to apply the revisions again.

[0613] Step 11:

[0614] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0615] These steps enable the efficient and high-quality creation of documents.

[0616] (Example 1)

[0617] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0618] In today's business environment, there is a demand for the rapid and efficient creation of high-quality documents. However, document creation typically requires a significant amount of time and effort, and especially when complex content or design is required, professional assistance is often necessary. Therefore, document creation is a major burden for many companies and individuals. Furthermore, if revisions are needed to a document, manually correcting it each time is inefficient and prone to errors.

[0619] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0620] In this invention, the server includes means for the user to input a request for document creation, means for sending the request from the terminal to the server in JSON format, and means for the server to analyze the request and send instructions for document creation to a generative artificial intelligence. This enables the user to create high-quality documents quickly and efficiently.

[0621] A "user" is the entity that uses the document creation system to create documents.

[0622] A "document creation request" is information in which the user enters the content and requirements of the document they wish to have created.

[0623] A "terminal" is an electronic device used by a user to access a document creation system, enter and submit requests, or view and modify generated documents.

[0624] A "server" is a computer system that receives requests sent from a terminal, sends instructions to a generative artificial intelligence to create documents, and provides the terminal with generated document samples and the final document.

[0625] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and representing data.

[0626] "Generative artificial intelligence" refers to artificial intelligence that automatically generates materials based on user requests.

[0627] "Document samples" refer to some or all of the multiple candidate documents created by a generative artificial intelligence.

[0628] "PDF format" is an abbreviation for Portable Document Format, and it is a file format for displaying and distributing electronic documents.

[0629] A "presentation format" is a file format used to visually present information in the form of slides.

[0630] "Modification details" refer to the changes and additional information added to the document sample selected by the user.

[0631] The "final version" refers to the completed document that incorporates the user's revisions.

[0632] The system of this invention is designed to enable users to efficiently create high-quality materials. Specifically, it includes a terminal in which the user inputs a request for material creation, a server that analyzes the received request and sends instructions for material creation to a generative artificial intelligence (AI), means for providing a sample of the generated material, and means for the user to select a sample, make modifications, and generate and provide the final material.

[0633] Hardware and software to be used

[0634] terminal

[0635] The device is used by the user to access the document creation system, enter and submit requests, and view and modify the generated documents. Specific examples of such devices include personal computers, tablets, and smartphones.

[0636] server

[0637] The server receives and analyzes user requests and sends instructions to the generative artificial intelligence to create the document. The server also converts the generated document sample into PDF or presentation format and provides it to the user's device. Technically, a web server, database server, and AI model server are required to perform these operations.

[0638] Generative artificial intelligence (AI)

[0639] Generative AI is used to generate multiple sample documents based on user requests. Specifically, it can utilize natural language processing (NLP) or machine learning models (e.g., GPT-3).

[0640] Software Tools

[0641] For sending and receiving data in JSON format, program libraries (e.g., fetch API or axios) are used, and for converting to PDF format, dedicated libraries such as PDFKit are used.

[0642] Data processing and data calculation

[0643] Request parsing

[0644] The server analyzes the request sent from the terminal. This analysis clarifies the purpose and necessary items for creating the document. For example, if the request is for a "new product introduction," the analysis results will include details such as key features, target market, and price comparison.

[0645] Generating document samples

[0646] The server sends the analysis results to the generative AI, which then generates multiple document samples based on those results. The generated documents will include different layouts and designs even for the same request.

[0647] Converting document samples

[0648] The server receives the sample documents created by the generative AI and converts them into an appropriate format (such as PDF or presentation format). This conversion process uses a dedicated software library.

[0649] Reflecting user-submitted changes

[0650] The user selects a provided sample document and enters the necessary revisions. The terminal sends the revisions to the server, which then incorporates them to generate the final document. If necessary, the generation AI may be used again. The final document is saved in PDF format or another format and provided from the server to the terminal.

[0651] Specific example

[0652] For example, a user might input a request such as, "I want to create a new product introduction document. It should include three main features, a target market, and a price comparison." The device sends this request to the server in JSON format. The server parses the request and instructs the generative AI to create the "new product introduction" document. The generative AI receives the instructions and generates several sample documents, each containing a description of the main features, a graphic of the target market, and a price comparison chart. The server saves the generated samples as PDF files and sends them to the device. The user selects one of the samples and inputs revisions, such as "add more detail to the description of the main features" or "revise the graphic of the target market." The device sends the ID of the selected sample and the revisions to the server, which then generates the final document incorporating these changes. The final document is sent to the device and displayed to the user.

[0653] Example of a prompt

[0654] Please create a product introduction document for the new product. It should include three key features, the target market, and a price comparison.

[0655] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0656] Step 1:

[0657] The user accesses the interface of the document creation system and enters a request for a new document. This includes the type of document and specific requirements (e.g., "I want to create a document introducing a new product. It should include three key features, the target market, and a price comparison"). The entered information is temporarily stored by the terminal.

[0658] Step 2:

[0659] The terminal converts the user's request into JSON format. For example, based on the input request, it converts it into a data format like {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}. This converted data becomes the input.

[0660] Step 3:

[0661] The terminal sends the request data, converted to JSON format, to the server. The transmission uses the HTTP protocol, delivering the request to the specified endpoint on the server side. This allows the server to begin parsing.

[0662] Step 4:

[0663] The server analyzes the request data it receives. It extracts information such as the type of request and detailed items, and generates the results as analysis data. For example, if the server receives a request for "new product introduction," it analyzes the items of main features, target market, and price comparison, and stores that data in list format.

[0664] Step 5:

[0665] The server sends instructions to the generating AI for document creation based on the analyzed data. The instructions include the type of document and the requested detailed items. For example, the generating AI might receive a prompt message such as, "Please create an introductory document for the new product. It should include three main features, the target market, and a price comparison."

[0666] Step 6:

[0667] The generation AI automatically generates multiple sample documents based on the instructions it receives. These sample documents include a variety of designs and layouts, such as explanations of key features, graphics of the target market, and price comparison charts. The generated sample documents become the output data.

[0668] Step 7:

[0669] The server receives sample documents generated by the generative AI and converts them into PDF or presentation formats. PDFKit or similar libraries are used for this conversion. The converted files become the new output.

[0670] Step 8:

[0671] The server sends the converted document sample to the terminal. HTTP or HTTPS protocols are used for transmission. The terminal loads the received document sample into an interface for displaying it to the user.

[0672] Step 9:

[0673] The user selects the most suitable document from the provided sample materials and enters any necessary modifications. For example, they might add information to a specific slide or change its design. The user's modifications are temporarily saved on the device.

[0674] Step 10:

[0675] The device converts the data containing the corrections into JSON format and sends it to the server along with the sample selection data. For example, it will be in the format {"sampleID": "12345", "corrections": ["Detailed description of key features", "Modified graphics for the target market"]}.

[0676] Step 11:

[0677] The server incorporates the received revisions and generates the final document. If necessary, the generation AI is used again to apply the final revisions. The generated final document becomes the output data.

[0678] Step 12:

[0679] The server sends the final document to the terminal, which then displays it to the user. The user can then review the final document and use it immediately.

[0680] (Application Example 1)

[0681] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0682] Planning production in a factory is extremely complex, requiring consideration of numerous factors. Furthermore, the efficient allocation of limited resources places a heavy burden on managers. Current systems require replanning every time changes or modifications occur, which is time-consuming and labor-intensive. Additionally, these plans are typically created manually, making them prone to human error. Therefore, a system is needed that can automatically generate production plans and manage them efficiently and accurately.

[0683] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0684] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate multiple document samples, means for providing the generated document samples to the user, means for the user to select and modify the samples, means for generating a final document reflecting the selected document samples and modifications, means for providing the final document to the user, means for inputting a request for production planning, means for analyzing the request and generating multiple production schedules, means for providing the generated production schedules to an administrator, means for the administrator to select and modify schedules, means for generating a final production plan reflecting the selected production schedules and modifications, and means for providing the final production plan to an administrator. This enables the automatic generation and efficient modification of production plans.

[0685] A "user request" is a specific request that a user enters into the system regarding document creation or production planning.

[0686] "Generative artificial intelligence" refers to artificial intelligence that has the ability to automatically generate documents and production schedules based on the data and instructions it receives.

[0687] A "document sample" is an initial draft document with multiple variations, created by a generative artificial intelligence.

[0688] A "production plan request" is when a manager enters specific production goals and requirements into the system.

[0689] "Generated production schedule" refers to an initial production schedule proposal with multiple variations, created by a generative artificial intelligence.

[0690] A "manager" is a person who is responsible for overseeing the planning and progress of a factory or production line.

[0691] "Final document" refers to the document that has been finalized to reflect user revisions.

[0692] The "final production plan" refers to the production schedule that has been finalized to reflect any revisions made by the manager.

[0693] This invention is a system for supporting the automatic generation of production plans and document creation in factories. It is designed to enable users to efficiently create high-quality documents and factory managers to formulate optimal production schedules. The following describes the program processing of this system, with specific examples.

[0694] System Configuration

[0695] The system consists of user terminals and a server. The user terminals provide an interface for inputting and modifying requests for document creation and production planning. The server works in conjunction with generative artificial intelligence (AI) to process requests and generate the final documents and production plans.

[0696] Hardware and software to be used

[0697] Generative AI: Uses generative AI models such as OpenAI's GPT-3.

[0698] Server-side framework: Use FastAPI to process requests.

[0699] Data format: JSON format is used for exchanging requests and generated data.

[0700] Data storage: Use a database (e.g., PostgreSQL) as needed.

[0701] Program Processing Description

[0702] 1. Entering a user request

[0703] Users enter requests for new materials and production plans through the terminal interface. For example, they might enter specific requirements such as, "I want to create a new product introduction document, including key features, target market, and price comparison."

[0704] 2. Sending and parsing requests

[0705] The terminal converts the input request into JSON format and sends it to the server. The server analyzes the request and sends instructions to the generative artificial intelligence for creating documents and production plans.

[0706] 3. Generation by generative artificial intelligence

[0707] Generative artificial intelligence automatically generates multiple document samples and production schedules based on transmitted instructions. It utilizes text generation models and schedule optimization algorithms as needed.

[0708] 4. Obtaining and providing samples

[0709] The server saves generated document samples and production schedules in PDF or presentation format and sends them to terminals. Users and administrators can then view these, select appropriate samples, and make modifications.

[0710] 5. Reflecting selections and modifications

[0711] Users and administrators select the best sample from the provided options and enter any necessary modifications. These modifications are sent to the server and reflected in the final documentation and production schedule.

[0712] 6. Provision of final documents and plans

[0713] The server generates the final documents and production plans reflecting the revisions and sends them to the terminal. Users and administrators can then view and use the finalized documents and plans.

[0714] Specific example

[0715] A user enters the following request into the system: "Generate a production schedule for a new smartphone. Production volume is 1000 units, deadline is December 31, 2024. Use a high-performance battery." The system analyzes this request and generates multiple production schedules. The user then selects the best schedule from the generated ones, makes any necessary modifications, and can review the final production plan.

[0716] Example of a prompt

[0717] "Please generate a production schedule for the new smartphone."

[0718] This allows users to easily enter requests, and generative artificial intelligence can generate documents and plans quickly and accurately.

[0719] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0720] Step 1:

[0721] Users enter requests for document creation and production planning.

[0722] Specific actions:

[0723] Users enter requests through the terminal's interface. For example, they might enter "new product information" or "production schedule for the new smartphone."

[0724] Input: Parent document name, requirements, and other conditions.

[0725] Output: Request data entered on the terminal.

[0726] Step 2:

[0727] Convert the request to JSON format and send it to the server.

[0728] Specific actions:

[0729] The terminal converts the user's input request into JSON format and sends that data to the server.

[0730] Input: Request data entered into the terminal by the user.

[0731] Output: Request data converted to JSON format, sent to the server.

[0732] Step 3:

[0733] The server analyzes the request and sends instructions to the generative artificial intelligence system for creating documents or generating production schedules.

[0734] Specific actions:

[0735] The server analyzes the received request data and sends instructions to the generative artificial intelligence based on the analysis results.

[0736] Input: Request data in JSON format.

[0737] Output: Analysis results, instruction data for the generative AI.

[0738] Step 4:

[0739] Generative artificial intelligence generates multiple document samples and production schedules.

[0740] Specific actions:

[0741] Generative artificial intelligence generates document samples and production schedules based on instructions received from a server. During this process, it performs necessary data processing and calculations.

[0742] Input: Instruction data from the server.

[0743] Output: Multiple document samples or production schedule.

[0744] Step 5:

[0745] The server retrieves the generated document samples and production schedules, saves them in PDF or presentation format, and then sends them to the terminal.

[0746] Specific actions:

[0747] The server receives samples and schedules generated by the generative AI, converts them to a standard file format, saves them, and then sends them to the terminal.

[0748] Input: Sample documents or production schedule data from a generative AI.

[0749] Output: Files converted to PDF or presentation format, data sent to the device.

[0750] Step 6:

[0751] Users and administrators select a sample provided and enter any necessary modifications.

[0752] Specific actions:

[0753] Users and administrators select the best option from several samples displayed on the terminal and enter the necessary modifications.

[0754] Input: Samples and modifications selected by the user or administrator.

[0755] Output: Data including correction instructions.

[0756] Step 7:

[0757] The terminal sends the selected document sample or production schedule and revision details to the server.

[0758] Specific actions:

[0759] The terminal converts the data, including the correction instructions, into JSON format and then sends it to the server.

[0760] Input: Data including correction instructions from users or administrators.

[0761] Output: Correction instruction data converted to JSON format, sent to the server.

[0762] Step 8:

[0763] The server incorporates the correction instructions and generates the final documents and production schedule.

[0764] Specific actions:

[0765] Based on the received correction instructions, the server uses generative artificial intelligence to generate the final documents and production schedules. Re-analysis is performed as needed during this process.

[0766] Input: Correction instruction data in JSON format.

[0767] Output: Final documents or production schedule.

[0768] Step 9:

[0769] The server sends the final documents or production plans to the terminal, which users and administrators then view and use.

[0770] Specific actions:

[0771] The server converts and saves final documents and production schedules in PDF or presentation format and sends them to the terminal. Users and administrators then view and use them on their terminals.

[0772] Input: Final documents or production schedule data.

[0773] Output: Final files converted to PDF or presentation format, sent to devices, and viewed and used by users and administrators.

[0774] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0775] This invention is a system that assists in document creation, designed to enable users to efficiently create high-quality documents. Furthermore, by incorporating an emotion engine that recognizes user emotions, it achieves optimal document creation tailored to the user's emotions. The following describes the system's program processing in natural language, along with specific examples.

[0776] Program processing

[0777] 1. User input of request

[0778] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and should include key features, target market, and a price comparison with competing products.

[0779] 2. Emotion recognition by the emotion engine

[0780] The device uses an emotion engine to recognize the user's emotions when they input data. For example, it might analyze facial expressions using a camera or analyze the tone of voice during voice input.

[0781] 3. Sending a request

[0782] The device converts the user's request and recognized emotions into an appropriate data format, such as JSON, and sends it to the server.

[0783] 4. Parsing the request

[0784] The server analyzes the received request and sentiment data, and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[0785] 5. Instructions for creating documents that take emotional data into consideration.

[0786] The server sends instructions to the generative AI for creating materials based on recognized emotion data. For example, if the user is feeling stressed, it recommends designs and tones that promote relaxation.

[0787] 6. Generating sample materials

[0788] The generative AI automatically generates multiple sample documents based on the instructions it receives. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[0789] 7. Obtaining sample materials

[0790] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[0791] 8. Provision of sample materials

[0792] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[0793] 9. User-selected and modified samples.

[0794] The user selects the most suitable sample material from the provided options and enters any necessary modifications. For example, they might specify changes such as adding more detail to the caption of a particular slide or modifying a graphic.

[0795] 10. Emotion monitoring using an emotion engine

[0796] The device monitors the user's emotional changes using an emotion engine while the user selects and modifies materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[0797] 11. Submit selected and modified data

[0798] The terminal sends the user-selected document sample, its modifications, and emotion data to the server. The data format is {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[0799] 12. Generating the final document

[0800] The server generates the final document, reflecting the user's revisions and final sentiment data. If necessary, it uses a generative AI to apply revisions again.

[0801] 13. Provision of final documents

[0802] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0803] Specific example

[0804] 1. User request input:

[0805] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[0806] 2. Emotion recognition by an emotion engine:

[0807] The device analyzes the user's facial expressions using its camera and recognizes that the user is feeling somewhat anxious.

[0808] 3. Submitting a request:

[0809] The device sends the request content and emotion data to the server as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0810] 4. Parsing the request:

[0811] The server analyzes the request and sends instructions to the generative AI to create new product introduction materials, taking into account the user's emotional data at this time.

[0812] 5. Instructions that take emotional data into consideration:

[0813] The server sends instructions to the generative AI for creating documents that recommend color schemes and fonts that will make the user feel comfortable.

[0814] 6. Generating sample materials:

[0815] The generative AI focuses on multiple document samples with a calm tone.

[0816] 7. Obtaining sample materials:

[0817] The server receives the generated sample in PDF format.

[0818] 8. Provision of sample materials:

[0819] The server sends sample documents to the terminal and displays them to the user.

[0820] 9. User selection and modification:

[0821] The user is instructed to select one sample and provide a detailed description of its main features.

[0822] 10. Emotional monitoring:

[0823] The device recognizes that the user is relaxed and makes correction suggestions.

[0824] 11. Submit selected and corrected data:

[0825] The terminal sends the selected sample and modifications to the server as {"sample_id": 1, "modifications": ["Added a function description"], "emotion": "relaxed"}.

[0826] 12. Generating the final document:

[0827] The server generates the final document, reflecting the changes and sentiment data.

[0828] 13. Provision of final documents:

[0829] The server sends the final document to the terminal and displays it to the user.

[0830] The following describes the processing flow.

[0831] Step 1:

[0832] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and include key features, target market, and a price comparison with competing products.

[0833] Step 2:

[0834] The device uses an emotion engine to recognize the user's emotions along with their input. For example, it uses the camera to analyze facial expressions to determine whether the user is feeling safe or anxious.

[0835] Step 3:

[0836] The device converts the user's request and recognized emotions into an appropriate data format such as JSON, and sends data to the server such as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0837] Step 4:

[0838] The server analyzes the received request and sentiment data. Based on the analysis, it understands that the document type is "new product introduction," and that the necessary items are three key features, the target market, and a price comparison. Furthermore, it considers that the user's sentiment is anxiety.

[0839] Step 5:

[0840] The server sends detailed instructions for document creation to the generative AI based on the analyzed content and emotional data. For example, it sends instructions in the format {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"], "desired_tone": "calm"}.

[0841] Step 6:

[0842] The generative AI receives instructions from the server and generates multiple sample documents with designs and tones that take user emotions into consideration. This results in documents that users can use with confidence.

[0843] Step 7:

[0844] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[0845] Step 8:

[0846] The server sends generated sample materials to the terminal. The terminal displays these samples to the user. The user can review multiple samples and select a design that matches their emotional state.

[0847] Step 9:

[0848] The user selects the most suitable document sample from those provided and enters the necessary modifications. For example, they might enter modifications such as "explain the main features in more detail" or "revise the target market graphic."

[0849] Step 10:

[0850] The device uses an emotion engine to monitor the user's emotional changes while they select and modify materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[0851] Step 11:

[0852] The device sends the user's selected sample document, its modifications, and emotion data to the server. For example, it sends data in the format {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[0853] Step 12:

[0854] The server generates the final document based on user feedback and final sentiment data. If necessary, it applies further revisions using a generative AI.

[0855] Step 13:

[0856] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0857] (Example 2)

[0858] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0859] In modern society, there is an increasing demand for rapid and high-quality document creation, but many users spend too much time on it. Furthermore, since users' emotions during document creation often influence the results, it is desirable to create documents that are emotionally resonant.

[0860] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0861] In this invention, the server includes means for recognizing the user's emotions and reflecting them in the document creation instructions, means for monitoring changes in the user's emotions, and means for sending and receiving document creation requests in JSON format. This enables the creation of optimal documents based on the user's emotions. Furthermore, because the document creation process is streamlined, the user can quickly create high-quality documents.

[0862] A "user" is a person or group that uses the document creation system, enters a document creation request, and reviews and modifies the document generated based on that request.

[0863] A "document creation request" is information that indicates the specific requirements regarding the type and content of the document the user wants to create, and it is entered into the system.

[0864] "Means for analyzing requests" refers to a device or program that receives a document creation request entered by a user, understands and interprets it, and sends the necessary instructions to a generative artificial intelligence.

[0865] "Generative artificial intelligence" refers to an algorithm or software system that automatically generates materials based on specified instructions.

[0866] "Document samples" refer to multiple candidate documents generated by a generative artificial intelligence system, serving as initial documents for the user to select and modify.

[0867] "Means of providing to the user" refers to a device or program for transmitting generated sample materials or final materials to the user's terminal and displaying them.

[0868] "Final document" refers to the document that has been finalized after reflecting the user's selections and revisions.

[0869] "Means of recognizing emotions" refers to a device or program that analyzes the user's facial expressions, voice tone, etc., to identify the user's emotional state.

[0870] "Means for monitoring emotional changes" refers to a device or program that monitors changes in a user's emotional state in real time during the document creation process and records and analyzes that data.

[0871] JSON format is a type of data format that describes data attributes and values ​​using key-value pairs, making it easy for humans to read and easy for machines to analyze.

[0872] "PDF format" is an abbreviation for Portable Document Format, and it is a digital file format for saving and viewing documents and images with high accuracy.

[0873] A "presentation format" is a type of material used for presentations and explanations, where information is visually organized in a slide format.

[0874] This invention is a document creation support system designed to enable users to efficiently create high-quality materials. The system automatically generates product introductions, presentation materials, and other documents based on user requests. Furthermore, it utilizes an emotion engine to recognize the user's emotions and create optimal materials accordingly. The following describes the program processing of this system.

[0875] Program processing

[0876] 1. User input of request

[0877] Users input the type and specific content of the required document through the document creation system's interface. If a user wants to create a document introducing a new product, they would specify items such as key features, target market, and price comparisons with competing products.

[0878] 2. Emotion recognition by the emotion engine

[0879] The device activates an emotion engine when the user inputs text, analyzing the user's facial expressions using the camera (using computer vision technology), or analyzing their voice tone when they input voice text. This allows the device to recognize the user's emotional state, such as whether they are relaxed, tense, or anxious.

[0880] 3. Sending a request

[0881] The device converts the user's request content and recognized emotion data into JSON format and sends it to the server. For example, if the request is for a new product introduction and the user is feeling somewhat anxious, the data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[0882] 4. Request parsing and instruction generation

[0883] The server analyzes the received request data and sentiment data. This clarifies the purpose and necessary items for document creation, and generates prompts for the AI ​​model. For example, if the user is feeling anxious, instructions such as "Create a new product introduction document using a design and tone that promotes relaxation" will be generated.

[0884] 5. Generating sample materials using a generative AI model

[0885] The generative AI automatically generates multiple sample documents based on instructions sent from the server. For example, documents containing key features, graphics of the target market, and price comparison charts of competing products can be generated.

[0886] 6. Sending and converting sample documents

[0887] The server receives the generated sample materials and converts them into PDF or presentation format. The samples are saved as PDF document or slide files.

[0888] 7. Provision of sample materials

[0889] The server sends the converted document sample to the terminal, which then displays it to the user. The user can view multiple document samples on the screen and select the most suitable one.

[0890] 8. User selection and modification of samples

[0891] The user selects the most suitable sample material from the provided options and enters the necessary modifications. For example, they might enter specific details such as enriching the caption on a particular slide or modifying a graphic.

[0892] 9. Emotion monitoring using an emotion engine

[0893] The device uses an emotion engine to monitor the user's emotional changes in real time while they select and modify sample materials. This allows for the provision of appropriate samples and modification suggestions based on the user's emotional state.

[0894] 10. Generating the final document

[0895] The server generates the final document, reflecting the user's selections and modifications, as well as sentiment data. If necessary, it re-applies modifications using the regenerating AI model.

[0896] 11. Provision of final documents

[0897] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[0898] Specific example

[0899] Here's a concrete example: A user enters the following request into the document creation system: "I want to create a document introducing a new product. It should include three main features, the target market, and a price comparison." The device uses its camera to analyze the user's facial expressions and recognizes that the user is feeling somewhat anxious. The device then sends the request and emotion data to the server.

[0900] The server analyzes the request and sends an instruction to the generative AI model to "create a new product introduction document using a design and tone that conveys a sense of relaxation." The generative AI model creates several sample documents and sends them to the server. The server converts the samples to PDF format and sends them to the user's device. The user reviews the sample documents, selects the most suitable one, and enters suggestions for revisions, such as adding more detail to the description of the main features.

[0901] The device continues to monitor the user's emotions while they are making revisions, recognizing that the user is relaxed. The device sends the revisions to the server, which generates the final document and sends it back to the device. The user can then review and use the final document.

[0902] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0903] Step 1:

[0904] Users input the type and specific content of the required documents through the document creation system's interface.

[0905] In terms of specific operations, if a user requests information about a new product, they will specify detailed items such as key features, target market, and price comparison with competing products. The input data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}.

[0906] Step 2:

[0907] The device activates an emotion engine when the user makes a input, and recognizes the user's emotions.

[0908] Specifically, the system uses computer vision technology to analyze the user's facial expressions with a camera, or analyzes their voice tone through a microphone during voice input. For example, if the user is feeling somewhat anxious, the recognition result will be "anxious." The input data is the user's facial expressions and voice tone, and the output data is their emotional state (e.g., "anxious").

[0909] Step 3:

[0910] The device converts the user's request and recognized sentiment data into JSON format and sends it to the server.

[0911] Specifically, the system receives user request details and emotion data (e.g., {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}) as input data and sends this to the server.

[0912] Step 4:

[0913] The server analyzes the received request data and sentiment data.

[0914] Specifically, the process involves parsing JSON data to identify the purpose of the document creation and the necessary items. Through this analysis, a prompt message is generated. The input data is the received JSON request data, and the output data is the prompt message (e.g., "Create a new product introduction document using a design and tone that conveys a sense of relaxation").

[0915] Step 5:

[0916] The server sends instructions to the AI ​​model for creating the document based on the generated prompt message.

[0917] In terms of specific operations, a prompt message is sent to a generating AI model, requesting it to generate sample materials based on the instructions. The input data is the prompt message, and the output data is the instructions sent to the generating AI model.

[0918] Step 6:

[0919] The generation AI model automatically generates multiple document samples based on instructions sent from the server.

[0920] Specifically, the system creates documents that include key features, graphics for the target market, and price comparison charts of competing products. The input data consists of instructions (prompt messages) from the server, and the output data is a sample of the generated document.

[0921] Step 7:

[0922] The server converts the generated sample documents into the appropriate format.

[0923] Specifically, the process involves converting sample documents into PDF or presentation formats. For example, it converts an HTML document sent from a generative AI model into a PDF. The input data is the generated sample document, and the output data is the sample document converted into the appropriate format.

[0924] Step 8:

[0925] The server sends the converted document sample to the terminal, which then displays it to the user.

[0926] Specifically, the system displays samples on the user interface, allowing the user to select from multiple sample documents. The input data consists of converted sample documents, while the output data is the sample document displayed to the user.

[0927] Step 9:

[0928] The user selects the most suitable document from the provided sample materials and enters any necessary modifications.

[0929] In terms of specific actions, users input details of modifications, such as enriching the description on a particular slide or correcting a graphic, through the interface. The input data consists of the selected document sample and the modifications, while the output data is the modified document sample.

[0930] Step 10:

[0931] The device uses an emotion engine to monitor changes in the user's emotions while they select and modify materials.

[0932] Specifically, the system uses cameras and microphones to analyze the user's emotions in real time and monitors their emotional state. Input data includes the user's facial expressions and voice tone, while output data is their real-time emotional state.

[0933] Step 11:

[0934] The device sends the user's selected document sample, its modifications, and sentiment data to the server.

[0935] Specifically, the terminal packages the revised document sample and sentiment data and sends them to the server. The input data consists of the revised document sample and sentiment data, while the output data is the data sent to the server.

[0936] Step 12:

[0937] The server generates the final document, incorporating user modifications and final sentiment data.

[0938] Specifically, the process involves modifying and creating the final document using an AI model. The input data consists of the modifications and sentiment data, while the output data is the final document.

[0939] Step 13:

[0940] The server sends the final generated document to the terminal, which then displays it to the user.

[0941] Specifically, the final document is displayed on the user interface for review. The input data is the final document, and the output data is the final document displayed to the user.

[0942] (Application Example 2)

[0943] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0944] Currently, when users create advertising materials, they need to consider the design and content individually, which is especially time-consuming when emotions are involved. Furthermore, the fluctuating emotions make it difficult to select the optimal design and content. As a result, creating advertising materials takes a long time, increasing user stress.

[0945] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0946] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate document samples, means for providing the generated document samples to the user, means for recognizing the user's emotions, means for the user to select and modify the samples, means for generating a final document that reflects the selected document samples, modifications, and recognized emotions, and means for providing the final document to the user. This enables the automatic generation of optimal advertising materials tailored to the user's emotions, improving work efficiency and reducing stress.

[0947] A "user" refers to an individual or legal entity that uses the system to request the creation of documents.

[0948] A "document creation request" refers to an operation in which a user specifies the type and content of the document they want to create.

[0949] "Methods for analyzing requests" refers to the process of analyzing the request content entered by the user and extracting the information necessary for creating the document.

[0950] "Generative artificial intelligence" refers to an algorithm or system that automatically creates necessary documents in response to instructions regarding document generation.

[0951] "Sample materials" refer to prototype advertising materials created by a generative artificial intelligence.

[0952] "Means of providing to the user" refers to the process of displaying or sending the generated sample materials to the user.

[0953] "Means of recognizing user emotions" refers to processing or devices that analyze and recognize emotions from a user's facial expressions, voice, etc.

[0954] "Method for selecting and modifying samples" refers to the operation of selecting a suitable sample from the provided materials and making necessary modifications.

[0955] "Method for generating final materials" refers to the process of creating final advertising materials based on the material samples selected and modified by the user.

[0956] The means of providing the "final document" to the user refers to the process of displaying or sending the generated final document to the user.

[0957] This invention is a system that allows users to efficiently create advertising materials, and comprises a cloud server, a terminal, and a generation AI model. The system receives user input from the terminal, and the terminal and cloud server work together to generate advertising materials, which are then ultimately provided to the user.

[0958] Program processing

[0959] 1. User input of request

[0960] The user uses a device (such as a smartphone) to input the specific details of the advertising material they want to create. For example, they might enter a request such as, "New product promotional advertisement. Includes key features, target market, and pricing information."

[0961] 2. Emotion recognition by the emotion engine

[0962] The device uses its camera and microphone to analyze the user's facial expressions and voice, recognizing their emotions (e.g., anxiety, excitement) in real time. This emotion data, along with the user's input, is sent to a cloud server.

[0963] 3. Sending a request

[0964] The user's input and recognized emotion data are converted to JSON format and sent to the cloud server.

[0965] 4. Request analysis and data sample generation

[0966] Generative artificial intelligence (AI) on a cloud server analyzes received requests and emotional data to generate sample advertising materials. The generative AI operates on a server with high-performance computing capabilities and incorporates emotionally responsive design elements such as relaxing color schemes and fonts.

[0967] 5. Provision of sample materials

[0968] The cloud server sends multiple generated document samples to the terminal and displays them to the user.

[0969] 6. User-selected and modified samples.

[0970] The user selects the most suitable document sample from those provided and makes revisions to its content. These revisions may include specific changes such as "explaining the main features in more detail" or "adding pricing information." During this process, the device re-evaluates the user's mood and, if the user is relaxed, suggests colorful designs, for example.

[0971] 7. Generation and provision of final materials

[0972] The cloud server generates the final advertising materials based on the selected sample materials and revisions. The generated final materials are sent to the device in PDF or presentation format and provided to the user.

[0973] Hardware and software to be used

[0974] hardware

[0975] Device: Smartphone (camera, microphone, display)

[0976] Cloud server: A computer with high-performance computing capabilities.

[0977] software

[0978] Emotion recognition engine: Face++ and Microsoft Azure Cognitive Services are used as examples.

[0979] Generative AI models: OpenAI GPT-4 and DALL-E are used.

[0980] Data format: JSON

[0981] API communication: RESTful API

[0982] Specific example

[0983] Example prompt: Create a promotional ad for your new product. Include key features, target market, and pricing information. Since users are feeling anxious, please use a design that emphasizes safety.

[0984] This system allows users to automatically generate optimal advertising materials that match their own emotions, significantly improving the efficiency of advertising material creation and reducing stress.

[0985] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0986] Step 1:

[0987] The user enters a request for document creation.

[0988] Input: Specific content of the advertising material (e.g., "New product promotional advertisement. Includes key features, target market, and pricing information.")

[0989] Output: Input request content

[0990] Specific actions: The user launches the application on their smartphone and enters detailed requirements for the advertising materials.

[0991] Step 2:

[0992] The device recognizes the user's emotions.

[0993] Input: User's facial expression or voice data

[0994] Output: Emotional data (e.g., anxiety, excitement, etc.)

[0995] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and voice in real time, and recognizes the user's emotions using an emotion engine.

[0996] Step 3:

[0997] The device sends the request details and sentiment data to the cloud server.

[0998] Input: Request details, sentiment data

[0999] Output: Data in JSON format (including request details and sentiment data)

[1000] Specific operation: The device converts the request content and recognized emotion data into JSON format and sends it to the cloud server.

[1001] Step 4:

[1002] The server analyzes the request content and emotional data, and sends instructions to the generative artificial intelligence to create the document.

[1003] Input: Request details and sentiment data in JSON format

[1004] Output: Instructions for generating documents to a generative artificial intelligence.

[1005] Specific operation: The server parses the received JSON data and sends instructions to the generative AI for creating materials based on the user's requests and emotions.

[1006] Step 5:

[1007] A generative AI model generates document samples.

[1008] Input: Document creation instructions

[1009] Output: Multiple document samples

[1010] Specific operation: Generative AI models (e.g., OpenAI GPT-4 and DALL-E) generate sample advertising materials based on instructions received from the server. In doing so, they also consider user sentiment data and incorporate designs and color schemes that emphasize safety.

[1011] Step 6:

[1012] The server sends the generated document sample to the terminal.

[1013] Input: Multiple document samples

[1014] Output: Data including sample materials (e.g., PDF or presentation format)

[1015] Specific operation: The server receives the generated document sample, converts it to an appropriate format (PDF or presentation format), and sends it to the terminal.

[1016] Step 7:

[1017] The user selects a sample document and makes modifications.

[1018] Input: Multiple document samples

[1019] Output: Selected sample and modifications

[1020] Specific operation: The user uses their smartphone to select the most suitable document from several provided sample documents and enter specific modification details (e.g., "Expand the description of the main features," "Modify the graphics," etc.).

[1021] Step 8:

[1022] The device then recognizes the user's emotions again and sends the corrections and emotion data to the server.

[1023] Input: User's facial expression or voice data, correction details

[1024] Output: Data in JSON format (including correction details and sentiment data)

[1025] Specific operation: The device uses the emotion engine again to recognize the user's emotions, converts the corrections and emotion data into JSON format, and sends them to the server.

[1026] Step 9:

[1027] The server generates the final document, reflecting the changes and sentiment data.

[1028] Input: Correction details and sentiment data in JSON format

[1029] Output: Final document

[1030] Specific operation: The server uses generative AI to generate the final document based on the received revisions and emotion data. If the user is relaxed, it will incorporate more colorful designs and color schemes.

[1031] Step 10:

[1032] The server sends the final document to the terminal and provides it to the user.

[1033] Input: Final document

[1034] Output: Data including the final document (e.g., PDF or presentation format)

[1035] Specific operation: The server converts the generated final document into the appropriate format and sends it to the terminal. The user can then view, download, or use the final document through the terminal.

[1036] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1037] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1038] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1039] [Third Embodiment]

[1040] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1041] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1042] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1043] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1044] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1045] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1046] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1047] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1048] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1049] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1050] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1051] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1052] This invention is a system designed to assist in document creation, enabling users to efficiently produce high-quality documents. The following describes the system's program processing in natural language, along with specific examples.

[1053] Program processing

[1054] 1. User input of request

[1055] Users enter requests for new documents through the document creation system's interface. For example, they might request a document titled "New Product Introduction," specifying that it should include key features, an overview of the target market, and a price comparison with competing products.

[1056] 2. Sending a request

[1057] The terminal converts the user's request into an appropriate data format (e.g., JSON) and sends it to the server. The data includes the type of document, specific items, and other requirements.

[1058] 3. Parsing the request

[1059] The server analyzes the request received from the terminal and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[1060] 4. Instructions for preparing documents

[1061] The server sends instructions to the generative AI for document creation based on the analyzed request. These instructions include an overview of the document, key elements, and specific content.

[1062] 5. Generating sample materials

[1063] The generative AI automatically generates multiple document samples based on the received instructions. This means that even with the same request, documents with different layouts and designs will be provided.

[1064] 6. Obtaining sample materials

[1065] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[1066] 7. Provision of sample materials

[1067] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[1068] 8. User-selected and modified samples.

[1069] Users select the most suitable document from the provided sample materials and enter any necessary modifications. These modifications can range from adding information to specific slides to changing the design.

[1070] 9. Submit selected and modified data

[1071] The terminal sends the selected document sample and the corrections to the server.

[1072] 10. Generating the final document

[1073] The server incorporates the user's modifications and generates the final document. Generative AI may be used again if necessary.

[1074] 11. Provision of final documents

[1075] The server sends the final document to the terminal, which then displays it to the user. The user can then view and use the final document.

[1076] Specific example

[1077] 1. User request input:

[1078] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[1079] 2. Submitting a request:

[1080] The device sends data to the server in the following format: {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1081] 3. Parsing the request:

[1082] The server analyzes this request and instructs the generative AI to create a document introducing the new product.

[1083] 4. Instructions for preparing the document:

[1084] The server sends instructions to the generative AI to create documents based on the analysis results.

[1085] 5. Generating sample materials:

[1086] The generative AI generates multiple sample documents that include descriptions of key features, graphics of the target market, and price comparison charts.

[1087] 6. Obtaining sample materials:

[1088] The server receives the generated sample and saves it as a PDF file.

[1089] 7. Provision of sample materials:

[1090] The server sends a PDF file to the terminal, and the terminal displays it to the user.

[1091] 8. User selection and modification of samples:

[1092] The user selects one sample and enters the necessary modifications, such as "add more detail to the description of the main features" or "revise the graphic for the target market."

[1093] 9. Submit selected and corrected data:

[1094] The terminal sends the ID of the selected sample and the modifications made to the server.

[1095] 10. Generating the final document:

[1096] The server generates the final document, incorporating the received revisions.

[1097] 11. Provision of final documents:

[1098] The server sends the final document to the terminal, which then displays it to the user.

[1099] The following describes the processing flow.

[1100] Step 1:

[1101] The user launches the document creation system and enters the type of document and its specific content. For example, they might instruct the system to include key features, target market, and a price comparison with competing products in a document titled "New Product Introduction."

[1102] Step 2:

[1103] The terminal converts the user's input into an appropriate data format such as JSON and sends it to the server as data in the format {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1104] Step 3:

[1105] The server analyzes the received request. The analysis reveals that the document type is "New Product Introduction," and that the required items are three key features, the target market, and a price comparison.

[1106] Step 4:

[1107] The server sends instructions to the generative AI to create materials based on the analysis results. Specifically, it sends instructions in the form of {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1108] Step 5:

[1109] The generative AI receives instructions from the server and generates multiple sample documents based on the request. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[1110] Step 6:

[1111] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[1112] Step 7:

[1113] The server sends the generated sample documents to the terminal. The terminal displays these samples to the user. The user can view multiple samples.

[1114] Step 8:

[1115] The user selects the most suitable document from the provided sample materials and enters the necessary modifications. For example, they might enter details about the caption on a particular slide or modify the graphics.

[1116] Step 9:

[1117] The terminal sends the user-selected document sample and its modifications to the server. The data format is {"sample_id": 1, "modifications": ["Added function description", "Graphic modification"]}.

[1118] Step 10:

[1119] The server generates the final document, incorporating the user's revisions. If necessary, it uses a generation AI to apply the revisions again.

[1120] Step 11:

[1121] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1122] These steps enable the efficient and high-quality creation of documents.

[1123] (Example 1)

[1124] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1125] In today's business environment, there is a demand for the rapid and efficient creation of high-quality documents. However, document creation typically requires a significant amount of time and effort, and especially when complex content or design is required, professional assistance is often necessary. Therefore, document creation is a major burden for many companies and individuals. Furthermore, if revisions are needed to a document, manually correcting it each time is inefficient and prone to errors.

[1126] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1127] In this invention, the server includes means for the user to input a request for document creation, means for sending the request from the terminal to the server in JSON format, and means for the server to analyze the request and send instructions for document creation to a generative artificial intelligence. This enables the user to create high-quality documents quickly and efficiently.

[1128] A "user" is the entity that uses the document creation system to create documents.

[1129] A "document creation request" is information in which the user enters the content and requirements of the document they wish to have created.

[1130] A "terminal" is an electronic device used by a user to access a document creation system, enter and submit requests, or view and modify generated documents.

[1131] A "server" is a computer system that receives requests sent from a terminal, sends instructions to a generative artificial intelligence to create documents, and provides the terminal with generated document samples and the final document.

[1132] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and representing data.

[1133] "Generative artificial intelligence" refers to artificial intelligence that automatically generates materials based on user requests.

[1134] "Document samples" refer to some or all of the multiple candidate documents created by a generative artificial intelligence.

[1135] "PDF format" is an abbreviation for Portable Document Format, and it is a file format for displaying and distributing electronic documents.

[1136] A "presentation format" is a file format used to visually present information in the form of slides.

[1137] "Modification details" refer to the changes and additional information added to the document sample selected by the user.

[1138] The "final version" refers to the completed document that incorporates the user's revisions.

[1139] The system of this invention is designed to enable users to efficiently create high-quality materials. Specifically, it includes a terminal in which the user inputs a request for material creation, a server that analyzes the received request and sends instructions for material creation to a generative artificial intelligence (AI), means for providing a sample of the generated material, and means for the user to select a sample, make modifications, and generate and provide the final material.

[1140] Hardware and software to be used

[1141] terminal

[1142] The device is used by the user to access the document creation system, enter and submit requests, and view and modify the generated documents. Specific examples of such devices include personal computers, tablets, and smartphones.

[1143] server

[1144] The server receives and analyzes user requests and sends instructions to the generative artificial intelligence to create the document. The server also converts the generated document sample into PDF or presentation format and provides it to the user's device. Technically, a web server, database server, and AI model server are required to perform these operations.

[1145] Generative artificial intelligence (AI)

[1146] Generative AI is used to generate multiple sample documents based on user requests. Specifically, it can utilize natural language processing (NLP) or machine learning models (e.g., GPT-3).

[1147] Software Tools

[1148] For sending and receiving data in JSON format, program libraries (e.g., fetch API or axios) are used, and for converting to PDF format, dedicated libraries such as PDFKit are used.

[1149] Data processing and data calculation

[1150] Request parsing

[1151] The server analyzes the request sent from the terminal. This analysis clarifies the purpose and necessary items for creating the document. For example, if the request is for a "new product introduction," the analysis results will include details such as key features, target market, and price comparison.

[1152] Generating document samples

[1153] The server sends the analysis results to the generative AI, which then generates multiple document samples based on those results. The generated documents will include different layouts and designs even for the same request.

[1154] Converting document samples

[1155] The server receives the sample documents created by the generative AI and converts them into an appropriate format (such as PDF or presentation format). This conversion process uses a dedicated software library.

[1156] Reflecting user-submitted changes

[1157] The user selects a provided sample document and enters the necessary revisions. The terminal sends the revisions to the server, which then incorporates them to generate the final document. If necessary, the generation AI may be used again. The final document is saved in PDF format or another format and provided from the server to the terminal.

[1158] Specific example

[1159] For example, a user might input a request such as, "I want to create a new product introduction document. It should include three main features, a target market, and a price comparison." The device sends this request to the server in JSON format. The server parses the request and instructs the generative AI to create the "new product introduction" document. The generative AI receives the instructions and generates several sample documents, each containing a description of the main features, a graphic of the target market, and a price comparison chart. The server saves the generated samples as PDF files and sends them to the device. The user selects one of the samples and inputs revisions, such as "add more detail to the description of the main features" or "revise the graphic of the target market." The device sends the ID of the selected sample and the revisions to the server, which then generates the final document incorporating these changes. The final document is sent to the device and displayed to the user.

[1160] Example of a prompt

[1161] Please create a product introduction document for the new product. It should include three key features, the target market, and a price comparison.

[1162] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1163] Step 1:

[1164] The user accesses the interface of the document creation system and enters a request for a new document. This includes the type of document and specific requirements (e.g., "I want to create a document introducing a new product. It should include three key features, the target market, and a price comparison"). The entered information is temporarily stored by the terminal.

[1165] Step 2:

[1166] The terminal converts the user's request into JSON format. For example, based on the input request, it converts it into a data format like {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}. This converted data becomes the input.

[1167] Step 3:

[1168] The terminal sends the request data, converted to JSON format, to the server. The transmission uses the HTTP protocol, delivering the request to the specified endpoint on the server side. This allows the server to begin parsing.

[1169] Step 4:

[1170] The server analyzes the request data it receives. It extracts information such as the type of request and detailed items, and generates the results as analysis data. For example, if the server receives a request for "new product introduction," it analyzes the items of main features, target market, and price comparison, and stores that data in list format.

[1171] Step 5:

[1172] The server sends instructions to the generating AI for document creation based on the analyzed data. The instructions include the type of document and the requested detailed items. For example, the generating AI might receive a prompt message such as, "Please create an introductory document for the new product. It should include three main features, the target market, and a price comparison."

[1173] Step 6:

[1174] The generation AI automatically generates multiple sample documents based on the instructions it receives. These sample documents include a variety of designs and layouts, such as explanations of key features, graphics of the target market, and price comparison charts. The generated sample documents become the output data.

[1175] Step 7:

[1176] The server receives sample documents generated by the generative AI and converts them into PDF or presentation formats. PDFKit or similar libraries are used for this conversion. The converted files become the new output.

[1177] Step 8:

[1178] The server sends the converted document sample to the terminal. HTTP or HTTPS protocols are used for transmission. The terminal loads the received document sample into an interface for displaying it to the user.

[1179] Step 9:

[1180] The user selects the most suitable document from the provided sample materials and enters any necessary modifications. For example, they might add information to a specific slide or change its design. The user's modifications are temporarily saved on the device.

[1181] Step 10:

[1182] The device converts the data containing the corrections into JSON format and sends it to the server along with the sample selection data. For example, it will be in the format {"sampleID": "12345", "corrections": ["Detailed description of key features", "Modified graphics for the target market"]}.

[1183] Step 11:

[1184] The server incorporates the received revisions and generates the final document. If necessary, the generation AI is used again to apply the final revisions. The generated final document becomes the output data.

[1185] Step 12:

[1186] The server sends the final document to the terminal, which then displays it to the user. The user can then review the final document and use it immediately.

[1187] (Application Example 1)

[1188] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1189] Planning production in a factory is extremely complex, requiring consideration of numerous factors. Furthermore, the efficient allocation of limited resources places a heavy burden on managers. Current systems require replanning every time changes or modifications occur, which is time-consuming and labor-intensive. Additionally, these plans are typically created manually, making them prone to human error. Therefore, a system is needed that can automatically generate production plans and manage them efficiently and accurately.

[1190] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1191] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate multiple document samples, means for providing the generated document samples to the user, means for the user to select and modify the samples, means for generating a final document reflecting the selected document samples and modifications, means for providing the final document to the user, means for inputting a request for production planning, means for analyzing the request and generating multiple production schedules, means for providing the generated production schedules to an administrator, means for the administrator to select and modify schedules, means for generating a final production plan reflecting the selected production schedules and modifications, and means for providing the final production plan to an administrator. This enables the automatic generation and efficient modification of production plans.

[1192] A "user request" is a specific request that a user enters into the system regarding document creation or production planning.

[1193] "Generative artificial intelligence" refers to artificial intelligence that has the ability to automatically generate documents and production schedules based on the data and instructions it receives.

[1194] A "document sample" is an initial draft document with multiple variations, created by a generative artificial intelligence.

[1195] A "production plan request" is when a manager enters specific production goals and requirements into the system.

[1196] "Generated production schedule" refers to an initial production schedule proposal with multiple variations, created by a generative artificial intelligence.

[1197] A "manager" is a person who is responsible for overseeing the planning and progress of a factory or production line.

[1198] "Final document" refers to the document that has been finalized to reflect user revisions.

[1199] The "final production plan" refers to the production schedule that has been finalized to reflect any revisions made by the manager.

[1200] This invention is a system for supporting the automatic generation of production plans and document creation in factories. It is designed to enable users to efficiently create high-quality documents and factory managers to formulate optimal production schedules. The following describes the program processing of this system, with specific examples.

[1201] System Configuration

[1202] The system consists of user terminals and a server. The user terminals provide an interface for inputting and modifying requests for document creation and production planning. The server works in conjunction with generative artificial intelligence (AI) to process requests and generate the final documents and production plans.

[1203] Hardware and software to be used

[1204] Generative AI: Uses generative AI models such as OpenAI's GPT-3.

[1205] Server-side framework: Use FastAPI to process requests.

[1206] Data format: JSON format is used for exchanging requests and generated data.

[1207] Data storage: Use a database (e.g., PostgreSQL) as needed.

[1208] Program Processing Description

[1209] 1. Entering a user request

[1210] Users enter requests for new materials and production plans through the terminal interface. For example, they might enter specific requirements such as, "I want to create a new product introduction document, including key features, target market, and price comparison."

[1211] 2. Sending and parsing requests

[1212] The terminal converts the input request into JSON format and sends it to the server. The server analyzes the request and sends instructions to the generative artificial intelligence for creating documents and production plans.

[1213] 3. Generation by generative artificial intelligence

[1214] Generative artificial intelligence automatically generates multiple document samples and production schedules based on transmitted instructions. It utilizes text generation models and schedule optimization algorithms as needed.

[1215] 4. Obtaining and providing samples

[1216] The server saves generated document samples and production schedules in PDF or presentation format and sends them to terminals. Users and administrators can then view these, select appropriate samples, and make modifications.

[1217] 5. Reflecting selections and modifications

[1218] Users and administrators select the best sample from the provided options and enter any necessary modifications. These modifications are sent to the server and reflected in the final documentation and production schedule.

[1219] 6. Provision of final documents and plans

[1220] The server generates the final documents and production plans reflecting the revisions and sends them to the terminal. Users and administrators can then view and use the finalized documents and plans.

[1221] Specific example

[1222] A user enters the following request into the system: "Generate a production schedule for a new smartphone. Production volume is 1000 units, deadline is December 31, 2024. Use a high-performance battery." The system analyzes this request and generates multiple production schedules. The user then selects the best schedule from the generated ones, makes any necessary modifications, and can review the final production plan.

[1223] Example of a prompt

[1224] "Please generate a production schedule for the new smartphone."

[1225] This allows users to easily enter requests, and generative artificial intelligence can generate documents and plans quickly and accurately.

[1226] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1227] Step 1:

[1228] Users enter requests for document creation and production planning.

[1229] Specific actions:

[1230] Users enter requests through the terminal's interface. For example, they might enter "new product information" or "production schedule for the new smartphone."

[1231] Input: Parent document name, requirements, and other conditions.

[1232] Output: Request data entered on the terminal.

[1233] Step 2:

[1234] Convert the request to JSON format and send it to the server.

[1235] Specific actions:

[1236] The terminal converts the user's input request into JSON format and sends that data to the server.

[1237] Input: Request data entered into the terminal by the user.

[1238] Output: Request data converted to JSON format, sent to the server.

[1239] Step 3:

[1240] The server analyzes the request and sends instructions to the generative artificial intelligence system for creating documents or generating production schedules.

[1241] Specific actions:

[1242] The server analyzes the received request data and sends instructions to the generative artificial intelligence based on the analysis results.

[1243] Input: Request data in JSON format.

[1244] Output: Analysis results, instruction data for the generative AI.

[1245] Step 4:

[1246] Generative artificial intelligence generates multiple document samples and production schedules.

[1247] Specific actions:

[1248] Generative artificial intelligence generates document samples and production schedules based on instructions received from a server. During this process, it performs necessary data processing and calculations.

[1249] Input: Instruction data from the server.

[1250] Output: Multiple document samples or production schedule.

[1251] Step 5:

[1252] The server retrieves the generated document samples and production schedules, saves them in PDF or presentation format, and then sends them to the terminal.

[1253] Specific actions:

[1254] The server receives samples and schedules generated by the generative AI, converts them to a standard file format, saves them, and then sends them to the terminal.

[1255] Input: Sample documents or production schedule data from a generative AI.

[1256] Output: Files converted to PDF or presentation format, data sent to the device.

[1257] Step 6:

[1258] Users and administrators select a sample provided and enter any necessary modifications.

[1259] Specific actions:

[1260] Users and administrators select the best option from several samples displayed on the terminal and enter the necessary modifications.

[1261] Input: Samples and modifications selected by the user or administrator.

[1262] Output: Data including correction instructions.

[1263] Step 7:

[1264] The terminal sends the selected document sample or production schedule and revision details to the server.

[1265] Specific actions:

[1266] The terminal converts the data, including the correction instructions, into JSON format and then sends it to the server.

[1267] Input: Data including correction instructions from users or administrators.

[1268] Output: Correction instruction data converted to JSON format, sent to the server.

[1269] Step 8:

[1270] The server incorporates the correction instructions and generates the final documents and production schedule.

[1271] Specific actions:

[1272] Based on the received correction instructions, the server uses generative artificial intelligence to generate the final documents and production schedules. Re-analysis is performed as needed during this process.

[1273] Input: Correction instruction data in JSON format.

[1274] Output: Final documents or production schedule.

[1275] Step 9:

[1276] The server sends the final documents or production plans to the terminal, which users and administrators then view and use.

[1277] Specific actions:

[1278] The server converts and saves final documents and production schedules in PDF or presentation format and sends them to the terminal. Users and administrators then view and use them on their terminals.

[1279] Input: Final documents or production schedule data.

[1280] Output: Final files converted to PDF or presentation format, sent to devices, and viewed and used by users and administrators.

[1281] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1282] This invention is a system that assists in document creation, designed to enable users to efficiently create high-quality documents. Furthermore, by incorporating an emotion engine that recognizes user emotions, it achieves optimal document creation tailored to the user's emotions. The following describes the system's program processing in natural language, along with specific examples.

[1283] Program processing

[1284] 1. User input of request

[1285] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and should include key features, target market, and a price comparison with competing products.

[1286] 2. Emotion recognition by the emotion engine

[1287] The device uses an emotion engine to recognize the user's emotions when they input data. For example, it might analyze facial expressions using a camera or analyze the tone of voice during voice input.

[1288] 3. Sending a request

[1289] The device converts the user's request and recognized emotions into an appropriate data format, such as JSON, and sends it to the server.

[1290] 4. Parsing the request

[1291] The server analyzes the received request and sentiment data, and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[1292] 5. Instructions for creating documents that take emotional data into consideration.

[1293] The server sends instructions to the generative AI for creating materials based on recognized emotion data. For example, if the user is feeling stressed, it recommends designs and tones that promote relaxation.

[1294] 6. Generating sample materials

[1295] The generative AI automatically generates multiple sample documents based on the instructions it receives. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[1296] 7. Obtaining sample materials

[1297] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[1298] 8. Provision of sample materials

[1299] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[1300] 9. User-selected and modified samples.

[1301] The user selects the most suitable sample material from the provided options and enters any necessary modifications. For example, they might specify changes such as adding more detail to the caption of a particular slide or modifying a graphic.

[1302] 10. Emotion monitoring using an emotion engine

[1303] The device monitors the user's emotional changes using an emotion engine while the user selects and modifies materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[1304] 11. Submit selected and modified data

[1305] The terminal sends the user-selected document sample, its modifications, and emotion data to the server. The data format is {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[1306] 12. Generating the final document

[1307] The server generates the final document, reflecting the user's revisions and final sentiment data. If necessary, it uses a generative AI to apply revisions again.

[1308] 13. Provision of final documents

[1309] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1310] Specific example

[1311] 1. User request input:

[1312] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[1313] 2. Emotion recognition by an emotion engine:

[1314] The device analyzes the user's facial expressions using its camera and recognizes that the user is feeling somewhat anxious.

[1315] 3. Submitting a request:

[1316] The device sends the request content and emotion data to the server as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1317] 4. Parsing the request:

[1318] The server analyzes the request and sends instructions to the generative AI to create new product introduction materials, taking into account the user's emotional data at this time.

[1319] 5. Instructions that take emotional data into consideration:

[1320] The server sends instructions to the generative AI for creating documents that recommend color schemes and fonts that will make the user feel comfortable.

[1321] 6. Generating sample materials:

[1322] The generative AI focuses on multiple document samples with a calm tone.

[1323] 7. Obtaining sample materials:

[1324] The server receives the generated sample in PDF format.

[1325] 8. Provision of sample materials:

[1326] The server sends sample documents to the terminal and displays them to the user.

[1327] 9. User selection and modification:

[1328] The user is instructed to select one sample and provide a detailed description of its main features.

[1329] 10. Emotional monitoring:

[1330] The device recognizes that the user is relaxed and makes correction suggestions.

[1331] 11. Submit selected and corrected data:

[1332] The terminal sends the selected sample and modifications to the server as {"sample_id": 1, "modifications": ["Added a function description"], "emotion": "relaxed"}.

[1333] 12. Generating the final document:

[1334] The server generates the final document, reflecting the changes and sentiment data.

[1335] 13. Provision of final documents:

[1336] The server sends the final document to the terminal and displays it to the user.

[1337] The following describes the processing flow.

[1338] Step 1:

[1339] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and include key features, target market, and a price comparison with competing products.

[1340] Step 2:

[1341] The device uses an emotion engine to recognize the user's emotions along with their input. For example, it uses the camera to analyze facial expressions to determine whether the user is feeling safe or anxious.

[1342] Step 3:

[1343] The device converts the user's request and recognized emotions into an appropriate data format such as JSON, and sends data to the server such as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1344] Step 4:

[1345] The server analyzes the received request and sentiment data. Based on the analysis, it understands that the document type is "new product introduction," and that the necessary items are three key features, the target market, and a price comparison. Furthermore, it considers that the user's sentiment is anxiety.

[1346] Step 5:

[1347] The server sends detailed instructions for document creation to the generative AI based on the analyzed content and emotional data. For example, it sends instructions in the format {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"], "desired_tone": "calm"}.

[1348] Step 6:

[1349] The generative AI receives instructions from the server and generates multiple sample documents with designs and tones that take user emotions into consideration. This results in documents that users can use with confidence.

[1350] Step 7:

[1351] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[1352] Step 8:

[1353] The server sends generated sample materials to the terminal. The terminal displays these samples to the user. The user can review multiple samples and select a design that matches their emotional state.

[1354] Step 9:

[1355] The user selects the most suitable document sample from those provided and enters the necessary modifications. For example, they might enter modifications such as "explain the main features in more detail" or "revise the target market graphic."

[1356] Step 10:

[1357] The device uses an emotion engine to monitor the user's emotional changes while they select and modify materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[1358] Step 11:

[1359] The device sends the user's selected sample document, its modifications, and emotion data to the server. For example, it sends data in the format {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[1360] Step 12:

[1361] The server generates the final document based on user feedback and final sentiment data. If necessary, it applies further revisions using a generative AI.

[1362] Step 13:

[1363] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1364] (Example 2)

[1365] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1366] In modern society, there is an increasing demand for rapid and high-quality document creation, but many users spend too much time on it. Furthermore, since users' emotions during document creation often influence the results, it is desirable to create documents that are emotionally resonant.

[1367] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1368] In this invention, the server includes means for recognizing the user's emotions and reflecting them in the document creation instructions, means for monitoring changes in the user's emotions, and means for sending and receiving document creation requests in JSON format. This enables the creation of optimal documents based on the user's emotions. Furthermore, because the document creation process is streamlined, the user can quickly create high-quality documents.

[1369] A "user" is a person or group that uses the document creation system, enters a document creation request, and reviews and modifies the document generated based on that request.

[1370] A "document creation request" is information that indicates the specific requirements regarding the type and content of the document the user wants to create, and it is entered into the system.

[1371] "Means for analyzing requests" refers to a device or program that receives a document creation request entered by a user, understands and interprets it, and sends the necessary instructions to a generative artificial intelligence.

[1372] "Generative artificial intelligence" refers to an algorithm or software system that automatically generates materials based on specified instructions.

[1373] "Document samples" refer to multiple candidate documents generated by a generative artificial intelligence system, serving as initial documents for the user to select and modify.

[1374] "Means of providing to the user" refers to a device or program for transmitting generated sample materials or final materials to the user's terminal and displaying them.

[1375] "Final document" refers to the document that has been finalized after reflecting the user's selections and revisions.

[1376] "Means of recognizing emotions" refers to a device or program that analyzes the user's facial expressions, voice tone, etc., to identify the user's emotional state.

[1377] "Means for monitoring emotional changes" refers to a device or program that monitors changes in a user's emotional state in real time during the document creation process and records and analyzes that data.

[1378] JSON format is a type of data format that describes data attributes and values ​​using key-value pairs, making it easy for humans to read and easy for machines to analyze.

[1379] "PDF format" is an abbreviation for Portable Document Format, and it is a digital file format for saving and viewing documents and images with high accuracy.

[1380] A "presentation format" is a type of material used for presentations and explanations, where information is visually organized in a slide format.

[1381] This invention is a document creation support system designed to enable users to efficiently create high-quality materials. The system automatically generates product introductions, presentation materials, and other documents based on user requests. Furthermore, it utilizes an emotion engine to recognize the user's emotions and create optimal materials accordingly. The following describes the program processing of this system.

[1382] Program processing

[1383] 1. User input of request

[1384] Users input the type and specific content of the required document through the document creation system's interface. If a user wants to create a document introducing a new product, they would specify items such as key features, target market, and price comparisons with competing products.

[1385] 2. Emotion recognition by the emotion engine

[1386] The device activates an emotion engine when the user inputs text, analyzing the user's facial expressions using the camera (using computer vision technology), or analyzing their voice tone when they input voice text. This allows the device to recognize the user's emotional state, such as whether they are relaxed, tense, or anxious.

[1387] 3. Sending a request

[1388] The device converts the user's request content and recognized emotion data into JSON format and sends it to the server. For example, if the request is for a new product introduction and the user is feeling somewhat anxious, the data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1389] 4. Request parsing and instruction generation

[1390] The server analyzes the received request data and sentiment data. This clarifies the purpose and necessary items for document creation, and generates prompts for the AI ​​model. For example, if the user is feeling anxious, instructions such as "Create a new product introduction document using a design and tone that promotes relaxation" will be generated.

[1391] 5. Generating sample materials using a generative AI model

[1392] The generative AI automatically generates multiple sample documents based on instructions sent from the server. For example, documents containing key features, graphics of the target market, and price comparison charts of competing products can be generated.

[1393] 6. Sending and converting sample documents

[1394] The server receives the generated sample materials and converts them into PDF or presentation format. The samples are saved as PDF document or slide files.

[1395] 7. Provision of sample materials

[1396] The server sends the converted document sample to the terminal, which then displays it to the user. The user can view multiple document samples on the screen and select the most suitable one.

[1397] 8. User selection and modification of samples

[1398] The user selects the most suitable sample material from the provided options and enters the necessary modifications. For example, they might enter specific details such as enriching the caption on a particular slide or modifying a graphic.

[1399] 9. Emotion monitoring using an emotion engine

[1400] The device uses an emotion engine to monitor the user's emotional changes in real time while they select and modify sample materials. This allows for the provision of appropriate samples and modification suggestions based on the user's emotional state.

[1401] 10. Generating the final document

[1402] The server generates the final document, reflecting the user's selections and modifications, as well as sentiment data. If necessary, it re-applies modifications using the regenerating AI model.

[1403] 11. Provision of final documents

[1404] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1405] Specific example

[1406] Here's a concrete example: A user enters the following request into the document creation system: "I want to create a document introducing a new product. It should include three main features, the target market, and a price comparison." The device uses its camera to analyze the user's facial expressions and recognizes that the user is feeling somewhat anxious. The device then sends the request and emotion data to the server.

[1407] The server analyzes the request and sends an instruction to the generative AI model to "create a new product introduction document using a design and tone that conveys a sense of relaxation." The generative AI model creates several sample documents and sends them to the server. The server converts the samples to PDF format and sends them to the user's device. The user reviews the sample documents, selects the most suitable one, and enters suggestions for revisions, such as adding more detail to the description of the main features.

[1408] The device continues to monitor the user's emotions while they are making revisions, recognizing that the user is relaxed. The device sends the revisions to the server, which generates the final document and sends it back to the device. The user can then review and use the final document.

[1409] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1410] Step 1:

[1411] Users input the type and specific content of the required documents through the document creation system's interface.

[1412] In terms of specific operations, if a user requests information about a new product, they will specify detailed items such as key features, target market, and price comparison with competing products. The input data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}.

[1413] Step 2:

[1414] The device activates an emotion engine when the user makes a input, and recognizes the user's emotions.

[1415] Specifically, the system uses computer vision technology to analyze the user's facial expressions with a camera, or analyzes their voice tone through a microphone during voice input. For example, if the user is feeling somewhat anxious, the recognition result will be "anxious." The input data is the user's facial expressions and voice tone, and the output data is their emotional state (e.g., "anxious").

[1416] Step 3:

[1417] The device converts the user's request and recognized sentiment data into JSON format and sends it to the server.

[1418] Specifically, the system receives user request details and emotion data (e.g., {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}) as input data and sends this to the server.

[1419] Step 4:

[1420] The server analyzes the received request data and sentiment data.

[1421] Specifically, the process involves parsing JSON data to identify the purpose of the document creation and the necessary items. Through this analysis, a prompt message is generated. The input data is the received JSON request data, and the output data is the prompt message (e.g., "Create a new product introduction document using a design and tone that conveys a sense of relaxation").

[1422] Step 5:

[1423] The server sends instructions to the AI ​​model for creating the document based on the generated prompt message.

[1424] In terms of specific operations, a prompt message is sent to a generating AI model, requesting it to generate sample materials based on the instructions. The input data is the prompt message, and the output data is the instructions sent to the generating AI model.

[1425] Step 6:

[1426] The generation AI model automatically generates multiple document samples based on instructions sent from the server.

[1427] Specifically, the system creates documents that include key features, graphics for the target market, and price comparison charts of competing products. The input data consists of instructions (prompt messages) from the server, and the output data is a sample of the generated document.

[1428] Step 7:

[1429] The server converts the generated sample documents into the appropriate format.

[1430] Specifically, the process involves converting sample documents into PDF or presentation formats. For example, it converts an HTML document sent from a generative AI model into a PDF. The input data is the generated sample document, and the output data is the sample document converted into the appropriate format.

[1431] Step 8:

[1432] The server sends the converted document sample to the terminal, which then displays it to the user.

[1433] Specifically, the system displays samples on the user interface, allowing the user to select from multiple sample documents. The input data consists of converted sample documents, while the output data is the sample document displayed to the user.

[1434] Step 9:

[1435] The user selects the most suitable document from the provided sample materials and enters any necessary modifications.

[1436] In terms of specific actions, users input details of modifications, such as enriching the description on a particular slide or correcting a graphic, through the interface. The input data consists of the selected document sample and the modifications, while the output data is the modified document sample.

[1437] Step 10:

[1438] The device uses an emotion engine to monitor changes in the user's emotions while they select and modify materials.

[1439] Specifically, the system uses cameras and microphones to analyze the user's emotions in real time and monitors their emotional state. Input data includes the user's facial expressions and voice tone, while output data is their real-time emotional state.

[1440] Step 11:

[1441] The device sends the user's selected document sample, its modifications, and sentiment data to the server.

[1442] Specifically, the terminal packages the revised document sample and sentiment data and sends them to the server. The input data consists of the revised document sample and sentiment data, while the output data is the data sent to the server.

[1443] Step 12:

[1444] The server generates the final document, incorporating user modifications and final sentiment data.

[1445] Specifically, the process involves modifying and creating the final document using an AI model. The input data consists of the modifications and sentiment data, while the output data is the final document.

[1446] Step 13:

[1447] The server sends the final generated document to the terminal, which then displays it to the user.

[1448] Specifically, the final document is displayed on the user interface for review. The input data is the final document, and the output data is the final document displayed to the user.

[1449] (Application Example 2)

[1450] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1451] Currently, when users create advertising materials, they need to consider the design and content individually, which is especially time-consuming when emotions are involved. Furthermore, the fluctuating emotions make it difficult to select the optimal design and content. As a result, creating advertising materials takes a long time, increasing user stress.

[1452] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1453] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate document samples, means for providing the generated document samples to the user, means for recognizing the user's emotions, means for the user to select and modify the samples, means for generating a final document that reflects the selected document samples, modifications, and recognized emotions, and means for providing the final document to the user. This enables the automatic generation of optimal advertising materials tailored to the user's emotions, improving work efficiency and reducing stress.

[1454] A "user" refers to an individual or legal entity that uses the system to request the creation of documents.

[1455] A "document creation request" refers to an operation in which a user specifies the type and content of the document they want to create.

[1456] "Methods for analyzing requests" refers to the process of analyzing the request content entered by the user and extracting the information necessary for creating the document.

[1457] "Generative artificial intelligence" refers to an algorithm or system that automatically creates necessary documents in response to instructions regarding document generation.

[1458] "Sample materials" refer to prototype advertising materials created by a generative artificial intelligence.

[1459] "Means of providing to the user" refers to the process of displaying or sending the generated sample materials to the user.

[1460] "Means of recognizing user emotions" refers to processing or devices that analyze and recognize emotions from a user's facial expressions, voice, etc.

[1461] "Method for selecting and modifying samples" refers to the operation of selecting a suitable sample from the provided materials and making necessary modifications.

[1462] "Method for generating final materials" refers to the process of creating final advertising materials based on the material samples selected and modified by the user.

[1463] The means of providing the "final document" to the user refers to the process of displaying or sending the generated final document to the user.

[1464] This invention is a system that allows users to efficiently create advertising materials, and comprises a cloud server, a terminal, and a generation AI model. The system receives user input from the terminal, and the terminal and cloud server work together to generate advertising materials, which are then ultimately provided to the user.

[1465] Program processing

[1466] 1. User input of request

[1467] The user uses a device (such as a smartphone) to input the specific details of the advertising material they want to create. For example, they might enter a request such as, "New product promotional advertisement. Includes key features, target market, and pricing information."

[1468] 2. Emotion recognition by the emotion engine

[1469] The device uses its camera and microphone to analyze the user's facial expressions and voice, recognizing their emotions (e.g., anxiety, excitement) in real time. This emotion data, along with the user's input, is sent to a cloud server.

[1470] 3. Sending a request

[1471] The user's input and recognized emotion data are converted to JSON format and sent to the cloud server.

[1472] 4. Request analysis and data sample generation

[1473] Generative artificial intelligence (AI) on a cloud server analyzes received requests and emotional data to generate sample advertising materials. The generative AI operates on a server with high-performance computing capabilities and incorporates emotionally responsive design elements such as relaxing color schemes and fonts.

[1474] 5. Provision of sample materials

[1475] The cloud server sends multiple generated document samples to the terminal and displays them to the user.

[1476] 6. User-selected and modified samples.

[1477] The user selects the most suitable document sample from those provided and makes revisions to its content. These revisions may include specific changes such as "explaining the main features in more detail" or "adding pricing information." During this process, the device re-evaluates the user's mood and, if the user is relaxed, suggests colorful designs, for example.

[1478] 7. Generation and provision of final materials

[1479] The cloud server generates the final advertising materials based on the selected sample materials and revisions. The generated final materials are sent to the device in PDF or presentation format and provided to the user.

[1480] Hardware and software to be used

[1481] hardware

[1482] Device: Smartphone (camera, microphone, display)

[1483] Cloud server: A computer with high-performance computing capabilities.

[1484] software

[1485] Emotion recognition engine: Face++ and Microsoft Azure Cognitive Services are used as examples.

[1486] Generative AI models: OpenAI GPT-4 and DALL-E are used.

[1487] Data format: JSON

[1488] API communication: RESTful API

[1489] Specific example

[1490] Example prompt: Create a promotional ad for your new product. Include key features, target market, and pricing information. Since users are feeling anxious, please use a design that emphasizes safety.

[1491] This system allows users to automatically generate optimal advertising materials that match their own emotions, significantly improving the efficiency of advertising material creation and reducing stress.

[1492] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1493] Step 1:

[1494] The user enters a request for document creation.

[1495] Input: Specific content of the advertising material (e.g., "New product promotional advertisement. Includes key features, target market, and pricing information.")

[1496] Output: Input request content

[1497] Specific actions: The user launches the application on their smartphone and enters detailed requirements for the advertising materials.

[1498] Step 2:

[1499] The device recognizes the user's emotions.

[1500] Input: User's facial expression or voice data

[1501] Output: Emotional data (e.g., anxiety, excitement, etc.)

[1502] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and voice in real time, and recognizes the user's emotions using an emotion engine.

[1503] Step 3:

[1504] The device sends the request details and sentiment data to the cloud server.

[1505] Input: Request details, sentiment data

[1506] Output: Data in JSON format (including request details and sentiment data)

[1507] Specific operation: The device converts the request content and recognized emotion data into JSON format and sends it to the cloud server.

[1508] Step 4:

[1509] The server analyzes the request content and emotional data, and sends instructions to the generative artificial intelligence to create the document.

[1510] Input: Request details and sentiment data in JSON format

[1511] Output: Instructions for generating documents to a generative artificial intelligence.

[1512] Specific operation: The server parses the received JSON data and sends instructions to the generative AI for creating materials based on the user's requests and emotions.

[1513] Step 5:

[1514] A generative AI model generates document samples.

[1515] Input: Document creation instructions

[1516] Output: Multiple document samples

[1517] Specific operation: Generative AI models (e.g., OpenAI GPT-4 and DALL-E) generate sample advertising materials based on instructions received from the server. In doing so, they also consider user sentiment data and incorporate designs and color schemes that emphasize safety.

[1518] Step 6:

[1519] The server sends the generated document sample to the terminal.

[1520] Input: Multiple document samples

[1521] Output: Data including sample materials (e.g., PDF or presentation format)

[1522] Specific operation: The server receives the generated document sample, converts it to an appropriate format (PDF or presentation format), and sends it to the terminal.

[1523] Step 7:

[1524] The user selects a sample document and makes modifications.

[1525] Input: Multiple document samples

[1526] Output: Selected sample and modifications

[1527] Specific operation: The user uses their smartphone to select the most suitable document from several provided sample documents and enter specific modification details (e.g., "Expand the description of the main features," "Modify the graphics," etc.).

[1528] Step 8:

[1529] The device then recognizes the user's emotions again and sends the corrections and emotion data to the server.

[1530] Input: User's facial expression or voice data, correction details

[1531] Output: Data in JSON format (including correction details and sentiment data)

[1532] Specific operation: The device uses the emotion engine again to recognize the user's emotions, converts the corrections and emotion data into JSON format, and sends them to the server.

[1533] Step 9:

[1534] The server generates the final document, reflecting the changes and sentiment data.

[1535] Input: Correction details and sentiment data in JSON format

[1536] Output: Final document

[1537] Specific operation: The server uses generative AI to generate the final document based on the received revisions and emotion data. If the user is relaxed, it will incorporate more colorful designs and color schemes.

[1538] Step 10:

[1539] The server sends the final document to the terminal and provides it to the user.

[1540] Input: Final document

[1541] Output: Data including the final document (e.g., PDF or presentation format)

[1542] Specific operation: The server converts the generated final document into the appropriate format and sends it to the terminal. The user can then view, download, or use the final document through the terminal.

[1543] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1544] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1545] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1546] [Fourth Embodiment]

[1547] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1548] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1549] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1550] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1551] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1552] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1553] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1554] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1555] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1556] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1557] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1558] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1559] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1560] This invention is a system designed to assist in document creation, enabling users to efficiently produce high-quality documents. The following describes the system's program processing in natural language, along with specific examples.

[1561] Program processing

[1562] 1. User input of request

[1563] Users enter requests for new documents through the document creation system's interface. For example, they might request a document titled "New Product Introduction," specifying that it should include key features, an overview of the target market, and a price comparison with competing products.

[1564] 2. Sending a request

[1565] The terminal converts the user's request into an appropriate data format (e.g., JSON) and sends it to the server. The data includes the type of document, specific items, and other requirements.

[1566] 3. Parsing the request

[1567] The server analyzes the request received from the terminal and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[1568] 4. Instructions for preparing documents

[1569] The server sends instructions to the generative AI for document creation based on the analyzed request. These instructions include an overview of the document, key elements, and specific content.

[1570] 5. Generating sample materials

[1571] The generative AI automatically generates multiple document samples based on the received instructions. This means that even with the same request, documents with different layouts and designs will be provided.

[1572] 6. Obtaining sample materials

[1573] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[1574] 7. Provision of sample materials

[1575] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[1576] 8. User-selected and modified samples.

[1577] Users select the most suitable document from the provided sample materials and enter any necessary modifications. These modifications can range from adding information to specific slides to changing the design.

[1578] 9. Submit selected and modified data

[1579] The terminal sends the selected document sample and the corrections to the server.

[1580] 10. Generating the final document

[1581] The server incorporates the user's modifications and generates the final document. Generative AI may be used again if necessary.

[1582] 11. Provision of final documents

[1583] The server sends the final document to the terminal, which then displays it to the user. The user can then view and use the final document.

[1584] Specific example

[1585] 1. User request input:

[1586] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[1587] 2. Submitting a request:

[1588] The device sends data to the server in the following format: {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1589] 3. Parsing the request:

[1590] The server analyzes this request and instructs the generative AI to create a document introducing the new product.

[1591] 4. Instructions for preparing the document:

[1592] The server sends instructions to the generative AI to create documents based on the analysis results.

[1593] 5. Generating sample materials:

[1594] The generative AI generates multiple sample documents that include descriptions of key features, graphics of the target market, and price comparison charts.

[1595] 6. Obtaining sample materials:

[1596] The server receives the generated sample and saves it as a PDF file.

[1597] 7. Provision of sample materials:

[1598] The server sends a PDF file to the terminal, and the terminal displays it to the user.

[1599] 8. User selection and modification of samples:

[1600] The user selects one sample and enters the necessary modifications, such as "add more detail to the description of the main features" or "revise the graphic for the target market."

[1601] 9. Submit selected and corrected data:

[1602] The terminal sends the ID of the selected sample and the modifications made to the server.

[1603] 10. Generating the final document:

[1604] The server generates the final document, incorporating the received revisions.

[1605] 11. Provision of final documents:

[1606] The server sends the final document to the terminal, which then displays it to the user.

[1607] The following describes the processing flow.

[1608] Step 1:

[1609] The user launches the document creation system and enters the type of document and its specific content. For example, they might instruct the system to include key features, target market, and a price comparison with competing products in a document titled "New Product Introduction."

[1610] Step 2:

[1611] The terminal converts the user's input into an appropriate data format such as JSON and sends it to the server as data in the format {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1612] Step 3:

[1613] The server analyzes the received request. The analysis reveals that the document type is "New Product Introduction," and that the required items are three key features, the target market, and a price comparison.

[1614] Step 4:

[1615] The server sends instructions to the generative AI to create materials based on the analysis results. Specifically, it sends instructions in the form of {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"]}.

[1616] Step 5:

[1617] The generative AI receives instructions from the server and generates multiple sample documents based on the request. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[1618] Step 6:

[1619] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[1620] Step 7:

[1621] The server sends the generated sample documents to the terminal. The terminal displays these samples to the user. The user can view multiple samples.

[1622] Step 8:

[1623] The user selects the most suitable document from the provided sample materials and enters the necessary modifications. For example, they might enter details about the caption on a particular slide or modify the graphics.

[1624] Step 9:

[1625] The terminal sends the user-selected document sample and its modifications to the server. The data format is {"sample_id": 1, "modifications": ["Added function description", "Graphic modification"]}.

[1626] Step 10:

[1627] The server generates the final document, incorporating the user's revisions. If necessary, it uses a generation AI to apply the revisions again.

[1628] Step 11:

[1629] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1630] These steps enable the efficient and high-quality creation of documents.

[1631] (Example 1)

[1632] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1633] In today's business environment, there is a demand for the rapid and efficient creation of high-quality documents. However, document creation typically requires a significant amount of time and effort, and especially when complex content or design is required, professional assistance is often necessary. Therefore, document creation is a major burden for many companies and individuals. Furthermore, if revisions are needed to a document, manually correcting it each time is inefficient and prone to errors.

[1634] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1635] In this invention, the server includes means for the user to input a request for document creation, means for sending the request from the terminal to the server in JSON format, and means for the server to analyze the request and send instructions for document creation to a generative artificial intelligence. This enables the user to create high-quality documents quickly and efficiently.

[1636] A "user" is the entity that uses the document creation system to create documents.

[1637] A "document creation request" is information in which the user enters the content and requirements of the document they wish to have created.

[1638] A "terminal" is an electronic device used by a user to access a document creation system, enter and submit requests, or view and modify generated documents.

[1639] A "server" is a computer system that receives requests sent from a terminal, sends instructions to a generative artificial intelligence to create documents, and provides the terminal with generated document samples and the final document.

[1640] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight data exchange format for structuring and representing data.

[1641] "Generative artificial intelligence" refers to artificial intelligence that automatically generates materials based on user requests.

[1642] "Document samples" refer to some or all of the multiple candidate documents created by a generative artificial intelligence.

[1643] "PDF format" is an abbreviation for Portable Document Format, and it is a file format for displaying and distributing electronic documents.

[1644] A "presentation format" is a file format used to visually present information in the form of slides.

[1645] "Modification details" refer to the changes and additional information added to the document sample selected by the user.

[1646] The "final version" refers to the completed document that incorporates the user's revisions.

[1647] The system of this invention is designed to enable users to efficiently create high-quality materials. Specifically, it includes a terminal in which the user inputs a request for material creation, a server that analyzes the received request and sends instructions for material creation to a generative artificial intelligence (AI), means for providing a sample of the generated material, and means for the user to select a sample, make modifications, and generate and provide the final material.

[1648] Hardware and software to be used

[1649] terminal

[1650] The device is used by the user to access the document creation system, enter and submit requests, and view and modify the generated documents. Specific examples of such devices include personal computers, tablets, and smartphones.

[1651] server

[1652] The server receives and analyzes user requests and sends instructions to the generative artificial intelligence to create the document. The server also converts the generated document sample into PDF or presentation format and provides it to the user's device. Technically, a web server, database server, and AI model server are required to perform these operations.

[1653] Generative artificial intelligence (AI)

[1654] Generative AI is used to generate multiple sample documents based on user requests. Specifically, it can utilize natural language processing (NLP) or machine learning models (e.g., GPT-3).

[1655] Software Tools

[1656] For sending and receiving data in JSON format, program libraries (e.g., fetch API or axios) are used, and for converting to PDF format, dedicated libraries such as PDFKit are used.

[1657] Data processing and data calculation

[1658] Request parsing

[1659] The server analyzes the request sent from the terminal. This analysis clarifies the purpose and necessary items for creating the document. For example, if the request is for a "new product introduction," the analysis results will include details such as key features, target market, and price comparison.

[1660] Generating document samples

[1661] The server sends the analysis results to the generative AI, which then generates multiple document samples based on those results. The generated documents will include different layouts and designs even for the same request.

[1662] Converting document samples

[1663] The server receives the sample documents created by the generative AI and converts them into an appropriate format (such as PDF or presentation format). This conversion process uses a dedicated software library.

[1664] Reflecting user-submitted changes

[1665] The user selects a provided sample document and enters the necessary revisions. The terminal sends the revisions to the server, which then incorporates them to generate the final document. If necessary, the generation AI may be used again. The final document is saved in PDF format or another format and provided from the server to the terminal.

[1666] Specific example

[1667] For example, a user might input a request such as, "I want to create a new product introduction document. It should include three main features, a target market, and a price comparison." The device sends this request to the server in JSON format. The server parses the request and instructs the generative AI to create the "new product introduction" document. The generative AI receives the instructions and generates several sample documents, each containing a description of the main features, a graphic of the target market, and a price comparison chart. The server saves the generated samples as PDF files and sends them to the device. The user selects one of the samples and inputs revisions, such as "add more detail to the description of the main features" or "revise the graphic of the target market." The device sends the ID of the selected sample and the revisions to the server, which then generates the final document incorporating these changes. The final document is sent to the device and displayed to the user.

[1668] Example of a prompt

[1669] Please create a product introduction document for the new product. It should include three key features, the target market, and a price comparison.

[1670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1671] Step 1:

[1672] The user accesses the interface of the document creation system and enters a request for a new document. This includes the type of document and specific requirements (e.g., "I want to create a document introducing a new product. It should include three key features, the target market, and a price comparison"). The entered information is temporarily stored by the terminal.

[1673] Step 2:

[1674] The terminal converts the user's request into JSON format. For example, based on the input request, it converts it into a data format like {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}. This converted data becomes the input.

[1675] Step 3:

[1676] The terminal sends the request data, converted to JSON format, to the server. The transmission uses the HTTP protocol, delivering the request to the specified endpoint on the server side. This allows the server to begin parsing.

[1677] Step 4:

[1678] The server analyzes the request data it receives. It extracts information such as the type of request and detailed items, and generates the results as analysis data. For example, if the server receives a request for "new product introduction," it analyzes the items of main features, target market, and price comparison, and stores that data in list format.

[1679] Step 5:

[1680] The server sends instructions to the generating AI for document creation based on the analyzed data. The instructions include the type of document and the requested detailed items. For example, the generating AI might receive a prompt message such as, "Please create an introductory document for the new product. It should include three main features, the target market, and a price comparison."

[1681] Step 6:

[1682] The generation AI automatically generates multiple sample documents based on the instructions it receives. These sample documents include a variety of designs and layouts, such as explanations of key features, graphics of the target market, and price comparison charts. The generated sample documents become the output data.

[1683] Step 7:

[1684] The server receives sample documents generated by the generative AI and converts them into PDF or presentation formats. PDFKit or similar libraries are used for this conversion. The converted files become the new output.

[1685] Step 8:

[1686] The server sends the converted document sample to the terminal. HTTP or HTTPS protocols are used for transmission. The terminal loads the received document sample into an interface for displaying it to the user.

[1687] Step 9:

[1688] The user selects the most suitable document from the provided sample materials and enters any necessary modifications. For example, they might add information to a specific slide or change its design. The user's modifications are temporarily saved on the device.

[1689] Step 10:

[1690] The device converts the data containing the corrections into JSON format and sends it to the server along with the sample selection data. For example, it will be in the format {"sampleID": "12345", "corrections": ["Detailed description of key features", "Modified graphics for the target market"]}.

[1691] Step 11:

[1692] The server incorporates the received revisions and generates the final document. If necessary, the generation AI is used again to apply the final revisions. The generated final document becomes the output data.

[1693] Step 12:

[1694] The server sends the final document to the terminal, which then displays it to the user. The user can then review the final document and use it immediately.

[1695] (Application Example 1)

[1696] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1697] Planning production in a factory is extremely complex, requiring consideration of numerous factors. Furthermore, the efficient allocation of limited resources places a heavy burden on managers. Current systems require replanning every time changes or modifications occur, which is time-consuming and labor-intensive. Additionally, these plans are typically created manually, making them prone to human error. Therefore, a system is needed that can automatically generate production plans and manage them efficiently and accurately.

[1698] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1699] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate multiple document samples, means for providing the generated document samples to the user, means for the user to select and modify the samples, means for generating a final document reflecting the selected document samples and modifications, means for providing the final document to the user, means for inputting a request for production planning, means for analyzing the request and generating multiple production schedules, means for providing the generated production schedules to an administrator, means for the administrator to select and modify schedules, means for generating a final production plan reflecting the selected production schedules and modifications, and means for providing the final production plan to an administrator. This enables the automatic generation and efficient modification of production plans.

[1700] A "user request" is a specific request that a user enters into the system regarding document creation or production planning.

[1701] "Generative artificial intelligence" refers to artificial intelligence that has the ability to automatically generate documents and production schedules based on the data and instructions it receives.

[1702] A "document sample" is an initial draft document with multiple variations, created by a generative artificial intelligence.

[1703] A "production plan request" is when a manager enters specific production goals and requirements into the system.

[1704] "Generated production schedule" refers to an initial production schedule proposal with multiple variations, created by a generative artificial intelligence.

[1705] A "manager" is a person who is responsible for overseeing the planning and progress of a factory or production line.

[1706] "Final document" refers to the document that has been finalized to reflect user revisions.

[1707] The "final production plan" refers to the production schedule that has been finalized to reflect any revisions made by the manager.

[1708] This invention is a system for supporting the automatic generation of production plans and document creation in factories. It is designed to enable users to efficiently create high-quality documents and factory managers to formulate optimal production schedules. The following describes the program processing of this system, with specific examples.

[1709] System Configuration

[1710] The system consists of user terminals and a server. The user terminals provide an interface for inputting and modifying requests for document creation and production planning. The server works in conjunction with generative artificial intelligence (AI) to process requests and generate the final documents and production plans.

[1711] Hardware and software to be used

[1712] Generative AI: Uses generative AI models such as OpenAI's GPT-3.

[1713] Server-side framework: Use FastAPI to process requests.

[1714] Data format: JSON format is used for exchanging requests and generated data.

[1715] Data storage: Use a database (e.g., PostgreSQL) as needed.

[1716] Program Processing Description

[1717] 1. Entering a user request

[1718] Users enter requests for new materials and production plans through the terminal interface. For example, they might enter specific requirements such as, "I want to create a new product introduction document, including key features, target market, and price comparison."

[1719] 2. Sending and parsing requests

[1720] The terminal converts the input request into JSON format and sends it to the server. The server analyzes the request and sends instructions to the generative artificial intelligence for creating documents and production plans.

[1721] 3. Generation by generative artificial intelligence

[1722] Generative artificial intelligence automatically generates multiple document samples and production schedules based on transmitted instructions. It utilizes text generation models and schedule optimization algorithms as needed.

[1723] 4. Obtaining and providing samples

[1724] The server saves generated document samples and production schedules in PDF or presentation format and sends them to terminals. Users and administrators can then view these, select appropriate samples, and make modifications.

[1725] 5. Reflecting selections and modifications

[1726] Users and administrators select the best sample from the provided options and enter any necessary modifications. These modifications are sent to the server and reflected in the final documentation and production schedule.

[1727] 6. Provision of final documents and plans

[1728] The server generates the final documents and production plans reflecting the revisions and sends them to the terminal. Users and administrators can then view and use the finalized documents and plans.

[1729] Specific example

[1730] A user enters the following request into the system: "Generate a production schedule for a new smartphone. Production volume is 1000 units, deadline is December 31, 2024. Use a high-performance battery." The system analyzes this request and generates multiple production schedules. The user then selects the best schedule from the generated ones, makes any necessary modifications, and can review the final production plan.

[1731] Example of a prompt

[1732] "Please generate a production schedule for the new smartphone."

[1733] This allows users to easily enter requests, and generative artificial intelligence can generate documents and plans quickly and accurately.

[1734] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1735] Step 1:

[1736] Users enter requests for document creation and production planning.

[1737] Specific actions:

[1738] Users enter requests through the terminal's interface. For example, they might enter "new product information" or "production schedule for the new smartphone."

[1739] Input: Parent document name, requirements, and other conditions.

[1740] Output: Request data entered on the terminal.

[1741] Step 2:

[1742] Convert the request to JSON format and send it to the server.

[1743] Specific actions:

[1744] The terminal converts the user's input request into JSON format and sends that data to the server.

[1745] Input: Request data entered into the terminal by the user.

[1746] Output: Request data converted to JSON format, sent to the server.

[1747] Step 3:

[1748] The server analyzes the request and sends instructions to the generative artificial intelligence system for creating documents or generating production schedules.

[1749] Specific actions:

[1750] The server analyzes the received request data and sends instructions to the generative artificial intelligence based on the analysis results.

[1751] Input: Request data in JSON format.

[1752] Output: Analysis results, instruction data for the generative AI.

[1753] Step 4:

[1754] Generative artificial intelligence generates multiple document samples and production schedules.

[1755] Specific actions:

[1756] Generative artificial intelligence generates document samples and production schedules based on instructions received from a server. During this process, it performs necessary data processing and calculations.

[1757] Input: Instruction data from the server.

[1758] Output: Multiple document samples or production schedule.

[1759] Step 5:

[1760] The server retrieves the generated document samples and production schedules, saves them in PDF or presentation format, and then sends them to the terminal.

[1761] Specific actions:

[1762] The server receives samples and schedules generated by the generative AI, converts them to a standard file format, saves them, and then sends them to the terminal.

[1763] Input: Sample documents or production schedule data from a generative AI.

[1764] Output: Files converted to PDF or presentation format, data sent to the device.

[1765] Step 6:

[1766] Users and administrators select a sample provided and enter any necessary modifications.

[1767] Specific actions:

[1768] Users and administrators select the best option from several samples displayed on the terminal and enter the necessary modifications.

[1769] Input: Samples and modifications selected by the user or administrator.

[1770] Output: Data including correction instructions.

[1771] Step 7:

[1772] The terminal sends the selected document sample or production schedule and revision details to the server.

[1773] Specific actions:

[1774] The terminal converts the data, including the correction instructions, into JSON format and then sends it to the server.

[1775] Input: Data including correction instructions from users or administrators.

[1776] Output: Correction instruction data converted to JSON format, sent to the server.

[1777] Step 8:

[1778] The server incorporates the correction instructions and generates the final documents and production schedule.

[1779] Specific actions:

[1780] Based on the received correction instructions, the server uses generative artificial intelligence to generate the final documents and production schedules. Re-analysis is performed as needed during this process.

[1781] Input: Correction instruction data in JSON format.

[1782] Output: Final documents or production schedule.

[1783] Step 9:

[1784] The server sends the final documents or production plans to the terminal, which users and administrators then view and use.

[1785] Specific actions:

[1786] The server converts and saves final documents and production schedules in PDF or presentation format and sends them to the terminal. Users and administrators then view and use them on their terminals.

[1787] Input: Final documents or production schedule data.

[1788] Output: Final files converted to PDF or presentation format, sent to devices, and viewed and used by users and administrators.

[1789] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1790] This invention is a system that assists in document creation, designed to enable users to efficiently create high-quality documents. Furthermore, by incorporating an emotion engine that recognizes user emotions, it achieves optimal document creation tailored to the user's emotions. The following describes the system's program processing in natural language, along with specific examples.

[1791] Program processing

[1792] 1. User input of request

[1793] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and should include key features, target market, and a price comparison with competing products.

[1794] 2. Emotion recognition by the emotion engine

[1795] The device uses an emotion engine to recognize the user's emotions when they input data. For example, it might analyze facial expressions using a camera or analyze the tone of voice during voice input.

[1796] 3. Sending a request

[1797] The device converts the user's request and recognized emotions into an appropriate data format, such as JSON, and sends it to the server.

[1798] 4. Parsing the request

[1799] The server analyzes the received request and sentiment data, and sends instructions for document creation to the generative artificial intelligence (AI). The analysis clarifies the purpose and necessary items for document creation.

[1800] 5. Instructions for creating documents that take emotional data into consideration.

[1801] The server sends instructions to the generative AI for creating materials based on recognized emotion data. For example, if the user is feeling stressed, it recommends designs and tones that promote relaxation.

[1802] 6. Generating sample materials

[1803] The generative AI automatically generates multiple sample documents based on the instructions it receives. Each sample includes key features, a graphic of the target market, and a price comparison chart of competing products.

[1804] 7. Obtaining sample materials

[1805] The server receives sample documents generated by the generative AI and converts them into an appropriate format (e.g., PDF or presentation format).

[1806] 8. Provision of sample materials

[1807] The server sends the generated document sample to the terminal, which then displays it to the user. The user can view multiple document samples.

[1808] 9. User-selected and modified samples.

[1809] The user selects the most suitable sample material from the provided options and enters any necessary modifications. For example, they might specify changes such as adding more detail to the caption of a particular slide or modifying a graphic.

[1810] 10. Emotion monitoring using an emotion engine

[1811] The device monitors the user's emotional changes using an emotion engine while the user selects and modifies materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[1812] 11. Submit selected and modified data

[1813] The terminal sends the user-selected document sample, its modifications, and emotion data to the server. The data format is {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[1814] 12. Generating the final document

[1815] The server generates the final document, reflecting the user's revisions and final sentiment data. If necessary, it uses a generative AI to apply revisions again.

[1816] 13. Provision of final documents

[1817] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1818] Specific example

[1819] 1. User request input:

[1820] The user inputs into the system, "I want to create a product introduction document for a new product. It should include three main features, the target market, and a price comparison."

[1821] 2. Emotion recognition by an emotion engine:

[1822] The device analyzes the user's facial expressions using its camera and recognizes that the user is feeling somewhat anxious.

[1823] 3. Submitting a request:

[1824] The device sends the request content and emotion data to the server as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1825] 4. Parsing the request:

[1826] The server analyzes the request and sends instructions to the generative AI to create new product introduction materials, taking into account the user's emotional data at this time.

[1827] 5. Instructions that take emotional data into consideration:

[1828] The server sends instructions to the generative AI for creating documents that recommend color schemes and fonts that will make the user feel comfortable.

[1829] 6. Generating sample materials:

[1830] The generative AI focuses on multiple document samples with a calm tone.

[1831] 7. Obtaining sample materials:

[1832] The server receives the generated sample in PDF format.

[1833] 8. Provision of sample materials:

[1834] The server sends sample documents to the terminal and displays them to the user.

[1835] 9. User selection and modification:

[1836] The user is instructed to select one sample and provide a detailed description of its main features.

[1837] 10. Emotional monitoring:

[1838] The device recognizes that the user is relaxed and makes correction suggestions.

[1839] 11. Submit selected and corrected data:

[1840] The terminal sends the selected sample and modifications to the server as {"sample_id": 1, "modifications": ["Added a function description"], "emotion": "relaxed"}.

[1841] 12. Generating the final document:

[1842] The server generates the final document, reflecting the changes and sentiment data.

[1843] 13. Provision of final documents:

[1844] The server sends the final document to the terminal and displays it to the user.

[1845] The following describes the processing flow.

[1846] Step 1:

[1847] The user launches the document creation system and enters the type of document and its specific content. For example, they might specify that the document should be a "new product introduction," and include key features, target market, and a price comparison with competing products.

[1848] Step 2:

[1849] The device uses an emotion engine to recognize the user's emotions along with their input. For example, it uses the camera to analyze facial expressions to determine whether the user is feeling safe or anxious.

[1850] Step 3:

[1851] The device converts the user's request and recognized emotions into an appropriate data format such as JSON, and sends data to the server such as {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1852] Step 4:

[1853] The server analyzes the received request and sentiment data. Based on the analysis, it understands that the document type is "new product introduction," and that the necessary items are three key features, the target market, and a price comparison. Furthermore, it considers that the user's sentiment is anxiety.

[1854] Step 5:

[1855] The server sends detailed instructions for document creation to the generative AI based on the analyzed content and emotional data. For example, it sends instructions in the format {"action": "generate", "type": "New Product Introduction", "content": ["Main Features: 3", "Target Market", "Price Comparison"], "desired_tone": "calm"}.

[1856] Step 6:

[1857] The generative AI receives instructions from the server and generates multiple sample documents with designs and tones that take user emotions into consideration. This results in documents that users can use with confidence.

[1858] Step 7:

[1859] The server receives multiple document samples generated by a generative AI. It converts the received samples into PDF or presentation formats and saves them.

[1860] Step 8:

[1861] The server sends generated sample materials to the terminal. The terminal displays these samples to the user. The user can review multiple samples and select a design that matches their emotional state.

[1862] Step 9:

[1863] The user selects the most suitable document sample from those provided and enters the necessary modifications. For example, they might enter modifications such as "explain the main features in more detail" or "revise the target market graphic."

[1864] Step 10:

[1865] The device uses an emotion engine to monitor the user's emotional changes while they select and modify materials. Based on these changes, the system provides appropriate samples and modification suggestions.

[1866] Step 11:

[1867] The device sends the user's selected sample document, its modifications, and emotion data to the server. For example, it sends data in the format {"sample_id": 1, "modifications": ["Added feature description", "Graphic modification"], "emotion": "satisfied"}.

[1868] Step 12:

[1869] The server generates the final document based on user feedback and final sentiment data. If necessary, it applies further revisions using a generative AI.

[1870] Step 13:

[1871] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1872] (Example 2)

[1873] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1874] In modern society, there is an increasing demand for rapid and high-quality document creation, but many users spend too much time on it. Furthermore, since users' emotions during document creation often influence the results, it is desirable to create documents that are emotionally resonant.

[1875] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1876] In this invention, the server includes means for recognizing the user's emotions and reflecting them in the document creation instructions, means for monitoring changes in the user's emotions, and means for sending and receiving document creation requests in JSON format. This enables the creation of optimal documents based on the user's emotions. Furthermore, because the document creation process is streamlined, the user can quickly create high-quality documents.

[1877] A "user" is a person or group that uses the document creation system, enters a document creation request, and reviews and modifies the document generated based on that request.

[1878] A "document creation request" is information that indicates the specific requirements regarding the type and content of the document the user wants to create, and it is entered into the system.

[1879] "Means for analyzing requests" refers to a device or program that receives a document creation request entered by a user, understands and interprets it, and sends the necessary instructions to a generative artificial intelligence.

[1880] "Generative artificial intelligence" refers to an algorithm or software system that automatically generates materials based on specified instructions.

[1881] "Document samples" refer to multiple candidate documents generated by a generative artificial intelligence system, serving as initial documents for the user to select and modify.

[1882] "Means of providing to the user" refers to a device or program for transmitting generated sample materials or final materials to the user's terminal and displaying them.

[1883] "Final document" refers to the document that has been finalized after reflecting the user's selections and revisions.

[1884] "Means of recognizing emotions" refers to a device or program that analyzes the user's facial expressions, voice tone, etc., to identify the user's emotional state.

[1885] "Means for monitoring emotional changes" refers to a device or program that monitors changes in a user's emotional state in real time during the document creation process and records and analyzes that data.

[1886] JSON format is a type of data format that describes data attributes and values ​​using key-value pairs, making it easy for humans to read and easy for machines to analyze.

[1887] "PDF format" is an abbreviation for Portable Document Format, and it is a digital file format for saving and viewing documents and images with high accuracy.

[1888] A "presentation format" is a type of material used for presentations and explanations, where information is visually organized in a slide format.

[1889] This invention is a document creation support system designed to enable users to efficiently create high-quality materials. The system automatically generates product introductions, presentation materials, and other documents based on user requests. Furthermore, it utilizes an emotion engine to recognize the user's emotions and create optimal materials accordingly. The following describes the program processing of this system.

[1890] Program processing

[1891] 1. User input of request

[1892] Users input the type and specific content of the required document through the document creation system's interface. If a user wants to create a document introducing a new product, they would specify items such as key features, target market, and price comparisons with competing products.

[1893] 2. Emotion recognition by the emotion engine

[1894] The device activates an emotion engine when the user inputs text, analyzing the user's facial expressions using the camera (using computer vision technology), or analyzing their voice tone when they input voice text. This allows the device to recognize the user's emotional state, such as whether they are relaxed, tense, or anxious.

[1895] 3. Sending a request

[1896] The device converts the user's request content and recognized emotion data into JSON format and sends it to the server. For example, if the request is for a new product introduction and the user is feeling somewhat anxious, the data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}.

[1897] 4. Request parsing and instruction generation

[1898] The server analyzes the received request data and sentiment data. This clarifies the purpose and necessary items for document creation, and generates prompts for the AI ​​model. For example, if the user is feeling anxious, instructions such as "Create a new product introduction document using a design and tone that promotes relaxation" will be generated.

[1899] 5. Generating sample materials using a generative AI model

[1900] The generative AI automatically generates multiple sample documents based on instructions sent from the server. For example, documents containing key features, graphics of the target market, and price comparison charts of competing products can be generated.

[1901] 6. Sending and converting sample documents

[1902] The server receives the generated sample materials and converts them into PDF or presentation format. The samples are saved as PDF document or slide files.

[1903] 7. Provision of sample materials

[1904] The server sends the converted document sample to the terminal, which then displays it to the user. The user can view multiple document samples on the screen and select the most suitable one.

[1905] 8. User selection and modification of samples

[1906] The user selects the most suitable sample material from the provided options and enters the necessary modifications. For example, they might enter specific details such as enriching the caption on a particular slide or modifying a graphic.

[1907] 9. Emotion monitoring using an emotion engine

[1908] The device uses an emotion engine to monitor the user's emotional changes in real time while they select and modify sample materials. This allows for the provision of appropriate samples and modification suggestions based on the user's emotional state.

[1909] 10. Generating the final document

[1910] The server generates the final document, reflecting the user's selections and modifications, as well as sentiment data. If necessary, it re-applies modifications using the regenerating AI model.

[1911] 11. Provision of final documents

[1912] The server sends the final generated document to the terminal, which then displays it to the user. The user can then review and use the final document.

[1913] Specific example

[1914] Here's a concrete example: A user enters the following request into the document creation system: "I want to create a document introducing a new product. It should include three main features, the target market, and a price comparison." The device uses its camera to analyze the user's facial expressions and recognizes that the user is feeling somewhat anxious. The device then sends the request and emotion data to the server.

[1915] The server analyzes the request and sends an instruction to the generative AI model to "create a new product introduction document using a design and tone that conveys a sense of relaxation." The generative AI model creates several sample documents and sends them to the server. The server converts the samples to PDF format and sends them to the user's device. The user reviews the sample documents, selects the most suitable one, and enters suggestions for revisions, such as adding more detail to the description of the main features.

[1916] The device continues to monitor the user's emotions while they are making revisions, recognizing that the user is relaxed. The device sends the revisions to the server, which generates the final document and sends it back to the device. The user can then review and use the final document.

[1917] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1918] Step 1:

[1919] Users input the type and specific content of the required documents through the document creation system's interface.

[1920] In terms of specific operations, if a user requests information about a new product, they will specify detailed items such as key features, target market, and price comparison with competing products. The input data will be in the format {"type": "New Product Introduction", "details": ["Key Features: 3", "Target Market", "Price Comparison"]}.

[1921] Step 2:

[1922] The device activates an emotion engine when the user makes a input, and recognizes the user's emotions.

[1923] Specifically, the system uses computer vision technology to analyze the user's facial expressions with a camera, or analyzes their voice tone through a microphone during voice input. For example, if the user is feeling somewhat anxious, the recognition result will be "anxious." The input data is the user's facial expressions and voice tone, and the output data is their emotional state (e.g., "anxious").

[1924] Step 3:

[1925] The device converts the user's request and recognized sentiment data into JSON format and sends it to the server.

[1926] Specifically, the system receives user request details and emotion data (e.g., {"type": "New Product Introduction", "details": ["Main Features: 3", "Target Market", "Price Comparison"], "emotion": "anxious"}) as input data and sends this to the server.

[1927] Step 4:

[1928] The server analyzes the received request data and sentiment data.

[1929] Specifically, the process involves parsing JSON data to identify the purpose of the document creation and the necessary items. Through this analysis, a prompt message is generated. The input data is the received JSON request data, and the output data is the prompt message (e.g., "Create a new product introduction document using a design and tone that conveys a sense of relaxation").

[1930] Step 5:

[1931] The server sends instructions to the AI ​​model for creating the document based on the generated prompt message.

[1932] In terms of specific operations, a prompt message is sent to a generating AI model, requesting it to generate sample materials based on the instructions. The input data is the prompt message, and the output data is the instructions sent to the generating AI model.

[1933] Step 6:

[1934] The generation AI model automatically generates multiple document samples based on instructions sent from the server.

[1935] Specifically, the system creates documents that include key features, graphics for the target market, and price comparison charts of competing products. The input data consists of instructions (prompt messages) from the server, and the output data is a sample of the generated document.

[1936] Step 7:

[1937] The server converts the generated sample documents into the appropriate format.

[1938] Specifically, the process involves converting sample documents into PDF or presentation formats. For example, it converts an HTML document sent from a generative AI model into a PDF. The input data is the generated sample document, and the output data is the sample document converted into the appropriate format.

[1939] Step 8:

[1940] The server sends the converted document sample to the terminal, which then displays it to the user.

[1941] Specifically, the system displays samples on the user interface, allowing the user to select from multiple sample documents. The input data consists of converted sample documents, while the output data is the sample document displayed to the user.

[1942] Step 9:

[1943] The user selects the most suitable document from the provided sample materials and enters any necessary modifications.

[1944] In terms of specific actions, users input details of modifications, such as enriching the description on a particular slide or correcting a graphic, through the interface. The input data consists of the selected document sample and the modifications, while the output data is the modified document sample.

[1945] Step 10:

[1946] The device uses an emotion engine to monitor changes in the user's emotions while they select and modify materials.

[1947] Specifically, the system uses cameras and microphones to analyze the user's emotions in real time and monitors their emotional state. Input data includes the user's facial expressions and voice tone, while output data is their real-time emotional state.

[1948] Step 11:

[1949] The device sends the user's selected document sample, its modifications, and sentiment data to the server.

[1950] Specifically, the terminal packages the revised document sample and sentiment data and sends them to the server. The input data consists of the revised document sample and sentiment data, while the output data is the data sent to the server.

[1951] Step 12:

[1952] The server generates the final document, incorporating user modifications and final sentiment data.

[1953] Specifically, the process involves modifying and creating the final document using an AI model. The input data consists of the modifications and sentiment data, while the output data is the final document.

[1954] Step 13:

[1955] The server sends the final generated document to the terminal, which then displays it to the user.

[1956] Specifically, the final document is displayed on the user interface for review. The input data is the final document, and the output data is the final document displayed to the user.

[1957] (Application Example 2)

[1958] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1959] Currently, when users create advertising materials, they need to consider the design and content individually, which is especially time-consuming when emotions are involved. Furthermore, the fluctuating emotions make it difficult to select the optimal design and content. As a result, creating advertising materials takes a long time, increasing user stress.

[1960] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1961] In this invention, the server includes means for the user to input a request for document creation, means for analyzing the request and sending instructions for document creation to a generative artificial intelligence, means for the generative artificial intelligence to generate document samples, means for providing the generated document samples to the user, means for recognizing the user's emotions, means for the user to select and modify the samples, means for generating a final document that reflects the selected document samples, modifications, and recognized emotions, and means for providing the final document to the user. This enables the automatic generation of optimal advertising materials tailored to the user's emotions, improving work efficiency and reducing stress.

[1962] A "user" refers to an individual or legal entity that uses the system to request the creation of documents.

[1963] A "document creation request" refers to an operation in which a user specifies the type and content of the document they want to create.

[1964] "Methods for analyzing requests" refers to the process of analyzing the request content entered by the user and extracting the information necessary for creating the document.

[1965] "Generative artificial intelligence" refers to an algorithm or system that automatically creates necessary documents in response to instructions regarding document generation.

[1966] "Sample materials" refer to prototype advertising materials created by a generative artificial intelligence.

[1967] "Means of providing to the user" refers to the process of displaying or sending the generated sample materials to the user.

[1968] "Means of recognizing user emotions" refers to processing or devices that analyze and recognize emotions from a user's facial expressions, voice, etc.

[1969] "Method for selecting and modifying samples" refers to the operation of selecting a suitable sample from the provided materials and making necessary modifications.

[1970] "Method for generating final materials" refers to the process of creating final advertising materials based on the material samples selected and modified by the user.

[1971] The means of providing the "final document" to the user refers to the process of displaying or sending the generated final document to the user.

[1972] This invention is a system that allows users to efficiently create advertising materials, and comprises a cloud server, a terminal, and a generation AI model. The system receives user input from the terminal, and the terminal and cloud server work together to generate advertising materials, which are then ultimately provided to the user.

[1973] Program processing

[1974] 1. User input of request

[1975] The user uses a device (such as a smartphone) to input the specific details of the advertising material they want to create. For example, they might enter a request such as, "New product promotional advertisement. Includes key features, target market, and pricing information."

[1976] 2. Emotion recognition by the emotion engine

[1977] The device uses its camera and microphone to analyze the user's facial expressions and voice, recognizing their emotions (e.g., anxiety, excitement) in real time. This emotion data, along with the user's input, is sent to a cloud server.

[1978] 3. Sending a request

[1979] The user's input and recognized emotion data are converted to JSON format and sent to the cloud server.

[1980] 4. Request analysis and data sample generation

[1981] Generative artificial intelligence (AI) on a cloud server analyzes received requests and emotional data to generate sample advertising materials. The generative AI operates on a server with high-performance computing capabilities and incorporates emotionally responsive design elements such as relaxing color schemes and fonts.

[1982] 5. Provision of sample materials

[1983] The cloud server sends multiple generated document samples to the terminal and displays them to the user.

[1984] 6. User-selected and modified samples.

[1985] The user selects the most suitable document sample from those provided and makes revisions to its content. These revisions may include specific changes such as "explaining the main features in more detail" or "adding pricing information." During this process, the device re-evaluates the user's mood and, if the user is relaxed, suggests colorful designs, for example.

[1986] 7. Generation and provision of final materials

[1987] The cloud server generates the final advertising materials based on the selected sample materials and revisions. The generated final materials are sent to the device in PDF or presentation format and provided to the user.

[1988] Hardware and software to be used

[1989] hardware

[1990] Device: Smartphone (camera, microphone, display)

[1991] Cloud server: A computer with high-performance computing capabilities.

[1992] software

[1993] Emotion recognition engine: Face++ and Microsoft Azure Cognitive Services are used as examples.

[1994] Generative AI models: OpenAI GPT-4 and DALL-E are used.

[1995] Data format: JSON

[1996] API communication: RESTful API

[1997] Specific example

[1998] Example prompt: Create a promotional ad for your new product. Include key features, target market, and pricing information. Since users are feeling anxious, please use a design that emphasizes safety.

[1999] This system allows users to automatically generate optimal advertising materials that match their own emotions, significantly improving the efficiency of advertising material creation and reducing stress.

[2000] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2001] Step 1:

[2002] The user enters a request for document creation.

[2003] Input: Specific content of the advertising material (e.g., "New product promotional advertisement. Includes key features, target market, and pricing information.")

[2004] Output: Input request content

[2005] Specific actions: The user launches the application on their smartphone and enters detailed requirements for the advertising materials.

[2006] Step 2:

[2007] The device recognizes the user's emotions.

[2008] Input: User's facial expression or voice data

[2009] Output: Emotional data (e.g., anxiety, excitement, etc.)

[2010] Specific operation: Using the smartphone's camera and microphone, the system analyzes the user's facial expressions and voice in real time, and recognizes the user's emotions using an emotion engine.

[2011] Step 3:

[2012] The device sends the request details and sentiment data to the cloud server.

[2013] Input: Request details, sentiment data

[2014] Output: Data in JSON format (including request details and sentiment data)

[2015] Specific operation: The device converts the request content and recognized emotion data into JSON format and sends it to the cloud server.

[2016] Step 4:

[2017] The server analyzes the request content and emotional data, and sends instructions to the generative artificial intelligence to create the document.

[2018] Input: Request details and sentiment data in JSON format

[2019] Output: Instructions for generating documents to a generative artificial intelligence.

[2020] Specific operation: The server parses the received JSON data and sends instructions to the generative AI for creating materials based on the user's requests and emotions.

[2021] Step 5:

[2022] A generative AI model generates document samples.

[2023] Input: Document creation instructions

[2024] Output: Multiple document samples

[2025] Specific operation: Generative AI models (e.g., OpenAI GPT-4 and DALL-E) generate sample advertising materials based on instructions received from the server. In doing so, they also consider user sentiment data and incorporate designs and color schemes that emphasize safety.

[2026] Step 6:

[2027] The server sends the generated document sample to the terminal.

[2028] Input: Multiple document samples

[2029] Output: Data including sample materials (e.g., PDF or presentation format)

[2030] Specific operation: The server receives the generated document sample, converts it to an appropriate format (PDF or presentation format), and sends it to the terminal.

[2031] Step 7:

[2032] The user selects a sample document and makes modifications.

[2033] Input: Multiple document samples

[2034] Output: Selected sample and modifications

[2035] Specific operation: The user uses their smartphone to select the most suitable document from several provided sample documents and enter specific modification details (e.g., "Expand the description of the main features," "Modify the graphics," etc.).

[2036] Step 8:

[2037] The device then recognizes the user's emotions again and sends the corrections and emotion data to the server.

[2038] Input: User's facial expression or voice data, correction details

[2039] Output: Data in JSON format (including correction details and sentiment data)

[2040] Specific operation: The device uses the emotion engine again to recognize the user's emotions, converts the corrections and emotion data into JSON format, and sends them to the server.

[2041] Step 9:

[2042] The server generates the final document, reflecting the changes and sentiment data.

[2043] Input: Correction details and sentiment data in JSON format

[2044] Output: Final document

[2045] Specific operation: The server uses generative AI to generate the final document based on the received revisions and emotion data. If the user is relaxed, it will incorporate more colorful designs and color schemes.

[2046] Step 10:

[2047] The server sends the final document to the terminal and provides it to the user.

[2048] Input: Final document

[2049] Output: Data including the final document (e.g., PDF or presentation format)

[2050] Specific operation: The server converts the generated final document into the appropriate format and sends it to the terminal. The user can then view, download, or use the final document through the terminal.

[2051] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2052] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2053] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2054] Furthermore, the emotion identification model 59, acting as an emotion engine, may dete...

Claims

1. A means for users to enter requests for document creation, A means of analyzing a request and sending instructions for document creation to a generative artificial intelligence, A means by which a generative artificial intelligence generates multiple document samples, A means of providing the generated document sample to the user, A means for the user to select a sample and make modifications, A means of generating the final document by reflecting the selected document samples and revisions, Means of providing the final document to the user, A system that includes this.

2. The system according to claim 1, comprising means for sending and receiving requests for document creation in JSON format.

3. The system according to claim 1, comprising means for providing the generated document sample in PDF format or presentation format.

Images