System

Abstract

A system is provided.SOLUTION: This system is provided with a means for receiving a material preparation request from a user in a voice or text format, a means for analyzing the intention of the user by a natural language processing engine, and a means for automatically generating an initial material constitution plan based on an analysis result, and for requesting feedback from the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's business environment, the numerous meetings held and the accompanying preparation of materials and presentations are a significant burden. This consumes a lot of time and resources, hindering efficient business operations. Furthermore, creating materials requires logical organization and appropriate visualization of data, which is difficult to do manually. There is a need for a system that can improve this situation and support effective and efficient material creation. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing the following means. Specifically, the system includes a means for receiving a user's request for creating a document in voice or text format, a means for analyzing the user's intent using a natural language processing engine, and a means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user. The system also includes a means for accessing a database or external data source to acquire necessary data, automatically converting the acquired data into a graph or table format, and inserting the data into the document. The system further includes a means for analyzing the overall structure and story flow of the document, determining weaknesses and areas for improvement, presenting suggestions to the user, and automatically optimizing the document based on the user's instructions. This system enables users to efficiently create high-quality documents.

[0006] "User" refers to the entity that creates materials using this system.

[0007] "Audio or text format" refers to the format of data input by the user, and refers to audio data or character data.

[0008] A "natural language processing engine" refers to a computer program that analyzes natural language text or speech input by a user and understands its intent and content.

[0009] "Document structure proposal" refers to a proposal for the overall structure and layout of a document that is automatically generated based on input from the user.

[0010] A "database" refers to a system that systematically collects, stores, and manages information and data.

[0011] "External data source" refers to information obtained from an external resource other than a database.

[0012] "Graphical and tabular formats" refer to formats for visually displaying data, such as diagrams and tables.

[0013] "Overall structure of the material" refers to the chapter and section structure of the entire material.

[0014] "Story flow" refers to how the content of a document unfolds appropriately based on a chronological or logical order.

[0015] "Optimization" refers to adjusting the content and structure of materials to be most effective and efficient. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to an AI system that assists users in creating documents efficiently. The system automatically creates document structure proposals and data visualizations based on user input in voice or text format. The system's program processing and specific examples are shown below.

[0038] System Overview

[0039] 1. User Registration

[0040] The user registers and enters the required information such as name, email address, and password via the terminal.

[0041] The server receives this information and stores it in a database.

[0042] 2. User Authentication

[0043] The user enters their email address and password on the login screen.

[0044] The server authenticates this and, if correct, starts the session.

[0045] 3. Start preparing materials

[0046] The user speaks or writes to the device, saying, "I want to create presentation materials."

[0047] The server parses the instructions using a natural language processing engine.

[0048] 4. Creating a draft of the materials

[0049] The server asks the user, "What is the purpose of the material?"

[0050] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[0051] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0052] 5. Inserting data and figures

[0053] The user indicates, "I want to insert a graph of sales data."

[0054] The server accesses the database or external data source to retrieve the required data.

[0055] Convert the acquired data into graph or table format and insert it into the document.

[0056] 6. Optimize your materials

[0057] The server analyzes the overall structure of the material and the flow of the story.

[0058] Determine weaknesses and areas for improvement and provide suggestions to users.

[0059] The user checks the suggestions, and the server adjusts to create the most suitable materials.

[0060] Specific examples

[0061] User Registration and Login

[0062] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[0063] 2. The device sends the input information to the server.

[0064] 3. The server receives the information and stores it in a database.

[0065] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[0066] 5. The server authenticates this and, if successful, starts the session.

[0067] From the start of document creation to the creation of a structure plan

[0068] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[0069] 2. The device converts the voice into text and sends it to the server.

[0070] 3. The server parses the instructions using a natural language processing engine.

[0071] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[0072] 5. The device presents the question to the user on the screen or by voice.

[0073] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[0074] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[0075] Inserting and optimizing data

[0076] 1. The user says, "Insert a graph of sales data."

[0077] 2. The server retrieves the sales data from the database and converts it into a graph.

[0078] 3. The graph generated by the server is inserted into the document and displayed on the terminal.

[0079] 4. The server checks the overall structure and story of the document and suggests improvements to the user.

[0080] 5. The user reviews and approves the proposal, and the final document is completed.

[0081] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials, saving a lot of time and effort.

[0082] The processing flow will be explained below.

[0083] User Registration and Authentication

[0084] User Registration

[0085] Step 1:

[0086] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[0087] Step 2:

[0088] The terminal sends the input information to the server.

[0089] Step 3:

[0090] The server validates the information it receives to ensure it is in the proper format.

[0091] Step 4:

[0092] The server stores the information in a database.

[0093] Step 5:

[0094] The server generates a registration completion message and sends it to the terminal.

[0095] Step 6:

[0096] The terminal displays a completion message.

[0097] User Authentication

[0098] Step 1:

[0099] The user enters their email address and password on the login screen and clicks the "Login" button.

[0100] Step 2:

[0101] The device sends the authentication information to the server.

[0102] Step 3:

[0103] The server will look up the database and match it with the information you entered.

[0104] Step 4:

[0105] If the server finds the match successful, it initiates the session and generates an authentication success message.

[0106] Step 5:

[0107] The server sends an authentication success message to the terminal.

[0108] Step 6:

[0109] The device will transition to the home screen and display a welcome message.

[0110] Conversational document creation

[0111] Start creating materials

[0112] Step 1:

[0113] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[0114] Step 2:

[0115] The server launches a natural language processing engine to analyze the user's intent.

[0116] Step 3:

[0117] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[0118] Step 4:

[0119] Your device will display or read the question aloud.

[0120] User Input

[0121] Step 1:

[0122] The user responds to the terminal with information about the purpose and content of the material.

[0123] Step 2:

[0124] The terminal sends the answer to the server.

[0125] Step 3:

[0126] The server analyzes the responses and generates an initial draft of the material structure.

[0127] Step 4:

[0128] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0129] Step 5:

[0130] The device displays the configuration plan.

[0131] feedback

[0132] Step 1:

[0133] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[0134] Step 2:

[0135] The terminal sends the instruction to the server.

[0136] Step 3:

[0137] The server recreates the draft document structure based on the instructions.

[0138] Step 4:

[0139] The server sends the updated configuration plan to the terminal.

[0140] Step 5:

[0141] The device displays the updated configuration.

[0142] Automatic data and diagram creation

[0143] Requesting data insertion

[0144] Step 1:

[0145] The user instructs the terminal to "insert a graph of sales data."

[0146] Step 2:

[0147] The terminal sends the instruction to the server.

[0148] Step 3:

[0149] The server accesses the database or external data source to retrieve the required data.

[0150] Step 4:

[0151] The server converts the retrieved data into an appropriate graph format.

[0152] Step 5:

[0153] The server sends the generated graph to the terminal.

[0154] Step 6:

[0155] The terminal displays the graph.

[0156] Data confirmation

[0157] Step 1:

[0158] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[0159] Step 2:

[0160] The terminal sends the instruction to the server.

[0161] Step 3:

[0162] The server regenerates the graph as needed and sends it to the device.

[0163] Step 4:

[0164] The terminal will display the regenerated graph.

[0165] Checking the direction of the material and the story

[0166] Check the story

[0167] Step 1:

[0168] The server analyzes the overall structure of the material and the flow of the story.

[0169] Step 2:

[0170] The server determines weak areas and areas for improvement and generates suggestions.

[0171] Step 3:

[0172] The server sends the proposal to the device.

[0173] Step 4:

[0174] The device will display suggestions.

[0175] User Verification

[0176] Step 1:

[0177] The user checks the proposal and inputs approval or correction instructions into the terminal.

[0178] Step 2:

[0179] The terminal sends the instruction to the server.

[0180] Step 3:

[0181] The server optimizes the materials based on the instructions.

[0182] Step 4:

[0183] The server sends the optimized material to the terminal.

[0184] Step 5:

[0185] The device displays optimized materials.

[0186] Optimization and concept review

[0187] Final Check

[0188] Step 1:

[0189] The server will thoroughly review the completed document and report any errors or improvements.

[0190] Step 2:

[0191] The server sends the report to the device.

[0192] Step 3:

[0193] The terminal displays the report.

[0194] Concept reconfirmation

[0195] Step 1:

[0196] The user initiates a dialogue with the terminal to reaffirm the concept or message.

[0197] Step 2:

[0198] The terminal sends the corresponding content to the server.

[0199] Step 3:

[0200] The server interacts with the user to reconfirm the concepts and messages of the materials.

[0201] Step 4:

[0202] The server will fine-tune the material as needed.

[0203] Step 5:

[0204] The server sends the final materials to the terminal.

[0205] Step 6:

[0206] The terminal displays the completed document.

[0207] Example 1

[0208] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0209] In conventional document creation systems, users had to manually plan the document structure, collect data, visualize it, and finally complete the document, which required a great deal of time and effort. During this process, it was difficult for users to maintain consistency in the document, and specialized knowledge was often required for data visualization and optimization. Therefore, a system that supports efficient and effective document creation was needed.

[0210] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0211] In this invention, the server includes means for receiving a user's request for creating a document in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for the user to register and enter authentication information via a terminal, which the server stores in a database, means for the user to enter an email address and password on a login screen, which the server authenticates and starts a session, means for automatically inserting data and figures specified by the user into the generated document structure proposal, which the server acquires, visualizes, and integrates into the document, and means for analyzing the overall structure and story flow of the document, determining weaknesses and areas for improvement, and presenting suggestions to the user. This enables users to efficiently create high-quality documents quickly and reduce the significant time and effort required in the document creation process.

[0212] A "material creation request" is a request for the creation of a material that a user submits to the system in voice or text format.

[0213] A "natural language processing engine" is an artificial intelligence technology that analyzes a user's voice and text instructions and understands their intentions.

[0214] The "initial document structure proposal" is an initial plan or outline of the document that the system automatically generates based on the user's instructions.

[0215] "Feedback" refers to evaluations and suggestions for improvement provided by users regarding the initial draft of the material structure.

[0216] "New registration" is the process in which a user enters personal information and authentication information for the first time to create an account in order to use the system.

[0217] "Authentication information" refers to information such as an email address and password used to identify a user.

[0218] "Database" means a system for efficiently and safely storing and managing user information and other necessary data.

[0219] "Logging in" is the process by which a user accesses a system using their credentials.

[0220] A "session" refers to the period of a series of operations and activities while a user is logged in to a system, and is a management mechanism for maintaining that state.

[0221] "Data visualization" is the process of converting acquired data into a form that is easy for users to understand, such as a graph or table.

[0222] "Overall structure of a document" is the process of organizing and arranging the content of the main parts and paragraphs of a document.

[0223] "Story flow" refers to the order and logical development of information within a document.

[0224] "Weaknesses and areas for improvement" refers to areas within the document that are insufficient or need to be improved.

[0225] "Suggestions" are advice and corrections for improvement that the system presents to the user.

[0226] "Optimization" is the process of adjusting materials to make them more effective and of higher quality.

[0227] MODE FOR CARRYING OUT THE INVENTION

[0228] This invention relates to an AI system that assists users in efficiently creating materials and obtaining high-quality output. This system receives instructions from the user in the form of voice or text, and automatically creates a document structure proposal and visualizes data. Specifically, the system functions with the server, terminal, and user playing their respective roles. The details are explained below.

[0229] Hardware and software used

[0230] The following hardware and software are used to operate this system.

[0231] Hardware

[0232] PC or smartphone

[0233] software

[0234] Web browser

[0235] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0236] Database management system (e.g. MySQL)

[0237] Speech recognition engine (e.g. Google Cloud Speech-to-Text API)

[0238] Data visualization libraries (e.g., Matplotlib)

[0239] Specific explanation of the system's processing

[0240] User Registration

[0241] The user enters their name, email address, and password on the new registration screen in their web browser. The device sends this data to the server. The server verifies the received data and saves it in a database. Once the data is saved, the server sends a registration completion message to the device, which the device displays to the user.

[0242] User Authentication

[0243] The user enters their email address and password on the login screen. The device sends the data to the server, which then queries the database for authentication. If authentication is successful, the server generates a session ID and sends it to the device. The device then displays a message indicating successful authentication.

[0244] Start creating materials

[0245] The user speaks to the device, saying, "I want to create presentation materials." The device converts this speech into text format and sends it to the server. The server analyzes it using a natural language processing engine, generates a question, "What is the purpose of the materials?" and sends it to the device. The device then displays this question to the user.

[0246] Creating a draft of the materials

[0247] The user responds to the terminal by saying, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server then analyzes the response again using a natural language processing engine and automatically generates an initial proposal for the composition of the materials. This proposal is then sent to the terminal, which then requests feedback from the user.

[0248] Inserting data and images

[0249] The user instructs the terminal to "insert a graph of sales data." The terminal sends this instruction to the server, which retrieves the required data from a database or external data source. The retrieved data is converted into a graph using a data visualization library and inserted into the document. The final draft of the document is sent to the terminal and displayed to the user.

[0250] Material optimization

[0251] The server uses a generative AI model to analyze the overall structure and story flow of the document. It identifies weaknesses and areas for improvement and sends suggestions to the user. The user reviews the suggestions and makes any necessary corrections. The server optimizes the document based on this feedback and sends the final draft to the device for the user to review and download.

[0252] Examples of prompt statements

[0253] "Please create promotional materials for our new product. The target audience is women in their 20s, and we would like to highlight the product's features."

[0254] "Please graph the following sales data and add it to your report."

[0255] As described above, the present invention is a system that supports users in efficiently creating high-quality materials, and reduces a great deal of time and effort in the process of creating materials.

[0256] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0257] Step 1:

[0258] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device receives this information and sends it to the server. The server registers this information in the database. This gives the user an account to access the system.

[0259] Step 2:

[0260] The user enters their email address and password on the login screen and clicks the "Login" button. The device receives the input information and sends it to the server. The server queries the database to confirm that the email address and password match, and if authentication is successful, generates a session ID and sends it to the device. The device displays a message to the user indicating successful authentication.

[0261] Step 3:

[0262] The user speaks to the device, saying, "I want to create presentation materials." The device converts the voice data into text and sends the text data to the server. The server uses a natural language processing engine to analyze the text data and identify the instruction. The server generates a question, "What is the purpose of the materials?" and sends it to the device.

[0263] Step 4:

[0264] The terminal displays a question from the server to the user. The user responds by typing, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server uses a natural language processing engine to analyze the user's response and automatically generates a proposal for the composition of the materials. The server then sends the generated proposal to the terminal and asks the user for feedback.

[0265] Step 5:

[0266] The user gives the instruction to "insert a graph of sales data." The terminal sends this instruction to the server. The server accesses the database or external data source to obtain sales data. Based on the obtained data, it generates a graph using a data visualization library. The server inserts the generated graph into the document and sends the result to the terminal. The terminal displays the updated document to the user.

[0267] Step 6:

[0268] The server uses a generative AI model to analyze the overall structure and story flow of the document. It determines weaknesses and areas for improvement and presents suggestions to the user. These suggestions are sent to the device, where the user can review and revise them. Based on the user's feedback, the server optimizes the document and sends the final draft to the device. The device displays the final document to the user and provides a download link if necessary.

[0269] (Application example 1)

[0270] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0271] Currently, production reports in factories are often created manually, which is time-consuming and labor-intensive. Furthermore, data aggregation and graphing require specialized knowledge and are inefficient. Furthermore, optimizing the overall structure and flow of data is time-consuming, which can lead to inconsistent quality in the reports created. Therefore, there is a need for a method that reduces this labor and time and allows for the rapid creation of high-quality production reports.

[0272] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0273] In this invention, the server includes means for receiving a document creation request from a user in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for accessing a factory production database to acquire production data, and means for converting the acquired production data into a graph format and inserting it into the document, thereby enabling the user to efficiently and automatically create high-quality production reports.

[0274] A "user" is a person who uses the system to make a document creation request.

[0275] A "material creation request" is an instruction from a user to create a material in voice or text format.

[0276] The "audio format" is a format in which the user inputs using spoken language.

[0277] The "text format" is a format in which the user inputs text such as letters and symbols.

[0278] A "natural language processing engine" is a technology that analyzes voice and text data and understands its intent.

[0279] A "document structure plan" is a plan that shows the initial structure and content layout of the entire document.

[0280] "Feedback" refers to opinions and ratings provided by users, and is information that the system uses to make further improvements.

[0281] A "database" is a storage medium for systematically organizing and storing information.

[0282] An "external data source" is a data source that exists outside the system.

[0283] A "production database" is a storage medium that collects various data related to production in a factory.

[0284] A "graphic format" is a chart format for visually representing numerical data.

[0285] "Natural language analysis" is the process by which a system understands and analyzes human language.

[0286] A "generative AI model" is an artificial intelligence model that automatically generates materials and content based on user intent and data.

[0287] A "prompt sentence" is an instruction sentence that is input to a generative AI model.

[0288] The present invention relates to an AI system that enables users to efficiently create documents such as production reports. This system receives instructions from the user in voice or text format, and automatically creates document structure proposals and data visualizations based on those instructions. Specific examples of how this system can be realized are shown below.

[0289] Overall system configuration

[0290] 1. User Registration

[0291] The user registers and enters the required information such as name, email address, and password.

[0292] The server completes the user registration by storing this information in a database.

[0293] 2. User Authentication

[0294] The user enters their email address and password on the login screen and performs authentication.

[0295] The server starts a session based on the entered information.

[0296] 3. Start preparing materials

[0297] The user issues a voice or text command to the device (smartphone or factory robot terminal) stating, "I want to create a production report."

[0298] The server parses the instructions using a natural language processing engine.

[0299] 4. Creating a draft of the materials

[0300] The server asks the user, "What is the purpose of the report?" and collects the user's response.

[0301] Based on the collected information, the server automatically generates an initial draft of the material structure and asks the user for feedback.

[0302] 5. Inserting data and graphs

[0303] The user instructs, "I want to insert a graph of production volume."

[0304] The server accesses the factory's production database and retrieves the necessary data.

[0305] The acquired data is converted into a graph format and inserted into the document.

[0306] 6. Optimize your materials

[0307] The server analyzes the overall structure of the document and the flow of the story, and suggests areas for improvement to the user.

[0308] Based on instructions from the user, the server optimizes the materials.

[0309] Technology and hardware used

[0310] 1. Natural Language Processing Engine

[0311] The system uses natural language processing to analyze the user's voice and text instructions, specifically using a natural language processing engine such as spaCy.

[0312] 2. Database Access

[0313] The server accesses the factory's production database and retrieves the necessary data, using database manipulation techniques such as SQL.

[0314] 3. Graph Generation

[0315] To visualize the acquired data, we use the pandas and matplotlib libraries to generate graphs.

[0316] Examples and prompts

[0317] Here is a concrete example:

[0318] Prompt: Create a production report for your factory and generate graphs based on the production data from the past two weeks. Also, briefly summarize your production plan for the future.

[0319] In this way, users can efficiently create high-quality production reports. Because various data is automatically acquired and visualized, it is possible to significantly reduce the amount of time and effort required.

[0320] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0321] Step 1:

[0322] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device sends the input information to the server. The server saves the sent information in a database. The input data is the name, email address, and password, and the output is the saved state in the database. This creates a user account.

[0323] Step 2:

[0324] The user enters an email address and password on the login screen and is authenticated. The device sends the entered information to the server. The server compares it with information in the database and, if it matches, starts a session. The input data is the email address and password, and the output is the session start status. This completes user authentication.

[0325] Step 3:

[0326] The user issues a voice or text command to the terminal, saying, "I want to create a production report." The terminal converts the voice into text format and sends the data to the server. The server uses a natural language processing engine to analyze the command. The input data is the voice or text command, and the output is the analysis result. This allows the user's intent to be understood.

[0327] Step 4:

[0328] The server asks the user, "What is the purpose of the report?" and communicates the question to the user via the terminal. The user answers the purpose and content using the terminal. The terminal sends the answer to the server. The server automatically generates an initial draft of the document structure based on the collected information, sends it to the terminal, and asks the user for feedback. The input data is the user's answer, and the output is the draft of the document structure. This generates the initial draft of the document structure.

[0329] Step 5:

[0330] The user gives a text command saying, "I want to insert a production volume graph." The terminal sends this command to the server. The server accesses the factory's production database and retrieves the data for the specified range. The retrieved data is converted into a data frame using pandas, and a graph is generated using matplotlib. The generated graph is inserted into the document and displayed on the terminal. The input data is the specified production data range, and the output is the generation and insertion of a graph. This achieves visualization of the production data.

[0331] Step 6:

[0332] The server analyzes the overall structure and story flow of the document and determines weaknesses and areas for improvement. Based on the analysis results, it presents suggestions to the user. The user reviews the suggestions and gives instructions for approval or revision. The server optimizes the document based on the user's instructions. The input data is the document to be analyzed, and the output is improvement suggestions and the optimized document. This results in the generation of a final, high-quality report.

[0333] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0334] This invention relates to an AI system that assists users in creating documents efficiently, and in particular provides a system equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. It can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions. The system's program processing and specific examples are shown below.

[0335] System Overview

[0336] 1. User Registration

[0337] The user registers and enters the required information such as name, email address, and password via the terminal.

[0338] The server receives this information and stores it in a database.

[0339] 2. User Authentication

[0340] The user enters their email address and password on the login screen.

[0341] The server authenticates this and, if correct, starts the session.

[0342] 3. Start preparing materials

[0343] The user speaks or writes to the device, saying, "I want to create presentation materials."

[0344] The server parses the instructions using a natural language processing engine.

[0345] 4. Creating a draft of the materials

[0346] The server asks the user, "What is the purpose of the material?"

[0347] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[0348] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0349] 5. Emotion recognition

[0350] When a user inputs text or voice, the server uses an emotion engine to analyze the user's emotions in real time.

[0351] Based on the recognized emotions, the server adjusts the proposed material composition.

[0352] 6. Inserting data and figures

[0353] The user indicates, "I want to insert a graph of sales data."

[0354] The server accesses the database or external data source to retrieve the required data.

[0355] Convert the acquired data into graph or table format and insert it into the document.

[0356] 7. Emotion-Based Optimization

[0357] The server analyzes the overall structure of the material and the flow of the story, optimizing the content of the feedback according to the recognized emotions.

[0358] The server determines areas for improvement and presents suggestions to the user.

[0359] Specific examples

[0360] User Registration and Login

[0361] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[0362] 2. The device sends the input information to the server.

[0363] 3. The server receives the information and stores it in a database.

[0364] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[0365] 5. The server authenticates this and, if successful, starts the session.

[0366] From the start of document creation to the creation of a structure plan

[0367] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[0368] 2. The device converts the voice into text and sends it to the server.

[0369] 3. The server parses the instructions using a natural language processing engine.

[0370] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[0371] 5. The device presents the question to the user on the screen or by voice.

[0372] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[0373] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[0374] Emotion recognition and data insertion, optimization

[0375] 1. The user provides input, including emotional expressions, via voice or text.

[0376] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[0377] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[0378] 4. The user says, "Insert a graph of sales data."

[0379] 5. The server retrieves the sales data from the database and converts it into a graph.

[0380] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[0381] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[0382] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials based on emotion recognition, saving users a lot of time and effort.

[0383] The processing flow will be explained below.

[0384] User Registration and Authentication

[0385] User Registration

[0386] Step 1:

[0387] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[0388] Step 2:

[0389] The terminal sends the input information to the server.

[0390] Step 3:

[0391] The server validates the information it receives to ensure it is in the proper format.

[0392] Step 4:

[0393] The server stores the information in a database.

[0394] Step 5:

[0395] The server generates a registration completion message and sends it to the terminal.

[0396] Step 6:

[0397] The terminal displays a completion message.

[0398] User Authentication

[0399] Step 1:

[0400] The user enters their email address and password on the login screen and clicks the "Login" button.

[0401] Step 2:

[0402] The device sends the authentication information to the server.

[0403] Step 3:

[0404] The server will look up the database and match it with the information you entered.

[0405] Step 4:

[0406] If the server finds the match successful, it initiates the session and generates an authentication success message.

[0407] Step 5:

[0408] The server sends an authentication success message to the terminal.

[0409] Step 6:

[0410] The device will transition to the home screen and display a welcome message.

[0411] Conversational document creation

[0412] Start creating materials

[0413] Step 1:

[0414] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[0415] Step 2:

[0416] The server launches a natural language processing engine to analyze the user's intent.

[0417] Step 3:

[0418] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[0419] Step 4:

[0420] Your device will display or read the question aloud.

[0421] User Input

[0422] Step 1:

[0423] The user responds to the terminal with information about the purpose and content of the material.

[0424] Step 2:

[0425] The terminal sends the answer to the server.

[0426] Step 3:

[0427] The server analyzes the responses and generates an initial draft of the material structure.

[0428] Step 4:

[0429] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0430] Step 5:

[0431] The device displays the configuration plan.

[0432] feedback

[0433] Step 1:

[0434] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[0435] Step 2:

[0436] The terminal sends the instruction to the server.

[0437] Step 3:

[0438] The server recreates the draft document structure based on the instructions.

[0439] Step 4:

[0440] The server sends the updated configuration plan to the terminal.

[0441] Step 5:

[0442] The device displays the updated configuration.

[0443] Automated data and diagram creation using emotion recognition

[0444] Emotion recognition and data insertion requests

[0445] Step 1:

[0446] When a user creates a document using voice or text, the device sends the input to the emotion engine.

[0447] Step 2:

[0448] The emotion engine analyzes the user's emotions in real time and sends the results to the server.

[0449] Step 3:

[0450] The server uses the emotion recognition results to adjust the composition of the material.

[0451] Step 4:

[0452] The user instructs the terminal to "insert a graph of sales data."

[0453] Step 5:

[0454] The terminal sends the instruction to the server.

[0455] Step 6:

[0456] The server accesses the database or external data source to retrieve the required data.

[0457] Step 7:

[0458] The server converts the retrieved data into an appropriate graph format.

[0459] Step 8:

[0460] The server sends the generated graph to the terminal.

[0461] Step 9:

[0462] The terminal displays the graph.

[0463] Emotion Recognition and Data Verification

[0464] Step 1:

[0465] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[0466] Step 2:

[0467] The terminal sends the instruction to the server.

[0468] Step 3:

[0469] The server regenerates the graph as needed and sends it to the device.

[0470] Step 4:

[0471] The terminal will display the regenerated graph.

[0472] Checking the direction of the material and the story

[0473] Check the story

[0474] Step 1:

[0475] The server analyzes the overall structure of the material and the flow of the story.

[0476] Step 2:

[0477] The server determines weak areas and areas for improvement and generates suggestions.

[0478] Step 3:

[0479] The server sends the proposal to the device.

[0480] Step 4:

[0481] The device will display suggestions.

[0482] User Verification

[0483] Step 1:

[0484] The user checks the proposal and inputs approval or correction instructions into the terminal.

[0485] Step 2:

[0486] The terminal sends the instruction to the server.

[0487] Step 3:

[0488] The server optimizes the materials based on the instructions.

[0489] Step 4:

[0490] The server sends the optimized material to the terminal.

[0491] Step 5:

[0492] The device displays optimized materials.

[0493] Optimization and concept review

[0494] Final Check

[0495] Step 1:

[0496] The server will thoroughly review the completed document and report any errors or improvements.

[0497] Step 2:

[0498] The server sends the report to the device.

[0499] Step 3:

[0500] The terminal displays the report.

[0501] Concept reconfirmation

[0502] Step 1:

[0503] The user initiates a dialogue at the terminal to reaffirm the concept or message.

[0504] Step 2:

[0505] The terminal sends the corresponding content to the server.

[0506] Step 3:

[0507] The server interacts with the user to reconfirm the concepts and messages of the materials.

[0508] Step 4:

[0509] The server will fine-tune the material as needed.

[0510] Step 5:

[0511] The server sends the final materials to the terminal.

[0512] Step 6:

[0513] The terminal displays the completed document.

[0514] Example 2

[0515] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0516] Conventional document creation systems require users to manually design document structures and convert data into graphs and tables, which requires a great deal of time and effort. Furthermore, it is difficult to create detailed documents that reflect the user's emotions and intentions. The present invention aims to solve these problems and provide a system that supports users in creating high-quality documents efficiently and easily.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0518] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, emotion recognition means for recognizing the user's emotions in real time, and means for automatically adjusting the material composition plan based on the recognized emotions. This makes it possible to automatically generate and adjust an appropriate material composition that reflects emotions simply by providing instructions from the user.

[0519] A "document creation request" refers to an instruction or request made by a user to the system to request the creation of a document.

[0520] "Voice or text format" refers to an interface that allows a user to give instructions to the system, and is in the form of voice input or text input.

[0521] A "natural language processing engine" refers to algorithms and software that analyze human language and understand its intent and meaning.

[0522] "Analysis results" refers to the information and data obtained by the natural language processing engine when it analyzes the user's instructions.

[0523] "Initial document structure proposal" refers to the initial structure and content of the document that the system automatically generates based on the analysis results.

[0524] "Feedback" refers to the evaluations, opinions, and correction requests that users provide to the system.

[0525] "Emotion recognition means" refers to algorithms and software for analyzing and understanding a user's emotions in real time.

[0526] "Means for automatically adjusting the proposed document structure" refers to functions or algorithms that automatically change or adjust the structure or content of a document based on recognized emotions.

[0527] A "database" refers to a system for systematically storing and managing information.

[0528] "External data source" refers to a source or service that provides data that exists outside the system.

[0529] "Converting into graph or table format" refers to the process or task of formatting acquired data into a form that is visually easy to understand.

[0530] "Means for inserting into documents" refers to functions and processes for placing the generated graphs and tables in appropriate positions in the documents.

[0531] "The overall structure of the material and the flow of the story" refers to the arrangement of each section or part within the material and ensuring that they have a continuous meaning.

[0532] "Measures for determining weaknesses and areas for improvement" refers to algorithms or processes used to identify gaps or areas in the content or structure of a document that need correction.

[0533] "Means for presenting suggestions" refers to the functions and processes for providing users with specific actions for improvement or change suggestions.

[0534] "Means for automatically optimizing materials" refers to functions and algorithms that automatically adjust and improve the content of materials to the most appropriate form based on user instructions and emotion recognition results.

[0535] This invention relates to an AI system that assists users in creating documents efficiently. The system is equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. The system can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions.

[0536] Hardware and Software Configuration

[0537] server

[0538] Database: A system for systematically storing and managing information

[0539] Natural language processing engine: Software for analyzing human language and understanding intent, such as the Google NLP API

[0540] Emotion recognition engine: Software for recognizing user emotions in real time, such as IBM Watson Tone Analyzer

[0541] Data visualization tools: Software for transforming data into graphs and tables, such as Matplotlib, Pandas DataFrame, etc.

[0542] Terminal

[0543] Speech recognition engine: Software that converts a user's speech into text, such as the Google Speech-to-Text API.

[0544] User interface: An interface for receiving voice commands, text input, and feedback

[0545] Example of implementation

[0546] User Registration and Login

[0547] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[0548] 2. The device sends the input information to the server.

[0549] 3. The server receives the information and stores it in a database.

[0550] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[0551] 5. The server authenticates this and, if successful, starts the session.

[0552] From the start of document creation to the creation of a structure plan

[0553] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[0554] 2. The device converts the voice into text and sends it to the server.

[0555] 3. The server parses the instructions using a natural language processing engine.

[0556] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[0557] 5. The device presents the question to the user on the screen or by voice.

[0558] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[0559] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[0560] Emotion recognition and data insertion, optimization

[0561] 1. The user provides input, including emotional expressions, via voice or text.

[0562] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[0563] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[0564] 4. The user says, "Insert a graph of sales data."

[0565] 5. The server retrieves the sales data from the database and converts it into a graph.

[0566] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[0567] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[0568] Prompt Sentence Examples

[0569] "I would like to register as a user. What is the process?"

[0570] "What are the steps to creating promotional materials for a new product?"

[0571] "How can I convert sales data into a graph and insert it into a presentation?"

[0572] "Show us how to optimize the overall structure of your materials based on emotions."

[0573] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0574] Step 1:

[0575] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[0576] The input contents are the user's name, email address, and password.

[0577] The terminal transmits the input information to the server.

[0578] The server stores the received information in a database, with the password hashed for security purposes.

[0579] Input: User information (name, email address, password)

[0580] Output: User information stored in the database

[0581] Step 2:

[0582] The user enters their email address and password on the login screen.

[0583] The input contents are the user's email address and password.

[0584] The terminal sends the input information to the server.

[0585] The server searches the database for the corresponding record and checks whether the entered information matches the hashed password.

[0586] When the authentication is successful, the server starts a session and sends a message of successful authentication to the terminal.

[0587] Input: Login information (email address, password)

[0588] Output: Session start message

[0589] Step 3:

[0590] The user speaks into the device and says, "I want to create a presentation," or inputs text.

[0591] The input may be a voice command or a text command.

[0592] The device converts the voice into text format and sends it to the server.

[0593] The server parses the input using a natural language processing engine.

[0594] Input: Voice or text instructions

[0595] Output: Analysis results (user's intentions and goals)

[0596] Step 4:

[0597] The server asks the user, "What is the purpose of the document?"

[0598] The terminal presents the question to the user.

[0599] When the user answers the question about the purpose and content, the device sends the answer to the server.

[0600] Input: User's answer (purpose of the material)

[0601] Output: Analysis results (information necessary for document creation)

[0602] Step 5:

[0603] The server generates an initial proposal for the material structure based on the answers.

[0604] The composition suggestions are generated using a natural language generation engine (e.g., a generative AI model).

[0605] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0606] Input: Analysis results (purpose and content of the data)

[0607] Output: Initial document structure proposal

[0608] Step 6:

[0609] The user provides feedback via voice or text.

[0610] The input is the user's feedback.

[0611] The server uses an emotion engine to recognize the user's emotions in real time.

[0612] The server adjusts the proposed document structure based on the feedback and the results of sentiment analysis.

[0613] Input: Feedback content, sentiment analysis results

[0614] Output: Adjusted document structure proposal

[0615] Step 7:

[0616] The user indicates, "I want to insert a graph of sales data."

[0617] The input contents are specific instructions from the user.

[0618] The terminal sends the instruction to the server.

[0619] The server accesses the database or external data source to retrieve the required data.

[0620] The server converts the acquired data into graph or table format.

[0621] The graph generated by the server is inserted into the document and displayed on the terminal.

[0622] Input: Data insertion instructions

[0623] Output: Inserted data (graphs and tables)

[0624] Step 8:

[0625] The server analyzes the overall structure and story flow of the material and identifies weaknesses and areas for improvement.

[0626] The server optimizes the content of the feedback based on the recognized emotion.

[0627] The server determines areas for improvement and presents suggestions to the user.

[0628] Input: Data structure, sentiment analysis results

[0629] Output: Optimized materials and recommendations

[0630] (Application example 2)

[0631] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0632] Current document creation systems have difficulty recognizing user emotions in real time and appropriately reflecting feedback, which results in users being unable to efficiently create high-quality documents. Similar problems exist in manufacturing sites when creating operator instructions, particularly due to a lack of emotion-based optimization, which is labor-intensive and time-consuming. This leads to issues such as reduced manufacturing efficiency and a lack of consistent quality.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0634] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, means for recognizing the user's emotions in real time and adjusting the material composition plan, and means for acquiring data and diagrams related to the manufacturing process and automatically inserting them into instructions, thereby enabling users to efficiently create high-quality materials and manufacturing instructions.

[0635] A "user" is an entity that uses the system to create materials and instructions.

[0636] A "material creation request" is a request for material creation made by a user to the system in voice or text format.

[0637] A "natural language processing engine" is a technology that analyzes a user's voice or text input and understands their intent.

[0638] The "initial document structure proposal" is a proposal for the initial structure of the document that is automatically generated by the system based on the user's input.

[0639] "Feedback" refers to opinions or requests for corrections made by the user regarding the configuration plan or data provided by the system.

[0640] "Recognizing emotions in real time" means instantly analyzing and determining the user's emotional state from input voice or text.

[0641] "Adjusting the proposed document structure" means optimizing the structure of the document based on emotions recognized in real time.

[0642] "Manufacturing process data" refers to specific data or information related to the manufacturing process.

[0643] "Automatically insert into instructions" means that the system automatically incorporates the acquired data into the instructions.

[0644] "Instructions" are documents that describe the steps and information required for an operator to follow a manufacturing process.

[0645] "Technology used" refers to the specific technology or method used in the invention.

[0646] The system embodying the present invention assists users in efficiently creating materials and manufacturing instructions. A specific embodiment of the system is described below.

[0647] First, a user inputs a document creation request in voice or text format using smart glasses, a smartphone, a head-mounted display, or a robot. The server receives this input and analyzes the user's intent using a natural language processing engine (e.g., spaCy). Based on the analysis results, an initial document structure proposal is automatically generated and presented to the user for feedback.

[0648] Next, an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) is activated to recognize emotions in real time from the user's voice and text input. This allows the proposed document structure to be adjusted based on the user's emotional state. For example, if the user is feeling impatient or frustrated, the proposed structure may be simplified or additional support may be provided.

[0649] During the manufacturing process, operators give instructions in voice or text format. These instructions are also analyzed by the server, and data and diagrams related to the manufacturing process are retrieved from a database (e.g., MongoDB) and automatically inserted into the instructions. This data is also converted into graphs and tables using libraries such as D3.js.

[0650] Furthermore, the server analyzes the overall structure and story flow of the materials, identifies weaknesses and areas for improvement, and presents suggestions to the user. This automatically optimizes the entire materials and instructions. The server uses a content analysis algorithm (e.g., the BERT model) to provide optimal suggestions for the user.

[0651] As a concrete example, consider the case where an operator gives a voice command such as, "I would like to create manufacturing process instructions for a new engine part." In this case, the system converts the voice into text format and analyzes it using a natural language processing engine. The system then asks, "What are the main contents of the manufacturing process?" If the operator responds with, "Assembly, testing, and shipping," an initial draft of the instruction structure is generated. If the system senses impatience in the operator's voice, it will suggest simplifying the process.

[0652] A concrete example of a prompt sentence is as follows:

[0653] “An operator says, ‘I want to create instructions for manufacturing an engine part.’ The system then asks, ‘What are the main components of the manufacturing process?’ The answer is, ‘Assembly, testing, and shipping.’ If the operator sounds impatient, how does the system adjust the proposed configuration?”

[0654] Such a system enables users and operators to efficiently create high-quality documents and manufacturing instructions.

[0655] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0656] Step 1:

[0657] The user uses smart glasses, a smartphone, a head-mounted display, or a robot to input a request for document creation in voice or text format. The input request is received by the device, and if it is in text format, it is converted to text using a speech recognition API (e.g., Google Speech-to-Text). Input: Voice / text request, Output: Text format request.

[0658] Step 2:

[0659] The device sends the received text request to the server. The server uses a natural language processing engine (e.g., spaCy) to analyze the request and understand the user's intent. Input: Text request, Output: Analyzed user intent.

[0660] Step 3:

[0661] The server automatically generates an initial document structure proposal based on the analysis results. The structure proposal is generated using a template engine (e.g., Jinja2). The generated structure proposal is sent to the terminal and the user is asked for feedback. Input: Analyzed user intent, Output: Initial document structure proposal.

[0662] Step 4:

[0663] The user inputs feedback in voice or text format, and the device receives it. Similarly, if the voice is input, it is converted into text format using a speech recognition API. Input: Voice / Text Feedback, Output: Text Feedback.

[0664] Step 5:

[0665] The device sends the received feedback to the server, which uses an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) to recognize the user's emotions in real time. Input: Text feedback, Output: Recognized emotions.

[0666] Step 6:

[0667] The server adjusts the proposed document composition based on the recognized emotion. If the emotion is impatience or dissatisfaction, the composition may be simplified or additional support may be provided. The adjusted composition is sent to the terminal and presented to the user. Input: Recognized emotion, Output: Adjusted document composition.

[0668] Step 7:

[0669] The user requests the insertion of data or diagrams related to the manufacturing process in voice or text format. The device receives the request and sends it to the server. The server accesses a database (e.g., MongoDB) or external data source to obtain the required data, and converts the data into a graph or table format using a library such as D3.js. Input: Data insertion request in text format, Output: Data in graph or table format.

[0670] Step 8:

[0671] The server automatically inserts the acquired and converted data into the instruction sheet. The instruction sheet is dynamically updated by the template engine, and after creation, it is sent to the terminal and presented to the user. Input: Data in graph or table format, Output: Updated instruction sheet.

[0672] Step 9:

[0673] The server analyzes the overall structure and story flow of the document, identifies weaknesses and areas for improvement, and presents suggestions to the user. Optimal improvement suggestions are generated using a content analysis algorithm (e.g., the BERT model). Input: Overall structure of the document, Output: Improvement suggestions.

[0674] Step 10:

[0675] The user receives suggestions from the server and confirms the final materials and instructions. If there is any final feedback, it is similarly entered by voice or text and final adjustments are made. Input: User's final confirmation and feedback, Output: Final confirmed materials or instructions.

[0676] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0677] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0678] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0679] [Second embodiment]

[0680] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0681] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0682] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0683] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0684] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0685] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0686] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0687] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0688] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0689] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0690] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0691] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0692] This invention relates to an AI system that assists users in creating documents efficiently. The system automatically creates document structure proposals and data visualizations based on user input in voice or text format. The system's program processing and specific examples are shown below.

[0693] System Overview

[0694] 1. User Registration

[0695] The user registers and enters the required information such as name, email address, and password via the terminal.

[0696] The server receives this information and stores it in a database.

[0697] 2. User Authentication

[0698] The user enters their email address and password on the login screen.

[0699] The server authenticates this and, if correct, starts the session.

[0700] 3. Start preparing materials

[0701] The user speaks or writes to the device, saying, "I want to create presentation materials."

[0702] The server parses the instructions using a natural language processing engine.

[0703] 4. Creating a draft of the materials

[0704] The server asks the user, "What is the purpose of the material?"

[0705] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[0706] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0707] 5. Inserting data and figures

[0708] The user indicates, "I want to insert a graph of sales data."

[0709] The server accesses the database or external data source to retrieve the required data.

[0710] Convert the acquired data into graph or table format and insert it into the document.

[0711] 6. Optimize your materials

[0712] The server analyzes the overall structure of the material and the flow of the story.

[0713] Determine weaknesses and areas for improvement and provide suggestions to users.

[0714] The user checks the suggestions, and the server adjusts to create the most suitable materials.

[0715] Specific examples

[0716] User Registration and Login

[0717] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[0718] 2. The device sends the input information to the server.

[0719] 3. The server receives the information and stores it in a database.

[0720] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[0721] 5. The server authenticates this and, if successful, starts the session.

[0722] From the start of document creation to the creation of a structure plan

[0723] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[0724] 2. The device converts the voice into text and sends it to the server.

[0725] 3. The server parses the instructions using a natural language processing engine.

[0726] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[0727] 5. The device presents the question to the user on the screen or by voice.

[0728] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[0729] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[0730] Inserting and optimizing data

[0731] 1. The user says, "Insert a graph of sales data."

[0732] 2. The server retrieves the sales data from the database and converts it into a graph.

[0733] 3. The graph generated by the server is inserted into the document and displayed on the terminal.

[0734] 4. The server checks the overall structure and story of the document and suggests improvements to the user.

[0735] 5. The user reviews and approves the proposal, and the final document is completed.

[0736] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials, saving a lot of time and effort.

[0737] The processing flow will be explained below.

[0738] User Registration and Authentication

[0739] User Registration

[0740] Step 1:

[0741] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[0742] Step 2:

[0743] The terminal sends the input information to the server.

[0744] Step 3:

[0745] The server validates the information it receives to ensure it is in the proper format.

[0746] Step 4:

[0747] The server stores the information in a database.

[0748] Step 5:

[0749] The server generates a registration completion message and sends it to the terminal.

[0750] Step 6:

[0751] The terminal displays a completion message.

[0752] User Authentication

[0753] Step 1:

[0754] The user enters their email address and password on the login screen and clicks the "Login" button.

[0755] Step 2:

[0756] The device sends the authentication information to the server.

[0757] Step 3:

[0758] The server will look up the database and match it with the information you entered.

[0759] Step 4:

[0760] If the server finds the match successful, it initiates the session and generates an authentication success message.

[0761] Step 5:

[0762] The server sends an authentication success message to the terminal.

[0763] Step 6:

[0764] The device will transition to the home screen and display a welcome message.

[0765] Conversational document creation

[0766] Start creating materials

[0767] Step 1:

[0768] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[0769] Step 2:

[0770] The server launches a natural language processing engine to analyze the user's intent.

[0771] Step 3:

[0772] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[0773] Step 4:

[0774] Your device will display or read the question aloud.

[0775] User Input

[0776] Step 1:

[0777] The user responds to the terminal with information about the purpose and content of the material.

[0778] Step 2:

[0779] The terminal sends the answer to the server.

[0780] Step 3:

[0781] The server analyzes the responses and generates an initial draft of the material structure.

[0782] Step 4:

[0783] The server sends the proposed configuration to the terminal and asks the user for feedback.

[0784] Step 5:

[0785] The device displays the configuration plan.

[0786] feedback

[0787] Step 1:

[0788] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[0789] Step 2:

[0790] The terminal sends the instruction to the server.

[0791] Step 3:

[0792] The server recreates the draft document structure based on the instructions.

[0793] Step 4:

[0794] The server sends the updated configuration plan to the terminal.

[0795] Step 5:

[0796] The device displays the updated configuration.

[0797] Automatic data and diagram creation

[0798] Requesting data insertion

[0799] Step 1:

[0800] The user instructs the terminal to "insert a graph of sales data."

[0801] Step 2:

[0802] The terminal sends the instruction to the server.

[0803] Step 3:

[0804] The server accesses the database or external data source to retrieve the required data.

[0805] Step 4:

[0806] The server converts the retrieved data into an appropriate graph format.

[0807] Step 5:

[0808] The server sends the generated graph to the terminal.

[0809] Step 6:

[0810] The terminal displays the graph.

[0811] Data confirmation

[0812] Step 1:

[0813] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[0814] Step 2:

[0815] The terminal sends the instruction to the server.

[0816] Step 3:

[0817] The server regenerates the graph as needed and sends it to the device.

[0818] Step 4:

[0819] The terminal will display the regenerated graph.

[0820] Checking the direction of the material and the story

[0821] Check the story

[0822] Step 1:

[0823] The server analyzes the overall structure of the material and the flow of the story.

[0824] Step 2:

[0825] The server determines weak areas and areas for improvement and generates suggestions.

[0826] Step 3:

[0827] The server sends the proposal to the device.

[0828] Step 4:

[0829] The device will display suggestions.

[0830] User Verification

[0831] Step 1:

[0832] The user checks the proposal and inputs approval or correction instructions into the terminal.

[0833] Step 2:

[0834] The terminal sends the instruction to the server.

[0835] Step 3:

[0836] The server optimizes the materials based on the instructions.

[0837] Step 4:

[0838] The server sends the optimized material to the terminal.

[0839] Step 5:

[0840] The device displays optimized materials.

[0841] Optimization and concept review

[0842] Final Check

[0843] Step 1:

[0844] The server will thoroughly review the completed document and report any errors or improvements.

[0845] Step 2:

[0846] The server sends the report to the device.

[0847] Step 3:

[0848] The terminal displays the report.

[0849] Concept reconfirmation

[0850] Step 1:

[0851] The user initiates a dialogue with the terminal to reaffirm the concept or message.

[0852] Step 2:

[0853] The terminal sends the corresponding content to the server.

[0854] Step 3:

[0855] The server interacts with the user to reconfirm the concepts and messages of the materials.

[0856] Step 4:

[0857] The server will fine-tune the material as needed.

[0858] Step 5:

[0859] The server sends the final materials to the terminal.

[0860] Step 6:

[0861] The terminal displays the completed document.

[0862] Example 1

[0863] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0864] In conventional document creation systems, users had to manually plan the document structure, collect data, visualize it, and finally complete the document, which required a great deal of time and effort. During this process, it was difficult for users to maintain consistency in the document, and specialized knowledge was often required for data visualization and optimization. Therefore, a system that supports efficient and effective document creation was needed.

[0865] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0866] In this invention, the server includes means for receiving a user's request for creating a document in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for the user to register and enter authentication information via a terminal, which the server stores in a database, means for the user to enter an email address and password on a login screen, which the server authenticates and starts a session, means for automatically inserting data and figures specified by the user into the generated document structure proposal, which the server acquires, visualizes, and integrates into the document, and means for analyzing the overall structure and story flow of the document, determining weaknesses and areas for improvement, and presenting suggestions to the user. This enables users to efficiently create high-quality documents quickly and reduce the significant time and effort required in the document creation process.

[0867] A "material creation request" is a request for the creation of a material that a user submits to the system in voice or text format.

[0868] A "natural language processing engine" is an artificial intelligence technology that analyzes a user's voice and text instructions and understands their intentions.

[0869] The "initial document structure proposal" is an initial plan or outline of the document that the system automatically generates based on the user's instructions.

[0870] "Feedback" refers to evaluations and suggestions for improvement provided by users regarding the initial draft of the material structure.

[0871] "New registration" is the process in which a user enters personal information and authentication information for the first time to create an account in order to use the system.

[0872] "Authentication information" refers to information such as an email address and password used to identify a user.

[0873] "Database" means a system for efficiently and safely storing and managing user information and other necessary data.

[0874] "Logging in" is the process by which a user accesses a system using their credentials.

[0875] A "session" refers to the period of a series of operations and activities while a user is logged in to a system, and is a management mechanism for maintaining that state.

[0876] "Data visualization" is the process of converting acquired data into a form that is easy for users to understand, such as a graph or table.

[0877] "Overall structure of a document" is the process of organizing and arranging the content of the main parts and paragraphs of a document.

[0878] "Story flow" refers to the order and logical development of information within a document.

[0879] "Weaknesses and areas for improvement" refers to areas within the document that are insufficient or need to be improved.

[0880] "Suggestions" are advice and corrections for improvement that the system presents to the user.

[0881] "Optimization" is the process of adjusting materials to make them more effective and of higher quality.

[0882] MODE FOR CARRYING OUT THE INVENTION

[0883] This invention relates to an AI system that assists users in efficiently creating materials and obtaining high-quality output. This system receives instructions from the user in the form of voice or text, and automatically creates a document structure proposal and visualizes data. Specifically, the system functions with the server, terminal, and user playing their respective roles. The details are explained below.

[0884] Hardware and software used

[0885] The following hardware and software are used to operate this system.

[0886] Hardware

[0887] PC or smartphone

[0888] software

[0889] Web browser

[0890] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0891] Database management system (e.g. MySQL)

[0892] Speech recognition engine (e.g. Google Cloud Speech-to-Text API)

[0893] Data visualization libraries (e.g., Matplotlib)

[0894] Specific explanation of the system's processing

[0895] User Registration

[0896] The user enters their name, email address, and password on the new registration screen in their web browser. The device sends this data to the server. The server verifies the received data and saves it in a database. Once the data is saved, the server sends a registration completion message to the device, which the device displays to the user.

[0897] User Authentication

[0898] The user enters their email address and password on the login screen. The device sends the data to the server, which then queries the database for authentication. If authentication is successful, the server generates a session ID and sends it to the device. The device then displays a message indicating successful authentication.

[0899] Start creating materials

[0900] The user speaks to the device, saying, "I want to create presentation materials." The device converts this speech into text format and sends it to the server. The server analyzes it using a natural language processing engine, generates a question, "What is the purpose of the materials?" and sends it to the device. The device then displays this question to the user.

[0901] Creating a draft of the materials

[0902] The user responds to the terminal by saying, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server then analyzes the response again using a natural language processing engine and automatically generates an initial proposal for the composition of the materials. This proposal is then sent to the terminal, which then requests feedback from the user.

[0903] Inserting data and images

[0904] The user instructs the terminal to "insert a graph of sales data." The terminal sends this instruction to the server, which retrieves the required data from a database or external data source. The retrieved data is converted into a graph using a data visualization library and inserted into the document. The final draft of the document is sent to the terminal and displayed to the user.

[0905] Material optimization

[0906] The server uses a generative AI model to analyze the overall structure and story flow of the document. It identifies weaknesses and areas for improvement and sends suggestions to the user. The user reviews the suggestions and makes any necessary corrections. The server optimizes the document based on this feedback and sends the final draft to the device for the user to review and download.

[0907] Examples of prompt statements

[0908] "Please create promotional materials for our new product. The target audience is women in their 20s, and we would like to highlight the product's features."

[0909] "Please graph the following sales data and add it to your report."

[0910] As described above, the present invention is a system that supports users in efficiently creating high-quality materials, and reduces a great deal of time and effort in the process of creating materials.

[0911] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0912] Step 1:

[0913] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device receives this information and sends it to the server. The server registers this information in the database. This gives the user an account to access the system.

[0914] Step 2:

[0915] The user enters their email address and password on the login screen and clicks the "Login" button. The device receives the input information and sends it to the server. The server queries the database to confirm that the email address and password match, and if authentication is successful, generates a session ID and sends it to the device. The device displays a message to the user indicating successful authentication.

[0916] Step 3:

[0917] The user speaks to the device, saying, "I want to create presentation materials." The device converts the voice data into text and sends the text data to the server. The server uses a natural language processing engine to analyze the text data and identify the instruction. The server generates a question, "What is the purpose of the materials?" and sends it to the device.

[0918] Step 4:

[0919] The terminal displays a question from the server to the user. The user responds by typing, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server uses a natural language processing engine to analyze the user's response and automatically generates a proposal for the composition of the materials. The server then sends the generated proposal to the terminal and asks the user for feedback.

[0920] Step 5:

[0921] The user gives the instruction to "insert a graph of sales data." The terminal sends this instruction to the server. The server accesses the database or external data source to obtain sales data. Based on the obtained data, it generates a graph using a data visualization library. The server inserts the generated graph into the document and sends the result to the terminal. The terminal displays the updated document to the user.

[0922] Step 6:

[0923] The server uses a generative AI model to analyze the overall structure and story flow of the document. It determines weaknesses and areas for improvement and presents suggestions to the user. These suggestions are sent to the device, where the user can review and revise them. Based on the user's feedback, the server optimizes the document and sends the final draft to the device. The device displays the final document to the user and provides a download link if necessary.

[0924] (Application example 1)

[0925] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0926] Currently, production reports in factories are often created manually, which is time-consuming and labor-intensive. Furthermore, data aggregation and graphing require specialized knowledge and are inefficient. Furthermore, optimizing the overall structure and flow of data is time-consuming, which can lead to inconsistent quality in the reports created. Therefore, there is a need for a method that reduces this labor and time and allows for the rapid creation of high-quality production reports.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0928] In this invention, the server includes means for receiving a document creation request from a user in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for accessing a factory production database to acquire production data, and means for converting the acquired production data into a graph format and inserting it into the document, thereby enabling the user to efficiently and automatically create high-quality production reports.

[0929] A "user" is a person who uses the system to make a document creation request.

[0930] A "material creation request" is an instruction from a user to create a material in voice or text format.

[0931] The "audio format" is a format in which the user inputs using spoken language.

[0932] The "text format" is a format in which the user inputs text such as letters and symbols.

[0933] A "natural language processing engine" is a technology that analyzes voice and text data and understands its intent.

[0934] A "document structure plan" is a plan that shows the initial structure and content layout of the entire document.

[0935] "Feedback" refers to opinions and ratings provided by users, and is information that the system uses to make further improvements.

[0936] A "database" is a storage medium for systematically organizing and storing information.

[0937] An "external data source" is a data source that exists outside the system.

[0938] A "production database" is a storage medium that collects various data related to production in a factory.

[0939] A "graphic format" is a chart format for visually representing numerical data.

[0940] "Natural language analysis" is the process by which a system understands and analyzes human language.

[0941] A "generative AI model" is an artificial intelligence model that automatically generates materials and content based on user intent and data.

[0942] A "prompt sentence" is an instruction sentence that is input to a generative AI model.

[0943] The present invention relates to an AI system that enables users to efficiently create documents such as production reports. This system receives instructions from the user in voice or text format, and automatically creates document structure proposals and data visualizations based on those instructions. Specific examples of how this system can be realized are shown below.

[0944] Overall system configuration

[0945] 1. User Registration

[0946] The user registers and enters the required information such as name, email address, and password.

[0947] The server completes the user registration by storing this information in a database.

[0948] 2. User Authentication

[0949] The user enters their email address and password on the login screen and performs authentication.

[0950] The server starts a session based on the entered information.

[0951] 3. Start preparing materials

[0952] The user issues a voice or text command to the device (smartphone or factory robot terminal) stating, "I want to create a production report."

[0953] The server parses the instructions using a natural language processing engine.

[0954] 4. Creating a draft of the materials

[0955] The server asks the user, "What is the purpose of the report?" and collects the user's response.

[0956] Based on the collected information, the server automatically generates an initial draft of the material structure and asks the user for feedback.

[0957] 5. Inserting data and graphs

[0958] The user instructs, "I want to insert a graph of production volume."

[0959] The server accesses the factory's production database and retrieves the necessary data.

[0960] The acquired data is converted into a graph format and inserted into the document.

[0961] 6. Optimize your materials

[0962] The server analyzes the overall structure of the document and the flow of the story, and suggests areas for improvement to the user.

[0963] Based on instructions from the user, the server optimizes the materials.

[0964] Technology and hardware used

[0965] 1. Natural Language Processing Engine

[0966] The system uses natural language processing to analyze the user's voice and text instructions, specifically using a natural language processing engine such as spaCy.

[0967] 2. Database Access

[0968] The server accesses the factory's production database and retrieves the necessary data, using database manipulation techniques such as SQL.

[0969] 3. Graph Generation

[0970] To visualize the acquired data, we use the pandas and matplotlib libraries to generate graphs.

[0971] Examples and prompts

[0972] Here is a concrete example:

[0973] Prompt: Create a production report for your factory and generate graphs based on the past two weeks' production data. Also, briefly summarize your future production plans.

[0974] In this way, users can efficiently create high-quality production reports. Because various data is automatically acquired and visualized, it is possible to significantly reduce the amount of time and effort required.

[0975] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0976] Step 1:

[0977] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device sends the input information to the server. The server saves the sent information in a database. The input data is the name, email address, and password, and the output is the saved state in the database. This creates a user account.

[0978] Step 2:

[0979] The user enters their email address and password on the login screen and is authenticated. The device sends the entered information to the server. The server compares it with the information in the database and starts a session if it matches. The input data is the email address and password, and the output is the session start status. This completes user authentication.

[0980] Step 3:

[0981] The user issues a voice or text command to the terminal, saying, "I want to create a production report." The terminal converts the voice into text format and sends the data to the server. The server uses a natural language processing engine to analyze the command. The input data is the voice or text command, and the output is the analysis result. This allows the user's intent to be understood.

[0982] Step 4:

[0983] The server asks the user, "What is the purpose of the report?" and communicates the question to the user via the terminal. The user answers the purpose and content using the terminal. The terminal sends the answer to the server. The server automatically generates an initial draft of the document structure based on the collected information, sends it to the terminal, and asks the user for feedback. The input data is the user's answer, and the output is the draft of the document structure. This generates the initial draft of the document structure.

[0984] Step 5:

[0985] The user gives a text command saying, "I want to insert a production volume graph." The terminal sends this command to the server. The server accesses the factory's production database and retrieves the data for the specified range. The retrieved data is converted into a data frame using pandas, and a graph is generated using matplotlib. The generated graph is inserted into the document and displayed on the terminal. The input data is the specified production data range, and the output is the generation and insertion of a graph. This achieves visualization of the production data.

[0986] Step 6:

[0987] The server analyzes the overall structure and story flow of the document and determines weaknesses and areas for improvement. Based on the analysis results, it presents suggestions to the user. The user reviews the suggestions and gives instructions for approval or revision. The server optimizes the document based on the user's instructions. The input data is the document to be analyzed, and the output is improvement suggestions and the optimized document. This results in the generation of a final, high-quality report.

[0988] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0989] This invention relates to an AI system that assists users in creating documents efficiently, and in particular provides a system equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. It can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions. The system's program processing and specific examples are shown below.

[0990] System Overview

[0991] 1. User Registration

[0992] The user registers and enters the required information such as name, email address, and password via the terminal.

[0993] The server receives this information and stores it in a database.

[0994] 2. User Authentication

[0995] The user enters their email address and password on the login screen.

[0996] The server authenticates this and, if correct, starts the session.

[0997] 3. Start preparing materials

[0998] The user speaks or writes to the device, saying, "I want to create presentation materials."

[0999] The server parses the instructions using a natural language processing engine.

[1000] 4. Creating a draft of the materials

[1001] The server asks the user, "What is the purpose of the material?"

[1002] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[1003] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1004] 5. Emotion recognition

[1005] When a user inputs text or voice, the server uses an emotion engine to analyze the user's emotions in real time.

[1006] Based on the recognized emotions, the server adjusts the proposed material composition.

[1007] 6. Inserting data and figures

[1008] The user indicates, "I want to insert a graph of sales data."

[1009] The server accesses the database or external data source to retrieve the required data.

[1010] Convert the acquired data into graph or table format and insert it into the document.

[1011] 7. Emotion-Based Optimization

[1012] The server analyzes the overall structure of the material and the flow of the story, optimizing the content of the feedback according to the recognized emotions.

[1013] The server determines areas for improvement and presents suggestions to the user.

[1014] Specific examples

[1015] User Registration and Login

[1016] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1017] 2. The device sends the input information to the server.

[1018] 3. The server receives the information and stores it in a database.

[1019] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[1020] 5. The server authenticates this and, if successful, starts the session.

[1021] From the start of document creation to the creation of a structure plan

[1022] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[1023] 2. The device converts the voice into text and sends it to the server.

[1024] 3. The server parses the instructions using a natural language processing engine.

[1025] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[1026] 5. The device presents the question to the user on the screen or by voice.

[1027] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[1028] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[1029] Emotion recognition and data insertion, optimization

[1030] 1. The user provides input, including emotional expressions, via voice or text.

[1031] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[1032] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[1033] 4. The user says, "Insert a graph of sales data."

[1034] 5. The server retrieves the sales data from the database and converts it into a graph.

[1035] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[1036] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[1037] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials based on emotion recognition, saving users a lot of time and effort.

[1038] The processing flow will be explained below.

[1039] User Registration and Authentication

[1040] User Registration

[1041] Step 1:

[1042] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[1043] Step 2:

[1044] The terminal sends the input information to the server.

[1045] Step 3:

[1046] The server validates the information it receives to ensure it is in the proper format.

[1047] Step 4:

[1048] The server stores the information in a database.

[1049] Step 5:

[1050] The server generates a registration completion message and sends it to the terminal.

[1051] Step 6:

[1052] The terminal displays a completion message.

[1053] User Authentication

[1054] Step 1:

[1055] The user enters their email address and password on the login screen and clicks the "Login" button.

[1056] Step 2:

[1057] The device sends the authentication information to the server.

[1058] Step 3:

[1059] The server will look up the database and match it with the information you entered.

[1060] Step 4:

[1061] If the server finds the match successful, it initiates the session and generates an authentication success message.

[1062] Step 5:

[1063] The server sends an authentication success message to the terminal.

[1064] Step 6:

[1065] The device will transition to the home screen and display a welcome message.

[1066] Conversational document creation

[1067] Start creating materials

[1068] Step 1:

[1069] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[1070] Step 2:

[1071] The server launches a natural language processing engine to analyze the user's intent.

[1072] Step 3:

[1073] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[1074] Step 4:

[1075] Your device will display or read the question aloud.

[1076] User Input

[1077] Step 1:

[1078] The user responds to the terminal with information about the purpose and content of the material.

[1079] Step 2:

[1080] The terminal sends the answer to the server.

[1081] Step 3:

[1082] The server analyzes the responses and generates an initial draft of the material structure.

[1083] Step 4:

[1084] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1085] Step 5:

[1086] The device displays the configuration plan.

[1087] feedback

[1088] Step 1:

[1089] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[1090] Step 2:

[1091] The terminal sends the instruction to the server.

[1092] Step 3:

[1093] The server recreates the draft document structure based on the instructions.

[1094] Step 4:

[1095] The server sends the updated configuration plan to the terminal.

[1096] Step 5:

[1097] The device displays the updated configuration.

[1098] Automated data and diagram creation using emotion recognition

[1099] Emotion recognition and data insertion requests

[1100] Step 1:

[1101] When a user creates a document using voice or text, the device sends the input to the emotion engine.

[1102] Step 2:

[1103] The emotion engine analyzes the user's emotions in real time and sends the results to the server.

[1104] Step 3:

[1105] The server uses the emotion recognition results to adjust the composition of the material.

[1106] Step 4:

[1107] The user instructs the terminal to "insert a graph of sales data."

[1108] Step 5:

[1109] The terminal sends the instruction to the server.

[1110] Step 6:

[1111] The server accesses the database or external data source to retrieve the required data.

[1112] Step 7:

[1113] The server converts the retrieved data into an appropriate graph format.

[1114] Step 8:

[1115] The server sends the generated graph to the terminal.

[1116] Step 9:

[1117] The terminal displays the graph.

[1118] Emotion Recognition and Data Verification

[1119] Step 1:

[1120] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[1121] Step 2:

[1122] The terminal sends the instruction to the server.

[1123] Step 3:

[1124] The server regenerates the graph as needed and sends it to the device.

[1125] Step 4:

[1126] The terminal will display the regenerated graph.

[1127] Checking the direction of the material and the story

[1128] Check the story

[1129] Step 1:

[1130] The server analyzes the overall structure of the material and the flow of the story.

[1131] Step 2:

[1132] The server determines weak areas and areas for improvement and generates suggestions.

[1133] Step 3:

[1134] The server sends the proposal to the device.

[1135] Step 4:

[1136] The device will display suggestions.

[1137] User Verification

[1138] Step 1:

[1139] The user checks the proposal and inputs approval or correction instructions into the terminal.

[1140] Step 2:

[1141] The terminal sends the instruction to the server.

[1142] Step 3:

[1143] The server optimizes the materials based on the instructions.

[1144] Step 4:

[1145] The server sends the optimized material to the terminal.

[1146] Step 5:

[1147] The device displays optimized materials.

[1148] Optimization and concept review

[1149] Final Check

[1150] Step 1:

[1151] The server will thoroughly review the completed document and report any errors or improvements.

[1152] Step 2:

[1153] The server sends the report to the device.

[1154] Step 3:

[1155] The terminal displays the report.

[1156] Concept reconfirmation

[1157] Step 1:

[1158] The user initiates a dialogue at the terminal to reaffirm the concept or message.

[1159] Step 2:

[1160] The terminal sends the corresponding content to the server.

[1161] Step 3:

[1162] The server interacts with the user to reconfirm the concepts and messages of the materials.

[1163] Step 4:

[1164] The server will fine-tune the material as needed.

[1165] Step 5:

[1166] The server sends the final materials to the terminal.

[1167] Step 6:

[1168] The terminal displays the completed document.

[1169] Example 2

[1170] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1171] Conventional document creation systems require users to manually design document structures and convert data into graphs and tables, which requires a great deal of time and effort. Furthermore, it is difficult to create detailed documents that reflect the user's emotions and intentions. The present invention aims to solve these problems and provide a system that supports users in creating high-quality documents efficiently and easily.

[1172] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1173] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, emotion recognition means for recognizing the user's emotions in real time, and means for automatically adjusting the material composition plan based on the recognized emotions. This makes it possible to automatically generate and adjust an appropriate material composition that reflects emotions simply by providing instructions from the user.

[1174] A "document creation request" refers to an instruction or request made by a user to the system to request the creation of a document.

[1175] "Voice or text format" refers to an interface that allows a user to give instructions to the system, and is in the form of voice input or text input.

[1176] A "natural language processing engine" refers to algorithms and software that analyze human language and understand its intent and meaning.

[1177] "Analysis results" refers to the information and data obtained by the natural language processing engine when it analyzes the user's instructions.

[1178] "Initial document structure proposal" refers to the initial structure and content of the document that the system automatically generates based on the analysis results.

[1179] "Feedback" refers to the evaluations, opinions, and correction requests that users provide to the system.

[1180] "Emotion recognition means" refers to algorithms and software for analyzing and understanding a user's emotions in real time.

[1181] "Means for automatically adjusting the proposed document structure" refers to functions or algorithms that automatically change or adjust the structure or content of a document based on recognized emotions.

[1182] A "database" refers to a system for systematically storing and managing information.

[1183] "External data source" refers to a source or service that provides data that exists outside the system.

[1184] "Converting into graph or table format" refers to the process or task of formatting acquired data into a form that is visually easy to understand.

[1185] "Means for inserting into documents" refers to functions and processes for placing the generated graphs and tables in appropriate positions in the documents.

[1186] "The overall structure of the material and the flow of the story" refers to the arrangement of each section or part within the material and ensuring that they have a continuous meaning.

[1187] "Measures for determining weaknesses and areas for improvement" refers to algorithms or processes used to identify gaps or areas in the content or structure of a document that need correction.

[1188] "Means for presenting suggestions" refers to the functions and processes for providing users with specific actions for improvement or change suggestions.

[1189] "Means for automatically optimizing materials" refers to functions and algorithms that automatically adjust and improve the content of materials to the most appropriate form based on user instructions and emotion recognition results.

[1190] This invention relates to an AI system that assists users in creating documents efficiently. The system is equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. The system can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions.

[1191] Hardware and Software Configuration

[1192] server

[1193] Database: A system for systematically storing and managing information

[1194] Natural language processing engine: Software for analyzing human language and understanding intent, such as the Google NLP API

[1195] Emotion recognition engine: Software for recognizing user emotions in real time, such as IBM Watson Tone Analyzer

[1196] Data visualization tools: Software for transforming data into graphs and tables, such as Matplotlib, Pandas DataFrame, etc.

[1197] Terminal

[1198] Speech recognition engine: Software that converts a user's speech into text, such as the Google Speech-to-Text API.

[1199] User interface: An interface for receiving voice commands, text input, and feedback

[1200] Example of implementation

[1201] User Registration and Login

[1202] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1203] 2. The device sends the input information to the server.

[1204] 3. The server receives the information and stores it in a database.

[1205] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[1206] 5. The server authenticates this and, if successful, starts the session.

[1207] From the start of document creation to the creation of a structure plan

[1208] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[1209] 2. The device converts the voice into text and sends it to the server.

[1210] 3. The server parses the instructions using a natural language processing engine.

[1211] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[1212] 5. The device presents the question to the user on the screen or by voice.

[1213] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[1214] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[1215] Emotion recognition and data insertion, optimization

[1216] 1. The user provides input, including emotional expressions, via voice or text.

[1217] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[1218] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[1219] 4. The user says, "Insert a graph of sales data."

[1220] 5. The server retrieves the sales data from the database and converts it into a graph.

[1221] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[1222] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[1223] Prompt Sentence Examples

[1224] "I would like to register as a user. What is the process?"

[1225] "What are the steps to creating promotional materials for a new product?"

[1226] "How can I convert sales data into a graph and insert it into a presentation?"

[1227] "Show us how to optimize the overall structure of your materials based on emotions."

[1228] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1229] Step 1:

[1230] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1231] The input contents are the user's name, email address, and password.

[1232] The terminal transmits the input information to the server.

[1233] The server stores the received information in a database, with the password hashed for security purposes.

[1234] Input: User information (name, email address, password)

[1235] Output: User information stored in the database

[1236] Step 2:

[1237] The user enters their email address and password on the login screen.

[1238] The input contents are the user's email address and password.

[1239] The terminal sends the input information to the server.

[1240] The server searches the database for the corresponding record and checks whether the entered information matches the hashed password.

[1241] When the authentication is successful, the server starts a session and sends a message of successful authentication to the terminal.

[1242] Input: Login information (email address, password)

[1243] Output: Session start message

[1244] Step 3:

[1245] The user speaks into the device and says, "I want to create a presentation," or inputs text.

[1246] The input may be a voice command or a text command.

[1247] The device converts the voice into text format and sends it to the server.

[1248] The server parses the input using a natural language processing engine.

[1249] Input: Voice or text instructions

[1250] Output: Analysis results (user's intentions and goals)

[1251] Step 4:

[1252] The server asks the user, "What is the purpose of the document?"

[1253] The terminal presents the question to the user.

[1254] When the user answers the question about the purpose and content, the device sends the answer to the server.

[1255] Input: User's answer (purpose of the material)

[1256] Output: Analysis results (information necessary for document creation)

[1257] Step 5:

[1258] The server generates an initial proposal for the material structure based on the answers.

[1259] The composition suggestions are generated using a natural language generation engine (e.g., a generative AI model).

[1260] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1261] Input: Analysis results (purpose and content of the data)

[1262] Output: Initial document structure proposal

[1263] Step 6:

[1264] The user provides feedback via voice or text.

[1265] The input is the user's feedback.

[1266] The server uses an emotion engine to recognize the user's emotions in real time.

[1267] The server adjusts the proposed document structure based on the feedback and the results of sentiment analysis.

[1268] Input: Feedback content, sentiment analysis results

[1269] Output: Adjusted document structure proposal

[1270] Step 7:

[1271] The user indicates, "I want to insert a graph of sales data."

[1272] The input contents are specific instructions from the user.

[1273] The terminal sends the instruction to the server.

[1274] The server accesses the database or external data source to retrieve the required data.

[1275] The server converts the acquired data into graph or table format.

[1276] The graph generated by the server is inserted into the document and displayed on the terminal.

[1277] Input: Data insertion instructions

[1278] Output: Inserted data (graphs and tables)

[1279] Step 8:

[1280] The server analyzes the overall structure and story flow of the material and identifies weaknesses and areas for improvement.

[1281] The server optimizes the content of the feedback based on the recognized emotion.

[1282] The server determines areas for improvement and presents suggestions to the user.

[1283] Input: Data structure, sentiment analysis results

[1284] Output: Optimized materials and recommendations

[1285] (Application example 2)

[1286] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1287] Current document creation systems have difficulty recognizing user emotions in real time and appropriately reflecting feedback, which results in users being unable to efficiently create high-quality documents. Similar problems exist in manufacturing sites when creating operator instructions, particularly due to a lack of emotion-based optimization, which is labor-intensive and time-consuming. This leads to issues such as reduced manufacturing efficiency and a lack of consistent quality.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1289] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, means for recognizing the user's emotions in real time and adjusting the material composition plan, and means for acquiring data and diagrams related to the manufacturing process and automatically inserting them into instructions, thereby enabling users to efficiently create high-quality materials and manufacturing instructions.

[1290] A "user" is an entity that uses the system to create materials and instructions.

[1291] A "material creation request" is a request for material creation made by a user to the system in voice or text format.

[1292] A "natural language processing engine" is a technology that analyzes a user's voice or text input and understands their intent.

[1293] The "initial document structure proposal" is a proposal for the initial structure of the document that is automatically generated by the system based on the user's input.

[1294] "Feedback" refers to opinions or requests for corrections made by the user regarding the configuration plan or data provided by the system.

[1295] "Recognizing emotions in real time" means instantly analyzing and determining the user's emotional state from input voice or text.

[1296] "Adjusting the proposed document structure" means optimizing the structure of the document based on emotions recognized in real time.

[1297] "Manufacturing process data" refers to specific data or information related to the manufacturing process.

[1298] "Automatically insert into instructions" means that the system automatically incorporates the acquired data into the instructions.

[1299] "Instructions" are documents that describe the steps and information required for an operator to follow a manufacturing process.

[1300] "Technology used" refers to the specific technology or method used in the invention.

[1301] The system embodying the present invention assists users in efficiently creating materials and manufacturing instructions. A specific embodiment of the system is described below.

[1302] First, a user inputs a document creation request in voice or text format using smart glasses, a smartphone, a head-mounted display, or a robot. The server receives this input and analyzes the user's intent using a natural language processing engine (e.g., spaCy). Based on the analysis results, an initial document structure proposal is automatically generated and presented to the user for feedback.

[1303] Next, an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) is activated to recognize emotions in real time from the user's voice and text input. This allows the proposed document structure to be adjusted based on the user's emotional state. For example, if the user is feeling impatient or frustrated, the proposed structure may be simplified or additional support may be provided.

[1304] During the manufacturing process, operators give instructions in voice or text format. These instructions are also analyzed by the server, and data and diagrams related to the manufacturing process are retrieved from a database (e.g., MongoDB) and automatically inserted into the instructions. This data is also converted into graphs and tables using libraries such as D3.js.

[1305] Furthermore, the server analyzes the overall structure and story flow of the materials, identifies weaknesses and areas for improvement, and presents suggestions to the user. This automatically optimizes the entire materials and instructions. The server uses a content analysis algorithm (e.g., the BERT model) to provide optimal suggestions for the user.

[1306] As a concrete example, consider the case where an operator gives a voice command such as, "I would like to create manufacturing process instructions for a new engine part." In this case, the system converts the voice into text format and analyzes it using a natural language processing engine. The system then asks, "What are the main contents of the manufacturing process?" If the operator responds with, "Assembly, testing, and shipping," an initial draft of the instruction structure is generated. If the system senses impatience in the operator's voice, it will suggest simplifying the process.

[1307] A concrete example of a prompt sentence is as follows:

[1308] “An operator says, ‘I want to create instructions for manufacturing an engine part.’ The system then asks, ‘What are the main components of the manufacturing process?’ The answer is, ‘Assembly, testing, and shipping.’ If the operator sounds impatient, how does the system adjust the proposed configuration?”

[1309] Such a system enables users and operators to efficiently create high-quality documents and manufacturing instructions.

[1310] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1311] Step 1:

[1312] The user uses smart glasses, a smartphone, a head-mounted display, or a robot to input a request for document creation in voice or text format. The input request is received by the device, and if it is in text format, it is converted to text using a speech recognition API (e.g., Google Speech-to-Text). Input: Voice / text request, Output: Text format request.

[1313] Step 2:

[1314] The device sends the received text request to the server. The server uses a natural language processing engine (e.g., spaCy) to analyze the request and understand the user's intent. Input: Text request, Output: Analyzed user intent.

[1315] Step 3:

[1316] The server automatically generates an initial document structure proposal based on the analysis results. The structure proposal is generated using a template engine (e.g., Jinja2). The generated structure proposal is sent to the terminal and the user is asked for feedback. Input: Analyzed user intent, Output: Initial document structure proposal.

[1317] Step 4:

[1318] The user inputs feedback in voice or text format, and the device receives it. Similarly, if the voice is input, it is converted into text format using a speech recognition API. Input: Voice / Text Feedback, Output: Text Feedback.

[1319] Step 5:

[1320] The device sends the received feedback to the server, which uses an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) to recognize the user's emotions in real time. Input: Text feedback, Output: Recognized emotions.

[1321] Step 6:

[1322] The server adjusts the proposed document composition based on the recognized emotion. If the emotion is impatience or dissatisfaction, the composition may be simplified or additional support may be provided. The adjusted composition is sent to the terminal and presented to the user. Input: Recognized emotion, Output: Adjusted document composition.

[1323] Step 7:

[1324] The user requests the insertion of data or diagrams related to the manufacturing process in voice or text format. The device receives the request and sends it to the server. The server accesses a database (e.g., MongoDB) or external data source to obtain the required data, and converts the data into a graph or table format using a library such as D3.js. Input: Data insertion request in text format, Output: Data in graph or table format.

[1325] Step 8:

[1326] The server automatically inserts the acquired and converted data into the instruction sheet. The instruction sheet is dynamically updated by the template engine, and after creation, it is sent to the terminal and presented to the user. Input: Data in graph or table format, Output: Updated instruction sheet.

[1327] Step 9:

[1328] The server analyzes the overall structure and story flow of the document, identifies weaknesses and areas for improvement, and presents suggestions to the user. Optimal improvement suggestions are generated using a content analysis algorithm (e.g., the BERT model). Input: Overall structure of the document, Output: Improvement suggestions.

[1329] Step 10:

[1330] The user receives suggestions from the server and confirms the final materials and instructions. If there is any final feedback, it is similarly entered by voice or text and final adjustments are made. Input: User's final confirmation and feedback, Output: Final confirmed materials or instructions.

[1331] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1332] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1333] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1334] [Third embodiment]

[1335] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1336] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1337] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1338] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1339] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1340] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1341] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1342] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1343] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1344] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1345] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1346] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1347] This invention relates to an AI system that assists users in creating documents efficiently. The system automatically creates document structure proposals and data visualizations based on user input in voice or text format. The system's program processing and specific examples are shown below.

[1348] System Overview

[1349] 1. User Registration

[1350] The user registers and enters the required information such as name, email address, and password via the terminal.

[1351] The server receives this information and stores it in a database.

[1352] 2. User Authentication

[1353] The user enters their email address and password on the login screen.

[1354] The server authenticates this and, if correct, starts the session.

[1355] 3. Start preparing materials

[1356] The user speaks or writes to the device, saying, "I want to create presentation materials."

[1357] The server parses the instructions using a natural language processing engine.

[1358] 4. Creating a draft of the materials

[1359] The server asks the user, "What is the purpose of the material?"

[1360] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[1361] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1362] 5. Inserting data and figures

[1363] The user indicates, "I want to insert a graph of sales data."

[1364] The server accesses the database or external data source to retrieve the required data.

[1365] Convert the acquired data into graph or table format and insert it into the document.

[1366] 6. Optimize your materials

[1367] The server analyzes the overall structure of the material and the flow of the story.

[1368] Determine weaknesses and areas for improvement and provide suggestions to users.

[1369] The user checks the suggestions, and the server adjusts to create the most suitable materials.

[1370] Specific examples

[1371] User Registration and Login

[1372] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1373] 2. The device sends the input information to the server.

[1374] 3. The server receives the information and stores it in a database.

[1375] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[1376] 5. The server authenticates this and, if successful, starts the session.

[1377] From the start of document creation to the creation of a structure plan

[1378] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[1379] 2. The device converts the voice into text and sends it to the server.

[1380] 3. The server parses the instructions using a natural language processing engine.

[1381] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[1382] 5. The device presents the question to the user on the screen or by voice.

[1383] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[1384] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[1385] Inserting and optimizing data

[1386] 1. The user says, "Insert a graph of sales data."

[1387] 2. The server retrieves the sales data from the database and converts it into a graph.

[1388] 3. The graph generated by the server is inserted into the document and displayed on the terminal.

[1389] 4. The server checks the overall structure and story of the document and suggests improvements to the user.

[1390] 5. The user reviews and approves the proposal, and the final document is completed.

[1391] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials, saving a lot of time and effort.

[1392] The processing flow will be explained below.

[1393] User Registration and Authentication

[1394] User Registration

[1395] Step 1:

[1396] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[1397] Step 2:

[1398] The terminal sends the input information to the server.

[1399] Step 3:

[1400] The server validates the information it receives to ensure it is in the proper format.

[1401] Step 4:

[1402] The server stores the information in a database.

[1403] Step 5:

[1404] The server generates a registration completion message and sends it to the terminal.

[1405] Step 6:

[1406] The terminal displays a completion message.

[1407] User Authentication

[1408] Step 1:

[1409] The user enters their email address and password on the login screen and clicks the "Login" button.

[1410] Step 2:

[1411] The device sends the authentication information to the server.

[1412] Step 3:

[1413] The server will look up the database and match it with the information you entered.

[1414] Step 4:

[1415] If the server finds the match successful, it initiates the session and generates an authentication success message.

[1416] Step 5:

[1417] The server sends an authentication success message to the terminal.

[1418] Step 6:

[1419] The device will transition to the home screen and display a welcome message.

[1420] Conversational document creation

[1421] Start creating materials

[1422] Step 1:

[1423] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[1424] Step 2:

[1425] The server launches a natural language processing engine to analyze the user's intent.

[1426] Step 3:

[1427] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[1428] Step 4:

[1429] Your device will display or read the question aloud.

[1430] User Input

[1431] Step 1:

[1432] The user responds to the terminal with information about the purpose and content of the material.

[1433] Step 2:

[1434] The terminal sends the answer to the server.

[1435] Step 3:

[1436] The server analyzes the responses and generates an initial draft of the material structure.

[1437] Step 4:

[1438] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1439] Step 5:

[1440] The device displays the configuration plan.

[1441] feedback

[1442] Step 1:

[1443] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[1444] Step 2:

[1445] The terminal sends the instruction to the server.

[1446] Step 3:

[1447] The server recreates the draft document structure based on the instructions.

[1448] Step 4:

[1449] The server sends the updated configuration plan to the terminal.

[1450] Step 5:

[1451] The device displays the updated configuration.

[1452] Automatic data and diagram creation

[1453] Requesting data insertion

[1454] Step 1:

[1455] The user instructs the terminal to "insert a graph of sales data."

[1456] Step 2:

[1457] The terminal sends the instruction to the server.

[1458] Step 3:

[1459] The server accesses the database or external data source to retrieve the required data.

[1460] Step 4:

[1461] The server converts the retrieved data into an appropriate graph format.

[1462] Step 5:

[1463] The server sends the generated graph to the terminal.

[1464] Step 6:

[1465] The terminal displays the graph.

[1466] Data confirmation

[1467] Step 1:

[1468] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[1469] Step 2:

[1470] The terminal sends the instruction to the server.

[1471] Step 3:

[1472] The server regenerates the graph as needed and sends it to the device.

[1473] Step 4:

[1474] The terminal will display the regenerated graph.

[1475] Checking the direction of the material and the story

[1476] Check the story

[1477] Step 1:

[1478] The server analyzes the overall structure of the material and the flow of the story.

[1479] Step 2:

[1480] The server determines weak areas and areas for improvement and generates suggestions.

[1481] Step 3:

[1482] The server sends the proposal to the device.

[1483] Step 4:

[1484] The device will display suggestions.

[1485] User Verification

[1486] Step 1:

[1487] The user checks the proposal and inputs approval or correction instructions into the terminal.

[1488] Step 2:

[1489] The terminal sends the instruction to the server.

[1490] Step 3:

[1491] The server optimizes the materials based on the instructions.

[1492] Step 4:

[1493] The server sends the optimized material to the terminal.

[1494] Step 5:

[1495] The device displays optimized materials.

[1496] Optimization and concept review

[1497] Final Check

[1498] Step 1:

[1499] The server will thoroughly review the completed document and report any errors or improvements.

[1500] Step 2:

[1501] The server sends the report to the device.

[1502] Step 3:

[1503] The terminal displays the report.

[1504] Concept reconfirmation

[1505] Step 1:

[1506] The user initiates a dialogue with the terminal to reaffirm the concept or message.

[1507] Step 2:

[1508] The terminal sends the corresponding content to the server.

[1509] Step 3:

[1510] The server interacts with the user to reconfirm the concepts and messages of the materials.

[1511] Step 4:

[1512] The server will fine-tune the material as needed.

[1513] Step 5:

[1514] The server sends the final materials to the terminal.

[1515] Step 6:

[1516] The terminal displays the completed document.

[1517] Example 1

[1518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1519] In conventional document creation systems, users had to manually plan the document structure, collect data, visualize it, and finally complete the document, which required a great deal of time and effort. During this process, it was difficult for users to maintain consistency in the document, and specialized knowledge was often required for data visualization and optimization. Therefore, a system that supports efficient and effective document creation was needed.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1521] In this invention, the server includes means for receiving a user's request for creating a document in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for the user to register and enter authentication information via a terminal, which the server stores in a database, means for the user to enter an email address and password on a login screen, which the server authenticates and starts a session, means for automatically inserting data and figures specified by the user into the generated document structure proposal, which the server acquires, visualizes, and integrates into the document, and means for analyzing the overall structure and story flow of the document, determining weaknesses and areas for improvement, and presenting suggestions to the user. This enables users to efficiently create high-quality documents quickly and reduce the significant time and effort required in the document creation process.

[1522] A "material creation request" is a request for the creation of a material that a user submits to the system in voice or text format.

[1523] A "natural language processing engine" is an artificial intelligence technology that analyzes a user's voice and text instructions and understands their intentions.

[1524] The "initial document structure proposal" is an initial plan or outline of the document that the system automatically generates based on the user's instructions.

[1525] "Feedback" refers to evaluations and suggestions for improvement provided by users regarding the initial draft of the material structure.

[1526] "New registration" is the process in which a user enters personal information and authentication information for the first time to create an account in order to use the system.

[1527] "Authentication information" refers to information such as an email address and password used to identify a user.

[1528] "Database" means a system for efficiently and safely storing and managing user information and other necessary data.

[1529] "Logging in" is the process by which a user accesses a system using their credentials.

[1530] A "session" refers to the period of a series of operations and activities while a user is logged in to a system, and is a management mechanism for maintaining that state.

[1531] "Data visualization" is the process of converting acquired data into a form that is easy for users to understand, such as a graph or table.

[1532] "Overall structure of a document" is the process of organizing and arranging the content of the main parts and paragraphs of a document.

[1533] "Story flow" refers to the order and logical development of information within a document.

[1534] "Weaknesses and areas for improvement" refers to areas within the document that are insufficient or need to be improved.

[1535] "Suggestions" are advice and corrections for improvement that the system presents to the user.

[1536] "Optimization" is the process of adjusting materials to make them more effective and of higher quality.

[1537] MODE FOR CARRYING OUT THE INVENTION

[1538] This invention relates to an AI system that assists users in efficiently creating materials and obtaining high-quality output. This system receives instructions from the user in the form of voice or text, and automatically creates a document structure proposal and visualizes data. Specifically, the system functions with the server, terminal, and user playing their respective roles. The details are explained below.

[1539] Hardware and software used

[1540] The following hardware and software are used to operate this system.

[1541] Hardware

[1542] PC or smartphone

[1543] software

[1544] Web browser

[1545] Natural language processing engine (e.g. Google Cloud Natural Language API)

[1546] Database management system (e.g. MySQL)

[1547] Speech recognition engine (e.g. Google Cloud Speech-to-Text API)

[1548] Data visualization libraries (e.g., Matplotlib)

[1549] Specific explanation of the system's processing

[1550] User Registration

[1551] The user enters their name, email address, and password on the new registration screen in their web browser. The device sends this data to the server. The server verifies the received data and saves it in a database. Once the data is saved, the server sends a registration completion message to the device, which the device displays to the user.

[1552] User Authentication

[1553] The user enters their email address and password on the login screen. The device sends the data to the server, which then queries the database for authentication. If authentication is successful, the server generates a session ID and sends it to the device. The device then displays a message indicating successful authentication.

[1554] Start creating materials

[1555] The user speaks to the device, saying, "I want to create presentation materials." The device converts this speech into text format and sends it to the server. The server analyzes it using a natural language processing engine, generates a question, "What is the purpose of the materials?" and sends it to the device. The device then displays this question to the user.

[1556] Creating a draft of the materials

[1557] The user responds to the terminal by saying, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server then analyzes the response again using a natural language processing engine and automatically generates an initial proposal for the composition of the materials. This proposal is then sent to the terminal, which then requests feedback from the user.

[1558] Inserting data and images

[1559] The user instructs the terminal to "insert a graph of sales data." The terminal sends this instruction to the server, which retrieves the required data from a database or external data source. The retrieved data is converted into a graph using a data visualization library and inserted into the document. The final draft of the document is sent to the terminal and displayed to the user.

[1560] Material optimization

[1561] The server uses a generative AI model to analyze the overall structure and story flow of the document. It identifies weaknesses and areas for improvement and sends suggestions to the user. The user reviews the suggestions and makes any necessary corrections. The server optimizes the document based on this feedback and sends the final draft to the device for the user to review and download.

[1562] Examples of prompt statements

[1563] "Please create promotional materials for our new product. The target audience is women in their 20s, and we would like to highlight the product's features."

[1564] "Please graph the following sales data and add it to your report."

[1565] As described above, the present invention is a system that supports users in efficiently creating high-quality materials, and reduces a great deal of time and effort in the process of creating materials.

[1566] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1567] Step 1:

[1568] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device receives this information and sends it to the server. The server registers this information in the database. This gives the user an account to access the system.

[1569] Step 2:

[1570] The user enters their email address and password on the login screen and clicks the "Login" button. The device receives the input information and sends it to the server. The server queries the database to confirm that the email address and password match, and if authentication is successful, generates a session ID and sends it to the device. The device displays a message to the user indicating successful authentication.

[1571] Step 3:

[1572] The user speaks to the device, saying, "I want to create presentation materials." The device converts the voice data into text and sends the text data to the server. The server uses a natural language processing engine to analyze the text data and identify the instruction. The server generates a question, "What is the purpose of the materials?" and sends it to the device.

[1573] Step 4:

[1574] The terminal displays a question from the server to the user. The user responds by typing, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server uses a natural language processing engine to analyze the user's response and automatically generates a proposal for the composition of the materials. The server then sends the generated proposal to the terminal and asks the user for feedback.

[1575] Step 5:

[1576] The user gives the instruction to "insert a graph of sales data." The terminal sends this instruction to the server. The server accesses the database or external data source to obtain sales data. Based on the obtained data, it generates a graph using a data visualization library. The server inserts the generated graph into the document and sends the result to the terminal. The terminal displays the updated document to the user.

[1577] Step 6:

[1578] The server uses a generative AI model to analyze the overall structure and story flow of the document. It determines weaknesses and areas for improvement and presents suggestions to the user. These suggestions are sent to the device, where the user can review and revise them. Based on the user's feedback, the server optimizes the document and sends the final draft to the device. The device displays the final document to the user and provides a download link if necessary.

[1579] (Application example 1)

[1580] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1581] Currently, production reports in factories are often created manually, which is time-consuming and labor-intensive. Furthermore, data aggregation and graphing require specialized knowledge and are inefficient. Furthermore, optimizing the overall structure and flow of data is time-consuming, which can lead to inconsistent quality in the reports created. Therefore, there is a need for a method that reduces this labor and time and allows for the rapid creation of high-quality production reports.

[1582] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1583] In this invention, the server includes means for receiving a document creation request from a user in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for accessing a factory production database to acquire production data, and means for converting the acquired production data into a graph format and inserting it into the document, thereby enabling the user to efficiently and automatically create high-quality production reports.

[1584] A "user" is a person who uses the system to make a document creation request.

[1585] A "material creation request" is an instruction from a user to create a material in voice or text format.

[1586] The "audio format" is a format in which the user inputs using spoken language.

[1587] The "text format" is a format in which the user inputs text such as letters and symbols.

[1588] A "natural language processing engine" is a technology that analyzes voice and text data and understands its intent.

[1589] A "document structure plan" is a plan that shows the initial structure and content layout of the entire document.

[1590] "Feedback" refers to opinions and ratings provided by users, and is information that the system uses to make further improvements.

[1591] A "database" is a storage medium for systematically organizing and storing information.

[1592] An "external data source" is a data source that exists outside the system.

[1593] A "production database" is a storage medium that collects various data related to production in a factory.

[1594] A "graphic format" is a chart format for visually representing numerical data.

[1595] "Natural language analysis" is the process by which a system understands and analyzes human language.

[1596] A "generative AI model" is an artificial intelligence model that automatically generates materials and content based on user intent and data.

[1597] A "prompt sentence" is an instruction sentence that is input to a generative AI model.

[1598] The present invention relates to an AI system that enables users to efficiently create documents such as production reports. This system receives instructions from the user in voice or text format, and automatically creates document structure proposals and data visualizations based on those instructions. Specific examples of how this system can be realized are shown below.

[1599] Overall system configuration

[1600] 1. User Registration

[1601] The user registers and enters the required information such as name, email address, and password.

[1602] The server completes the user registration by storing this information in a database.

[1603] 2. User Authentication

[1604] The user enters their email address and password on the login screen and performs authentication.

[1605] The server starts a session based on the entered information.

[1606] 3. Start preparing materials

[1607] The user issues a voice or text command to the device (smartphone or factory robot terminal) stating, "I want to create a production report."

[1608] The server parses the instructions using a natural language processing engine.

[1609] 4. Creating a draft of the materials

[1610] The server asks the user, "What is the purpose of the report?" and collects the user's response.

[1611] Based on the collected information, the server automatically generates an initial draft of the material structure and asks the user for feedback.

[1612] 5. Inserting data and graphs

[1613] The user instructs, "I want to insert a graph of production volume."

[1614] The server accesses the factory's production database and retrieves the necessary data.

[1615] The acquired data is converted into a graph format and inserted into the document.

[1616] 6. Optimize your materials

[1617] The server analyzes the overall structure of the document and the flow of the story, and suggests areas for improvement to the user.

[1618] Based on instructions from the user, the server optimizes the materials.

[1619] Technology and hardware used

[1620] 1. Natural Language Processing Engine

[1621] The system uses natural language processing to analyze the user's voice and text instructions, specifically using a natural language processing engine such as spaCy.

[1622] 2. Database Access

[1623] The server accesses the factory's production database and retrieves the necessary data, using database manipulation techniques such as SQL.

[1624] 3. Graph Generation

[1625] To visualize the acquired data, we use the pandas and matplotlib libraries to generate graphs.

[1626] Examples and prompts

[1627] Here is a concrete example:

[1628] Prompt: Create a production report for your factory and generate graphs based on the production data from the past two weeks. Also, briefly summarize your production plan for the future.

[1629] In this way, users can efficiently create high-quality production reports. Because various data is automatically acquired and visualized, it is possible to significantly reduce the amount of time and effort required.

[1630] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1631] Step 1:

[1632] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device sends the input information to the server. The server saves the sent information in a database. The input data is the name, email address, and password, and the output is the saved state in the database. This creates a user account.

[1633] Step 2:

[1634] The user enters an email address and password on the login screen and is authenticated. The device sends the entered information to the server. The server compares it with information in the database and, if it matches, starts a session. The input data is the email address and password, and the output is the session start status. This completes user authentication.

[1635] Step 3:

[1636] The user issues a voice or text command to the terminal, saying, "I want to create a production report." The terminal converts the voice into text format and sends the data to the server. The server uses a natural language processing engine to analyze the command. The input data is the voice or text command, and the output is the analysis result. This allows the user's intent to be understood.

[1637] Step 4:

[1638] The server asks the user, "What is the purpose of the report?" and communicates the question to the user via the terminal. The user answers the purpose and content using the terminal. The terminal sends the answer to the server. The server automatically generates an initial draft of the document structure based on the collected information, sends it to the terminal, and asks the user for feedback. The input data is the user's answer, and the output is the draft of the document structure. This generates the initial draft of the document structure.

[1639] Step 5:

[1640] The user gives a text command saying, "I want to insert a production volume graph." The terminal sends this command to the server. The server accesses the factory's production database and retrieves the data for the specified range. The retrieved data is converted into a data frame using pandas, and a graph is generated using matplotlib. The generated graph is inserted into the document and displayed on the terminal. The input data is the specified production data range, and the output is the generation and insertion of a graph. This achieves visualization of the production data.

[1641] Step 6:

[1642] The server analyzes the overall structure and story flow of the document and determines weaknesses and areas for improvement. Based on the analysis results, it presents suggestions to the user. The user reviews the suggestions and gives instructions for approval or revision. The server optimizes the document based on the user's instructions. The input data is the document to be analyzed, and the output is improvement suggestions and the optimized document. This results in the generation of a final, high-quality report.

[1643] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1644] This invention relates to an AI system that assists users in creating documents efficiently, and in particular provides a system equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. It can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions. The system's program processing and specific examples are shown below.

[1645] System Overview

[1646] 1. User Registration

[1647] The user registers and enters the required information such as name, email address, and password via the terminal.

[1648] The server receives this information and stores it in a database.

[1649] 2. User Authentication

[1650] The user enters their email address and password on the login screen.

[1651] The server authenticates this and, if correct, starts the session.

[1652] 3. Start preparing materials

[1653] The user speaks or writes to the device, saying, "I want to create presentation materials."

[1654] The server parses the instructions using a natural language processing engine.

[1655] 4. Creating a draft of the materials

[1656] The server asks the user, "What is the purpose of the material?"

[1657] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[1658] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1659] 5. Emotion recognition

[1660] When a user inputs text or voice, the server uses an emotion engine to analyze the user's emotions in real time.

[1661] Based on the recognized emotions, the server adjusts the proposed material composition.

[1662] 6. Inserting data and figures

[1663] The user indicates, "I want to insert a graph of sales data."

[1664] The server accesses the database or external data source to retrieve the required data.

[1665] Convert the acquired data into graph or table format and insert it into the document.

[1666] 7. Emotion-Based Optimization

[1667] The server analyzes the overall structure of the material and the flow of the story, optimizing the content of the feedback according to the recognized emotions.

[1668] The server determines areas for improvement and presents suggestions to the user.

[1669] Specific examples

[1670] User Registration and Login

[1671] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1672] 2. The device sends the input information to the server.

[1673] 3. The server receives the information and stores it in a database.

[1674] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[1675] 5. The server authenticates this and, if successful, starts the session.

[1676] From the start of document creation to the creation of a structure plan

[1677] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[1678] 2. The device converts the voice into text and sends it to the server.

[1679] 3. The server parses the instructions using a natural language processing engine.

[1680] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[1681] 5. The device presents the question to the user on the screen or by voice.

[1682] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[1683] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[1684] Emotion recognition and data insertion, optimization

[1685] 1. The user provides input, including emotional expressions, via voice or text.

[1686] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[1687] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[1688] 4. The user says, "Insert a graph of sales data."

[1689] 5. The server retrieves the sales data from the database and converts it into a graph.

[1690] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[1691] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[1692] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials based on emotion recognition, saving users a lot of time and effort.

[1693] The processing flow will be explained below.

[1694] User Registration and Authentication

[1695] User Registration

[1696] Step 1:

[1697] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[1698] Step 2:

[1699] The terminal sends the input information to the server.

[1700] Step 3:

[1701] The server validates the information it receives to ensure it is in the proper format.

[1702] Step 4:

[1703] The server stores the information in a database.

[1704] Step 5:

[1705] The server generates a registration completion message and sends it to the terminal.

[1706] Step 6:

[1707] The terminal displays a completion message.

[1708] User Authentication

[1709] Step 1:

[1710] The user enters their email address and password on the login screen and clicks the "Login" button.

[1711] Step 2:

[1712] The device sends the authentication information to the server.

[1713] Step 3:

[1714] The server will look up the database and match it with the information you entered.

[1715] Step 4:

[1716] If the server finds the match successful, it initiates the session and generates an authentication success message.

[1717] Step 5:

[1718] The server sends an authentication success message to the terminal.

[1719] Step 6:

[1720] The device will transition to the home screen and display a welcome message.

[1721] Conversational document creation

[1722] Start creating materials

[1723] Step 1:

[1724] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[1725] Step 2:

[1726] The server launches a natural language processing engine to analyze the user's intent.

[1727] Step 3:

[1728] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[1729] Step 4:

[1730] Your device will display or read the question aloud.

[1731] User Input

[1732] Step 1:

[1733] The user responds to the terminal with information about the purpose and content of the material.

[1734] Step 2:

[1735] The terminal sends the answer to the server.

[1736] Step 3:

[1737] The server analyzes the responses and generates an initial draft of the material structure.

[1738] Step 4:

[1739] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1740] Step 5:

[1741] The device displays the configuration plan.

[1742] feedback

[1743] Step 1:

[1744] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[1745] Step 2:

[1746] The terminal sends the instruction to the server.

[1747] Step 3:

[1748] The server recreates the draft document structure based on the instructions.

[1749] Step 4:

[1750] The server sends the updated configuration plan to the terminal.

[1751] Step 5:

[1752] The device displays the updated configuration.

[1753] Automated data and diagram creation using emotion recognition

[1754] Emotion recognition and data insertion requests

[1755] Step 1:

[1756] When a user creates a document using voice or text, the device sends the input to the emotion engine.

[1757] Step 2:

[1758] The emotion engine analyzes the user's emotions in real time and sends the results to the server.

[1759] Step 3:

[1760] The server uses the emotion recognition results to adjust the composition of the material.

[1761] Step 4:

[1762] The user instructs the terminal to "insert a graph of sales data."

[1763] Step 5:

[1764] The terminal sends the instruction to the server.

[1765] Step 6:

[1766] The server accesses the database or external data source to retrieve the required data.

[1767] Step 7:

[1768] The server converts the retrieved data into an appropriate graph format.

[1769] Step 8:

[1770] The server sends the generated graph to the terminal.

[1771] Step 9:

[1772] The terminal displays the graph.

[1773] Emotion Recognition and Data Verification

[1774] Step 1:

[1775] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[1776] Step 2:

[1777] The terminal sends the instruction to the server.

[1778] Step 3:

[1779] The server regenerates the graph as needed and sends it to the device.

[1780] Step 4:

[1781] The terminal will display the regenerated graph.

[1782] Checking the direction of the material and the story

[1783] Check the story

[1784] Step 1:

[1785] The server analyzes the overall structure of the material and the flow of the story.

[1786] Step 2:

[1787] The server determines weak areas and areas for improvement and generates suggestions.

[1788] Step 3:

[1789] The server sends the proposal to the device.

[1790] Step 4:

[1791] The device will display suggestions.

[1792] User Verification

[1793] Step 1:

[1794] The user checks the proposal and inputs approval or correction instructions into the terminal.

[1795] Step 2:

[1796] The terminal sends the instruction to the server.

[1797] Step 3:

[1798] The server optimizes the materials based on the instructions.

[1799] Step 4:

[1800] The server sends the optimized material to the terminal.

[1801] Step 5:

[1802] The device displays optimized materials.

[1803] Optimization and concept review

[1804] Final Check

[1805] Step 1:

[1806] The server will thoroughly review the completed document and report any errors or improvements.

[1807] Step 2:

[1808] The server sends the report to the device.

[1809] Step 3:

[1810] The terminal displays the report.

[1811] Concept reconfirmation

[1812] Step 1:

[1813] The user initiates a dialogue at the terminal to reaffirm the concept or message.

[1814] Step 2:

[1815] The terminal sends the corresponding content to the server.

[1816] Step 3:

[1817] The server interacts with the user to reconfirm the concepts and messages of the materials.

[1818] Step 4:

[1819] The server will fine-tune the material as needed.

[1820] Step 5:

[1821] The server sends the final materials to the terminal.

[1822] Step 6:

[1823] The terminal displays the completed document.

[1824] Example 2

[1825] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1826] Conventional document creation systems require users to manually design document structures and convert data into graphs and tables, which requires a great deal of time and effort. Furthermore, it is difficult to create detailed documents that reflect the user's emotions and intentions. The present invention aims to solve these problems and provide a system that supports users in creating high-quality documents efficiently and easily.

[1827] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1828] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, emotion recognition means for recognizing the user's emotions in real time, and means for automatically adjusting the material composition plan based on the recognized emotions. This makes it possible to automatically generate and adjust an appropriate material composition that reflects emotions simply by providing instructions from the user.

[1829] A "document creation request" refers to an instruction or request made by a user to the system to request the creation of a document.

[1830] "Voice or text format" refers to an interface that allows a user to give instructions to the system, and is in the form of voice input or text input.

[1831] A "natural language processing engine" refers to algorithms and software that analyze human language and understand its intent and meaning.

[1832] "Analysis results" refers to the information and data obtained by the natural language processing engine when it analyzes the user's instructions.

[1833] "Initial document structure proposal" refers to the initial structure and content of the document that the system automatically generates based on the analysis results.

[1834] "Feedback" refers to the evaluations, opinions, and correction requests that users provide to the system.

[1835] "Emotion recognition means" refers to algorithms and software for analyzing and understanding a user's emotions in real time.

[1836] "Means for automatically adjusting the proposed document structure" refers to functions or algorithms that automatically change or adjust the structure or content of a document based on recognized emotions.

[1837] A "database" refers to a system for systematically storing and managing information.

[1838] "External data source" refers to a source or service that provides data that exists outside the system.

[1839] "Converting into graph or table format" refers to the process or task of formatting acquired data into a form that is visually easy to understand.

[1840] "Means for inserting into documents" refers to functions and processes for placing the generated graphs and tables in appropriate positions in the documents.

[1841] "The overall structure of the material and the flow of the story" refers to the arrangement of each section or part within the material and ensuring that they have a continuous meaning.

[1842] "Measures for determining weaknesses and areas for improvement" refers to algorithms or processes used to identify gaps or areas in the content or structure of a document that need correction.

[1843] "Means for presenting suggestions" refers to the functions and processes for providing users with specific actions for improvement or change suggestions.

[1844] "Means for automatically optimizing materials" refers to functions and algorithms that automatically adjust and improve the content of materials to the most appropriate form based on user instructions and emotion recognition results.

[1845] This invention relates to an AI system that assists users in creating documents efficiently. The system is equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. The system can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions.

[1846] Hardware and Software Configuration

[1847] server

[1848] Database: A system for systematically storing and managing information

[1849] Natural language processing engine: Software for analyzing human language and understanding intent, such as the Google NLP API

[1850] Emotion recognition engine: Software for recognizing user emotions in real time, such as IBM Watson Tone Analyzer

[1851] Data visualization tools: Software for transforming data into graphs and tables, such as Matplotlib, Pandas DataFrame, etc.

[1852] Terminal

[1853] Speech recognition engine: Software that converts a user's speech into text, such as the Google Speech-to-Text API.

[1854] User interface: An interface for receiving voice commands, text input, and feedback

[1855] Example of implementation

[1856] User Registration and Login

[1857] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1858] 2. The device sends the input information to the server.

[1859] 3. The server receives the information and stores it in a database.

[1860] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[1861] 5. The server authenticates this and, if successful, starts the session.

[1862] From the start of document creation to the creation of a structure plan

[1863] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[1864] 2. The device converts the voice into text and sends it to the server.

[1865] 3. The server parses the instructions using a natural language processing engine.

[1866] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[1867] 5. The device presents the question to the user on the screen or by voice.

[1868] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[1869] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[1870] Emotion recognition and data insertion, optimization

[1871] 1. The user provides input, including emotional expressions, via voice or text.

[1872] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[1873] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[1874] 4. The user says, "Insert a graph of sales data."

[1875] 5. The server retrieves the sales data from the database and converts it into a graph.

[1876] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[1877] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[1878] Prompt Sentence Examples

[1879] "I would like to register as a user. What is the process?"

[1880] "What are the steps to creating promotional materials for a new product?"

[1881] "How can I convert sales data into a graph and insert it into a presentation?"

[1882] "Show us how to optimize the overall structure of your materials based on emotions."

[1883] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1884] Step 1:

[1885] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[1886] The input contents are the user's name, email address, and password.

[1887] The terminal transmits the input information to the server.

[1888] The server stores the received information in a database, with the password hashed for security purposes.

[1889] Input: User information (name, email address, password)

[1890] Output: User information stored in the database

[1891] Step 2:

[1892] The user enters their email address and password on the login screen.

[1893] The input contents are the user's email address and password.

[1894] The terminal sends the input information to the server.

[1895] The server searches the database for the corresponding record and checks whether the entered information matches the hashed password.

[1896] When the authentication is successful, the server starts a session and sends a message of successful authentication to the terminal.

[1897] Input: Login information (email address, password)

[1898] Output: Session start message

[1899] Step 3:

[1900] The user speaks into the device and says, "I want to create a presentation," or inputs text.

[1901] The input may be a voice command or a text command.

[1902] The device converts the voice into text format and sends it to the server.

[1903] The server parses the input using a natural language processing engine.

[1904] Input: Voice or text instructions

[1905] Output: Analysis results (user's intentions and goals)

[1906] Step 4:

[1907] The server asks the user, "What is the purpose of the document?"

[1908] The terminal presents the question to the user.

[1909] When the user answers the question about the purpose and content, the device sends the answer to the server.

[1910] Input: User's answer (purpose of the material)

[1911] Output: Analysis results (information necessary for document creation)

[1912] Step 5:

[1913] The server generates an initial proposal for the material structure based on the answers.

[1914] The composition suggestions are generated using a natural language generation engine (e.g., a generative AI model).

[1915] The server sends the proposed configuration to the terminal and asks the user for feedback.

[1916] Input: Analysis results (purpose and content of the data)

[1917] Output: Initial document structure proposal

[1918] Step 6:

[1919] The user provides feedback via voice or text.

[1920] The input is the user's feedback.

[1921] The server uses an emotion engine to recognize the user's emotions in real time.

[1922] The server adjusts the proposed document structure based on the feedback and the results of sentiment analysis.

[1923] Input: Feedback content, sentiment analysis results

[1924] Output: Adjusted document structure proposal

[1925] Step 7:

[1926] The user indicates, "I want to insert a graph of sales data."

[1927] The input contents are specific instructions from the user.

[1928] The terminal sends the instruction to the server.

[1929] The server accesses the database or external data source to retrieve the required data.

[1930] The server converts the acquired data into graph or table format.

[1931] The graph generated by the server is inserted into the document and displayed on the terminal.

[1932] Input: Data insertion instructions

[1933] Output: Inserted data (graphs and tables)

[1934] Step 8:

[1935] The server analyzes the overall structure and story flow of the material and identifies weaknesses and areas for improvement.

[1936] The server optimizes the content of the feedback based on the recognized emotion.

[1937] The server determines areas for improvement and presents suggestions to the user.

[1938] Input: Data structure, sentiment analysis results

[1939] Output: Optimized materials and recommendations

[1940] (Application example 2)

[1941] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1942] Current document creation systems have difficulty recognizing user emotions in real time and appropriately reflecting feedback, which results in users being unable to efficiently create high-quality documents. Similar problems exist in manufacturing sites when creating operator instructions, particularly due to a lack of emotion-based optimization, which is labor-intensive and time-consuming. This leads to issues such as reduced manufacturing efficiency and a lack of consistent quality.

[1943] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1944] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, means for recognizing the user's emotions in real time and adjusting the material composition plan, and means for acquiring data and diagrams related to the manufacturing process and automatically inserting them into instructions, thereby enabling users to efficiently create high-quality materials and manufacturing instructions.

[1945] A "user" is an entity that uses the system to create materials and instructions.

[1946] A "material creation request" is a request for material creation made by a user to the system in voice or text format.

[1947] A "natural language processing engine" is a technology that analyzes a user's voice or text input and understands their intent.

[1948] The "initial document structure proposal" is a proposal for the initial structure of the document that is automatically generated by the system based on the user's input.

[1949] "Feedback" refers to opinions or requests for corrections made by the user regarding the configuration plan or data provided by the system.

[1950] "Recognizing emotions in real time" means instantly analyzing and determining the user's emotional state from input voice or text.

[1951] "Adjusting the proposed document structure" means optimizing the structure of the document based on emotions recognized in real time.

[1952] "Manufacturing process data" refers to specific data or information related to the manufacturing process.

[1953] "Automatically insert into instructions" means that the system automatically incorporates the acquired data into the instructions.

[1954] "Instructions" are documents that describe the steps and information required for an operator to follow a manufacturing process.

[1955] "Technology used" refers to the specific technology or method used in the invention.

[1956] The system embodying the present invention assists users in efficiently creating materials and manufacturing instructions. A specific embodiment of the system is described below.

[1957] First, a user inputs a document creation request in voice or text format using smart glasses, a smartphone, a head-mounted display, or a robot. The server receives this input and analyzes the user's intent using a natural language processing engine (e.g., spaCy). Based on the analysis results, an initial document structure proposal is automatically generated and presented to the user for feedback.

[1958] Next, an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) is activated to recognize emotions in real time from the user's voice and text input. This allows the proposed document structure to be adjusted based on the user's emotional state. For example, if the user is feeling impatient or frustrated, the proposed structure may be simplified or additional support may be provided.

[1959] During the manufacturing process, operators give instructions in voice or text format. These instructions are also analyzed by the server, and data and diagrams related to the manufacturing process are retrieved from a database (e.g., MongoDB) and automatically inserted into the instructions. This data is also converted into graphs and tables using libraries such as D3.js.

[1960] Furthermore, the server analyzes the overall structure and story flow of the materials, identifies weaknesses and areas for improvement, and presents suggestions to the user. This automatically optimizes the entire materials and instructions. The server uses a content analysis algorithm (e.g., the BERT model) to provide optimal suggestions for the user.

[1961] As a concrete example, consider the case where an operator gives a voice command such as, "I would like to create manufacturing process instructions for a new engine part." In this case, the system converts the voice into text format and analyzes it using a natural language processing engine. The system then asks, "What are the main contents of the manufacturing process?" If the operator responds with, "Assembly, testing, and shipping," an initial draft of the instruction structure is generated. If the system senses impatience in the operator's voice, it will suggest simplifying the process.

[1962] A concrete example of a prompt sentence is as follows:

[1963] “An operator says, ‘I want to create instructions for manufacturing an engine part.’ The system then asks, ‘What are the main components of the manufacturing process?’ The answer is, ‘Assembly, testing, and shipping.’ If the operator sounds impatient, how does the system adjust the proposed configuration?”

[1964] Such a system enables users and operators to efficiently create high-quality documents and manufacturing instructions.

[1965] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1966] Step 1:

[1967] The user uses smart glasses, a smartphone, a head-mounted display, or a robot to input a request for document creation in voice or text format. The input request is received by the device, and if it is in text format, it is converted to text using a speech recognition API (e.g., Google Speech-to-Text). Input: Voice / text request, Output: Text format request.

[1968] Step 2:

[1969] The device sends the received text request to the server. The server uses a natural language processing engine (e.g., spaCy) to analyze the request and understand the user's intent. Input: Text request, Output: Analyzed user intent.

[1970] Step 3:

[1971] The server automatically generates an initial document structure proposal based on the analysis results. The structure proposal is generated using a template engine (e.g., Jinja2). The generated structure proposal is sent to the terminal and the user is asked for feedback. Input: Analyzed user intent, Output: Initial document structure proposal.

[1972] Step 4:

[1973] The user inputs feedback in voice or text format, and the device receives it. Similarly, if the voice is input, it is converted into text format using a speech recognition API. Input: Voice / Text Feedback, Output: Text Feedback.

[1974] Step 5:

[1975] The device sends the received feedback to the server, which uses an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) to recognize the user's emotions in real time. Input: Text feedback, Output: Recognized emotions.

[1976] Step 6:

[1977] The server adjusts the proposed document composition based on the recognized emotion. If the emotion is impatience or dissatisfaction, the composition may be simplified or additional support may be provided. The adjusted composition is sent to the terminal and presented to the user. Input: Recognized emotion, Output: Adjusted document composition.

[1978] Step 7:

[1979] The user requests the insertion of data or diagrams related to the manufacturing process in voice or text format. The device receives the request and sends it to the server. The server accesses a database (e.g., MongoDB) or external data source to obtain the required data, and converts the data into a graph or table format using a library such as D3.js. Input: Data insertion request in text format, Output: Data in graph or table format.

[1980] Step 8:

[1981] The server automatically inserts the acquired and converted data into the instruction sheet. The instruction sheet is dynamically updated by the template engine, and after creation, it is sent to the terminal and presented to the user. Input: Data in graph or table format, Output: Updated instruction sheet.

[1982] Step 9:

[1983] The server analyzes the overall structure and story flow of the document, identifies weaknesses and areas for improvement, and presents suggestions to the user. Optimal improvement suggestions are generated using a content analysis algorithm (e.g., the BERT model). Input: Overall structure of the document, Output: Improvement suggestions.

[1984] Step 10:

[1985] The user receives suggestions from the server and confirms the final materials and instructions. If there is any final feedback, it is similarly entered by voice or text and final adjustments are made. Input: User's final confirmation and feedback, Output: Final confirmed materials or instructions.

[1986] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1987] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1988] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1989] [Fourth embodiment]

[1990] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1991] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1992] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1993] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1994] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1995] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1996] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1997] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1998] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1999] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2000] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2001] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2002] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2003] This invention relates to an AI system that assists users in creating documents efficiently. The system automatically creates document structure proposals and data visualizations based on user input in voice or text format. The system's program processing and specific examples are shown below.

[2004] System Overview

[2005] 1. User Registration

[2006] The user registers and enters the required information such as name, email address, and password via the terminal.

[2007] The server receives this information and stores it in a database.

[2008] 2. User Authentication

[2009] The user enters their email address and password on the login screen.

[2010] The server authenticates this and, if correct, starts the session.

[2011] 3. Start preparing materials

[2012] The user speaks or writes to the device, saying, "I want to create presentation materials."

[2013] The server parses the instructions using a natural language processing engine.

[2014] 4. Creating a draft of the materials

[2015] The server asks the user, "What is the purpose of the material?"

[2016] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[2017] The server sends the proposed configuration to the terminal and asks the user for feedback.

[2018] 5. Inserting data and figures

[2019] The user indicates, "I want to insert a graph of sales data."

[2020] The server accesses the database or external data source to retrieve the required data.

[2021] Convert the acquired data into graph or table format and insert it into the document.

[2022] 6. Optimize your materials

[2023] The server analyzes the overall structure of the material and the flow of the story.

[2024] Determine weaknesses and areas for improvement and provide suggestions to users.

[2025] The user checks the suggestions, and the server adjusts to create the most suitable materials.

[2026] Specific examples

[2027] User Registration and Login

[2028] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[2029] 2. The device sends the input information to the server.

[2030] 3. The server receives the information and stores it in a database.

[2031] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[2032] 5. The server authenticates this and, if successful, starts the session.

[2033] From the start of document creation to the creation of a structure plan

[2034] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[2035] 2. The device converts the voice into text and sends it to the server.

[2036] 3. The server parses the instructions using a natural language processing engine.

[2037] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[2038] 5. The device presents the question to the user on the screen or by voice.

[2039] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[2040] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[2041] Inserting and optimizing data

[2042] 1. The user says, "Insert a graph of sales data."

[2043] 2. The server retrieves the sales data from the database and converts it into a graph.

[2044] 3. The graph generated by the server is inserted into the document and displayed on the terminal.

[2045] 4. The server checks the overall structure and story of the document and suggests improvements to the user.

[2046] 5. The user reviews and approves the proposal, and the final document is completed.

[2047] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials, saving a lot of time and effort.

[2048] The processing flow will be explained below.

[2049] User Registration and Authentication

[2050] User Registration

[2051] Step 1:

[2052] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[2053] Step 2:

[2054] The terminal sends the input information to the server.

[2055] Step 3:

[2056] The server validates the information it receives to ensure it is in the proper format.

[2057] Step 4:

[2058] The server stores the information in a database.

[2059] Step 5:

[2060] The server generates a registration completion message and sends it to the terminal.

[2061] Step 6:

[2062] The terminal displays a completion message.

[2063] User Authentication

[2064] Step 1:

[2065] The user enters their email address and password on the login screen and clicks the "Login" button.

[2066] Step 2:

[2067] The device sends the authentication information to the server.

[2068] Step 3:

[2069] The server will look up the database and match it with the information you entered.

[2070] Step 4:

[2071] If the server finds the match successful, it initiates the session and generates an authentication success message.

[2072] Step 5:

[2073] The server sends an authentication success message to the terminal.

[2074] Step 6:

[2075] The device will transition to the home screen and display a welcome message.

[2076] Conversational document creation

[2077] Start creating materials

[2078] Step 1:

[2079] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[2080] Step 2:

[2081] The server launches a natural language processing engine to analyze the user's intent.

[2082] Step 3:

[2083] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[2084] Step 4:

[2085] Your device will display or read the question aloud.

[2086] User Input

[2087] Step 1:

[2088] The user responds to the terminal with information about the purpose and content of the material.

[2089] Step 2:

[2090] The terminal sends the answer to the server.

[2091] Step 3:

[2092] The server analyzes the responses and generates an initial draft of the material structure.

[2093] Step 4:

[2094] The server sends the proposed configuration to the terminal and asks the user for feedback.

[2095] Step 5:

[2096] The device displays the configuration plan.

[2097] feedback

[2098] Step 1:

[2099] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[2100] Step 2:

[2101] The terminal sends the instruction to the server.

[2102] Step 3:

[2103] The server recreates the draft document structure based on the instructions.

[2104] Step 4:

[2105] The server sends the updated configuration plan to the terminal.

[2106] Step 5:

[2107] The device displays the updated configuration.

[2108] Automatic data and diagram creation

[2109] Requesting data insertion

[2110] Step 1:

[2111] The user instructs the terminal to "insert a graph of sales data."

[2112] Step 2:

[2113] The terminal sends the instruction to the server.

[2114] Step 3:

[2115] The server accesses the database or external data source to retrieve the required data.

[2116] Step 4:

[2117] The server converts the retrieved data into an appropriate graph format.

[2118] Step 5:

[2119] The server sends the generated graph to the terminal.

[2120] Step 6:

[2121] The terminal displays the graph.

[2122] Data confirmation

[2123] Step 1:

[2124] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[2125] Step 2:

[2126] The terminal sends the instruction to the server.

[2127] Step 3:

[2128] The server regenerates the graph as needed and sends it to the device.

[2129] Step 4:

[2130] The terminal will display the regenerated graph.

[2131] Checking the direction of the material and the story

[2132] Check the story

[2133] Step 1:

[2134] The server analyzes the overall structure of the material and the flow of the story.

[2135] Step 2:

[2136] The server determines weak areas and areas for improvement and generates suggestions.

[2137] Step 3:

[2138] The server sends the proposal to the device.

[2139] Step 4:

[2140] The device will display suggestions.

[2141] User Verification

[2142] Step 1:

[2143] The user checks the proposal and inputs approval or correction instructions into the terminal.

[2144] Step 2:

[2145] The terminal sends the instruction to the server.

[2146] Step 3:

[2147] The server optimizes the materials based on the instructions.

[2148] Step 4:

[2149] The server sends the optimized material to the terminal.

[2150] Step 5:

[2151] The device displays optimized materials.

[2152] Optimization and concept review

[2153] Final Check

[2154] Step 1:

[2155] The server will thoroughly review the completed document and report any errors or improvements.

[2156] Step 2:

[2157] The server sends the report to the device.

[2158] Step 3:

[2159] The terminal displays the report.

[2160] Concept reconfirmation

[2161] Step 1:

[2162] The user initiates a dialogue with the terminal to reaffirm the concept or message.

[2163] Step 2:

[2164] The terminal sends the corresponding content to the server.

[2165] Step 3:

[2166] The server interacts with the user to reconfirm the concepts and messages of the materials.

[2167] Step 4:

[2168] The server will fine-tune the material as needed.

[2169] Step 5:

[2170] The server sends the final materials to the terminal.

[2171] Step 6:

[2172] The terminal displays the completed document.

[2173] Example 1

[2174] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2175] In conventional document creation systems, users had to manually plan the document structure, collect data, visualize it, and finally complete the document, which required a great deal of time and effort. During this process, it was difficult for users to maintain consistency in the document, and specialized knowledge was often required for data visualization and optimization. Therefore, a system that supports efficient and effective document creation was needed.

[2176] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2177] In this invention, the server includes means for receiving a user's request for creating a document in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for the user to register and enter authentication information via a terminal, which the server stores in a database, means for the user to enter an email address and password on a login screen, which the server authenticates and starts a session, means for automatically inserting data and figures specified by the user into the generated document structure proposal, which the server acquires, visualizes, and integrates into the document, and means for analyzing the overall structure and story flow of the document, determining weaknesses and areas for improvement, and presenting suggestions to the user. This enables users to efficiently create high-quality documents quickly and reduce the significant time and effort required in the document creation process.

[2178] A "material creation request" is a request for the creation of a material that a user submits to the system in voice or text format.

[2179] A "natural language processing engine" is an artificial intelligence technology that analyzes a user's voice and text instructions and understands their intentions.

[2180] The "initial document structure proposal" is an initial plan or outline of the document that the system automatically generates based on the user's instructions.

[2181] "Feedback" refers to evaluations and suggestions for improvement provided by users regarding the initial draft of the material structure.

[2182] "New registration" is the process in which a user enters personal information and authentication information for the first time to create an account in order to use the system.

[2183] "Authentication information" refers to information such as an email address and password used to identify a user.

[2184] "Database" means a system for efficiently and safely storing and managing user information and other necessary data.

[2185] "Logging in" is the process by which a user accesses a system using their credentials.

[2186] A "session" refers to the period of a series of operations and activities while a user is logged in to a system, and is a management mechanism for maintaining that state.

[2187] "Data visualization" is the process of converting acquired data into a form that is easy for users to understand, such as a graph or table.

[2188] "Overall structure of a document" is the process of organizing and arranging the content of the main parts and paragraphs of a document.

[2189] "Story flow" refers to the order and logical development of information within a document.

[2190] "Weaknesses and areas for improvement" refers to areas within the document that are insufficient or need to be improved.

[2191] "Suggestions" are advice and corrections for improvement that the system presents to the user.

[2192] "Optimization" is the process of adjusting materials to make them more effective and of higher quality.

[2193] MODE FOR CARRYING OUT THE INVENTION

[2194] This invention relates to an AI system that assists users in efficiently creating materials and obtaining high-quality output. This system receives instructions from the user in the form of voice or text, and automatically creates a document structure proposal and visualizes data. Specifically, the system functions with the server, terminal, and user playing their respective roles. The details are explained below.

[2195] Hardware and software used

[2196] The following hardware and software are used to operate this system.

[2197] Hardware

[2198] PC or smartphone

[2199] software

[2200] Web browser

[2201] Natural language processing engine (e.g. Google Cloud Natural Language API)

[2202] Database management system (e.g. MySQL)

[2203] Speech recognition engine (e.g. Google Cloud Speech-to-Text API)

[2204] Data visualization libraries (e.g., Matplotlib)

[2205] Specific explanation of the system's processing

[2206] User Registration

[2207] The user enters their name, email address, and password on the new registration screen in their web browser. The device sends this data to the server. The server verifies the received data and saves it in a database. Once the data is saved, the server sends a registration completion message to the device, which the device displays to the user.

[2208] User Authentication

[2209] The user enters their email address and password on the login screen. The device sends the data to the server, which then queries the database for authentication. If authentication is successful, the server generates a session ID and sends it to the device. The device then displays a message indicating successful authentication.

[2210] Start creating materials

[2211] The user speaks to the device, saying, "I want to create presentation materials." The device converts this speech into text format and sends it to the server. The server analyzes it using a natural language processing engine, generates a question, "What is the purpose of the materials?" and sends it to the device. The device then displays this question to the user.

[2212] Creating a draft of the materials

[2213] The user responds to the terminal by saying, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server then analyzes the response again using a natural language processing engine and automatically generates an initial proposal for the composition of the materials. This proposal is then sent to the terminal, which then requests feedback from the user.

[2214] Inserting data and images

[2215] The user instructs the terminal to "insert a graph of sales data." The terminal sends this instruction to the server, which retrieves the required data from a database or external data source. The retrieved data is converted into a graph using a data visualization library and inserted into the document. The final draft of the document is sent to the terminal and displayed to the user.

[2216] Material optimization

[2217] The server uses a generative AI model to analyze the overall structure and story flow of the document. It identifies weaknesses and areas for improvement and sends suggestions to the user. The user reviews the suggestions and makes any necessary corrections. The server optimizes the document based on this feedback and sends the final draft to the device for the user to review and download.

[2218] Examples of prompt statements

[2219] "Please create promotional materials for our new product. The target audience is women in their 20s, and we would like to highlight the product's features."

[2220] "Please graph the following sales data and add it to your report."

[2221] As described above, the present invention is a system that supports users in efficiently creating high-quality materials, and reduces a great deal of time and effort in the process of creating materials.

[2222] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2223] Step 1:

[2224] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device receives this information and sends it to the server. The server registers this information in the database. This gives the user an account to access the system.

[2225] Step 2:

[2226] The user enters their email address and password on the login screen and clicks the "Login" button. The device receives the input information and sends it to the server. The server queries the database to confirm that the email address and password match, and if authentication is successful, generates a session ID and sends it to the device. The device displays a message to the user indicating successful authentication.

[2227] Step 3:

[2228] The user speaks to the device, saying, "I want to create presentation materials." The device converts the voice data into text and sends the text data to the server. The server uses a natural language processing engine to analyze the text data and identify the instruction. The server generates a question, "What is the purpose of the materials?" and sends it to the device.

[2229] Step 4:

[2230] The terminal displays a question from the server to the user. The user responds by typing, "I would like to explain the features of a new product and promote sales." The terminal then sends this response to the server. The server uses a natural language processing engine to analyze the user's response and automatically generates a proposal for the composition of the materials. The server then sends the generated proposal to the terminal and asks the user for feedback.

[2231] Step 5:

[2232] The user gives the instruction to "insert a graph of sales data." The terminal sends this instruction to the server. The server accesses the database or external data source to obtain sales data. Based on the obtained data, it generates a graph using a data visualization library. The server inserts the generated graph into the document and sends the result to the terminal. The terminal displays the updated document to the user.

[2233] Step 6:

[2234] The server uses a generative AI model to analyze the overall structure and story flow of the document. It determines weaknesses and areas for improvement and presents suggestions to the user. These suggestions are sent to the device, where the user can review and revise them. Based on the user's feedback, the server optimizes the document and sends the final draft to the device. The device displays the final document to the user and provides a download link if necessary.

[2235] (Application example 1)

[2236] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2237] Currently, production reports in factories are often created manually, which is time-consuming and labor-intensive. Furthermore, data aggregation and graphing require specialized knowledge and are inefficient. Furthermore, optimizing the overall structure and flow of data is time-consuming, which can lead to inconsistent quality in the reports created. Therefore, there is a need for a method that reduces this labor and time and allows for the rapid creation of high-quality production reports.

[2238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2239] In this invention, the server includes means for receiving a document creation request from a user in voice or text format, means for analyzing the user's intent using a natural language processing engine, means for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user, means for accessing a factory production database to acquire production data, and means for converting the acquired production data into a graph format and inserting it into the document, thereby enabling the user to efficiently and automatically create high-quality production reports.

[2240] A "user" is a person who uses the system to make a document creation request.

[2241] A "material creation request" is an instruction from a user to create a material in voice or text format.

[2242] The "audio format" is a format in which the user inputs using spoken language.

[2243] The "text format" is a format in which the user inputs text such as letters and symbols.

[2244] A "natural language processing engine" is a technology that analyzes voice and text data and understands its intent.

[2245] A "document structure plan" is a plan that shows the initial structure and content layout of the entire document.

[2246] "Feedback" refers to opinions and ratings provided by users, and is information that the system uses to make further improvements.

[2247] A "database" is a storage medium for systematically organizing and storing information.

[2248] An "external data source" is a data source that exists outside the system.

[2249] A "production database" is a storage medium that collects various data related to production in a factory.

[2250] A "graphic format" is a chart format for visually representing numerical data.

[2251] "Natural language analysis" is the process by which a system understands and analyzes human language.

[2252] A "generative AI model" is an artificial intelligence model that automatically generates materials and content based on user intent and data.

[2253] A "prompt sentence" is an instruction sentence that is input to a generative AI model.

[2254] The present invention relates to an AI system that enables users to efficiently create documents such as production reports. This system receives instructions from the user in voice or text format, and automatically creates document structure proposals and data visualizations based on those instructions. Specific examples of how this system can be realized are shown below.

[2255] Overall system configuration

[2256] 1. User Registration

[2257] The user registers and enters the required information such as name, email address, and password.

[2258] The server completes the user registration by storing this information in a database.

[2259] 2. User Authentication

[2260] The user enters their email address and password on the login screen and performs authentication.

[2261] The server starts a session based on the entered information.

[2262] 3. Start preparing materials

[2263] The user issues a voice or text command to the device (smartphone or factory robot terminal) stating, "I want to create a production report."

[2264] The server parses the instructions using a natural language processing engine.

[2265] 4. Creating a draft of the materials

[2266] The server asks the user, "What is the purpose of the report?" and collects the user's response.

[2267] Based on the collected information, the server automatically generates an initial draft of the material structure and asks the user for feedback.

[2268] 5. Inserting data and graphs

[2269] The user instructs, "I want to insert a graph of production volume."

[2270] The server accesses the factory's production database and retrieves the necessary data.

[2271] The acquired data is converted into a graph format and inserted into the document.

[2272] 6. Optimize your materials

[2273] The server analyzes the overall structure of the document and the flow of the story, and suggests areas for improvement to the user.

[2274] Based on instructions from the user, the server optimizes the materials.

[2275] Technology and hardware used

[2276] 1. Natural Language Processing Engine

[2277] The system uses natural language processing to analyze the user's voice and text instructions, specifically using a natural language processing engine such as spaCy.

[2278] 2. Database Access

[2279] The server accesses the factory's production database and retrieves the necessary data, using database manipulation techniques such as SQL.

[2280] 3. Graph Generation

[2281] To visualize the acquired data, we use the pandas and matplotlib libraries to generate graphs.

[2282] Examples and prompts

[2283] Here is a concrete example:

[2284] Prompt: Create a production report for your factory and generate graphs based on the production data from the past two weeks. Also, briefly summarize your production plan for the future.

[2285] In this way, users can efficiently create high-quality production reports. Because various data is automatically acquired and visualized, it is possible to significantly reduce the amount of time and effort required.

[2286] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2287] Step 1:

[2288] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button. The device sends the input information to the server. The server saves the sent information in a database. The input data is the name, email address, and password, and the output is the saved state in the database. This creates a user account.

[2289] Step 2:

[2290] The user enters an email address and password on the login screen and is authenticated. The device sends the entered information to the server. The server compares it with information in the database and, if it matches, starts a session. The input data is the email address and password, and the output is the session start status. This completes user authentication.

[2291] Step 3:

[2292] The user issues a voice or text command to the terminal, saying, "I want to create a production report." The terminal converts the voice into text format and sends the data to the server. The server uses a natural language processing engine to analyze the command. The input data is the voice or text command, and the output is the analysis result. This allows the user's intent to be understood.

[2293] Step 4:

[2294] The server asks the user, "What is the purpose of the report?" and communicates the question to the user via the terminal. The user answers the purpose and content using the terminal. The terminal sends the answer to the server. The server automatically generates an initial draft of the document structure based on the collected information, sends it to the terminal, and asks the user for feedback. The input data is the user's answer, and the output is the draft of the document structure. This generates the initial draft of the document structure.

[2295] Step 5:

[2296] The user gives a text command saying, "I want to insert a production volume graph." The terminal sends this command to the server. The server accesses the factory's production database and retrieves the data for the specified range. The retrieved data is converted into a data frame using pandas, and a graph is generated using matplotlib. The generated graph is inserted into the document and displayed on the terminal. The input data is the specified production data range, and the output is the generation and insertion of a graph. This achieves visualization of the production data.

[2297] Step 6:

[2298] The server analyzes the overall structure and story flow of the document and determines weaknesses and areas for improvement. Based on the analysis results, it presents suggestions to the user. The user reviews the suggestions and gives instructions for approval or revision. The server optimizes the document based on the user's instructions. The input data is the document to be analyzed, and the output is improvement suggestions and the optimized document. This results in the generation of a final, high-quality report.

[2299] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2300] This invention relates to an AI system that assists users in creating documents efficiently, and in particular provides a system equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. It can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions. The system's program processing and specific examples are shown below.

[2301] System Overview

[2302] 1. User Registration

[2303] The user registers and enters the required information such as name, email address, and password via the terminal.

[2304] The server receives this information and stores it in a database.

[2305] 2. User Authentication

[2306] The user enters their email address and password on the login screen.

[2307] The server authenticates this and, if correct, starts the session.

[2308] 3. Start preparing materials

[2309] The user speaks or writes to the device, saying, "I want to create presentation materials."

[2310] The server parses the instructions using a natural language processing engine.

[2311] 4. Creating a draft of the materials

[2312] The server asks the user, "What is the purpose of the material?"

[2313] When the user answers questions about the purpose and content, the server generates an initial proposal for the material structure based on the answers.

[2314] The server sends the proposed configuration to the terminal and asks the user for feedback.

[2315] 5. Emotion recognition

[2316] When a user inputs text or voice, the server uses an emotion engine to analyze the user's emotions in real time.

[2317] Based on the recognized emotions, the server adjusts the proposed material composition.

[2318] 6. Inserting data and figures

[2319] The user indicates, "I want to insert a graph of sales data."

[2320] The server accesses the database or external data source to retrieve the required data.

[2321] Convert the acquired data into graph or table format and insert it into the document.

[2322] 7. Emotion-Based Optimization

[2323] The server analyzes the overall structure of the material and the flow of the story, optimizing the content of the feedback according to the recognized emotions.

[2324] The server determines areas for improvement and presents suggestions to the user.

[2325] Specific examples

[2326] User Registration and Login

[2327] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[2328] 2. The device sends the input information to the server.

[2329] 3. The server receives the information and stores it in a database.

[2330] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[2331] 5. The server authenticates this and, if successful, starts the session.

[2332] From the start of document creation to the creation of a structure plan

[2333] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[2334] 2. The device converts the voice into text and sends it to the server.

[2335] 3. The server parses the instructions using a natural language processing engine.

[2336] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[2337] 5. The device presents the question to the user on the screen or by voice.

[2338] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[2339] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[2340] Emotion recognition and data insertion, optimization

[2341] 1. The user provides input, including emotional expressions, via voice or text.

[2342] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[2343] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[2344] 4. The user says, "Insert a graph of sales data."

[2345] 5. The server retrieves the sales data from the database and converts it into a graph.

[2346] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[2347] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[2348] This system allows users to efficiently create high-quality materials. The server analyzes the user's intentions and automatically optimizes the materials based on emotion recognition, saving users a lot of time and effort.

[2349] The processing flow will be explained below.

[2350] User Registration and Authentication

[2351] User Registration

[2352] Step 1:

[2353] The user enters the required information such as name, email address, and password on the new registration screen and clicks the "Register" button.

[2354] Step 2:

[2355] The terminal sends the input information to the server.

[2356] Step 3:

[2357] The server validates the information it receives to ensure it is in the proper format.

[2358] Step 4:

[2359] The server stores the information in a database.

[2360] Step 5:

[2361] The server generates a registration completion message and sends it to the terminal.

[2362] Step 6:

[2363] The terminal displays a completion message.

[2364] User Authentication

[2365] Step 1:

[2366] The user enters their email address and password on the login screen and clicks the "Login" button.

[2367] Step 2:

[2368] The device sends the authentication information to the server.

[2369] Step 3:

[2370] The server will look up the database and match it with the information you entered.

[2371] Step 4:

[2372] If the server finds the match successful, it initiates the session and generates an authentication success message.

[2373] Step 5:

[2374] The server sends an authentication success message to the terminal.

[2375] Step 6:

[2376] The device will transition to the home screen and display a welcome message.

[2377] Conversational document creation

[2378] Start creating materials

[2379] Step 1:

[2380] The user communicates to the server via voice or text through the terminal that "I want to create presentation materials."

[2381] Step 2:

[2382] The server launches a natural language processing engine to analyze the user's intent.

[2383] Step 3:

[2384] The server generates a question, "What is the purpose of the document?" and sends it to the terminal.

[2385] Step 4:

[2386] Your device will display or read the question aloud.

[2387] User Input

[2388] Step 1:

[2389] The user responds to the terminal with information about the purpose and content of the material.

[2390] Step 2:

[2391] The terminal sends the answer to the server.

[2392] Step 3:

[2393] The server analyzes the responses and generates an initial draft of the material structure.

[2394] Step 4:

[2395] The server sends the proposed configuration to the terminal and asks the user for feedback.

[2396] Step 5:

[2397] The device displays the configuration plan.

[2398] feedback

[2399] Step 1:

[2400] The user checks the proposed document structure and inputs corrections and additional instructions into the terminal.

[2401] Step 2:

[2402] The terminal sends the instruction to the server.

[2403] Step 3:

[2404] The server recreates the draft document structure based on the instructions.

[2405] Step 4:

[2406] The server sends the updated configuration plan to the terminal.

[2407] Step 5:

[2408] The device displays the updated configuration.

[2409] Automated data and diagram creation using emotion recognition

[2410] Emotion recognition and data insertion requests

[2411] Step 1:

[2412] When a user creates a document using voice or text, the device sends the input to the emotion engine.

[2413] Step 2:

[2414] The emotion engine analyzes the user's emotions in real time and sends the results to the server.

[2415] Step 3:

[2416] The server uses the emotion recognition results to adjust the composition of the material.

[2417] Step 4:

[2418] The user instructs the terminal to "insert a graph of sales data."

[2419] Step 5:

[2420] The terminal sends the instruction to the server.

[2421] Step 6:

[2422] The server accesses the database or external data source to retrieve the required data.

[2423] Step 7:

[2424] The server converts the retrieved data into an appropriate graph format.

[2425] Step 8:

[2426] The server sends the generated graph to the terminal.

[2427] Step 9:

[2428] The terminal displays the graph.

[2429] Emotion Recognition and Data Verification

[2430] Step 1:

[2431] The user checks the displayed graph and inputs corrections or additional instructions into the terminal.

[2432] Step 2:

[2433] The terminal sends the instruction to the server.

[2434] Step 3:

[2435] The server regenerates the graph as needed and sends it to the device.

[2436] Step 4:

[2437] The terminal will display the regenerated graph.

[2438] Checking the direction of the material and the story

[2439] Check the story

[2440] Step 1:

[2441] The server analyzes the overall structure of the material and the flow of the story.

[2442] Step 2:

[2443] The server determines weak areas and areas for improvement and generates suggestions.

[2444] Step 3:

[2445] The server sends the proposal to the device.

[2446] Step 4:

[2447] The device will display suggestions.

[2448] User Verification

[2449] Step 1:

[2450] The user checks the proposal and inputs approval or correction instructions into the terminal.

[2451] Step 2:

[2452] The terminal sends the instruction to the server.

[2453] Step 3:

[2454] The server optimizes the materials based on the instructions.

[2455] Step 4:

[2456] The server sends the optimized material to the terminal.

[2457] Step 5:

[2458] The device displays optimized materials.

[2459] Optimization and concept review

[2460] Final Check

[2461] Step 1:

[2462] The server will thoroughly review the completed document and report any errors or improvements.

[2463] Step 2:

[2464] The server sends the report to the device.

[2465] Step 3:

[2466] The terminal displays the report.

[2467] Concept reconfirmation

[2468] Step 1:

[2469] The user initiates a dialogue at the terminal to reaffirm the concept or message.

[2470] Step 2:

[2471] The terminal sends the corresponding content to the server.

[2472] Step 3:

[2473] The server interacts with the user to reconfirm the concepts and messages of the materials.

[2474] Step 4:

[2475] The server will fine-tune the material as needed.

[2476] Step 5:

[2477] The server sends the final materials to the terminal.

[2478] Step 6:

[2479] The terminal displays the completed document.

[2480] Example 2

[2481] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2482] Conventional document creation systems require users to manually design document structures and convert data into graphs and tables, which requires a great deal of time and effort. Furthermore, it is difficult to create detailed documents that reflect the user's emotions and intentions. The present invention aims to solve these problems and provide a system that supports users in creating high-quality documents efficiently and easily.

[2483] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2484] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, emotion recognition means for recognizing the user's emotions in real time, and means for automatically adjusting the material composition plan based on the recognized emotions. This makes it possible to automatically generate and adjust an appropriate material composition that reflects emotions simply by providing instructions from the user.

[2485] A "document creation request" refers to an instruction or request made by a user to the system to request the creation of a document.

[2486] "Voice or text format" refers to an interface that allows a user to give instructions to the system, and is in the form of voice input or text input.

[2487] A "natural language processing engine" refers to algorithms and software that analyze human language and understand its intent and meaning.

[2488] "Analysis results" refers to the information and data obtained by the natural language processing engine when it analyzes the user's instructions.

[2489] "Initial document structure proposal" refers to the initial structure and content of the document that the system automatically generates based on the analysis results.

[2490] "Feedback" refers to the evaluations, opinions, and correction requests that users provide to the system.

[2491] "Emotion recognition means" refers to algorithms and software for analyzing and understanding a user's emotions in real time.

[2492] "Means for automatically adjusting the proposed document structure" refers to functions or algorithms that automatically change or adjust the structure or content of a document based on recognized emotions.

[2493] A "database" refers to a system for systematically storing and managing information.

[2494] "External data source" refers to a source or service that provides data that exists outside the system.

[2495] "Converting into graph or table format" refers to the process or task of formatting acquired data into a form that is visually easy to understand.

[2496] "Means for inserting into documents" refers to functions and processes for placing the generated graphs and tables in appropriate positions in the documents.

[2497] "The overall structure of the material and the flow of the story" refers to the arrangement of each section or part within the material and ensuring that they have a continuous meaning.

[2498] "Measures for determining weaknesses and areas for improvement" refers to algorithms or processes used to identify gaps or areas in the content or structure of a document that need correction.

[2499] "Means for presenting suggestions" refers to the functions and processes for providing users with specific actions for improvement or change suggestions.

[2500] "Means for automatically optimizing materials" refers to functions and algorithms that automatically adjust and improve the content of materials to the most appropriate form based on user instructions and emotion recognition results.

[2501] This invention relates to an AI system that assists users in creating documents efficiently. The system is equipped with an emotion engine that recognizes the user's emotions and optimizes the document creation process. The system allows the user to input instructions in voice or text format, and automatically creates document structure proposals and visualizes data based on those instructions. The system can also recognize emotions from the user's input and adjust the feedback content and document structure based on those emotions.

[2502] Hardware and Software Configuration

[2503] server

[2504] Database: A system for systematically storing and managing information

[2505] Natural language processing engine: Software for analyzing human language and understanding intent, such as the Google NLP API

[2506] Emotion recognition engine: Software for recognizing user emotions in real time, such as IBM Watson Tone Analyzer

[2507] Data visualization tools: Software for transforming data into graphs and tables, such as Matplotlib, Pandas DataFrame, etc.

[2508] Terminal

[2509] Speech recognition engine: Software that converts a user's speech into text, such as the Google Speech-to-Text API.

[2510] User interface: An interface for receiving voice commands, text input, and feedback

[2511] Example of implementation

[2512] User Registration and Login

[2513] 1. The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[2514] 2. The device sends the input information to the server.

[2515] 3. The server receives the information and stores it in a database.

[2516] 4. After registration, the user is taken to the login screen and enters their email address and password again.

[2517] 5. The server authenticates this and, if successful, starts the session.

[2518] From the start of document creation to the creation of a structure plan

[2519] 1. The user speaks into the terminal, saying, "I want to create promotional materials for a new product."

[2520] 2. The device converts the voice into text and sends it to the server.

[2521] 3. The server parses the instructions using a natural language processing engine.

[2522] 4. The server generates a question such as "What is the purpose of the material?" and sends it to the terminal.

[2523] 5. The device presents the question to the user on the screen or by voice.

[2524] 6. The user answers, "I would like to explain the features of the new product and promote sales."

[2525] 7. The server analyzes the answers and automatically generates an initial proposal for the material structure.

[2526] Emotion recognition and data insertion, optimization

[2527] 1. The user provides input, including emotional expressions, via voice or text.

[2528] 2. The server uses an emotion engine to recognize the user's emotions in real time.

[2529] 3. Based on the recognized emotions, the server adjusts the proposed document composition in real time.

[2530] 4. The user says, "Insert a graph of sales data."

[2531] 5. The server retrieves the sales data from the database and converts it into a graph.

[2532] 6. The graph generated by the server is inserted into the document and displayed on the terminal.

[2533] 7. The server checks the overall structure of the document and the flow of the story, determines areas for improvement based on user emotional data, and makes optimization suggestions.

[2534] Prompt Sentence Examples

[2535] "I would like to register as a user. What is the process?"

[2536] "What are the steps to creating promotional materials for a new product?"

[2537] "How can I convert sales data into a graph and insert it into a presentation?"

[2538] "Show us how to optimize the overall structure of your materials based on emotions."

[2539] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2540] Step 1:

[2541] The user enters their name, email address, and password on the new registration screen and clicks the "Register" button.

[2542] The input contents are the user's name, email address, and password.

[2543] The terminal transmits the input information to the server.

[2544] The server stores the received information in a database, with the password hashed for security purposes.

[2545] Input: User information (name, email address, password)

[2546] Output: User information stored in the database

[2547] Step 2:

[2548] The user enters their email address and password on the login screen.

[2549] The input contents are the user's email address and password.

[2550] The terminal sends the input information to the server.

[2551] The server searches the database for the corresponding record and checks whether the entered information matches the hashed password.

[2552] When the authentication is successful, the server starts a session and sends a message of successful authentication to the terminal.

[2553] Input: Login information (email address, password)

[2554] Output: Session start message

[2555] Step 3:

[2556] The user speaks into the device and says, "I want to create a presentation," or inputs text.

[2557] The input may be a voice command or a text command.

[2558] The device converts the voice into text format and sends it to the server.

[2559] The server parses the input using a natural language processing engine.

[2560] Input: Voice or text instructions

[2561] Output: Analysis results (user's intentions and goals)

[2562] Step 4:

[2563] The server asks the user, "What is the purpose of the document?"

[2564] The terminal presents the question to the user.

[2565] When the user answers the question about the purpose and content, the device sends the answer to the server.

[2566] Input: User's answer (purpose of the material)

[2567] Output: Analysis results (information necessary for document creation)

[2568] Step 5:

[2569] The server generates an initial proposal for the material structure based on the answers.

[2570] The composition suggestions are generated using a natural language generation engine (e.g., a generative AI model).

[2571] The server sends the proposed configuration to the terminal and asks the user for feedback.

[2572] Input: Analysis results (purpose and content of the data)

[2573] Output: Initial document structure proposal

[2574] Step 6:

[2575] The user provides feedback via voice or text.

[2576] The input is the user's feedback.

[2577] The server uses an emotion engine to recognize the user's emotions in real time.

[2578] The server adjusts the proposed document structure based on the feedback and the results of sentiment analysis.

[2579] Input: Feedback content, sentiment analysis results

[2580] Output: Adjusted document structure proposal

[2581] Step 7:

[2582] The user indicates, "I want to insert a graph of sales data."

[2583] The input contents are specific instructions from the user.

[2584] The terminal sends the instruction to the server.

[2585] The server accesses the database or external data source to retrieve the required data.

[2586] The server converts the acquired data into graph or table format.

[2587] The graph generated by the server is inserted into the document and displayed on the terminal.

[2588] Input: Data insertion instructions

[2589] Output: Inserted data (graphs and tables)

[2590] Step 8:

[2591] The server analyzes the overall structure and story flow of the material and identifies weaknesses and areas for improvement.

[2592] The server optimizes the content of the feedback based on the recognized emotion.

[2593] The server determines areas for improvement and presents suggestions to the user.

[2594] Input: Data structure, sentiment analysis results

[2595] Output: Optimized materials and recommendations

[2596] (Application example 2)

[2597] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2598] Current document creation systems have difficulty recognizing user emotions in real time and appropriately reflecting feedback, which results in users being unable to efficiently create high-quality documents. Similar problems exist in manufacturing sites when creating operator instructions, particularly due to a lack of emotion-based optimization, which is labor-intensive and time-consuming. This leads to issues such as reduced manufacturing efficiency and a lack of consistent quality.

[2599] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2600] In this invention, the server includes means for receiving a material creation request from a user in voice or text format, means for analyzing the user's intentions using a natural language processing engine, means for automatically generating an initial material composition plan based on the analysis results and requesting feedback from the user, means for recognizing the user's emotions in real time and adjusting the material composition plan, and means for acquiring data and diagrams related to the manufacturing process and automatically inserting them into instructions, thereby enabling users to efficiently create high-quality materials and manufacturing instructions.

[2601] A "user" is an entity that uses the system to create materials and instructions.

[2602] A "material creation request" is a request for material creation made by a user to the system in voice or text format.

[2603] A "natural language processing engine" is a technology that analyzes a user's voice or text input and understands their intent.

[2604] The "initial document structure proposal" is a proposal for the initial structure of the document that is automatically generated by the system based on the user's input.

[2605] "Feedback" refers to opinions or requests for corrections made by the user regarding the configuration plan or data provided by the system.

[2606] "Recognizing emotions in real time" means instantly analyzing and determining the user's emotional state from input voice or text.

[2607] "Adjusting the proposed document structure" means optimizing the structure of the document based on emotions recognized in real time.

[2608] "Manufacturing process data" refers to specific data or information related to the manufacturing process.

[2609] "Automatically insert into instructions" means that the system automatically incorporates the acquired data into the instructions.

[2610] "Instructions" are documents that describe the steps and information required for an operator to follow a manufacturing process.

[2611] "Technology used" refers to the specific technology or method used in the invention.

[2612] The system embodying the present invention assists users in efficiently creating materials and manufacturing instructions. A specific embodiment of the system is described below.

[2613] First, a user inputs a document creation request in voice or text format using smart glasses, a smartphone, a head-mounted display, or a robot. The server receives this input and analyzes the user's intent using a natural language processing engine (e.g., spaCy). Based on the analysis results, an initial document structure proposal is automatically generated and presented to the user for feedback.

[2614] Next, an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) is activated to recognize emotions in real time from the user's voice and text input. This allows the proposed document structure to be adjusted based on the user's emotional state. For example, if the user is feeling impatient or frustrated, the proposed structure may be simplified or additional support may be provided.

[2615] During the manufacturing process, operators give instructions in voice or text format. These instructions are also analyzed by the server, and data and diagrams related to the manufacturing process are retrieved from a database (e.g., MongoDB) and automatically inserted into the instructions. This data is also converted into graphs and tables using libraries such as D3.js.

[2616] Furthermore, the server analyzes the overall structure and story flow of the materials, identifies weaknesses and areas for improvement, and presents suggestions to the user. This automatically optimizes the entire materials and instructions. The server uses a content analysis algorithm (e.g., the BERT model) to provide optimal suggestions for the user.

[2617] As a concrete example, consider the case where an operator gives a voice command such as, "I would like to create manufacturing process instructions for a new engine part." In this case, the system converts the voice into text format and analyzes it using a natural language processing engine. The system then asks, "What are the main contents of the manufacturing process?" If the operator responds with, "Assembly, testing, and shipping," an initial draft of the instruction structure is generated. If the system senses impatience in the operator's voice, it will suggest simplifying the process.

[2618] A concrete example of a prompt sentence is as follows:

[2619] “An operator says, ‘I want to create instructions for manufacturing an engine part.’ The system then asks, ‘What are the main components of the manufacturing process?’ The answer is, ‘Assembly, testing, and shipping.’ If the operator sounds impatient, how does the system adjust the proposed configuration?”

[2620] Such a system enables users and operators to efficiently create high-quality documents and manufacturing instructions.

[2621] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2622] Step 1:

[2623] The user uses smart glasses, a smartphone, a head-mounted display, or a robot to input a request for document creation in voice or text format. The input request is received by the device, and if it is in text format, it is converted to text using a speech recognition API (e.g., Google Speech-to-Text). Input: Voice / text request, Output: Text format request.

[2624] Step 2:

[2625] The device sends the received text request to the server. The server uses a natural language processing engine (e.g., spaCy) to analyze the request and understand the user's intent. Input: Text request, Output: Analyzed user intent.

[2626] Step 3:

[2627] The server automatically generates an initial document structure proposal based on the analysis results. The structure proposal is generated using a template engine (e.g., Jinja2). The generated structure proposal is sent to the terminal and the user is asked for feedback. Input: Analyzed user intent, Output: Initial document structure proposal.

[2628] Step 4:

[2629] The user inputs feedback in voice or text format, and the device receives it. Similarly, if the voice is input, it is converted into text format using a speech recognition API. Input: Voice / Text Feedback, Output: Text Feedback.

[2630] Step 5:

[2631] The device sends the received feedback to the server, which uses an emotion recognition engine (e.g., an emotion analysis model using TensorFlow) to recognize the user's emotions in real time. Input: Text feedback, Output: Recognized emotions.

[2632] Step 6:

[2633] The server adjusts the proposed document composition based on the recognized emotion. If the emotion is impatience or dissatisfaction, the composition may be simplified or additional support may be provided. The adjusted composition is sent to the terminal and presented to the user. Input: Recognized emotion, Output: Adjusted document composition.

[2634] Step 7:

[2635] The user requests the insertion of data or diagrams related to the manufacturing process in voice or text format. The device receives the request and sends it to the server. The server accesses a database (e.g., MongoDB) or external data source to obtain the required data, and converts the data into a graph or table format using a library such as D3.js. Input: Data insertion request in text format, Output: Data in graph or table format.

[2636] Step 8:

[2637] The server automatically inserts the acquired and converted data into the instruction sheet. The instruction sheet is dynamically updated by the template engine, and after creation, it is sent to the terminal and presented to the user. Input: Data in graph or table format, Output: Updated instruction sheet.

[2638] Step 9:

[2639] The server analyzes the overall structure and story flow of the document, identifies weaknesses and areas for improvement, and presents suggestions to the user. Optimal improvement suggestions are generated using a content analysis algorithm (e.g., the BERT model). Input: Overall structure of the document, Output: Improvement suggestions.

[2640] Step 10:

[2641] The user receives suggestions from the server and confirms the final materials and instructions. If there is any final feedback, it is similarly entered by voice or text and final adjustments are made. Input: User's final confirmation and feedback, Output: Final confirmed materials or instructions.

[2642] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2643] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2644] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2645] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2646] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2647] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2648] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2649] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discom...

Claims

1. means for receiving a document creation request from a user in voice or text format; A means for analyzing user intent using a natural language processing engine; A method for automatically generating an initial document structure proposal based on the analysis results and requesting feedback from the user; A system including:

2. Access a database or external data source to retrieve the required data, Including means for automatically converting acquired data into graphs or tables and inserting them into documents; The system of claim 1 .

3. Analyze the overall structure of the material and the flow of the story, A means of determining weaknesses and areas for improvement and providing suggestions to users; including means for automatically optimizing the material based on user instructions; The system of claim 1 .

Images