System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving unknown points from a user; means for analyzing the received unknown points using a generative AI model; means for automatically generating a manual based on the analysis results; and means for transmitting the generated manual to the user's device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional EOS data entry, when users encountered questions, there was a lack of tools to quickly and accurately resolve those questions. Users had to manually look up the manual, which took time and effort, and they sometimes couldn't find the information they needed. Creating the manual itself was also time-consuming, hindering efficient work progress. This invention aims to solve these problems by quickly analyzing users' questions and automatically generating appropriate manuals, thereby improving the efficiency of resolving questions and reducing the effort required to create manuals. [Means for solving the problem]

[0005] The system includes a means for receiving unclear data from users, a means for analyzing the received unclear data using a generative AI model, a means for automatically generating a manual based on the analysis results, and a means for sending the generated manual to the user's device. This makes it possible to quickly analyze the unclear data entered by the user and automatically generate and provide an easily understandable manual in natural language. Furthermore, the generated manual is provided in HTML or PDF format, allowing users to immediately check the information they need. As a result, the resolution of unclear data becomes more efficient and users' work efficiency improves.

[0006] "User" refers to the entity that operates the system and inputs any questions.

[0007] "Unclear data" refers to information entered by users about matters or questions they do not understand while operating the system.

[0008] A "generative AI model" refers to an artificial intelligence model that analyzes received unknown data and automatically generates the necessary information and procedures.

[0009] "Analysis" refers to the process of evaluating unknown data and identifying information needed by the user.

[0010] A "manual" refers to a document or guideline that provides explanations and procedures for solving unclear points.

[0011] "Means of receiving" refers to the functions and methods for incorporating user input data and unclear data into the system.

[0012] "Means of analysis" refers to the functions and methods of using a generative AI model to analyze received unknown data and extract the necessary information.

[0013] "Means for automatic generation" refers to functions and methods for automatically creating manuals using the analysis results of a generative AI model.

[0014] "Means of transmission" refers to the functions and methods for delivering the generated manual to the user's device.

[0015] "Terminal" refers to the device (e.g., PC, smartphone, tablet, etc.) used by a user to operate the system.

[0016] "HTML format" is one of the standard markup languages ​​for constructing web pages, and refers to the format in which the generated manual is displayed in a web browser.

[0017] "PDF format" is a standard format for electronic documents, and refers to the format in which generated manuals are provided in a form that is easy to print and share. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The system of the present invention is designed to quickly and accurately resolve any questions users may encounter while entering EOS data. The system receives questions from users, analyzes them using a generative AI model, and automatically generates a manual based on the results to provide to the user.

[0040] Server Processing

[0041] 1. Receiving input from the user

[0042] The server receives EOS input data and unclear data sent from the user's terminal. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[0043] 2. Analysis using generative AI models

[0044] The received unknown data is passed to the generative AI model, which then uses natural language processing technology to deeply understand the unknown content and identify its key points.

[0045] 3. Generate manual

[0046] Based on the analysis results, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed.

[0047] 4. Provision to Users

[0048] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[0049] Terminal handling

[0050] 1. Sending input data

[0051] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[0052] 2. Receiving the manual

[0053] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[0054] User Actions

[0055] 1. EOS input

[0056] The user inputs the EOS data using the terminal.

[0057] 2. Enter any questions

[0058] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[0059] 3. Check the manual

[0060] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[0061] Specific examples

[0062] Example 1: New user registration procedure

[0063] 1. Users

[0064] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0065] 2. Terminal

[0066] The entered unknown data is sent to the server.

[0067] 3. Server

[0068] The received unknown data is passed to the generative AI model for analysis.

[0069] Based on the analysis results, a manual including user registration procedures is automatically generated.

[0070] The generated manual is sent to the terminal in HTML format.

[0071] 4. Terminal

[0072] Receive the manual and view it in your web browser.

[0073] 5. Users

[0074] Please refer to the manual and proceed while understanding the new user registration procedure.

[0075] Example 2: How to set up data backup

[0076] 1. Users

[0077] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[0078] 2. Terminal

[0079] The entered unknown data is sent to the server.

[0080] 3. Server

[0081] The received unknown data is passed to the generative AI model for analysis.

[0082] Based on the analysis results, a manual including data backup setup procedures is automatically generated.

[0083] The generated manual is sent to the terminal in PDF format.

[0084] 4. Terminal

[0085] Receive the manual and view it in a PDF viewer.

[0086] 5. Users

[0087] Refer to the manual and proceed while understanding how to set up data backup.

[0088] In this way, this system analyzes unclear points based on the data entered by the user and provides immediate solutions, thereby improving work efficiency and reducing work stress.

[0089] The processing flow will be explained below.

[0090] Step 1:

[0091] user

[0092] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the details in the text box (e.g., "I don't know how to register in the system").

[0093] Step 2:

[0094] Terminal

[0095] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the submit button, the data is sent to the endpoint API.

[0096] Step 3:

[0097] server

[0098] Receives data sent from the device. The API receives the request and checks the data. Validates that the received data contains any unknowns.

[0099] Step 4:

[0100] server

[0101] Initialize the generative AI model and prepare to input the received unknown data into the model.

[0102] Step 5:

[0103] Generative AI Models

[0104] Input the data of the unknown points and start the analysis. Natural language processing technology is used for the analysis to deeply understand the content of the unknown points and identify the problem points.

[0105] Step 6:

[0106] Generative AI Models

[0107] Based on the analysis results, necessary information and procedures are extracted. For example, if a user is unsure about how to register in the system, specific procedures for registering in the system are generated.

[0108] Step 7:

[0109] server

[0110] The analysis results from the generative AI model are used to construct the content of the manual, which includes explanatory text, procedures, illustrations, and other information, formatted in a user-friendly format.

[0111] Step 8:

[0112] server

[0113] Converts the generated manual into HTML or PDF format, depending on the system settings or user requirements.

[0114] Step 9:

[0115] server

[0116] Formatted manual data is sent to the terminal using an endpoint, ensuring that it is delivered to the user quickly.

[0117] Step 10:

[0118] Terminal

[0119] Receives manual data sent from the server. The received manual is processed for display in a web browser or PDF viewer.

[0120] Step 11:

[0121] user

[0122] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions, resolves any questions, and continues working. For example, the user can refer to the system registration procedure to complete registration.

[0123] In this way, the entire process flow is achieved quickly and efficiently.

[0124] Example 1

[0125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] Previous systems lacked a means to quickly and accurately resolve any questions users might encounter while entering EOS data. This resulted in users spending a lot of time resolving problems, reducing work efficiency. There was also the risk that users would rely on inaccurate information when conducting their own research.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0128] In this invention, the server includes a means for receiving input data from a user, a means for analyzing the received unclear point data using a generative AI model, and a means for automatically generating a manual based on the analysis results. This makes it possible to quickly analyze the user's unclear points and provide appropriate solutions. In addition, the generated manual is provided in HTML or PDF format, improving the convenience of users by allowing them to refer to it immediately.

[0129] "Means for receiving input data from a user" is a function that allows the server to receive information that a user has input using a terminal.

[0130] "Means for analyzing received unclear data using a generative AI model" refers to a function that uses a generative AI model that employs natural language processing technology to perform a detailed analysis of the unclear data entered by the user and understand its content.

[0131] "Means for automatically generating manuals based on analysis results" is a function that automatically creates manuals in a format that is easy for users to understand, based on the results of analysis by the generative AI model.

[0132] "Means for sending the generated manual to the user's terminal" refers to a function for sending the automatically generated manual to the user's terminal in an appropriate format (HTML or PDF).

[0133] "Means for sending input data to a server in JSON format" is a function for converting data entered by a user on a terminal into JSON format and sending it to a server.

[0134] "Means for providing to users in HTML or PDF format" refers to a function for converting the generated manual into HTML or PDF format and providing it to users in a format that is easily accessible.

[0135] The system of the present invention is designed to quickly and accurately resolve any unclear points users may encounter while entering EOS data. The system receives input data from the user, analyzes it using a generative AI model, and automatically generates a manual based on the analysis results to provide to the user. A specific implementation of the present invention is described below.

[0136] Server Processing

[0137] The server includes hardware and software for receiving EOS input data and unknown data sent from the user's device. For example, the server has an API endpoint for processing HTTP requests and has the functionality to appropriately receive input data from the user. This data is often sent in JSON format, so the server parses and stores the JSON data.

[0138] Next, the server passes the received unknown data to a generative AI model to begin analysis. The generative AI model uses natural language processing technologies such as GPT-3 and BERT. This allows it to deeply understand the content of the unknown and analyze it. At this time, the specific prompt text passed to the model is the user's input, such as "I don't know how to register in the system."

[0139] Based on the analysis results, the server automatically generates a manual using the information obtained from the generative AI model. This manual contains specific procedures and explanations, and inserts diagrams and tables as needed. The server then converts the generated manual into HTML or PDF format using an HTML template or a PDF generation library (e.g., Weasyprint).

[0140] The completed manual is sent to the user's device in HTML or PDF format, where the user can view it immediately. For example, the manual data generated by "POST / api / manuals" is sent to the user's device.

[0141] Terminal handling

[0142] The terminal provides an interface for users to input EOS data. When a user inputs an unknown point, the data is sent to the server in JSON format. For example, the unknown point data is sent by "POST / api / queries".

[0143] The device then receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the device's browser or PDF viewer, allowing the user to view detailed instructions and explanations and solve problems on the spot.

[0144] User Actions

[0145] Users enter EOS data using a terminal. They enter the data in detail on the interface, and if they have any questions, they enter them and send them to the server. For example, they can enter a specific question such as, "I don't know how to register in the system."

[0146] Once the manual sent from the server is received by the terminal, the user can open it in a browser or PDF viewer and check the contents. For example, the user can read the manual that details the new user registration procedure and proceed by following the steps.

[0147] Specific examples

[0148] Example: New user registration procedure

[0149] 1. Users

[0150] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0151] 2. Terminal

[0152] The inputted unknown data is sent to the server.

[0153] 3. Server

[0154] The received unknown data is passed to the generative AI model for analysis.

[0155] Based on the analysis results, a manual including user registration procedures is automatically generated.

[0156] The generated manual is sent to the terminal in HTML format.

[0157] 4. Terminal

[0158] Receive the manual and view it in your web browser.

[0159] 5. Users

[0160] Please refer to the manual and proceed while understanding the new user registration procedure.

[0161] In this way, the system analyzes unclear points based on the data entered by the user and provides prompt solutions, thereby improving work efficiency and reducing work stress.

[0162] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0163] Step 1: Sending input data from the user

[0164] 1. Input: The user inputs EOS data using the terminal, and if there is anything unclear, they enter the details.

[0165] 2. Specific action: The user enters specific questions, such as "I don't know how to register in the system," into the input form on the interface and presses the submit button.

[0166] 3. Data processing / calculation: The terminal converts the data entered by the user into JSON format.

[0167] 4. Output: Send the unknown data in JSON format to the server (e.g., send to "POST / api / queries").

[0168] Step 2: Receiving input data from the user

[0169] 1. Input: Unknown data in JSON format sent from the terminal.

[0170] 2. Specific operation: The server receives the HTTP request and retrieves the unknown data.

[0171] 3. Data processing / calculation: Parse (analyze) the received JSON data and save it in the internal database.

[0172] 4. Output: The saved instances of the unknown data.

[0173] Step 3: Analysis by generative AI model

[0174] 1. Input: Stored unknown data.

[0175] 2. Specific operation: The server passes the stored unknown data to the generative AI model (e.g., GPT-3) as a prompt sentence.

[0176] 3. Data processing / calculation: The generative AI model performs analysis based on the prompt text and generates appropriate answers and procedures.

[0177] 4. Output: Analysis results from the generative AI model (e.g., specific operating procedures).

[0178] Step 4: Automatic generation of manuals

[0179] 1. Input: Analysis results from the generative AI model.

[0180] 2. Specific operation: The server generates a manual including text and illustrations based on the analysis results.

[0181] 3. Data processing / calculation: Format the analysis results using HTML templates or a PDF generation library (e.g., Weasyprint).

[0182] 4. Output: Manual in HTML or PDF format.

[0183] Step 5: Manually send to users

[0184] 1. Input: Manual in HTML or PDF format.

[0185] 2. Specific operation: The server creates an HTTP response to return the generated manual to the user's device.

[0186] 3. Data processing / calculation: Include manual data in the HTTP response body.

[0187] 4. Output: An HTTP response containing the manual is sent to the user's device (e.g., sent with "POST / api / manuals").

[0188] Step 6: Receive and view the manual

[0189] 1. Input: The HTTP response sent by the server (including manual data).

[0190] 2. Specific operation: The terminal receives the HTTP response and extracts the included manual data.

[0191] 3. Data processing / calculation: Convert the extracted manual data into a format that can be displayed in an HTML or PDF viewer.

[0192] 4. Output: The manual that will be displayed on the user's device.

[0193] Step 7: Refer to your user manual

[0194] 1. Input: Manual displayed on the terminal.

[0195] 2. Specific actions: The user refers to the manual displayed on the device and checks the steps to resolve any questions.

[0196] 3. Data processing / calculation: The user performs the operations according to the contents of the manual.

[0197] 4. Output: Resolved issues and smooth progress of work. The end result is that issues are resolved and work proceeds smoothly.

[0198] (Application example 1)

[0199] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0200] This invention relates to technology for quickly and accurately resolving questions and problems that arise during the operation of industrial robots. In conventional industrial robot systems, when a problem occurs, the intervention of an operator with specialized knowledge is required to resolve the problem, which inevitably leads to interruptions in work and reduced efficiency. Such problems reduce the productivity of industrial processes and increase maintenance costs. Meanwhile, there is a demand for systems in which robots themselves have the ability to solve problems and respond in real time.

[0201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0202] In this invention, the server includes means for receiving unclear point data from a user, means for analyzing the received unclear point data using a generative AI model, means for automatically generating a manual based on the analysis results, means for transmitting the generated manual to the user's terminal, means for the industrial robot to recognize problems through sensors or cameras while in operation and transmit the data in JSON format to the server, means for the generative AI model to identify solutions based on the unclear point data and generate a manual in HTML or PDF format that details the solutions, and means for providing the generated manual to the industrial robot's display device in real time. This enables the industrial robot to recognize unclear points on its own while in operation, obtain solutions in real time via the server, and quickly solve problems.

[0203] "Means for receiving unclear data from users" refers to an input interface that allows users to report unclear or questionable points to the system, and typically handles data in JSON format.

[0204] "Means of analyzing received unknown data using a generative AI model" refers to the process of passing received data to a generative AI model that uses natural language processing technology, and analyzing and understanding its contents.

[0205] "Means for automatically generating manuals based on analysis results" refers to algorithms or scripts for automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from the generative AI model.

[0206] "Means for sending the generated manual to the user's terminal" refers to a sending function for providing the generated manual to the user in HTML or PDF format in real time.

[0207] "A means for industrial robots to detect problems through sensors and cameras while in operation and send the data to a server in JSON format" refers to the process of using sensors and cameras to detect the status and problems of robots and send that information to a server.

[0208] "Means for the generative AI model to identify a solution and generate a manual in HTML or PDF format detailing that solution" refers to the function of using a generative AI model to find a solution to a problem and automatically generating a manual in HTML or PDF format detailing that solution in an easy-to-understand manner.

[0209] "Means for providing the generated manual to the display device of the industrial robot in real time" refers to the function of displaying the generated manual on the display of the industrial robot in real time, so that the operator and the robot itself can immediately refer to it.

[0210] The system of the present invention is a support system for quickly and accurately resolving questions and problems that industrial robots operating in factories encounter during operation. This system uses hardware such as industrial robots, servers, sensors, cameras, and displays, as well as software such as generative AI models (e.g., OpenAI GPT-4), Python, Flask (a web framework), and a JSON-formatted data interface.

[0211] The server includes the following means:

[0212] 1. Means of receiving unclear data from users

[0213] It is a means for users or robots to send and receive problems or questions they encounter in JSON format to the server.

[0214] 2. A means of analyzing incoming unknown data using a generative AI model

[0215] This is the process of passing the received data to a generative AI model (e.g., OpenAI GPT-4) that uses natural language processing technology to analyze and understand its content.

[0216] 3. A method for automatically generating manuals based on analysis results

[0217] It is a means of automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from a generative AI model.

[0218] 4. Means for sending the generated manual to the user's device

[0219] This function allows the generated manual to be sent to the user in HTML or PDF format for viewing.

[0220] 5. A way for industrial robots to detect problems through sensors and cameras while in operation and send that data in JSON format to a server

[0221] This is the process of detecting the robot's status and problems using sensors and cameras, and sending that information to a server in JSON format.

[0222] 6. A means for the generative AI model to identify solutions and generate detailed solutions manuals in HTML or PDF format.

[0223] This function uses generative AI models to find solutions to problems and automatically generates detailed and easy-to-understand manuals describing the solutions in HTML or PDF format.

[0224] 7. Means for providing the generated manual to the display device of the industrial robot in real time

[0225] This function allows the generated manual to be displayed in real time on the industrial robot's display, making it available for immediate reference by the operator and the robot itself.

[0226] For example, if an industrial robot detects a problem while in operation, such as "the equipment connection is not working properly," it will send this information, obtained through sensors and cameras, in JSON format to a server. The server will then use a generative AI model based on the received data to analyze the problem and find a solution. Based on the analysis results, it will automatically generate a manual including operating procedures, and display this manual in real time on the industrial robot's display. If a user inputs a point of uncertainty, such as "I don't know how to connect the equipment," the same procedure will be followed, and the generated manual will be sent to the user's device.

[0227] Here are some example prompts:

[0228] Please analyze the following problem and provide a specific solution:

[0229] "Device connection is not working properly"

[0230] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0231] Step 1:

[0232] While industrial robots are operating, they detect problems through sensors and cameras and send the data in JSON format to a server.

[0233] Specifically, the robot's sensors and cameras detect abnormal behavior or error messages. Detailed problem data (e.g., "Device connection is not working properly") is received as input, packaged in JSON format, and sent to the server via an HTTP POST request. The server receives this data and proceeds with further processing.

[0234] Step 2:

[0235] The server passes the received JSON-formatted unknown data to the generative AI model for analysis.

[0236] Specifically, the server reads the received data and sends it to a generative AI model (e.g., OpenAI GPT-4). The input is problem data from the user or robot. The generative AI model analyzes this data and identifies unclear points and solutions. For example, in response to the problem of "equipment connection not working properly," the generative AI model outputs "the correct procedure for connecting cables" and "how to check the connection."

[0237] Step 3:

[0238] Based on the analysis results, the server automatically generates a manual.

[0239] Specifically, it receives the solution from the generative AI model and formats it as detailed instructions in natural language. It uses the analysis results of the generative AI model obtained as input (e.g., "Please double-check the cable and connect it to the correct port") and automatically generates a manual in HTML or PDF format. The output is a detailed manual including operating procedures and illustrations.

[0240] Step 4:

[0241] The generated manual is provided in real time to the display device of the industrial robot.

[0242] Specifically, the server sends the generated manual to the robot's display. The input is the manual data generated in step 3, and the output is the manual displayed on the robot's display. This makes it available for immediate reference by users and robots.

[0243] Step 5:

[0244] If necessary, the generated manual is also sent to the user's terminal.

[0245] Specifically, the server sends the generated manual in HTML or PDF format to the user's device. The input is the manual data generated in step 3, and the output is the manual displayed on the user's device. For example, if a user inquires, "I don't know how to connect the equipment," this manual is sent to the user's device and can be viewed in a browser or PDF viewer.

[0246] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0247] The system of the present invention quickly and accurately resolves any questions users may encounter while entering data into the EOS system. It also recognizes the user's emotions and provides a manual tailored to their state, providing more effective support. The system receives data on the user's questions, analyzes them using a generative AI model, and automatically generates and provides a manual based on the results. Furthermore, by incorporating an emotion engine, the system adjusts the tone and expression of the manual according to the user's emotional state.

[0248] Server Processing

[0249] 1. Receiving input from the user

[0250] The server receives EOS input data and unclear data sent from the user's device. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[0251] 2. Emotion Recognition by Emotion Engine

[0252] The received unknown data is input into the emotion engine to recognize the user's emotion. The emotion engine analyzes the user's emotional state (e.g., anger, sadness, joy) from the text data.

[0253] 3. Analysis using generative AI models

[0254] The recognized user emotion data along with the difficulty data is passed to the generative AI model, which then uses natural language processing technology to identify the unclear points, taking into account the content of the unclear points and the user's emotions.

[0255] 4. Generate manual

[0256] Based on the analysis results and the user's emotions, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed, and adjusts the tone and expression according to the user's emotions.

[0257] 5. Provision to Users

[0258] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[0259] Terminal handling

[0260] 1. Sending input data

[0261] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[0262] 2. Receiving the manual

[0263] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[0264] User Actions

[0265] 1. EOS input

[0266] The user inputs the EOS data using the terminal.

[0267] 2. Enter any questions

[0268] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[0269] 3. Check the manual

[0270] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[0271] Specific examples

[0272] Example 1: New user registration procedure

[0273] 1. Users

[0274] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0275] 2. Terminal

[0276] The entered unknown data is sent to the server.

[0277] 3. Server

[0278] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in an "anger" state.

[0279] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[0280] Based on the analysis results, a manual including user registration procedures is automatically generated, where instructions are explained in more polite language and a more friendly tone depending on the user's emotions.

[0281] The generated manual is sent to the terminal in HTML format.

[0282] 4. Terminal

[0283] Receive the manual and view it in your web browser.

[0284] 5. Users

[0285] Please refer to the manual and proceed while understanding the new user registration procedure.

[0286] Example 2: How to set up data backup

[0287] 1. Users

[0288] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[0289] 2. Terminal

[0290] The entered unknown data is sent to the server.

[0291] 3. Server

[0292] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in a "sad" state.

[0293] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[0294] Based on the analysis results, a manual including instructions for setting up data backups is automatically generated, which includes encouraging words and reassuring expressions depending on the user's emotions.

[0295] The generated manual is sent to the terminal in PDF format.

[0296] 4. Terminal

[0297] Receive the manual and view it in a PDF viewer.

[0298] 5. Users

[0299] Refer to the manual and proceed while understanding how to set up data backup.

[0300] In this way, this system, which incorporates an emotion engine, analyzes the unclear points entered by the user and provides the optimal manual based on their emotions, thereby improving work efficiency and the user experience.

[0301] The processing flow will be explained below.

[0302] Step 1:

[0303] user

[0304] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the specific details in the text box (e.g., "I don't know how to register in the system").

[0305] Step 2:

[0306] Terminal

[0307] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the send button, the data is sent to the server via the REST API.

[0308] Step 3:

[0309] server

[0310] The data sent from the device is received. The API processes the request, validates the data, and then passes any unclear data to the emotion engine.

[0311] Step 4:

[0312] Emotion Engine

[0313] Analyzes the received unknown data to recognize the user's emotional state. Analyzes text data to identify the user's emotions (e.g., anger, sadness, joy).

[0314] Step 5:

[0315] server

[0316] Along with the emotion data obtained from the emotion engine, the unknown data is passed to the generative AI model.

[0317] Step 6:

[0318] Generative AI Models

[0319] Input the unclear point data and emotion data and begin analysis. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and identify the necessary information.

[0320] Step 7:

[0321] Generative AI Models

[0322] Based on the analysis results, necessary steps and explanations are extracted, and the content is adjusted to suit the user's emotions using a tone and expression.

[0323] Step 8:

[0324] server

[0325] Using information from the generative AI model, the system automatically generates a manual that includes instructions, steps, and illustrations, and adjusts tone and expression based on emotional data.

[0326] Step 9:

[0327] server

[0328] Convert the generated manual into HTML or PDF format, the format is selected based on the user's requirements and system settings.

[0329] Step 10:

[0330] server

[0331] Formatted manual data is sent to the terminal. For quick delivery, a REST API is used to send the data.

[0332] Step 11:

[0333] Terminal

[0334] Receives the manual data sent from the server. The received manual is then processed so that it can be displayed in a web browser or PDF viewer.

[0335] Step 12:

[0336] user

[0337] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions and resolves any questions. For example, the user can refer to the system registration procedure to complete the registration.

[0338] This system not only enables users to quickly resolve any questions they may have, but also allows them to receive optimal support tailored to their emotional state at the time.

[0339] Example 2

[0340] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional user support systems simply display simple FAQs or existing manuals in response to data input by users, making it difficult to provide customized support that reflects the situation and emotions of each individual user. Furthermore, there was a lack of systems that could properly recognize the emotional state of users when they input their questions and take appropriate approaches. As a result, the quality of the user experience declined and support efficiency declined.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0343] In this invention, the server includes means for receiving uncertainty data and emotion data from the user, means for analyzing the received uncertainty data and emotion data using an emotion recognition engine, means for generating a prompt sentence based on the received uncertainty data and the analyzed emotion data using a generative AI model, means for obtaining an analysis result using the generative AI model based on the generated prompt sentence, means for automatically generating a manual based on the analysis result and the user's emotion, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide optimal support that takes into account the user's uncertainty and emotional state.

[0344] "User's unclear point data" is text data describing problems and questions that users have encountered while using the system.

[0345] "Emotion data" is data that indicates the emotional state that the user is feeling when inputting the unclear point.

[0346] An "emotion recognition engine" is a program or algorithm that analyzes and identifies a user's emotional state from text data.

[0347] A "generative AI model" is an artificial intelligence model that performs natural language processing based on large amounts of text data, and specifically has the ability to understand the content of a user's inquiry and generate an appropriate response.

[0348] A "prompt" is a textual instruction given to a generative AI model, and is a document that includes an introductory section for analysis.

[0349] "Analysis results" refers to the information and answers analyzed by the generative AI model, and indicate solutions to any questions users may have.

[0350] A "manual" is a document that describes solutions and procedures for problems that users are facing.

[0351] "HTML" is a standard markup language for building web pages.

[0352] "PDF" is a file format designed for displaying and printing electronic documents, and allows documents to be shared while maintaining their format.

[0353] "User device" refers to an electronic device, such as a PC, smartphone, or tablet, that a user uses to operate the system.

[0354] This invention is a support system that quickly and accurately resolves any questions users may have when using a system. The system utilizes a generative AI model to analyze the user's questions and emotional state, providing the most appropriate manual.

[0355] Main hardware and software used

[0356] Hardware: User devices (PCs, smartphones, tablets, etc.), servers (high-performance computing servers)

[0357] Software: Emotion recognition engine (natural language processing library), generative AI model (natural language generation model)

[0358] Data processing and calculation

[0359] Server Processing

[0360] 1. Receiving input data:

[0361] The server receives EOS input data and unclear data sent from the user's device in JSON format. For example, if a user enters "I don't know how to register in the system," this data is parsed by the server.

[0362] 2. Emotion Recognition:

[0363] The server inputs the received unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine is used to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[0364] 3. Generate prompt sentence:

[0365] The server generates a prompt for the generative AI model based on the recognized emotion data and the unclear point data. This prompt contains specific questions and instructions to clearly communicate the unclear point to the generative AI model. For example, it could be something like, "Please create a manual that explains how to register in the system in the most detailed way possible. The user is feeling angry."

[0366] 4. Run the analysis:

[0367] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[0368] 5. Generate manual:

[0369] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[0370] 6. Offerings to Users:

[0371] The server then sends the completed manual to the user's device in HTML or PDF format, allowing the user to instantly view the information they need.

[0372] Terminal handling

[0373] 1. User input submission:

[0374] The terminal sends the EOS data and unknown data entered by the user to the server in JSON format.

[0375] 2. Receiving the manual:

[0376] The terminal receives the manual sent from the server. The manual is displayed in HTML or PDF format, so users can easily view it without needing special software.

[0377] User Actions

[0378] 1. EOS input:

[0379] The user inputs the EOS data using the terminal.

[0380] 2. Submit any questions:

[0381] When a user encounters a problem, they enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system" in the input form.

[0382] 3. Check the manual:

[0383] The user checks the manual sent from the server on the terminal and obtains the necessary information. For example, the received manual contains detailed instructions on how to register the system, and the user can proceed with the work by following the instructions.

[0384] Specific examples

[0385] Example 1: New user registration procedure

[0386] 1. User:

[0387] Enter your question, such as "I don't know how to register as a new user."

[0388] 2. Terminal:

[0389] The inputted unknown data is sent to the server.

[0390] 3. Server:

[0391] The received unknown data is passed to the emotion engine, which recognizes the user's emotion as "anger."

[0392] Generate prompt sentences based on emotion data and unclear point data.

[0393] The prompt sentence, "Please create a manual that explains how to register in the system in the most detailed way possible. Users are feeling angry," is passed to the generative AI model.

[0394] Based on the analysis results obtained from the generative AI model, a manual written in a friendly tone is generated.

[0395] Send the manual to the terminal in HTML format.

[0396] 4. Terminal:

[0397] Receive the manual and view it in your browser.

[0398] 5. User:

[0399] Please refer to the manual and proceed while understanding the new user registration procedure.

[0400] This system makes it possible to quickly provide the most appropriate manual that takes into account the user's uncertainties and emotional state.

[0401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0402] Step 1: Receiving input data

[0403] The server receives EOS input data and unknown data sent from the user's device. The input data is in JSON format, and the data structure consists of key-value pairs. The received data includes the user ID, unknown content, timestamp, etc. The server parses this data and stores each item in the appropriate variable.

[0404] input:

[0405] User EOS input data

[0406] Uncertainty data (e.g., "I don't know how to register in the system")

[0407] output:

[0408] Parsed uncertainty data and other meta information

[0409] Specific operation:

[0410] The server parses the received JSON data and stores the user ID, EOS data, and unknown data in appropriate variables.

[0411] Step 2: Recognize emotions

[0412] The server inputs the parsed unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine uses a natural language processing library to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[0413] input:

[0414] Unknown data

[0415] output:

[0416] Emotion recognition result (e.g., "anger" state)

[0417] Specific operation:

[0418] The emotion recognition engine inputs the unknown text and determines its emotional state, which is then passed to the next processing step.

[0419] Step 3: Generate a prompt statement

[0420] The server generates prompts for the generative AI model based on the recognized emotion data and unclear point data. These prompts include specific questions and instructions to communicate the unclear points to the generative AI model in an easy-to-understand manner.

[0421] input:

[0422] Unknown data

[0423] Emotion recognition results

[0424] output:

[0425] Prompt statement (e.g., "Please create the most thorough manual possible explaining how to register the system. Users are angry.")

[0426] Specific operation:

[0427] The server uses the unknown data and emotion recognition results to generate prompt sentences based on dedicated templates.

[0428] Step 4: Perform the analysis

[0429] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[0430] input:

[0431] Prompt statement

[0432] output:

[0433] Analysis results (e.g. system registration procedure)

[0434] Specific operation:

[0435] The server inputs a prompt sentence into the generative AI model and obtains a solution as an analysis result.

[0436] Step 5: Generate the manual

[0437] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[0438] input:

[0439] Analysis results

[0440] Emotion recognition results

[0441] output:

[0442] Auto-generated manual

[0443] Specific operation:

[0444] The server uses the analysis results and emotion data to generate a manual based on an appropriate template.

[0445] Step 6: Deliver to users

[0446] The server then sends the completed manual to the user's device in HTML or PDF format, with a link and notification automatically sent for immediate access.

[0447] input:

[0448] Auto-generated manual

[0449] output:

[0450] Manual sent to the user's device (HTML or PDF format)

[0451] Specific operation:

[0452] The server converts the manual into an appropriate format and sends it to the user's device. It also generates a notification of completion and an access link, which it provides to the user.

[0453] (Application example 2)

[0454] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0455] A support system that can quickly and appropriately respond to technical questions and problems that drivers and users encounter in autonomous vehicles is needed. Conventional support systems do not provide manuals that take into account the user's emotional state, which can result in increased stress and dissatisfaction for users. Furthermore, conventional technologies take time to resolve users' questions, and there are insufficient means to mitigate safety risks that arise during this time.

[0456] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving unclear data from the user, means for analyzing the received unclear data using a generative AI model, means for automatically generating a manual based on the analysis results and the user's emotional data, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide an appropriate manual using a tone and expression that matches the user's emotional state, enabling immediate responses to technical questions and problems in autonomous vehicles and improving user satisfaction and safety.

[0457] "User" refers to a person who uses the system, specifically a driver or user.

[0458] "Uncertainty data" refers to data about questions or problems users encounter while using the system.

[0459] A "generative AI model" is an artificial intelligence model that analyzes received unknown data and generates corresponding information and solutions.

[0460] "Analysis" refers to the process of examining data in detail and understanding its contents.

[0461] "Emotional data" is data that represents a user's emotional state extracted from their text or speech.

[0462] A "manual" is a document that contains procedures and explanations for resolving user questions and problems.

[0463] "Auto-generation" refers to the process by which a program automatically creates output information from specific input data.

[0464] "Terminal" refers to a device used to use the system, and specifically includes smartphones and in-car tablets.

[0465] This invention is a system that provides appropriate support in real time for questions or problems that users encounter in autonomous vehicles. This system uses smartphones or in-car tablets as terminals, and sends input data from users to a server, which then analyzes it.

[0466] The server uses the following methods:

[0467] A means for receiving data on questions from users: Users use their devices to input text or voice about questions or problems. This input data is sent to the server.

[0468] A method for analyzing received unknown data using a generative AI model: A generative AI model is used to analyze the received data. The generative AI model uses natural language processing techniques to analyze the unknown data and identify its points.

[0469] A means of automatically generating manuals based on analysis results and user emotional data: An emotion engine is used to extract user emotional data, which is then combined with the analysis results to automatically generate manuals that take into account appropriate tone and expression.

[0470] Means for sending the generated manual to the user's terminal: The generated manual is sent to the user's terminal in HTML or PDF format.

[0471] As a concrete example, consider the case where the vehicle camera displays an error message during autonomous driving. When the user types "The vehicle camera is displaying an error message, what should I do?" into the terminal, the following process is executed.

[0472] The server receives the user's input and uses an emotion engine to recognize the user's emotion as "impatience." The generative AI model then analyzes the input and automatically generates a manual with specific steps to deal with the camera's error message. The manual includes a gentle tone and expressions to help users respond calmly to their emotions. The generated manual is sent to the user's device in HTML format and displayed in a window.

[0473] An implementation example shows the use of the following prompt:

[0474] My car's camera is showing an error message, what should I do?

[0475] Emotion: "Impatience"

[0476] This embodiment allows users to respond appropriately and quickly to any problems that may occur during automated driving, reducing stress and improving overall safety and satisfaction.

[0477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0478] Step 1:

[0479] The user uses a device (smartphone or in-car tablet) to input text or voice information about a problem or issue. For example, the user might input, "The vehicle camera is displaying an error message. What should I do?" This input data is sent to the server in an appropriate format, such as JSON.

[0480] Step 2:

[0481] The server receives input data from the user and passes it to the emotion engine for analysis. The emotion engine extracts the user's emotional state from the text data and recognizes it as "impatience," for example. This emotion data is returned to the server as the analysis result.

[0482] Step 3:

[0483] The server inputs the received unclear point data and emotion data into the generative AI model. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and generate a manual containing appropriate solutions and information. Specifically, it generates text containing how to deal with camera errors.

[0484] Step 4:

[0485] The server automatically generates a manual in HTML or PDF format based on the analysis results obtained from the generative AI model. The manual is adjusted according to the user's emotions, including a gentler tone and more specific instructions for users who are feeling impatient.

[0486] Step 5:

[0487] The server sends the completed manual to the user's device, where the user can view the HTML manual in a browser or download and read it in PDF format. The user can refer to this manual and follow the specific steps to solve the problem.

[0488] This series of processes enables users to respond quickly and appropriately to any problems that may occur during automated driving, thereby improving safety and satisfaction.

[0489] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0490] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0491] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0492] [Second embodiment]

[0493] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0494] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0495] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0496] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0497] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0498] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0499] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0500] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0501] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0502] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0503] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0504] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0505] The system of the present invention is designed to quickly and accurately resolve any questions users may encounter while entering EOS data. The system receives questions from users, analyzes them using a generative AI model, and automatically generates a manual based on the results to provide to the user.

[0506] Server Processing

[0507] 1. Receiving input from the user

[0508] The server receives EOS input data and unclear data sent from the user's terminal. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[0509] 2. Analysis using generative AI models

[0510] The received unknown data is passed to the generative AI model, which then uses natural language processing technology to deeply understand the unknown content and identify its key points.

[0511] 3. Generate manual

[0512] Based on the analysis results, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed.

[0513] 4. Provision to Users

[0514] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[0515] Terminal handling

[0516] 1. Sending input data

[0517] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[0518] 2. Receiving the manual

[0519] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[0520] User Actions

[0521] 1. EOS input

[0522] The user inputs the EOS data using the terminal.

[0523] 2. Enter any questions

[0524] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[0525] 3. Check the manual

[0526] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[0527] Specific examples

[0528] Example 1: New user registration procedure

[0529] 1. Users

[0530] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0531] 2. Terminal

[0532] The entered unknown data is sent to the server.

[0533] 3. Server

[0534] The received unknown data is passed to the generative AI model for analysis.

[0535] Based on the analysis results, a manual including user registration procedures is automatically generated.

[0536] The generated manual is sent to the terminal in HTML format.

[0537] 4. Terminal

[0538] Receive the manual and view it in your web browser.

[0539] 5. Users

[0540] Please refer to the manual and proceed while understanding the new user registration procedure.

[0541] Example 2: How to set up data backup

[0542] 1. Users

[0543] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[0544] 2. Terminal

[0545] The entered unknown data is sent to the server.

[0546] 3. Server

[0547] The received unknown data is passed to the generative AI model for analysis.

[0548] Based on the analysis results, a manual including data backup setup procedures is automatically generated.

[0549] The generated manual is sent to the terminal in PDF format.

[0550] 4. Terminal

[0551] Receive the manual and view it in a PDF viewer.

[0552] 5. Users

[0553] Refer to the manual and proceed while understanding how to set up data backup.

[0554] In this way, this system analyzes unclear points based on the data entered by the user and provides immediate solutions, thereby improving work efficiency and reducing work stress.

[0555] The processing flow will be explained below.

[0556] Step 1:

[0557] user

[0558] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the details in the text box (e.g., "I don't know how to register in the system").

[0559] Step 2:

[0560] Terminal

[0561] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the submit button, the data is sent to the endpoint API.

[0562] Step 3:

[0563] server

[0564] Receives data sent from the device. The API receives the request and checks the data. Validates that the received data contains any unknowns.

[0565] Step 4:

[0566] server

[0567] Initialize the generative AI model and prepare to input the received unknown data into the model.

[0568] Step 5:

[0569] Generative AI Models

[0570] Input the data of the unknown points and start the analysis. Natural language processing technology is used for the analysis to deeply understand the content of the unknown points and identify the problem points.

[0571] Step 6:

[0572] Generative AI Models

[0573] Based on the analysis results, necessary information and procedures are extracted. For example, if a user is unsure about how to register in the system, specific procedures for registering in the system are generated.

[0574] Step 7:

[0575] server

[0576] The analysis results from the generative AI model are used to construct the content of the manual, which includes explanatory text, procedures, illustrations, and other information, formatted in a user-friendly format.

[0577] Step 8:

[0578] server

[0579] Converts the generated manual into HTML or PDF format, depending on the system settings or user requirements.

[0580] Step 9:

[0581] server

[0582] Formatted manual data is sent to the terminal using an endpoint, ensuring that it is delivered to the user quickly.

[0583] Step 10:

[0584] Terminal

[0585] Receives manual data sent from the server. The received manual is processed for display in a web browser or PDF viewer.

[0586] Step 11:

[0587] user

[0588] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions, resolves any questions, and continues working. For example, the user can refer to the system registration procedure to complete registration.

[0589] In this way, the entire process flow is achieved quickly and efficiently.

[0590] Example 1

[0591] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0592] Previous systems lacked a means to quickly and accurately resolve any questions users might encounter while entering EOS data. This resulted in users spending a lot of time resolving problems, reducing work efficiency. There was also the risk that users would rely on inaccurate information when conducting their own research.

[0593] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0594] In this invention, the server includes a means for receiving input data from a user, a means for analyzing the received unclear point data using a generative AI model, and a means for automatically generating a manual based on the analysis results. This makes it possible to quickly analyze the user's unclear points and provide appropriate solutions. In addition, the generated manual is provided in HTML or PDF format, improving the convenience of users by allowing them to refer to it immediately.

[0595] "Means for receiving input data from a user" is a function that allows the server to receive information that a user has input using a terminal.

[0596] "Means for analyzing received unclear data using a generative AI model" refers to a function that uses a generative AI model that employs natural language processing technology to perform a detailed analysis of the unclear data entered by the user and understand its content.

[0597] "Means for automatically generating manuals based on analysis results" is a function that automatically creates manuals in a format that is easy for users to understand, based on the results of analysis by the generative AI model.

[0598] "Means for sending the generated manual to the user's terminal" refers to a function for sending the automatically generated manual to the user's terminal in an appropriate format (HTML or PDF).

[0599] "Means for sending input data to a server in JSON format" is a function for converting data entered by a user on a terminal into JSON format and sending it to a server.

[0600] "Means for providing to users in HTML or PDF format" refers to a function for converting the generated manual into HTML or PDF format and providing it to users in a format that is easily accessible.

[0601] The system of the present invention is designed to quickly and accurately resolve any unclear points users may encounter while entering EOS data. The system receives input data from the user, analyzes it using a generative AI model, and automatically generates a manual based on the analysis results to provide to the user. A specific implementation of the present invention is described below.

[0602] Server Processing

[0603] The server includes hardware and software for receiving EOS input data and unknown data sent from the user's device. For example, the server has an API endpoint for processing HTTP requests and has the functionality to appropriately receive input data from the user. This data is often sent in JSON format, so the server parses and stores the JSON data.

[0604] Next, the server passes the received unknown data to a generative AI model to begin analysis. The generative AI model uses natural language processing technologies such as GPT-3 and BERT. This allows it to deeply understand the content of the unknown and analyze it. At this time, the specific prompt text passed to the model is the user's input, such as "I don't know how to register in the system."

[0605] Based on the analysis results, the server automatically generates a manual using the information obtained from the generative AI model. This manual contains specific procedures and explanations, and inserts diagrams and tables as needed. The server then converts the generated manual into HTML or PDF format using an HTML template or a PDF generation library (e.g., Weasyprint).

[0606] The completed manual is sent to the user's device in HTML or PDF format, where the user can view it immediately. For example, the manual data generated by "POST / api / manuals" is sent to the user's device.

[0607] Terminal handling

[0608] The terminal provides an interface for users to input EOS data. When a user inputs an unknown point, the data is sent to the server in JSON format. For example, the unknown point data is sent by "POST / api / queries".

[0609] The device then receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the device's browser or PDF viewer, allowing the user to view detailed instructions and explanations and solve problems on the spot.

[0610] User Actions

[0611] Users enter EOS data using a terminal. They enter the data in detail on the interface, and if they have any questions, they enter them and send them to the server. For example, they can enter a specific question such as, "I don't know how to register in the system."

[0612] Once the manual sent from the server is received by the terminal, the user can open it in a browser or PDF viewer and check the contents. For example, the user can read the manual that details the new user registration procedure and proceed by following the steps.

[0613] Specific examples

[0614] Example: New user registration procedure

[0615] 1. Users

[0616] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0617] 2. Terminal

[0618] The inputted unknown data is sent to the server.

[0619] 3. Server

[0620] The received unknown data is passed to the generative AI model for analysis.

[0621] Based on the analysis results, a manual including user registration procedures is automatically generated.

[0622] The generated manual is sent to the terminal in HTML format.

[0623] 4. Terminal

[0624] Receive the manual and view it in your web browser.

[0625] 5. Users

[0626] Please refer to the manual and proceed while understanding the new user registration procedure.

[0627] In this way, the system analyzes unclear points based on the data entered by the user and provides prompt solutions, thereby improving work efficiency and reducing work stress.

[0628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0629] Step 1: Sending input data from the user

[0630] 1. Input: The user inputs EOS data using the terminal, and if there is anything unclear, they enter the details.

[0631] 2. Specific action: The user enters specific questions, such as "I don't know how to register in the system," into the input form on the interface and presses the submit button.

[0632] 3. Data processing / calculation: The terminal converts the data entered by the user into JSON format.

[0633] 4. Output: Send the unknown data in JSON format to the server (e.g., send to "POST / api / queries").

[0634] Step 2: Receiving input data from the user

[0635] 1. Input: Unknown data in JSON format sent from the terminal.

[0636] 2. Specific operation: The server receives the HTTP request and retrieves the unknown data.

[0637] 3. Data processing / calculation: Parse (analyze) the received JSON data and save it in the internal database.

[0638] 4. Output: The saved instances of the unknown data.

[0639] Step 3: Analysis by generative AI model

[0640] 1. Input: Stored unknown data.

[0641] 2. Specific operation: The server passes the stored unknown data to the generative AI model (e.g., GPT-3) as a prompt sentence.

[0642] 3. Data processing / calculation: The generative AI model performs analysis based on the prompt text and generates appropriate answers and procedures.

[0643] 4. Output: Analysis results from the generative AI model (e.g., specific operating procedures).

[0644] Step 4: Automatic generation of manuals

[0645] 1. Input: Analysis results from the generative AI model.

[0646] 2. Specific operation: The server generates a manual including text and illustrations based on the analysis results.

[0647] 3. Data processing / calculation: Format the analysis results using HTML templates or a PDF generation library (e.g., Weasyprint).

[0648] 4. Output: Manual in HTML or PDF format.

[0649] Step 5: Manually send to users

[0650] 1. Input: Manual in HTML or PDF format.

[0651] 2. Specific operation: The server creates an HTTP response to return the generated manual to the user's device.

[0652] 3. Data processing / calculation: Include manual data in the HTTP response body.

[0653] 4. Output: An HTTP response containing the manual is sent to the user's device (e.g., sent with "POST / api / manuals").

[0654] Step 6: Receive and view the manual

[0655] 1. Input: The HTTP response sent by the server (including manual data).

[0656] 2. Specific operation: The terminal receives the HTTP response and extracts the included manual data.

[0657] 3. Data processing / calculation: Convert the extracted manual data into a format that can be displayed in an HTML or PDF viewer.

[0658] 4. Output: The manual that will be displayed on the user's device.

[0659] Step 7: Refer to your user manual

[0660] 1. Input: Manual displayed on the terminal.

[0661] 2. Specific actions: The user refers to the manual displayed on the device and checks the steps to resolve any questions.

[0662] 3. Data processing / calculation: The user performs the operations according to the contents of the manual.

[0663] 4. Output: Resolved issues and smooth progress of work. The end result is that issues are resolved and work proceeds smoothly.

[0664] (Application example 1)

[0665] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] This invention relates to technology for quickly and accurately resolving questions and problems that arise during the operation of industrial robots. In conventional industrial robot systems, when a problem occurs, the intervention of an operator with specialized knowledge is required to resolve the problem, which inevitably leads to interruptions in work and reduced efficiency. Such problems reduce the productivity of industrial processes and increase maintenance costs. Meanwhile, there is a demand for systems in which robots themselves have the ability to solve problems and respond in real time.

[0667] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0668] In this invention, the server includes means for receiving unclear point data from a user, means for analyzing the received unclear point data using a generative AI model, means for automatically generating a manual based on the analysis results, means for transmitting the generated manual to the user's terminal, means for the industrial robot to recognize problems through sensors or cameras while in operation and transmit the data in JSON format to the server, means for the generative AI model to identify solutions based on the unclear point data and generate a manual in HTML or PDF format that details the solutions, and means for providing the generated manual to the industrial robot's display device in real time. This enables the industrial robot to recognize unclear points on its own while in operation, obtain solutions in real time via the server, and quickly solve problems.

[0669] "Means for receiving unclear data from users" refers to an input interface that allows users to report unclear or questionable points to the system, and typically handles data in JSON format.

[0670] "Means of analyzing received unknown data using a generative AI model" refers to the process of passing received data to a generative AI model that uses natural language processing technology, and analyzing and understanding its contents.

[0671] "Means for automatically generating manuals based on analysis results" refers to algorithms or scripts for automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from the generative AI model.

[0672] "Means for sending the generated manual to the user's terminal" refers to a sending function for providing the generated manual to the user in HTML or PDF format in real time.

[0673] "A means for industrial robots to detect problems through sensors and cameras while in operation and send the data to a server in JSON format" refers to the process of using sensors and cameras to detect the status and problems of robots and send that information to a server.

[0674] "Means for the generative AI model to identify a solution and generate a manual in HTML or PDF format detailing that solution" refers to the function of using a generative AI model to find a solution to a problem and automatically generating a manual in HTML or PDF format detailing that solution in an easy-to-understand manner.

[0675] "Means for providing the generated manual to the display device of the industrial robot in real time" refers to the function of displaying the generated manual on the display of the industrial robot in real time, so that the operator and the robot itself can immediately refer to it.

[0676] The system of the present invention is a support system for quickly and accurately resolving questions and problems that industrial robots operating in factories encounter during operation. This system uses hardware such as industrial robots, servers, sensors, cameras, and displays, as well as software such as generative AI models (e.g., OpenAI GPT-4), Python, Flask (a web framework), and a JSON-formatted data interface.

[0677] The server includes the following means:

[0678] 1. Means of receiving unclear data from users

[0679] It is a means for users or robots to send and receive problems or questions they encounter in JSON format to the server.

[0680] 2. A means of analyzing incoming unknown data using a generative AI model

[0681] This is the process of passing the received data to a generative AI model (e.g., OpenAI GPT-4) that uses natural language processing technology to analyze and understand its content.

[0682] 3. A method for automatically generating manuals based on analysis results

[0683] It is a means of automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from a generative AI model.

[0684] 4. Means for sending the generated manual to the user's device

[0685] This function allows the generated manual to be sent to the user in HTML or PDF format for viewing.

[0686] 5. A way for industrial robots to detect problems through sensors and cameras while in operation and send that data in JSON format to a server

[0687] This is the process of detecting the robot's status and problems using sensors and cameras, and sending that information to a server in JSON format.

[0688] 6. A means for the generative AI model to identify solutions and generate detailed solutions manuals in HTML or PDF format.

[0689] This function uses generative AI models to find solutions to problems and automatically generates detailed and easy-to-understand manuals describing the solutions in HTML or PDF format.

[0690] 7. Means for providing the generated manual to the display device of the industrial robot in real time

[0691] This function allows the generated manual to be displayed in real time on the industrial robot's display, making it available for immediate reference by the operator and the robot itself.

[0692] For example, if an industrial robot detects a problem while in operation, such as "the equipment connection is not working properly," it will send this information, obtained through sensors and cameras, in JSON format to a server. The server will then use a generative AI model based on the received data to analyze the problem and find a solution. Based on the analysis results, it will automatically generate a manual including operating procedures, and display this manual in real time on the industrial robot's display. If a user inputs a point of uncertainty, such as "I don't know how to connect the equipment," the same procedure will be followed, and the generated manual will be sent to the user's device.

[0693] Here are some example prompts:

[0694] Please analyze the following problem and provide a specific solution:

[0695] "Device connection is not working properly"

[0696] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0697] Step 1:

[0698] While industrial robots are operating, they detect problems through sensors and cameras and send the data in JSON format to a server.

[0699] Specifically, the robot's sensors and cameras detect abnormal behavior or error messages. Detailed problem data (e.g., "Device connection is not working properly") is received as input, packaged in JSON format, and sent to the server via an HTTP POST request. The server receives this data and proceeds with further processing.

[0700] Step 2:

[0701] The server passes the received JSON-formatted unknown data to the generative AI model for analysis.

[0702] Specifically, the server reads the received data and sends it to a generative AI model (e.g., OpenAI GPT-4). The input is problem data from the user or robot. The generative AI model analyzes this data and identifies unclear points and solutions. For example, in response to the problem of "equipment connection not working properly," the generative AI model outputs "the correct procedure for connecting cables" and "how to check the connection."

[0703] Step 3:

[0704] Based on the analysis results, the server automatically generates a manual.

[0705] Specifically, it receives the solution from the generative AI model and formats it as detailed instructions in natural language. It uses the analysis results of the generative AI model obtained as input (e.g., "Please double-check the cable and connect it to the correct port") and automatically generates a manual in HTML or PDF format. The output is a detailed manual including operating procedures and illustrations.

[0706] Step 4:

[0707] The generated manual is provided in real time to the display device of the industrial robot.

[0708] Specifically, the server sends the generated manual to the robot's display. The input is the manual data generated in step 3, and the output is the manual displayed on the robot's display. This makes it available for immediate reference by users and robots.

[0709] Step 5:

[0710] If necessary, the generated manual is also sent to the user's terminal.

[0711] Specifically, the server sends the generated manual in HTML or PDF format to the user's device. The input is the manual data generated in step 3, and the output is the manual displayed on the user's device. For example, if a user inquires, "I don't know how to connect the equipment," this manual is sent to the user's device and can be viewed in a browser or PDF viewer.

[0712] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0713] The system of the present invention quickly and accurately resolves any questions users may encounter while entering data into the EOS system. It also recognizes the user's emotions and provides a manual tailored to their state, providing more effective support. The system receives data on the user's questions, analyzes them using a generative AI model, and automatically generates and provides a manual based on the results. Furthermore, by incorporating an emotion engine, the system adjusts the tone and expression of the manual according to the user's emotional state.

[0714] Server Processing

[0715] 1. Receiving input from the user

[0716] The server receives EOS input data and unclear data sent from the user's device. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[0717] 2. Emotion Recognition by Emotion Engine

[0718] The received unknown data is input into the emotion engine to recognize the user's emotion. The emotion engine analyzes the user's emotional state (e.g., anger, sadness, joy) from the text data.

[0719] 3. Analysis using generative AI models

[0720] The recognized user emotion data along with the difficulty data is passed to the generative AI model, which then uses natural language processing technology to identify the unclear points, taking into account the content of the unclear points and the user's emotions.

[0721] 4. Generate manual

[0722] Based on the analysis results and the user's emotions, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed, and adjusts the tone and expression according to the user's emotions.

[0723] 5. Provision to Users

[0724] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[0725] Terminal handling

[0726] 1. Sending input data

[0727] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[0728] 2. Receiving the manual

[0729] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[0730] User Actions

[0731] 1. EOS input

[0732] The user inputs the EOS data using the terminal.

[0733] 2. Enter any questions

[0734] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[0735] 3. Check the manual

[0736] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[0737] Specific examples

[0738] Example 1: New user registration procedure

[0739] 1. Users

[0740] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0741] 2. Terminal

[0742] The entered unknown data is sent to the server.

[0743] 3. Server

[0744] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in an "anger" state.

[0745] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[0746] Based on the analysis results, a manual including user registration procedures is automatically generated, where instructions are explained in more polite language and a more friendly tone depending on the user's emotions.

[0747] The generated manual is sent to the terminal in HTML format.

[0748] 4. Terminal

[0749] Receive the manual and view it in your web browser.

[0750] 5. Users

[0751] Please refer to the manual and proceed while understanding the new user registration procedure.

[0752] Example 2: How to set up data backup

[0753] 1. Users

[0754] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[0755] 2. Terminal

[0756] The entered unknown data is sent to the server.

[0757] 3. Server

[0758] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in a "sad" state.

[0759] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[0760] Based on the analysis results, a manual including instructions for setting up data backups is automatically generated, which includes encouraging words and reassuring expressions depending on the user's emotions.

[0761] The generated manual is sent to the terminal in PDF format.

[0762] 4. Terminal

[0763] Receive the manual and view it in a PDF viewer.

[0764] 5. Users

[0765] Refer to the manual and proceed while understanding how to set up data backup.

[0766] In this way, this system, which incorporates an emotion engine, analyzes the unclear points entered by the user and provides the optimal manual based on their emotions, thereby improving work efficiency and the user experience.

[0767] The processing flow will be explained below.

[0768] Step 1:

[0769] user

[0770] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the specific details in the text box (e.g., "I don't know how to register in the system").

[0771] Step 2:

[0772] Terminal

[0773] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the send button, the data is sent to the server via the REST API.

[0774] Step 3:

[0775] server

[0776] The data sent from the device is received. The API processes the request, validates the data, and then passes any unclear data to the emotion engine.

[0777] Step 4:

[0778] Emotion Engine

[0779] Analyzes the received unknown data to recognize the user's emotional state. Analyzes text data to identify the user's emotions (e.g., anger, sadness, joy).

[0780] Step 5:

[0781] server

[0782] Along with the emotion data obtained from the emotion engine, the unknown data is passed to the generative AI model.

[0783] Step 6:

[0784] Generative AI Models

[0785] Input the unclear point data and emotion data and begin analysis. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and identify the necessary information.

[0786] Step 7:

[0787] Generative AI Models

[0788] Based on the analysis results, necessary steps and explanations are extracted, and the content is adjusted to suit the user's emotions using a tone and expression.

[0789] Step 8:

[0790] server

[0791] Using information from the generative AI model, the system automatically generates a manual that includes instructions, steps, and illustrations, and adjusts tone and expression based on emotional data.

[0792] Step 9:

[0793] server

[0794] Convert the generated manual into HTML or PDF format, the format is selected based on the user's requirements and system settings.

[0795] Step 10:

[0796] server

[0797] Formatted manual data is sent to the terminal. For quick delivery, a REST API is used to send the data.

[0798] Step 11:

[0799] Terminal

[0800] Receives the manual data sent from the server. The received manual is then processed so that it can be displayed in a web browser or PDF viewer.

[0801] Step 12:

[0802] user

[0803] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions and resolves any questions. For example, the user can refer to the system registration procedure to complete the registration.

[0804] This system not only enables users to quickly resolve any questions they may have, but also allows them to receive optimal support tailored to their emotional state at the time.

[0805] Example 2

[0806] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] Conventional user support systems simply display simple FAQs or existing manuals in response to data input by users, making it difficult to provide customized support that reflects the situation and emotions of each individual user. Furthermore, there was a lack of systems that could properly recognize the emotional state of users when they input their questions and take appropriate approaches. As a result, the quality of the user experience declined and support efficiency declined.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0809] In this invention, the server includes means for receiving uncertainty data and emotion data from the user, means for analyzing the received uncertainty data and emotion data using an emotion recognition engine, means for generating a prompt sentence based on the received uncertainty data and the analyzed emotion data using a generative AI model, means for obtaining an analysis result using the generative AI model based on the generated prompt sentence, means for automatically generating a manual based on the analysis result and the user's emotion, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide optimal support that takes into account the user's uncertainty and emotional state.

[0810] "User's unclear point data" is text data describing problems and questions that users have encountered while using the system.

[0811] "Emotion data" is data that indicates the emotional state that the user is feeling when inputting the unclear point.

[0812] An "emotion recognition engine" is a program or algorithm that analyzes and identifies a user's emotional state from text data.

[0813] A "generative AI model" is an artificial intelligence model that performs natural language processing based on large amounts of text data, and specifically has the ability to understand the content of a user's inquiry and generate an appropriate response.

[0814] A "prompt" is a textual instruction given to a generative AI model, and is a document that includes an introductory section for analysis.

[0815] "Analysis results" refers to the information and answers analyzed by the generative AI model, and indicate solutions to any questions users may have.

[0816] A "manual" is a document that describes solutions and procedures for problems that users are facing.

[0817] "HTML" is a standard markup language for building web pages.

[0818] "PDF" is a file format designed for displaying and printing electronic documents, and allows documents to be shared while maintaining their format.

[0819] "User device" refers to an electronic device, such as a PC, smartphone, or tablet, that a user uses to operate the system.

[0820] This invention is a support system that quickly and accurately resolves any questions users may have when using a system. The system utilizes a generative AI model to analyze the user's questions and emotional state, providing the most appropriate manual.

[0821] Main hardware and software used

[0822] Hardware: User devices (PCs, smartphones, tablets, etc.), servers (high-performance computing servers)

[0823] Software: Emotion recognition engine (natural language processing library), generative AI model (natural language generation model)

[0824] Data processing and calculation

[0825] Server Processing

[0826] 1. Receiving input data:

[0827] The server receives EOS input data and unclear data sent from the user's device in JSON format. For example, if a user enters "I don't know how to register in the system," this data is parsed by the server.

[0828] 2. Emotion Recognition:

[0829] The server inputs the received unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine is used to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[0830] 3. Generate prompt sentence:

[0831] The server generates a prompt for the generative AI model based on the recognized emotion data and the unclear point data. This prompt contains specific questions and instructions to clearly communicate the unclear point to the generative AI model. For example, it could be something like, "Please create a manual that explains how to register in the system in the most detailed way possible. The user is feeling angry."

[0832] 4. Run the analysis:

[0833] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[0834] 5. Generate manual:

[0835] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[0836] 6. Offerings to Users:

[0837] The server then sends the completed manual to the user's device in HTML or PDF format, allowing the user to instantly view the information they need.

[0838] Terminal handling

[0839] 1. User input submission:

[0840] The terminal sends the EOS data and unknown data entered by the user to the server in JSON format.

[0841] 2. Receiving the manual:

[0842] The terminal receives the manual sent from the server. The manual is displayed in HTML or PDF format, so users can easily view it without needing special software.

[0843] User Actions

[0844] 1. EOS input:

[0845] The user inputs the EOS data using the terminal.

[0846] 2. Submit any questions:

[0847] When a user encounters a problem, they enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system" in the input form.

[0848] 3. Check the manual:

[0849] The user checks the manual sent from the server on the terminal and obtains the necessary information. For example, the received manual contains detailed instructions on how to register the system, and the user can proceed with the work by following the instructions.

[0850] Specific examples

[0851] Example 1: New user registration procedure

[0852] 1. User:

[0853] Enter your question, such as "I don't know how to register as a new user."

[0854] 2. Terminal:

[0855] The inputted unknown data is sent to the server.

[0856] 3. Server:

[0857] The received unknown data is passed to the emotion engine, which recognizes the user's emotion as "anger."

[0858] Generate prompt sentences based on emotion data and unclear point data.

[0859] The prompt sentence, "Please create a manual that explains how to register in the system in the most detailed way possible. Users are feeling angry," is passed to the generative AI model.

[0860] Based on the analysis results obtained from the generative AI model, a manual written in a friendly tone is generated.

[0861] Send the manual to the terminal in HTML format.

[0862] 4. Terminal:

[0863] Receive the manual and view it in your browser.

[0864] 5. User:

[0865] Please refer to the manual and proceed while understanding the new user registration procedure.

[0866] This system makes it possible to quickly provide the most appropriate manual that takes into account the user's uncertainties and emotional state.

[0867] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0868] Step 1: Receiving input data

[0869] The server receives EOS input data and unknown data sent from the user's device. The input data is in JSON format, and the data structure consists of key-value pairs. The received data includes the user ID, unknown content, timestamp, etc. The server parses this data and stores each item in the appropriate variable.

[0870] input:

[0871] User EOS input data

[0872] Uncertainty data (e.g., "I don't know how to register in the system")

[0873] output:

[0874] Parsed uncertainty data and other meta information

[0875] Specific operation:

[0876] The server parses the received JSON data and stores the user ID, EOS data, and unknown data in appropriate variables.

[0877] Step 2: Recognize emotions

[0878] The server inputs the parsed unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine uses a natural language processing library to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[0879] input:

[0880] Unknown data

[0881] output:

[0882] Emotion recognition result (e.g., "anger" state)

[0883] Specific operation:

[0884] The emotion recognition engine inputs the unknown text and determines its emotional state, which is then passed to the next processing step.

[0885] Step 3: Generate a prompt statement

[0886] The server generates prompts for the generative AI model based on the recognized emotion data and unclear point data. These prompts include specific questions and instructions to communicate the unclear points to the generative AI model in an easy-to-understand manner.

[0887] input:

[0888] Unknown data

[0889] Emotion recognition results

[0890] output:

[0891] Prompt statement (e.g., "Please create the most thorough manual possible explaining how to register the system. Users are angry.")

[0892] Specific operation:

[0893] The server uses the unknown data and emotion recognition results to generate prompt sentences based on dedicated templates.

[0894] Step 4: Perform the analysis

[0895] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[0896] input:

[0897] Prompt statement

[0898] output:

[0899] Analysis results (e.g. system registration procedure)

[0900] Specific operation:

[0901] The server inputs a prompt sentence into the generative AI model and obtains a solution as an analysis result.

[0902] Step 5: Generate the manual

[0903] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[0904] input:

[0905] Analysis results

[0906] Emotion recognition results

[0907] output:

[0908] Auto-generated manual

[0909] Specific operation:

[0910] The server uses the analysis results and emotion data to generate a manual based on an appropriate template.

[0911] Step 6: Deliver to users

[0912] The server then sends the completed manual to the user's device in HTML or PDF format, with a link and notification automatically sent for immediate access.

[0913] input:

[0914] Auto-generated manual

[0915] output:

[0916] Manual sent to the user's device (HTML or PDF format)

[0917] Specific operation:

[0918] The server converts the manual into an appropriate format and sends it to the user's device. It also generates a notification of completion and an access link, which it provides to the user.

[0919] (Application example 2)

[0920] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] A support system that can quickly and appropriately respond to technical questions and problems that drivers and users encounter in autonomous vehicles is needed. Conventional support systems do not provide manuals that take into account the user's emotional state, which can result in increased stress and dissatisfaction for users. Furthermore, conventional technologies take time to resolve users' questions, and there are insufficient means to mitigate safety risks that arise during this time.

[0922] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving unclear data from the user, means for analyzing the received unclear data using a generative AI model, means for automatically generating a manual based on the analysis results and the user's emotional data, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide an appropriate manual using a tone and expression that matches the user's emotional state, enabling immediate responses to technical questions and problems in autonomous vehicles and improving user satisfaction and safety.

[0923] "User" refers to a person who uses the system, specifically a driver or user.

[0924] "Uncertainty data" refers to data about questions or problems users encounter while using the system.

[0925] A "generative AI model" is an artificial intelligence model that analyzes received unknown data and generates corresponding information and solutions.

[0926] "Analysis" refers to the process of examining data in detail and understanding its contents.

[0927] "Emotional data" is data that represents a user's emotional state extracted from their text or speech.

[0928] A "manual" is a document that contains procedures and explanations for resolving user questions and problems.

[0929] "Auto-generation" refers to the process by which a program automatically creates output information from specific input data.

[0930] "Terminal" refers to a device used to use the system, and specifically includes smartphones and in-car tablets.

[0931] This invention is a system that provides appropriate support in real time for questions or problems that users encounter in autonomous vehicles. This system uses smartphones or in-car tablets as terminals, and sends input data from users to a server, which then analyzes it.

[0932] The server uses the following methods:

[0933] A means for receiving data on questions from users: Users use their devices to input text or voice about questions or problems. This input data is sent to the server.

[0934] A method for analyzing received unknown data using a generative AI model: A generative AI model is used to analyze the received data. The generative AI model uses natural language processing techniques to analyze the unknown data and identify its points.

[0935] A means of automatically generating manuals based on analysis results and user emotional data: An emotion engine is used to extract user emotional data, which is then combined with the analysis results to automatically generate manuals that take into account appropriate tone and expression.

[0936] Means for sending the generated manual to the user's terminal: The generated manual is sent to the user's terminal in HTML or PDF format.

[0937] As a concrete example, consider the case where the vehicle camera displays an error message during autonomous driving. When the user types "The vehicle camera is displaying an error message, what should I do?" into the terminal, the following process is executed.

[0938] The server receives the user's input and uses an emotion engine to recognize the user's emotion as "impatience." The generative AI model then analyzes the input and automatically generates a manual with specific steps to deal with the camera's error message. The manual includes a gentle tone and expressions to help users respond calmly to their emotions. The generated manual is sent to the user's device in HTML format and displayed in a window.

[0939] An implementation example shows the use of the following prompt:

[0940] My car's camera is showing an error message, what should I do?

[0941] Emotion: "Impatience"

[0942] This embodiment allows users to respond appropriately and quickly to any problems that may occur during automated driving, reducing stress and improving overall safety and satisfaction.

[0943] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0944] Step 1:

[0945] The user uses a device (smartphone or in-car tablet) to input text or voice information about a problem or issue. For example, the user might input, "The vehicle camera is displaying an error message. What should I do?" This input data is sent to the server in an appropriate format, such as JSON.

[0946] Step 2:

[0947] The server receives input data from the user and passes it to the emotion engine for analysis. The emotion engine extracts the user's emotional state from the text data and recognizes it as "impatience," for example. This emotion data is returned to the server as the analysis result.

[0948] Step 3:

[0949] The server inputs the received unclear point data and emotion data into the generative AI model. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and generate a manual containing appropriate solutions and information. Specifically, it generates text containing how to deal with camera errors.

[0950] Step 4:

[0951] The server automatically generates a manual in HTML or PDF format based on the analysis results obtained from the generative AI model. The manual is adjusted according to the user's emotions, including a gentler tone and more specific instructions for users who are feeling impatient.

[0952] Step 5:

[0953] The server sends the completed manual to the user's device, where the user can view the HTML manual in a browser or download and read it in PDF format. The user can refer to this manual and follow the specific steps to solve the problem.

[0954] This series of processes enables users to respond quickly and appropriately to any problems that may occur during automated driving, thereby improving safety and satisfaction.

[0955] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0956] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0957] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0958] [Third embodiment]

[0959] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0960] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0961] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0962] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0963] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0965] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0966] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0967] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0968] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0969] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0970] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0971] The system of the present invention is designed to quickly and accurately resolve any questions users may encounter while entering EOS data. The system receives questions from users, analyzes them using a generative AI model, and automatically generates a manual based on the results to provide to the user.

[0972] Server Processing

[0973] 1. Receiving input from the user

[0974] The server receives EOS input data and unclear data sent from the user's terminal. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[0975] 2. Analysis using generative AI models

[0976] The received unknown data is passed to the generative AI model, which then uses natural language processing technology to deeply understand the unknown content and identify its key points.

[0977] 3. Generate manual

[0978] Based on the analysis results, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed.

[0979] 4. Provision to Users

[0980] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[0981] Terminal handling

[0982] 1. Sending input data

[0983] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[0984] 2. Receiving the manual

[0985] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[0986] User Actions

[0987] 1. EOS input

[0988] The user inputs the EOS data using the terminal.

[0989] 2. Enter any questions

[0990] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[0991] 3. Check the manual

[0992] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[0993] Specific examples

[0994] Example 1: New user registration procedure

[0995] 1. Users

[0996] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[0997] 2. Terminal

[0998] The entered unknown data is sent to the server.

[0999] 3. Server

[1000] The received unknown data is passed to the generative AI model for analysis.

[1001] Based on the analysis results, a manual including user registration procedures is automatically generated.

[1002] The generated manual is sent to the terminal in HTML format.

[1003] 4. Terminal

[1004] Receive the manual and view it in your web browser.

[1005] 5. Users

[1006] Please refer to the manual and proceed while understanding the new user registration procedure.

[1007] Example 2: How to set up data backup

[1008] 1. Users

[1009] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[1010] 2. Terminal

[1011] The entered unknown data is sent to the server.

[1012] 3. Server

[1013] The received unknown data is passed to the generative AI model for analysis.

[1014] Based on the analysis results, a manual including data backup setup procedures is automatically generated.

[1015] The generated manual is sent to the terminal in PDF format.

[1016] 4. Terminal

[1017] Receive the manual and view it in a PDF viewer.

[1018] 5. Users

[1019] Refer to the manual and proceed while understanding how to set up data backup.

[1020] In this way, this system analyzes unclear points based on the data entered by the user and provides immediate solutions, thereby improving work efficiency and reducing work stress.

[1021] The processing flow will be explained below.

[1022] Step 1:

[1023] user

[1024] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the details in the text box (e.g., "I don't know how to register in the system").

[1025] Step 2:

[1026] Terminal

[1027] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the submit button, the data is sent to the endpoint API.

[1028] Step 3:

[1029] server

[1030] Receives data sent from the device. The API receives the request and checks the data. Validates that the received data contains any unknowns.

[1031] Step 4:

[1032] server

[1033] Initialize the generative AI model and prepare to input the received unknown data into the model.

[1034] Step 5:

[1035] Generative AI Models

[1036] Input the data of the unknown points and start the analysis. Natural language processing technology is used for the analysis to deeply understand the content of the unknown points and identify the problem points.

[1037] Step 6:

[1038] Generative AI Models

[1039] Based on the analysis results, necessary information and procedures are extracted. For example, if a user is unsure about how to register in the system, specific procedures for registering in the system are generated.

[1040] Step 7:

[1041] server

[1042] The analysis results from the generative AI model are used to construct the content of the manual, which includes explanatory text, procedures, illustrations, and other information, formatted in a user-friendly format.

[1043] Step 8:

[1044] server

[1045] Converts the generated manual into HTML or PDF format, depending on the system settings or user requirements.

[1046] Step 9:

[1047] server

[1048] Formatted manual data is sent to the terminal using an endpoint, ensuring that it is delivered to the user quickly.

[1049] Step 10:

[1050] Terminal

[1051] Receives manual data sent from the server. The received manual is processed for display in a web browser or PDF viewer.

[1052] Step 11:

[1053] user

[1054] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions, resolves any questions, and continues working. For example, the user can refer to the system registration procedure to complete registration.

[1055] In this way, the entire process flow is achieved quickly and efficiently.

[1056] Example 1

[1057] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1058] Previous systems lacked a means to quickly and accurately resolve any questions users might encounter while entering EOS data. This resulted in users spending a lot of time resolving problems, reducing work efficiency. There was also the risk that users would rely on inaccurate information when conducting their own research.

[1059] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1060] In this invention, the server includes a means for receiving input data from a user, a means for analyzing the received unclear point data using a generative AI model, and a means for automatically generating a manual based on the analysis results. This makes it possible to quickly analyze the user's unclear points and provide appropriate solutions. In addition, the generated manual is provided in HTML or PDF format, improving the convenience of users by allowing them to refer to it immediately.

[1061] "Means for receiving input data from a user" is a function that allows the server to receive information that a user has input using a terminal.

[1062] "Means for analyzing received unclear data using a generative AI model" refers to a function that uses a generative AI model that employs natural language processing technology to perform a detailed analysis of the unclear data entered by the user and understand its content.

[1063] "Means for automatically generating manuals based on analysis results" is a function that automatically creates manuals in a format that is easy for users to understand, based on the results of analysis by the generative AI model.

[1064] "Means for sending the generated manual to the user's terminal" refers to a function for sending the automatically generated manual to the user's terminal in an appropriate format (HTML or PDF).

[1065] "Means for sending input data to a server in JSON format" is a function for converting data entered by a user on a terminal into JSON format and sending it to a server.

[1066] "Means for providing to users in HTML or PDF format" refers to a function for converting the generated manual into HTML or PDF format and providing it to users in a format that is easily accessible.

[1067] The system of the present invention is designed to quickly and accurately resolve any unclear points users may encounter while entering EOS data. The system receives input data from the user, analyzes it using a generative AI model, and automatically generates a manual based on the analysis results to provide to the user. A specific implementation of the present invention is described below.

[1068] Server Processing

[1069] The server includes hardware and software for receiving EOS input data and unknown data sent from the user's device. For example, the server has an API endpoint for processing HTTP requests and has the functionality to appropriately receive input data from the user. This data is often sent in JSON format, so the server parses and stores the JSON data.

[1070] Next, the server passes the received unknown data to a generative AI model to begin analysis. The generative AI model uses natural language processing technologies such as GPT-3 and BERT. This allows it to deeply understand the content of the unknown and analyze it. At this time, the specific prompt text passed to the model is the user's input, such as "I don't know how to register in the system."

[1071] Based on the analysis results, the server automatically generates a manual using the information obtained from the generative AI model. This manual contains specific procedures and explanations, and inserts diagrams and tables as needed. The server then converts the generated manual into HTML or PDF format using an HTML template or a PDF generation library (e.g., Weasyprint).

[1072] The completed manual is sent to the user's device in HTML or PDF format, where the user can view it immediately. For example, the manual data generated by "POST / api / manuals" is sent to the user's device.

[1073] Terminal handling

[1074] The terminal provides an interface for users to input EOS data. When a user inputs an unknown point, the data is sent to the server in JSON format. For example, the unknown point data is sent by "POST / api / queries".

[1075] The device then receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the device's browser or PDF viewer, allowing the user to view detailed instructions and explanations and solve problems on the spot.

[1076] User Actions

[1077] Users enter EOS data using a terminal. They enter the data in detail on the interface, and if they have any questions, they enter them and send them to the server. For example, they can enter a specific question such as, "I don't know how to register in the system."

[1078] Once the manual sent from the server is received by the terminal, the user can open it in a browser or PDF viewer and check the contents. For example, the user can read the manual that details the new user registration procedure and proceed by following the steps.

[1079] Specific examples

[1080] Example: New user registration procedure

[1081] 1. Users

[1082] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[1083] 2. Terminal

[1084] The inputted unknown data is sent to the server.

[1085] 3. Server

[1086] The received unknown data is passed to the generative AI model for analysis.

[1087] Based on the analysis results, a manual including user registration procedures is automatically generated.

[1088] The generated manual is sent to the terminal in HTML format.

[1089] 4. Terminal

[1090] Receive the manual and view it in your web browser.

[1091] 5. Users

[1092] Please refer to the manual and proceed while understanding the new user registration procedure.

[1093] In this way, the system analyzes unclear points based on the data entered by the user and provides prompt solutions, thereby improving work efficiency and reducing work stress.

[1094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1095] Step 1: Sending input data from the user

[1096] 1. Input: The user inputs EOS data using the terminal, and if there is anything unclear, they enter the details.

[1097] 2. Specific action: The user enters specific questions, such as "I don't know how to register in the system," into the input form on the interface and presses the submit button.

[1098] 3. Data processing / calculation: The terminal converts the data entered by the user into JSON format.

[1099] 4. Output: Send the unknown data in JSON format to the server (e.g., send to "POST / api / queries").

[1100] Step 2: Receiving input data from the user

[1101] 1. Input: Unknown data in JSON format sent from the terminal.

[1102] 2. Specific operation: The server receives the HTTP request and retrieves the unknown data.

[1103] 3. Data processing / calculation: Parse (analyze) the received JSON data and save it in the internal database.

[1104] 4. Output: The saved instances of the unknown data.

[1105] Step 3: Analysis by generative AI model

[1106] 1. Input: Stored unknown data.

[1107] 2. Specific operation: The server passes the stored unknown data to the generative AI model (e.g., GPT-3) as a prompt sentence.

[1108] 3. Data processing / calculation: The generative AI model performs analysis based on the prompt text and generates appropriate answers and procedures.

[1109] 4. Output: Analysis results from the generative AI model (e.g., specific operating procedures).

[1110] Step 4: Automatic generation of manuals

[1111] 1. Input: Analysis results from the generative AI model.

[1112] 2. Specific operation: The server generates a manual including text and illustrations based on the analysis results.

[1113] 3. Data processing / calculation: Format the analysis results using HTML templates or a PDF generation library (e.g., Weasyprint).

[1114] 4. Output: Manual in HTML or PDF format.

[1115] Step 5: Manually send to users

[1116] 1. Input: Manual in HTML or PDF format.

[1117] 2. Specific operation: The server creates an HTTP response to return the generated manual to the user's device.

[1118] 3. Data processing / calculation: Include manual data in the HTTP response body.

[1119] 4. Output: An HTTP response containing the manual is sent to the user's device (e.g., sent with "POST / api / manuals").

[1120] Step 6: Receive and view the manual

[1121] 1. Input: The HTTP response sent by the server (including manual data).

[1122] 2. Specific operation: The terminal receives the HTTP response and extracts the included manual data.

[1123] 3. Data processing / calculation: Convert the extracted manual data into a format that can be displayed in an HTML or PDF viewer.

[1124] 4. Output: The manual that will be displayed on the user's device.

[1125] Step 7: Refer to your user manual

[1126] 1. Input: Manual displayed on the terminal.

[1127] 2. Specific actions: The user refers to the manual displayed on the device and checks the steps to resolve any questions.

[1128] 3. Data processing / calculation: The user performs the operations according to the contents of the manual.

[1129] 4. Output: Resolved issues and smooth progress of work. The end result is that issues are resolved and work proceeds smoothly.

[1130] (Application example 1)

[1131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1132] This invention relates to technology for quickly and accurately resolving questions and problems that arise during the operation of industrial robots. In conventional industrial robot systems, when a problem occurs, the intervention of an operator with specialized knowledge is required to resolve the problem, which inevitably leads to interruptions in work and reduced efficiency. Such problems reduce the productivity of industrial processes and increase maintenance costs. Meanwhile, there is a demand for systems in which robots themselves have the ability to solve problems and respond in real time.

[1133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1134] In this invention, the server includes means for receiving unclear point data from a user, means for analyzing the received unclear point data using a generative AI model, means for automatically generating a manual based on the analysis results, means for transmitting the generated manual to the user's terminal, means for the industrial robot to recognize problems through sensors or cameras while in operation and transmit the data in JSON format to the server, means for the generative AI model to identify solutions based on the unclear point data and generate a manual in HTML or PDF format that details the solutions, and means for providing the generated manual to the industrial robot's display device in real time. This enables the industrial robot to recognize unclear points on its own while in operation, obtain solutions in real time via the server, and quickly solve problems.

[1135] "Means for receiving unclear data from users" refers to an input interface that allows users to report unclear or questionable points to the system, and typically handles data in JSON format.

[1136] "Means of analyzing received unknown data using a generative AI model" refers to the process of passing received data to a generative AI model that uses natural language processing technology, and analyzing and understanding its contents.

[1137] "Means for automatically generating manuals based on analysis results" refers to algorithms or scripts for automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from the generative AI model.

[1138] "Means for sending the generated manual to the user's terminal" refers to a sending function for providing the generated manual to the user in HTML or PDF format in real time.

[1139] "A means for industrial robots to detect problems through sensors and cameras while in operation and send the data to a server in JSON format" refers to the process of using sensors and cameras to detect the status and problems of robots and send that information to a server.

[1140] "Means for the generative AI model to identify a solution and generate a manual in HTML or PDF format detailing that solution" refers to the function of using a generative AI model to find a solution to a problem and automatically generating a manual in HTML or PDF format detailing that solution in an easy-to-understand manner.

[1141] "Means for providing the generated manual to the display device of the industrial robot in real time" refers to the function of displaying the generated manual on the display of the industrial robot in real time, so that the operator and the robot itself can immediately refer to it.

[1142] The system of the present invention is a support system for quickly and accurately resolving questions and problems that industrial robots operating in factories encounter during operation. This system uses hardware such as industrial robots, servers, sensors, cameras, and displays, as well as software such as generative AI models (e.g., OpenAI GPT-4), Python, Flask (a web framework), and a JSON-formatted data interface.

[1143] The server includes the following means:

[1144] 1. Means of receiving unclear data from users

[1145] It is a means for users or robots to send and receive problems or questions they encounter in JSON format to the server.

[1146] 2. A means of analyzing incoming unknown data using a generative AI model

[1147] This is the process of passing the received data to a generative AI model (e.g., OpenAI GPT-4) that uses natural language processing technology to analyze and understand its content.

[1148] 3. A method for automatically generating manuals based on analysis results

[1149] It is a means of automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from a generative AI model.

[1150] 4. Means for sending the generated manual to the user's device

[1151] This function allows the generated manual to be sent to the user in HTML or PDF format for viewing.

[1152] 5. A way for industrial robots to detect problems through sensors and cameras while in operation and send that data in JSON format to a server

[1153] This is the process of detecting the robot's status and problems using sensors and cameras, and sending that information to a server in JSON format.

[1154] 6. A means for the generative AI model to identify solutions and generate detailed solutions manuals in HTML or PDF format.

[1155] This function uses generative AI models to find solutions to problems and automatically generates detailed and easy-to-understand manuals describing the solutions in HTML or PDF format.

[1156] 7. Means for providing the generated manual to the display device of the industrial robot in real time

[1157] This function allows the generated manual to be displayed in real time on the industrial robot's display, making it available for immediate reference by the operator and the robot itself.

[1158] For example, if an industrial robot detects a problem while in operation, such as "the equipment connection is not working properly," it will send this information, obtained through sensors and cameras, in JSON format to a server. The server will then use a generative AI model based on the received data to analyze the problem and find a solution. Based on the analysis results, it will automatically generate a manual including operating procedures, and display this manual in real time on the industrial robot's display. If a user inputs a point of uncertainty, such as "I don't know how to connect the equipment," the same procedure will be followed, and the generated manual will be sent to the user's device.

[1159] Here are some example prompts:

[1160] Please analyze the following problem and provide a specific solution:

[1161] "Device connection is not working properly"

[1162] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1163] Step 1:

[1164] While industrial robots are operating, they detect problems through sensors and cameras and send the data in JSON format to a server.

[1165] Specifically, the robot's sensors and cameras detect abnormal behavior or error messages. Detailed problem data (e.g., "Device connection is not working properly") is received as input, packaged in JSON format, and sent to the server via an HTTP POST request. The server receives this data and proceeds with further processing.

[1166] Step 2:

[1167] The server passes the received JSON-formatted unknown data to the generative AI model for analysis.

[1168] Specifically, the server reads the received data and sends it to a generative AI model (e.g., OpenAI GPT-4). The input is problem data from the user or robot. The generative AI model analyzes this data and identifies unclear points and solutions. For example, in response to the problem of "equipment connection not working properly," the generative AI model outputs "the correct procedure for connecting cables" and "how to check the connection."

[1169] Step 3:

[1170] Based on the analysis results, the server automatically generates a manual.

[1171] Specifically, it receives the solution from the generative AI model and formats it as detailed instructions in natural language. It uses the analysis results of the generative AI model obtained as input (e.g., "Please double-check the cable and connect it to the correct port") and automatically generates a manual in HTML or PDF format. The output is a detailed manual including operating procedures and illustrations.

[1172] Step 4:

[1173] The generated manual is provided in real time to the display device of the industrial robot.

[1174] Specifically, the server sends the generated manual to the robot's display. The input is the manual data generated in step 3, and the output is the manual displayed on the robot's display. This makes it available for immediate reference by users and robots.

[1175] Step 5:

[1176] If necessary, the generated manual is also sent to the user's terminal.

[1177] Specifically, the server sends the generated manual in HTML or PDF format to the user's device. The input is the manual data generated in step 3, and the output is the manual displayed on the user's device. For example, if a user inquires, "I don't know how to connect the equipment," this manual is sent to the user's device and can be viewed in a browser or PDF viewer.

[1178] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1179] The system of the present invention quickly and accurately resolves any questions users may encounter while entering data into the EOS system. It also recognizes the user's emotions and provides a manual tailored to their state, providing more effective support. The system receives data on the user's questions, analyzes them using a generative AI model, and automatically generates and provides a manual based on the results. Furthermore, by incorporating an emotion engine, the system adjusts the tone and expression of the manual according to the user's emotional state.

[1180] Server Processing

[1181] 1. Receiving input from the user

[1182] The server receives EOS input data and unclear data sent from the user's device. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[1183] 2. Emotion Recognition by Emotion Engine

[1184] The received unknown data is input into the emotion engine to recognize the user's emotion. The emotion engine analyzes the user's emotional state (e.g., anger, sadness, joy) from the text data.

[1185] 3. Analysis using generative AI models

[1186] The recognized user emotion data along with the difficulty data is passed to the generative AI model, which then uses natural language processing technology to identify the unclear points, taking into account the content of the unclear points and the user's emotions.

[1187] 4. Generate manual

[1188] Based on the analysis results and the user's emotions, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed, and adjusts the tone and expression according to the user's emotions.

[1189] 5. Provision to Users

[1190] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[1191] Terminal handling

[1192] 1. Sending input data

[1193] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[1194] 2. Receiving the manual

[1195] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[1196] User Actions

[1197] 1. EOS input

[1198] The user inputs the EOS data using the terminal.

[1199] 2. Enter any questions

[1200] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[1201] 3. Check the manual

[1202] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[1203] Specific examples

[1204] Example 1: New user registration procedure

[1205] 1. Users

[1206] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[1207] 2. Terminal

[1208] The entered unknown data is sent to the server.

[1209] 3. Server

[1210] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in an "anger" state.

[1211] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[1212] Based on the analysis results, a manual including user registration procedures is automatically generated, where instructions are explained in more polite language and a more friendly tone depending on the user's emotions.

[1213] The generated manual is sent to the terminal in HTML format.

[1214] 4. Terminal

[1215] Receive the manual and view it in your web browser.

[1216] 5. Users

[1217] Please refer to the manual and proceed while understanding the new user registration procedure.

[1218] Example 2: How to set up data backup

[1219] 1. Users

[1220] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[1221] 2. Terminal

[1222] The entered unknown data is sent to the server.

[1223] 3. Server

[1224] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in a "sad" state.

[1225] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[1226] Based on the analysis results, a manual including instructions for setting up data backups is automatically generated, which includes encouraging words and reassuring expressions depending on the user's emotions.

[1227] The generated manual is sent to the terminal in PDF format.

[1228] 4. Terminal

[1229] Receive the manual and view it in a PDF viewer.

[1230] 5. Users

[1231] Refer to the manual and proceed while understanding how to set up data backup.

[1232] In this way, this system, which incorporates an emotion engine, analyzes the unclear points entered by the user and provides the optimal manual based on their emotions, thereby improving work efficiency and the user experience.

[1233] The processing flow will be explained below.

[1234] Step 1:

[1235] user

[1236] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the specific details in the text box (e.g., "I don't know how to register in the system").

[1237] Step 2:

[1238] Terminal

[1239] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the send button, the data is sent to the server via the REST API.

[1240] Step 3:

[1241] server

[1242] The data sent from the device is received. The API processes the request, validates the data, and then passes any unclear data to the emotion engine.

[1243] Step 4:

[1244] Emotion Engine

[1245] Analyzes the received unknown data to recognize the user's emotional state. Analyzes text data to identify the user's emotions (e.g., anger, sadness, joy).

[1246] Step 5:

[1247] server

[1248] Along with the emotion data obtained from the emotion engine, the unknown data is passed to the generative AI model.

[1249] Step 6:

[1250] Generative AI Models

[1251] Input the unclear point data and emotion data and begin analysis. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and identify the necessary information.

[1252] Step 7:

[1253] Generative AI Models

[1254] Based on the analysis results, necessary steps and explanations are extracted, and the content is adjusted to suit the user's emotions using a tone and expression.

[1255] Step 8:

[1256] server

[1257] Using information from the generative AI model, the system automatically generates a manual that includes instructions, steps, and illustrations, and adjusts tone and expression based on emotional data.

[1258] Step 9:

[1259] server

[1260] Convert the generated manual into HTML or PDF format, the format is selected based on the user's requirements and system settings.

[1261] Step 10:

[1262] server

[1263] Formatted manual data is sent to the terminal. For quick delivery, a REST API is used to send the data.

[1264] Step 11:

[1265] Terminal

[1266] Receives the manual data sent from the server. The received manual is then processed so that it can be displayed in a web browser or PDF viewer.

[1267] Step 12:

[1268] user

[1269] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions and resolves any questions. For example, the user can refer to the system registration procedure to complete the registration.

[1270] This system not only enables users to quickly resolve any questions they may have, but also allows them to receive optimal support tailored to their emotional state at the time.

[1271] Example 2

[1272] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1273] Conventional user support systems simply display simple FAQs or existing manuals in response to data input by users, making it difficult to provide customized support that reflects the situation and emotions of each individual user. Furthermore, there was a lack of systems that could properly recognize the emotional state of users when they input their questions and take appropriate approaches. As a result, the quality of the user experience declined and support efficiency declined.

[1274] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1275] In this invention, the server includes means for receiving uncertainty data and emotion data from the user, means for analyzing the received uncertainty data and emotion data using an emotion recognition engine, means for generating a prompt sentence based on the received uncertainty data and the analyzed emotion data using a generative AI model, means for obtaining an analysis result using the generative AI model based on the generated prompt sentence, means for automatically generating a manual based on the analysis result and the user's emotion, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide optimal support that takes into account the user's uncertainty and emotional state.

[1276] "User's unclear point data" is text data describing problems and questions that users have encountered while using the system.

[1277] "Emotion data" is data that indicates the emotional state that the user is feeling when inputting the unclear point.

[1278] An "emotion recognition engine" is a program or algorithm that analyzes and identifies a user's emotional state from text data.

[1279] A "generative AI model" is an artificial intelligence model that performs natural language processing based on large amounts of text data, and specifically has the ability to understand the content of a user's inquiry and generate an appropriate response.

[1280] A "prompt" is a textual instruction given to a generative AI model, and is a document that includes an introductory section for analysis.

[1281] "Analysis results" refers to the information and answers analyzed by the generative AI model, and indicate solutions to any questions users may have.

[1282] A "manual" is a document that describes solutions and procedures for problems that users are facing.

[1283] "HTML" is a standard markup language for building web pages.

[1284] "PDF" is a file format designed for displaying and printing electronic documents, and allows documents to be shared while maintaining their format.

[1285] "User device" refers to an electronic device, such as a PC, smartphone, or tablet, that a user uses to operate the system.

[1286] This invention is a support system that quickly and accurately resolves any questions users may have when using a system. The system utilizes a generative AI model to analyze the user's questions and emotional state, providing the most appropriate manual.

[1287] Main hardware and software used

[1288] Hardware: User devices (PCs, smartphones, tablets, etc.), servers (high-performance computing servers)

[1289] Software: Emotion recognition engine (natural language processing library), generative AI model (natural language generation model)

[1290] Data processing and calculation

[1291] Server Processing

[1292] 1. Receiving input data:

[1293] The server receives EOS input data and unclear data sent from the user's device in JSON format. For example, if a user enters "I don't know how to register in the system," this data is parsed by the server.

[1294] 2. Emotion Recognition:

[1295] The server inputs the received unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine is used to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[1296] 3. Generate prompt sentence:

[1297] The server generates a prompt for the generative AI model based on the recognized emotion data and the unclear point data. This prompt contains specific questions and instructions to clearly communicate the unclear point to the generative AI model. For example, it could be something like, "Please create a manual that explains how to register in the system in the most detailed way possible. The user is feeling angry."

[1298] 4. Run the analysis:

[1299] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[1300] 5. Generate manual:

[1301] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[1302] 6. Offerings to Users:

[1303] The server then sends the completed manual to the user's device in HTML or PDF format, allowing the user to instantly view the information they need.

[1304] Terminal handling

[1305] 1. User input submission:

[1306] The terminal sends the EOS data and unknown data entered by the user to the server in JSON format.

[1307] 2. Receiving the manual:

[1308] The terminal receives the manual sent from the server. The manual is displayed in HTML or PDF format, so users can easily view it without needing special software.

[1309] User Actions

[1310] 1. EOS input:

[1311] The user inputs the EOS data using the terminal.

[1312] 2. Submit any questions:

[1313] When a user encounters a problem, they enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system" in the input form.

[1314] 3. Check the manual:

[1315] The user checks the manual sent from the server on the terminal and obtains the necessary information. For example, the received manual contains detailed instructions on how to register the system, and the user can proceed with the work by following the instructions.

[1316] Specific examples

[1317] Example 1: New user registration procedure

[1318] 1. User:

[1319] Enter your question, such as "I don't know how to register as a new user."

[1320] 2. Terminal:

[1321] The inputted unknown data is sent to the server.

[1322] 3. Server:

[1323] The received unknown data is passed to the emotion engine, which recognizes the user's emotion as "anger."

[1324] Generate prompt sentences based on emotion data and unclear point data.

[1325] The prompt sentence, "Please create a manual that explains how to register in the system in the most detailed way possible. Users are feeling angry," is passed to the generative AI model.

[1326] Based on the analysis results obtained from the generative AI model, a manual written in a friendly tone is generated.

[1327] Send the manual to the terminal in HTML format.

[1328] 4. Terminal:

[1329] Receive the manual and view it in your browser.

[1330] 5. User:

[1331] Please refer to the manual and proceed while understanding the new user registration procedure.

[1332] This system makes it possible to quickly provide the most appropriate manual that takes into account the user's uncertainties and emotional state.

[1333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1334] Step 1: Receiving input data

[1335] The server receives EOS input data and unknown data sent from the user's device. The input data is in JSON format, and the data structure consists of key-value pairs. The received data includes the user ID, unknown content, timestamp, etc. The server parses this data and stores each item in the appropriate variable.

[1336] input:

[1337] User EOS input data

[1338] Uncertainty data (e.g., "I don't know how to register in the system")

[1339] output:

[1340] Parsed uncertainty data and other meta information

[1341] Specific operation:

[1342] The server parses the received JSON data and stores the user ID, EOS data, and unknown data in appropriate variables.

[1343] Step 2: Recognize emotions

[1344] The server inputs the parsed unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine uses a natural language processing library to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[1345] input:

[1346] Unknown data

[1347] output:

[1348] Emotion recognition result (e.g., "anger" state)

[1349] Specific operation:

[1350] The emotion recognition engine inputs the unknown text and determines its emotional state, which is then passed to the next processing step.

[1351] Step 3: Generate a prompt statement

[1352] The server generates prompts for the generative AI model based on the recognized emotion data and unclear point data. These prompts include specific questions and instructions to communicate the unclear points to the generative AI model in an easy-to-understand manner.

[1353] input:

[1354] Unknown data

[1355] Emotion recognition results

[1356] output:

[1357] Prompt statement (e.g., "Please create the most thorough manual possible explaining how to register the system. Users are angry.")

[1358] Specific operation:

[1359] The server uses the unknown data and emotion recognition results to generate prompt sentences based on dedicated templates.

[1360] Step 4: Perform the analysis

[1361] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[1362] input:

[1363] Prompt statement

[1364] output:

[1365] Analysis results (e.g. system registration procedure)

[1366] Specific operation:

[1367] The server inputs a prompt sentence into the generative AI model and obtains a solution as an analysis result.

[1368] Step 5: Generate the manual

[1369] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[1370] input:

[1371] Analysis results

[1372] Emotion recognition results

[1373] output:

[1374] Auto-generated manual

[1375] Specific operation:

[1376] The server uses the analysis results and emotion data to generate a manual based on an appropriate template.

[1377] Step 6: Deliver to users

[1378] The server then sends the completed manual to the user's device in HTML or PDF format, with a link and notification automatically sent for immediate access.

[1379] input:

[1380] Auto-generated manual

[1381] output:

[1382] Manual sent to the user's device (HTML or PDF format)

[1383] Specific operation:

[1384] The server converts the manual into an appropriate format and sends it to the user's device. It also generates a notification of completion and an access link, which it provides to the user.

[1385] (Application example 2)

[1386] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1387] A support system that can quickly and appropriately respond to technical questions and problems that drivers and users encounter in autonomous vehicles is needed. Conventional support systems do not provide manuals that take into account the user's emotional state, which can result in increased stress and dissatisfaction for users. Furthermore, conventional technologies take time to resolve users' questions, and there are insufficient means to mitigate safety risks that arise during this time.

[1388] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving unclear data from the user, means for analyzing the received unclear data using a generative AI model, means for automatically generating a manual based on the analysis results and the user's emotional data, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide an appropriate manual using a tone and expression that matches the user's emotional state, enabling immediate responses to technical questions and problems in autonomous vehicles and improving user satisfaction and safety.

[1389] "User" refers to a person who uses the system, specifically a driver or user.

[1390] "Uncertainty data" refers to data about questions or problems users encounter while using the system.

[1391] A "generative AI model" is an artificial intelligence model that analyzes received unknown data and generates corresponding information and solutions.

[1392] "Analysis" refers to the process of examining data in detail and understanding its contents.

[1393] "Emotional data" is data that represents a user's emotional state extracted from their text or speech.

[1394] A "manual" is a document that contains procedures and explanations for resolving user questions and problems.

[1395] "Auto-generation" refers to the process by which a program automatically creates output information from specific input data.

[1396] "Terminal" refers to a device used to use the system, and specifically includes smartphones and in-car tablets.

[1397] This invention is a system that provides appropriate support in real time for questions or problems that users encounter in autonomous vehicles. This system uses smartphones or in-car tablets as terminals, and sends input data from users to a server, which then analyzes it.

[1398] The server uses the following methods:

[1399] A means for receiving data on questions from users: Users use their devices to input text or voice about questions or problems. This input data is sent to the server.

[1400] A method for analyzing received unknown data using a generative AI model: A generative AI model is used to analyze the received data. The generative AI model uses natural language processing techniques to analyze the unknown data and identify its points.

[1401] A means of automatically generating manuals based on analysis results and user emotional data: An emotion engine is used to extract user emotional data, which is then combined with the analysis results to automatically generate manuals that take into account appropriate tone and expression.

[1402] Means for sending the generated manual to the user's terminal: The generated manual is sent to the user's terminal in HTML or PDF format.

[1403] As a concrete example, consider the case where the vehicle camera displays an error message during autonomous driving. When the user types "The vehicle camera is displaying an error message, what should I do?" into the terminal, the following process is executed.

[1404] The server receives the user's input and uses an emotion engine to recognize the user's emotion as "impatience." The generative AI model then analyzes the input and automatically generates a manual with specific steps to deal with the camera's error message. The manual includes a gentle tone and expressions to help users respond calmly to their emotions. The generated manual is sent to the user's device in HTML format and displayed in a window.

[1405] An implementation example shows the use of the following prompt:

[1406] My car's camera is showing an error message, what should I do?

[1407] Emotion: "Impatience"

[1408] This embodiment allows users to respond appropriately and quickly to any problems that may occur during automated driving, reducing stress and improving overall safety and satisfaction.

[1409] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1410] Step 1:

[1411] The user uses a device (smartphone or in-car tablet) to input text or voice information about a problem or issue. For example, the user might input, "The vehicle camera is displaying an error message. What should I do?" This input data is sent to the server in an appropriate format, such as JSON.

[1412] Step 2:

[1413] The server receives input data from the user and passes it to the emotion engine for analysis. The emotion engine extracts the user's emotional state from the text data and recognizes it as "impatience," for example. This emotion data is returned to the server as the analysis result.

[1414] Step 3:

[1415] The server inputs the received unclear point data and emotion data into the generative AI model. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and generate a manual containing appropriate solutions and information. Specifically, it generates text containing how to deal with camera errors.

[1416] Step 4:

[1417] The server automatically generates a manual in HTML or PDF format based on the analysis results obtained from the generative AI model. The manual is adjusted according to the user's emotions, including a gentler tone and more specific instructions for users who are feeling impatient.

[1418] Step 5:

[1419] The server sends the completed manual to the user's device, where the user can view the HTML manual in a browser or download and read it in PDF format. The user can refer to this manual and follow the specific steps to solve the problem.

[1420] This series of processes enables users to respond quickly and appropriately to any problems that may occur during automated driving, thereby improving safety and satisfaction.

[1421] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1422] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1423] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1424] [Fourth embodiment]

[1425] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1426] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1427] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1428] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1429] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1430] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1431] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1432] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1433] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1434] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1435] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1436] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1437] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1438] The system of the present invention is designed to quickly and accurately resolve any questions users may encounter while entering EOS data. The system receives questions from users, analyzes them using a generative AI model, and automatically generates a manual based on the results to provide to the user.

[1439] Server Processing

[1440] 1. Receiving input from the user

[1441] The server receives EOS input data and unclear data sent from the user's terminal. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[1442] 2. Analysis using generative AI models

[1443] The received unknown data is passed to the generative AI model, which then uses natural language processing technology to deeply understand the unknown content and identify its key points.

[1444] 3. Generate manual

[1445] Based on the analysis results, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed.

[1446] 4. Provision to Users

[1447] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[1448] Terminal handling

[1449] 1. Sending input data

[1450] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[1451] 2. Receiving the manual

[1452] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[1453] User Actions

[1454] 1. EOS input

[1455] The user inputs the EOS data using the terminal.

[1456] 2. Enter any questions

[1457] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[1458] 3. Check the manual

[1459] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[1460] Specific examples

[1461] Example 1: New user registration procedure

[1462] 1. Users

[1463] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[1464] 2. Terminal

[1465] The entered unknown data is sent to the server.

[1466] 3. Server

[1467] The received unknown data is passed to the generative AI model for analysis.

[1468] Based on the analysis results, a manual including user registration procedures is automatically generated.

[1469] The generated manual is sent to the terminal in HTML format.

[1470] 4. Terminal

[1471] Receive the manual and view it in your web browser.

[1472] 5. Users

[1473] Please refer to the manual and proceed while understanding the new user registration procedure.

[1474] Example 2: How to set up data backup

[1475] 1. Users

[1476] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[1477] 2. Terminal

[1478] The entered unknown data is sent to the server.

[1479] 3. Server

[1480] The received unknown data is passed to the generative AI model for analysis.

[1481] Based on the analysis results, a manual including data backup setup procedures is automatically generated.

[1482] The generated manual is sent to the terminal in PDF format.

[1483] 4. Terminal

[1484] Receive the manual and view it in a PDF viewer.

[1485] 5. Users

[1486] Refer to the manual and proceed while understanding how to set up data backup.

[1487] In this way, this system analyzes unclear points based on the data entered by the user and provides immediate solutions, thereby improving work efficiency and reducing work stress.

[1488] The processing flow will be explained below.

[1489] Step 1:

[1490] user

[1491] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the details in the text box (e.g., "I don't know how to register in the system").

[1492] Step 2:

[1493] Terminal

[1494] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the submit button, the data is sent to the endpoint API.

[1495] Step 3:

[1496] server

[1497] Receives data sent from the device. The API receives the request and checks the data. Validates that the received data contains any unknowns.

[1498] Step 4:

[1499] server

[1500] Initialize the generative AI model and prepare to input the received unknown data into the model.

[1501] Step 5:

[1502] Generative AI Models

[1503] Input the data of the unknown points and start the analysis. Natural language processing technology is used for the analysis to deeply understand the content of the unknown points and identify the problem points.

[1504] Step 6:

[1505] Generative AI Models

[1506] Based on the analysis results, necessary information and procedures are extracted. For example, if a user is unsure about how to register in the system, specific procedures for registering in the system are generated.

[1507] Step 7:

[1508] server

[1509] The analysis results from the generative AI model are used to construct the content of the manual, which includes explanatory text, procedures, illustrations, and other information, formatted in a user-friendly format.

[1510] Step 8:

[1511] server

[1512] Converts the generated manual into HTML or PDF format, depending on the system settings or user requirements.

[1513] Step 9:

[1514] server

[1515] Formatted manual data is sent to the terminal using an endpoint, ensuring that it is delivered to the user quickly.

[1516] Step 10:

[1517] Terminal

[1518] Receives manual data sent from the server. The received manual is processed for display in a web browser or PDF viewer.

[1519] Step 11:

[1520] user

[1521] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions, resolves any questions, and continues working. For example, the user can refer to the system registration procedure to complete registration.

[1522] In this way, the entire process flow is achieved quickly and efficiently.

[1523] Example 1

[1524] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1525] Previous systems lacked a means to quickly and accurately resolve any questions users might encounter while entering EOS data. This resulted in users spending a lot of time resolving problems, reducing work efficiency. There was also the risk that users would rely on inaccurate information when conducting their own research.

[1526] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1527] In this invention, the server includes a means for receiving input data from a user, a means for analyzing the received unclear point data using a generative AI model, and a means for automatically generating a manual based on the analysis results. This makes it possible to quickly analyze the user's unclear points and provide appropriate solutions. In addition, the generated manual is provided in HTML or PDF format, improving the convenience of users by allowing them to refer to it immediately.

[1528] "Means for receiving input data from a user" is a function that allows the server to receive information that a user has input using a terminal.

[1529] "Means for analyzing received unclear data using a generative AI model" refers to a function that uses a generative AI model that employs natural language processing technology to perform a detailed analysis of the unclear data entered by the user and understand its content.

[1530] "Means for automatically generating manuals based on analysis results" is a function that automatically creates manuals in a format that is easy for users to understand, based on the results of analysis by the generative AI model.

[1531] "Means for sending the generated manual to the user's terminal" refers to a function for sending the automatically generated manual to the user's terminal in an appropriate format (HTML or PDF).

[1532] "Means for sending input data to a server in JSON format" is a function for converting data entered by a user on a terminal into JSON format and sending it to a server.

[1533] "Means for providing to users in HTML or PDF format" refers to a function for converting the generated manual into HTML or PDF format and providing it to users in a format that is easily accessible.

[1534] The system of the present invention is designed to quickly and accurately resolve any unclear points users may encounter while entering EOS data. The system receives input data from the user, analyzes it using a generative AI model, and automatically generates a manual based on the analysis results to provide to the user. A specific implementation of the present invention is described below.

[1535] Server Processing

[1536] The server includes hardware and software for receiving EOS input data and unknown data sent from the user's device. For example, the server has an API endpoint for processing HTTP requests and has the functionality to appropriately receive input data from the user. This data is often sent in JSON format, so the server parses and stores the JSON data.

[1537] Next, the server passes the received unknown data to a generative AI model to begin analysis. The generative AI model uses natural language processing technologies such as GPT-3 and BERT. This allows it to deeply understand the content of the unknown and analyze it. At this time, the specific prompt text passed to the model is the user's input, such as "I don't know how to register in the system."

[1538] Based on the analysis results, the server automatically generates a manual using the information obtained from the generative AI model. This manual contains specific procedures and explanations, and inserts diagrams and tables as needed. The server then converts the generated manual into HTML or PDF format using an HTML template or a PDF generation library (e.g., Weasyprint).

[1539] The completed manual is sent to the user's device in HTML or PDF format, where the user can view it immediately. For example, the manual data generated by "POST / api / manuals" is sent to the user's device.

[1540] Terminal handling

[1541] The terminal provides an interface for users to input EOS data. When a user inputs an unknown point, the data is sent to the server in JSON format. For example, the unknown point data is sent by "POST / api / queries".

[1542] The device then receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the device's browser or PDF viewer, allowing the user to view detailed instructions and explanations and solve problems on the spot.

[1543] User Actions

[1544] Users enter EOS data using a terminal. They enter the data in detail on the interface, and if they have any questions, they enter them and send them to the server. For example, they can enter a specific question such as, "I don't know how to register in the system."

[1545] Once the manual sent from the server is received by the terminal, the user can open it in a browser or PDF viewer and check the contents. For example, the user can read the manual that details the new user registration procedure and proceed by following the steps.

[1546] Specific examples

[1547] Example: New user registration procedure

[1548] 1. Users

[1549] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[1550] 2. Terminal

[1551] The inputted unknown data is sent to the server.

[1552] 3. Server

[1553] The received unknown data is passed to the generative AI model for analysis.

[1554] Based on the analysis results, a manual including user registration procedures is automatically generated.

[1555] The generated manual is sent to the terminal in HTML format.

[1556] 4. Terminal

[1557] Receive the manual and view it in your web browser.

[1558] 5. Users

[1559] Please refer to the manual and proceed while understanding the new user registration procedure.

[1560] In this way, the system analyzes unclear points based on the data entered by the user and provides prompt solutions, thereby improving work efficiency and reducing work stress.

[1561] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1562] Step 1: Sending input data from the user

[1563] 1. Input: The user inputs EOS data using the terminal, and if there is anything unclear, they enter the details.

[1564] 2. Specific action: The user enters specific questions, such as "I don't know how to register in the system," into the input form on the interface and presses the submit button.

[1565] 3. Data processing / calculation: The terminal converts the data entered by the user into JSON format.

[1566] 4. Output: Send the unknown data in JSON format to the server (e.g., send to "POST / api / queries").

[1567] Step 2: Receiving input data from the user

[1568] 1. Input: Unknown data in JSON format sent from the terminal.

[1569] 2. Specific operation: The server receives the HTTP request and retrieves the unknown data.

[1570] 3. Data processing / calculation: Parse (analyze) the received JSON data and save it in the internal database.

[1571] 4. Output: The saved instances of the unknown data.

[1572] Step 3: Analysis by generative AI model

[1573] 1. Input: Stored unknown data.

[1574] 2. Specific operation: The server passes the stored unknown data to the generative AI model (e.g., GPT-3) as a prompt sentence.

[1575] 3. Data processing / calculation: The generative AI model performs analysis based on the prompt text and generates appropriate answers and procedures.

[1576] 4. Output: Analysis results from the generative AI model (e.g., specific operating procedures).

[1577] Step 4: Automatic generation of manuals

[1578] 1. Input: Analysis results from the generative AI model.

[1579] 2. Specific operation: The server generates a manual including text and illustrations based on the analysis results.

[1580] 3. Data processing / calculation: Format the analysis results using HTML templates or a PDF generation library (e.g., Weasyprint).

[1581] 4. Output: Manual in HTML or PDF format.

[1582] Step 5: Manually send to users

[1583] 1. Input: Manual in HTML or PDF format.

[1584] 2. Specific operation: The server creates an HTTP response to return the generated manual to the user's device.

[1585] 3. Data processing / calculation: Include manual data in the HTTP response body.

[1586] 4. Output: An HTTP response containing the manual is sent to the user's device (e.g., sent with "POST / api / manuals").

[1587] Step 6: Receive and view the manual

[1588] 1. Input: The HTTP response sent by the server (including manual data).

[1589] 2. Specific operation: The terminal receives the HTTP response and extracts the included manual data.

[1590] 3. Data processing / calculation: Convert the extracted manual data into a format that can be displayed in an HTML or PDF viewer.

[1591] 4. Output: The manual that will be displayed on the user's device.

[1592] Step 7: Refer to your user manual

[1593] 1. Input: Manual displayed on the terminal.

[1594] 2. Specific actions: The user refers to the manual displayed on the device and checks the steps to resolve any questions.

[1595] 3. Data processing / calculation: The user performs the operations according to the contents of the manual.

[1596] 4. Output: Resolved issues and smooth progress of work. The end result is that issues are resolved and work proceeds smoothly.

[1597] (Application example 1)

[1598] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1599] This invention relates to technology for quickly and accurately resolving questions and problems that arise during the operation of industrial robots. In conventional industrial robot systems, when a problem occurs, the intervention of an operator with specialized knowledge is required to resolve the problem, which inevitably leads to interruptions in work and reduced efficiency. Such problems reduce the productivity of industrial processes and increase maintenance costs. Meanwhile, there is a demand for systems in which robots themselves have the ability to solve problems and respond in real time.

[1600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1601] In this invention, the server includes means for receiving unclear point data from a user, means for analyzing the received unclear point data using a generative AI model, means for automatically generating a manual based on the analysis results, means for transmitting the generated manual to the user's terminal, means for the industrial robot to recognize problems through sensors or cameras while in operation and transmit the data in JSON format to the server, means for the generative AI model to identify solutions based on the unclear point data and generate a manual in HTML or PDF format that details the solutions, and means for providing the generated manual to the industrial robot's display device in real time. This enables the industrial robot to recognize unclear points on its own while in operation, obtain solutions in real time via the server, and quickly solve problems.

[1602] "Means for receiving unclear data from users" refers to an input interface that allows users to report unclear or questionable points to the system, and typically handles data in JSON format.

[1603] "Means of analyzing received unknown data using a generative AI model" refers to the process of passing received data to a generative AI model that uses natural language processing technology, and analyzing and understanding its contents.

[1604] "Means for automatically generating manuals based on analysis results" refers to algorithms or scripts for automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from the generative AI model.

[1605] "Means for sending the generated manual to the user's terminal" refers to a sending function for providing the generated manual to the user in HTML or PDF format in real time.

[1606] "A means for industrial robots to detect problems through sensors and cameras while in operation and send the data to a server in JSON format" refers to the process of using sensors and cameras to detect the status and problems of robots and send that information to a server.

[1607] "Means for the generative AI model to identify a solution and generate a manual in HTML or PDF format detailing that solution" refers to the function of using a generative AI model to find a solution to a problem and automatically generating a manual in HTML or PDF format detailing that solution in an easy-to-understand manner.

[1608] "Means for providing the generated manual to the display device of the industrial robot in real time" refers to the function of displaying the generated manual on the display of the industrial robot in real time, so that the operator and the robot itself can immediately refer to it.

[1609] The system of the present invention is a support system for quickly and accurately resolving questions and problems that industrial robots operating in factories encounter during operation. This system uses hardware such as industrial robots, servers, sensors, cameras, and displays, as well as software such as generative AI models (e.g., OpenAI GPT-4), Python, Flask (a web framework), and a JSON-formatted data interface.

[1610] The server includes the following means:

[1611] 1. Means of receiving unclear data from users

[1612] It is a means for users or robots to send and receive problems or questions they encounter in JSON format to the server.

[1613] 2. A means of analyzing incoming unknown data using a generative AI model

[1614] This is the process of passing the received data to a generative AI model (e.g., OpenAI GPT-4) that uses natural language processing technology to analyze and understand its content.

[1615] 3. A method for automatically generating manuals based on analysis results

[1616] It is a means of automatically generating manuals containing specific operating procedures and solutions based on the analysis results obtained from a generative AI model.

[1617] 4. Means for sending the generated manual to the user's device

[1618] This function allows the generated manual to be sent to the user in HTML or PDF format for viewing.

[1619] 5. A way for industrial robots to detect problems through sensors and cameras while in operation and send that data in JSON format to a server

[1620] This is the process of detecting the robot's status and problems using sensors and cameras, and sending that information to a server in JSON format.

[1621] 6. A means for the generative AI model to identify solutions and generate detailed solutions manuals in HTML or PDF format.

[1622] This function uses generative AI models to find solutions to problems and automatically generates detailed and easy-to-understand manuals describing the solutions in HTML or PDF format.

[1623] 7. Means for providing the generated manual to the display device of the industrial robot in real time

[1624] This function allows the generated manual to be displayed in real time on the industrial robot's display, making it available for immediate reference by the operator and the robot itself.

[1625] For example, if an industrial robot detects a problem while in operation, such as "the equipment connection is not working properly," it will send this information, obtained through sensors and cameras, in JSON format to a server. The server will then use a generative AI model based on the received data to analyze the problem and find a solution. Based on the analysis results, it will automatically generate a manual including operating procedures, and display this manual in real time on the industrial robot's display. If a user inputs a point of uncertainty, such as "I don't know how to connect the equipment," the same procedure will be followed, and the generated manual will be sent to the user's device.

[1626] Here are some example prompts:

[1627] Please analyze the following problem and provide a specific solution:

[1628] "Device connection is not working properly"

[1629] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1630] Step 1:

[1631] While industrial robots are operating, they detect problems through sensors and cameras and send the data in JSON format to a server.

[1632] Specifically, the robot's sensors and cameras detect abnormal behavior or error messages. Detailed problem data (e.g., "Device connection is not working properly") is received as input, packaged in JSON format, and sent to the server via an HTTP POST request. The server receives this data and proceeds with further processing.

[1633] Step 2:

[1634] The server passes the received JSON-formatted unknown data to the generative AI model for analysis.

[1635] Specifically, the server reads the received data and sends it to a generative AI model (e.g., OpenAI GPT-4). The input is problem data from the user or robot. The generative AI model analyzes this data and identifies unclear points and solutions. For example, in response to the problem of "equipment connection not working properly," the generative AI model outputs "the correct procedure for connecting cables" and "how to check the connection."

[1636] Step 3:

[1637] Based on the analysis results, the server automatically generates a manual.

[1638] Specifically, it receives the solution from the generative AI model and formats it as detailed instructions in natural language. It uses the analysis results of the generative AI model obtained as input (e.g., "Please double-check the cable and connect it to the correct port") and automatically generates a manual in HTML or PDF format. The output is a detailed manual including operating procedures and illustrations.

[1639] Step 4:

[1640] The generated manual is provided in real time to the display device of the industrial robot.

[1641] Specifically, the server sends the generated manual to the robot's display. The input is the manual data generated in step 3, and the output is the manual displayed on the robot's display. This makes it available for immediate reference by users and robots.

[1642] Step 5:

[1643] If necessary, the generated manual is also sent to the user's terminal.

[1644] Specifically, the server sends the generated manual in HTML or PDF format to the user's device. The input is the manual data generated in step 3, and the output is the manual displayed on the user's device. For example, if a user inquires, "I don't know how to connect the equipment," this manual is sent to the user's device and can be viewed in a browser or PDF viewer.

[1645] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1646] The system of the present invention quickly and accurately resolves any questions users may encounter while entering data into the EOS system. It also recognizes the user's emotions and provides a manual tailored to their state, providing more effective support. The system receives data on the user's questions, analyzes them using a generative AI model, and automatically generates and provides a manual based on the results. Furthermore, by incorporating an emotion engine, the system adjusts the tone and expression of the manual according to the user's emotional state.

[1647] Server Processing

[1648] 1. Receiving input from the user

[1649] The server receives EOS input data and unclear data sent from the user's device. It captures the unclear data entered by the user (e.g., "I don't know how to register in the system").

[1650] 2. Emotion Recognition by Emotion Engine

[1651] The received unknown data is input into the emotion engine to recognize the user's emotion. The emotion engine analyzes the user's emotional state (e.g., anger, sadness, joy) from the text data.

[1652] 3. Analysis using generative AI models

[1653] The recognized user emotion data along with the difficulty data is passed to the generative AI model, which then uses natural language processing technology to identify the unclear points, taking into account the content of the unclear points and the user's emotions.

[1654] 4. Generate manual

[1655] Based on the analysis results and the user's emotions, the server uses information obtained from the generative AI model to automatically generate a manual, which includes specific steps and explanations, as well as diagrams and tables as needed, and adjusts the tone and expression according to the user's emotions.

[1656] 5. Provision to Users

[1657] The completed manual is sent to the user's device in HTML or PDF format, allowing the user to instantly refer to the information they need.

[1658] Terminal handling

[1659] 1. Sending input data

[1660] When a user inputs EOS data using a terminal, if there is anything unclear, they can enter the details and send it to the server. The input is done quickly and sent to the server in an appropriate format (e.g., JSON format).

[1661] 2. Receiving the manual

[1662] The terminal receives the manual sent from the server. The manual is provided in HTML or PDF format and can be displayed in the terminal's browser or PDF viewer.

[1663] User Actions

[1664] 1. EOS input

[1665] The user inputs the EOS data using the terminal.

[1666] 2. Enter any questions

[1667] When a user encounters a problem, they can enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system."

[1668] 3. Check the manual

[1669] The user can check the manual sent from the server on the terminal and obtain the necessary information. For example, the user can check the manual that contains detailed instructions for system registration and proceed with the work according to the contents.

[1670] Specific examples

[1671] Example 1: New user registration procedure

[1672] 1. Users

[1673] If you do not know how to register as a new user on the system, enter your question by writing "I do not know how to register as a new user."

[1674] 2. Terminal

[1675] The entered unknown data is sent to the server.

[1676] 3. Server

[1677] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in an "anger" state.

[1678] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[1679] Based on the analysis results, a manual including user registration procedures is automatically generated, where instructions are explained in more polite language and a more friendly tone depending on the user's emotions.

[1680] The generated manual is sent to the terminal in HTML format.

[1681] 4. Terminal

[1682] Receive the manual and view it in your web browser.

[1683] 5. Users

[1684] Please refer to the manual and proceed while understanding the new user registration procedure.

[1685] Example 2: How to set up data backup

[1686] 1. Users

[1687] If you do not know how to set up data backup, enter your question by writing "I do not know how to set up data backup."

[1688] 2. Terminal

[1689] The entered unknown data is sent to the server.

[1690] 3. Server

[1691] The received unknown data is passed to the emotion engine to analyze the user's emotions. For example, it detects whether the user is in a "sad" state.

[1692] Along with the emotion data, the unknown data is passed to the generative AI model for analysis.

[1693] Based on the analysis results, a manual including instructions for setting up data backups is automatically generated, which includes encouraging words and reassuring expressions depending on the user's emotions.

[1694] The generated manual is sent to the terminal in PDF format.

[1695] 4. Terminal

[1696] Receive the manual and view it in a PDF viewer.

[1697] 5. Users

[1698] Refer to the manual and proceed while understanding how to set up data backup.

[1699] In this way, this system, which incorporates an emotion engine, analyzes the unclear points entered by the user and provides the optimal manual based on their emotions, thereby improving work efficiency and the user experience.

[1700] The processing flow will be explained below.

[1701] Step 1:

[1702] user

[1703] The user enters EOS data using the terminal. If there is anything unclear, the user can enter the specific details in the text box (e.g., "I don't know how to register in the system").

[1704] Step 2:

[1705] Terminal

[1706] The EOS data and unknown data entered by the user are sent to the server in an appropriate format (e.g., JSON format). When the user presses the send button, the data is sent to the server via the REST API.

[1707] Step 3:

[1708] server

[1709] The data sent from the device is received. The API processes the request, validates the data, and then passes any unclear data to the emotion engine.

[1710] Step 4:

[1711] Emotion Engine

[1712] Analyzes the received unknown data to recognize the user's emotional state. Analyzes text data to identify the user's emotions (e.g., anger, sadness, joy).

[1713] Step 5:

[1714] server

[1715] Along with the emotion data obtained from the emotion engine, the unknown data is passed to the generative AI model.

[1716] Step 6:

[1717] Generative AI Models

[1718] Input the unclear point data and emotion data and begin analysis. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and identify the necessary information.

[1719] Step 7:

[1720] Generative AI Models

[1721] Based on the analysis results, necessary steps and explanations are extracted, and the content is adjusted to suit the user's emotions using a tone and expression.

[1722] Step 8:

[1723] server

[1724] Using information from the generative AI model, the system automatically generates a manual that includes instructions, steps, and illustrations, and adjusts tone and expression based on emotional data.

[1725] Step 9:

[1726] server

[1727] Convert the generated manual into HTML or PDF format, the format is selected based on the user's requirements and system settings.

[1728] Step 10:

[1729] server

[1730] Formatted manual data is sent to the terminal. For quick delivery, a REST API is used to send the data.

[1731] Step 11:

[1732] Terminal

[1733] Receives the manual data sent from the server. The received manual is then processed so that it can be displayed in a web browser or PDF viewer.

[1734] Step 12:

[1735] user

[1736] The user checks the manual displayed on the terminal and obtains the necessary information. The user follows the instructions and resolves any questions. For example, the user can refer to the system registration procedure to complete the registration.

[1737] This system not only enables users to quickly resolve any questions they may have, but also allows them to receive optimal support tailored to their emotional state at the time.

[1738] Example 2

[1739] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1740] Conventional user support systems simply display simple FAQs or existing manuals in response to data input by users, making it difficult to provide customized support that reflects the situation and emotions of each individual user. Furthermore, there was a lack of systems that could properly recognize the emotional state of users when they input their questions and take appropriate approaches. As a result, the quality of the user experience declined and support efficiency declined.

[1741] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1742] In this invention, the server includes means for receiving uncertainty data and emotion data from the user, means for analyzing the received uncertainty data and emotion data using an emotion recognition engine, means for generating a prompt sentence based on the received uncertainty data and the analyzed emotion data using a generative AI model, means for obtaining an analysis result using the generative AI model based on the generated prompt sentence, means for automatically generating a manual based on the analysis result and the user's emotion, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide optimal support that takes into account the user's uncertainty and emotional state.

[1743] "User's unclear point data" is text data describing problems and questions that users have encountered while using the system.

[1744] "Emotion data" is data that indicates the emotional state that the user is feeling when inputting the unclear point.

[1745] An "emotion recognition engine" is a program or algorithm that analyzes and identifies a user's emotional state from text data.

[1746] A "generative AI model" is an artificial intelligence model that performs natural language processing based on large amounts of text data, and specifically has the ability to understand the content of a user's inquiry and generate an appropriate response.

[1747] A "prompt" is a textual instruction given to a generative AI model, and is a document that includes an introductory section for analysis.

[1748] "Analysis results" refers to the information and answers analyzed by the generative AI model, and indicate solutions to any questions users may have.

[1749] A "manual" is a document that describes solutions and procedures for problems that users are facing.

[1750] "HTML" is a standard markup language for building web pages.

[1751] "PDF" is a file format designed for displaying and printing electronic documents, and allows documents to be shared while maintaining their format.

[1752] "User device" refers to an electronic device, such as a PC, smartphone, or tablet, that a user uses to operate the system.

[1753] This invention is a support system that quickly and accurately resolves any questions users may have when using a system. The system utilizes a generative AI model to analyze the user's questions and emotional state, providing the most appropriate manual.

[1754] Main hardware and software used

[1755] Hardware: User devices (PCs, smartphones, tablets, etc.), servers (high-performance computing servers)

[1756] Software: Emotion recognition engine (natural language processing library), generative AI model (natural language generation model)

[1757] Data processing and calculation

[1758] Server Processing

[1759] 1. Receiving input data:

[1760] The server receives EOS input data and unclear data sent from the user's device in JSON format. For example, if a user enters "I don't know how to register in the system," this data is parsed by the server.

[1761] 2. Emotion Recognition:

[1762] The server inputs the received unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine is used to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[1763] 3. Generate prompt sentence:

[1764] The server generates a prompt for the generative AI model based on the recognized emotion data and the unclear point data. This prompt contains specific questions and instructions to clearly communicate the unclear point to the generative AI model. For example, it could be something like, "Please create a manual that explains how to register in the system in the most detailed way possible. The user is feeling angry."

[1765] 4. Run the analysis:

[1766] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[1767] 5. Generate manual:

[1768] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[1769] 6. Offerings to Users:

[1770] The server then sends the completed manual to the user's device in HTML or PDF format, allowing the user to instantly view the information they need.

[1771] Terminal handling

[1772] 1. User input submission:

[1773] The terminal sends the EOS data and unknown data entered by the user to the server in JSON format.

[1774] 2. Receiving the manual:

[1775] The terminal receives the manual sent from the server. The manual is displayed in HTML or PDF format, so users can easily view it without needing special software.

[1776] User Actions

[1777] 1. EOS input:

[1778] The user inputs the EOS data using the terminal.

[1779] 2. Submit any questions:

[1780] When a user encounters a problem, they enter the details of the problem and send it to the server. For example, they can enter a specific inquiry such as "I don't know how to register in the system" in the input form.

[1781] 3. Check the manual:

[1782] The user checks the manual sent from the server on the terminal and obtains the necessary information. For example, the received manual contains detailed instructions on how to register the system, and the user can proceed with the work by following the instructions.

[1783] Specific examples

[1784] Example 1: New user registration procedure

[1785] 1. User:

[1786] Enter your question, such as "I don't know how to register as a new user."

[1787] 2. Terminal:

[1788] The inputted unknown data is sent to the server.

[1789] 3. Server:

[1790] The received unknown data is passed to the emotion engine, which recognizes the user's emotion as "anger."

[1791] Generate prompt sentences based on emotion data and unclear point data.

[1792] The prompt sentence, "Please create a manual that explains how to register in the system in the most detailed way possible. Users are feeling angry," is passed to the generative AI model.

[1793] Based on the analysis results obtained from the generative AI model, a manual written in a friendly tone is generated.

[1794] Send the manual to the terminal in HTML format.

[1795] 4. Terminal:

[1796] Receive the manual and view it in your browser.

[1797] 5. User:

[1798] Please refer to the manual and proceed while understanding the new user registration procedure.

[1799] This system makes it possible to quickly provide the most appropriate manual that takes into account the user's uncertainties and emotional state.

[1800] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1801] Step 1: Receiving input data

[1802] The server receives EOS input data and unknown data sent from the user's device. The input data is in JSON format, and the data structure consists of key-value pairs. The received data includes the user ID, unknown content, timestamp, etc. The server parses this data and stores each item in the appropriate variable.

[1803] input:

[1804] User EOS input data

[1805] Uncertainty data (e.g., "I don't know how to register in the system")

[1806] output:

[1807] Parsed uncertainty data and other meta information

[1808] Specific operation:

[1809] The server parses the received JSON data and stores the user ID, EOS data, and unknown data in appropriate variables.

[1810] Step 2: Recognize emotions

[1811] The server inputs the parsed unknown data into an emotion recognition engine to analyze the user's emotions. The emotion recognition engine uses a natural language processing library to identify emotional states such as "anger," "sadness," and "joy" from the text data.

[1812] input:

[1813] Unknown data

[1814] output:

[1815] Emotion recognition result (e.g., "anger" state)

[1816] Specific operation:

[1817] The emotion recognition engine inputs the unknown text and determines its emotional state, which is then passed to the next processing step.

[1818] Step 3: Generate a prompt statement

[1819] The server generates prompts for the generative AI model based on the recognized emotion data and unclear point data. These prompts include specific questions and instructions to communicate the unclear points to the generative AI model in an easy-to-understand manner.

[1820] input:

[1821] Unknown data

[1822] Emotion recognition results

[1823] output:

[1824] Prompt statement (e.g., "Please create the most thorough manual possible explaining how to register the system. Users are angry.")

[1825] Specific operation:

[1826] The server uses the unknown data and emotion recognition results to generate prompt sentences based on dedicated templates.

[1827] Step 4: Perform the analysis

[1828] The server passes the generated prompt to the generative AI model, which then uses natural language processing technology to analyze the user's concerns and propose appropriate solutions.

[1829] input:

[1830] Prompt statement

[1831] output:

[1832] Analysis results (e.g. system registration procedure)

[1833] Specific operation:

[1834] The server inputs a prompt sentence into the generative AI model and obtains a solution as an analysis result.

[1835] Step 5: Generate the manual

[1836] Based on the analysis results obtained from the generative AI model, the server automatically generates a manual that includes specific procedures, explanations, diagrams, tables, and other information, with a tone and expressions that correspond to the user's emotional state.

[1837] input:

[1838] Analysis results

[1839] Emotion recognition results

[1840] output:

[1841] Auto-generated manual

[1842] Specific operation:

[1843] The server uses the analysis results and emotion data to generate a manual based on an appropriate template.

[1844] Step 6: Deliver to users

[1845] The server then sends the completed manual to the user's device in HTML or PDF format, with a link and notification automatically sent for immediate access.

[1846] input:

[1847] Auto-generated manual

[1848] output:

[1849] Manual sent to the user's device (HTML or PDF format)

[1850] Specific operation:

[1851] The server converts the manual into an appropriate format and sends it to the user's device. It also generates a notification of completion and an access link, which it provides to the user.

[1852] (Application example 2)

[1853] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1854] A support system that can quickly and appropriately respond to technical questions and problems that drivers and users encounter in autonomous vehicles is needed. Conventional support systems do not provide manuals that take into account the user's emotional state, which can result in increased stress and dissatisfaction for users. Furthermore, conventional technologies take time to resolve users' questions, and there are insufficient means to mitigate safety risks that arise during this time.

[1855] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving unclear data from the user, means for analyzing the received unclear data using a generative AI model, means for automatically generating a manual based on the analysis results and the user's emotional data, and means for transmitting the generated manual to the user's terminal. This makes it possible to quickly provide an appropriate manual using a tone and expression that matches the user's emotional state, enabling immediate responses to technical questions and problems in autonomous vehicles and improving user satisfaction and safety.

[1856] "User" refers to a person who uses the system, specifically a driver or user.

[1857] "Uncertainty data" refers to data about questions or problems users encounter while using the system.

[1858] A "generative AI model" is an artificial intelligence model that analyzes received unknown data and generates corresponding information and solutions.

[1859] "Analysis" refers to the process of examining data in detail and understanding its contents.

[1860] "Emotional data" is data that represents a user's emotional state extracted from their text or speech.

[1861] A "manual" is a document that contains procedures and explanations for resolving user questions and problems.

[1862] "Auto-generation" refers to the process by which a program automatically creates output information from specific input data.

[1863] "Terminal" refers to a device used to use the system, and specifically includes smartphones and in-car tablets.

[1864] This invention is a system that provides appropriate support in real time for questions or problems that users encounter in autonomous vehicles. This system uses smartphones or in-car tablets as terminals, and sends input data from users to a server, which then analyzes it.

[1865] The server uses the following methods:

[1866] A means for receiving data on questions from users: Users use their devices to input text or voice about questions or problems. This input data is sent to the server.

[1867] A method for analyzing received unknown data using a generative AI model: A generative AI model is used to analyze the received data. The generative AI model uses natural language processing techniques to analyze the unknown data and identify its points.

[1868] A means of automatically generating manuals based on analysis results and user emotional data: An emotion engine is used to extract user emotional data, which is then combined with the analysis results to automatically generate manuals that take into account appropriate tone and expression.

[1869] Means for sending the generated manual to the user's terminal: The generated manual is sent to the user's terminal in HTML or PDF format.

[1870] As a concrete example, consider the case where the vehicle camera displays an error message during autonomous driving. When the user types "The vehicle camera is displaying an error message, what should I do?" into the terminal, the following process is executed.

[1871] The server receives the user's input and uses an emotion engine to recognize the user's emotion as "impatience." The generative AI model then analyzes the input and automatically generates a manual with specific steps to deal with the camera's error message. The manual includes a gentle tone and expressions to help users respond calmly to their emotions. The generated manual is sent to the user's device in HTML format and displayed in a window.

[1872] An implementation example shows the use of the following prompt:

[1873] My car's camera is showing an error message, what should I do?

[1874] Emotion: "Impatience"

[1875] This embodiment allows users to respond appropriately and quickly to any problems that may occur during automated driving, reducing stress and improving overall safety and satisfaction.

[1876] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1877] Step 1:

[1878] The user uses a device (smartphone or in-car tablet) to input text or voice information about a problem or issue. For example, the user might input, "The vehicle camera is displaying an error message. What should I do?" This input data is sent to the server in an appropriate format, such as JSON.

[1879] Step 2:

[1880] The server receives input data from the user and passes it to the emotion engine for analysis. The emotion engine extracts the user's emotional state from the text data and recognizes it as "impatience," for example. This emotion data is returned to the server as the analysis result.

[1881] Step 3:

[1882] The server inputs the received unclear point data and emotion data into the generative AI model. The generative AI model uses natural language processing technology to consider the content of the unclear point and the user's emotion and generate a manual containing appropriate solutions and information. Specifically, it generates text containing how to deal with camera errors.

[1883] Step 4:

[1884] The server automatically generates a manual in HTML or PDF format based on the analysis results obtained from the generative AI model. The manual is adjusted according to the user's emotions, including a gentler tone and more specific instructions for users who are feeling impatient.

[1885] Step 5:

[1886] The server sends the completed manual to the user's device, where the user can view the HTML manual in a browser or download and read it in PDF format. The user can refer to this manual and follow the specific steps to solve the problem.

[1887] This series of processes enables users to respond quickly and appropriately to any problems that may occur during automated driving, thereby improving safety and satisfaction.

[1888] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1889] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1890] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1891] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1892] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1893] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1894] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1895] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1896] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1897] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1898] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1899] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1900] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1901] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1902] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1903] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1904] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1905] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1906] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1907] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1908] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1909] The following is further disclosed regarding the above embodiment.

[1910] (Claim 1)

[1911] a means for receiving question data from a user;

[1912] a means for analyzing the received unknown data using a generative AI model;

[1913] A means of automatically generating manuals based on the analysis results,

[1914] A means for transmitting the generated manual to a user's terminal;

[1915] A system including:

[1916] (Claim 2)

[1917] 10. The system of claim 1, further comprising means for formatting the analysis results as a manual in natural language based on the received uncertainty data.

[1918] (Claim 3)

[1919] 10. The system of claim 1, further comprising means for providing the generated manual to the user in HTML or PDF format.

[1920] "Example 1"

[1921] (Claim 1)

[1922] means for receiving input data from a user;

[1923] a means for analyzing the received unknown data using a generative AI model;

[1924] A means for automatically generating a manual based on the analysis results;

[1925] A means for transmitting the generated manual to a user's terminal;

[1926] A system including:

[1927] (Claim 2)

[1928] 10. The system of claim 1, further comprising: means for transmitting the received uncertainty data to a server in JSON format.

[1929] (Claim 3)

[1930] 10. The system of claim 1, further comprising means for providing the generated manual to the user in HTML or PDF format.

[1931] "Application Example 1"

[1932] (Claim 1)

[1933] a means for receiving question data from a user;

[1934] a means for analyzing the received unknown data using a generative AI model;

[1935] A means of automatically generating manuals based on the analysis results,

[1936] A means for transmitting the generated manual to a user's terminal;

[1937] A method for industrial robots to detect problems through sensors and cameras while in operation and send the data in JSON format to a server.

[1938] A means for the generative AI model to identify solutions based on the unknown data and generate a manual detailing the solutions in HTML or PDF format;

[1939] means for providing the generated manual to a display device of the industrial robot in real time;

[1940] A system including:

[1941] (Claim 2)

[1942] 2. The system according to claim 1, further comprising means for formatting the analysis results as a manual in natural language based on the received unclear point data.

[1943] (Claim 3)

[1944] 10. The system of claim 1, further comprising means for providing the generated manual to the user in HTML or PDF format.

[1945] "Example 2: Combining Emotion Engines"

[1946] (Claim 1)

[1947] means for receiving uncertainty data and emotion data from a user;

[1948] means for analyzing the received unclear point data and emotion data using an emotion recognition engine;

[1949] A means for generating a prompt sentence based on the received unclear point data and the analyzed emotion data using a generative AI model;

[1950] A means for obtaining analysis results using a generative AI model based on the generated prompt sentence;

[1951] A means for automatically generating a manual based on the analysis results and user sentiment;

[1952] A means for transmitting the generated manual to a user's terminal;

[1953] A system including:

[1954] (Claim 2)

[1955] 10. The system of claim 1, further comprising means for formatting the analysis results as a manual in natural language.

[1956] (Claim 3)

[1957] 10. The system of claim 1, further comprising means for providing the generated manual to the user in HTML or PDF format.

[1958] "Application example 2 when combining emotion engines"

[1959] (Claim 1)

[1960] a means for receiving question data from a user;

[1961] a means for analyzing the received unknown data using a generative AI model;

[1962] A method for automatically generating manuals based on the analysis results and user emotion data,

[1963] A means for transmitting the generated manual to a user's terminal;

[1964] A system including:

[1965] (Claim 2)

[1966] 10. The system of claim 1, further comprising means for formatting the analysis results as a manual in natural language based on the received uncertainty data and emotion data.

[1967] (Claim 3)

[1968] 10. The system of claim 1, further comprising means for providing the generated manual to the user in HTML or PDF format. [Explanation of symbols]

[1969] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving question data from a user; a means for analyzing the received unknown data using a generative AI model; A means of automatically generating manuals based on the analysis results, A means for transmitting the generated manual to a user's terminal; A system including:

2. The system of claim 1 , further comprising means for formatting the analysis results as a manual in natural language based on the received uncertainty data.

3. The system of claim 1 , further comprising means for providing the generated manual to the user in HTML or PDF format.

Images