System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving inquiries from user terminals; means for analyzing the received inquiries; means for retrieving related information from a database based on the analysis results; means for generating solutions by applying a generative AI model using the retrieval results; means for providing the generated solutions to users; and means for providing appropriate employee information based on the solutions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] This invention relates to a system that quickly and accurately identifies who to contact when an unclear point or technical issue arises in a corporate project, and provides an appropriate solution. Conventionally, when faced with a project-specific unclear point or technical issue, it is often unclear who the person in charge should contact, and it takes time and effort to get an appropriate answer. To solve this problem, the objective is to provide a system that uses a generative AI model to efficiently provide solutions, identify the appropriate employee, and respond quickly. [Means for solving the problem]

[0005] The present invention solves the above problems by using the following means.

[0006] means for receiving an inquiry from a user terminal;

[0007] means for analyzing the received inquiry;

[0008] A means for searching a database for related information based on the analysis results;

[0009] a means for applying a generative AI model using the search results to generate a solution; and

[0010] a means for providing the generated solution to a user;

[0011] a means for providing appropriate employee information based on said solution;

[0012] The present invention provides a system including:

[0013] This allows users to quickly and accurately obtain solutions when they encounter project-specific questions or technical challenges, and enables efficient responses by contacting the appropriate employee.

[0014] A "user terminal" is a device through which a user accesses the system and inputs queries.

[0015] An "Inquiry" is a specific technical or project-related question or concern submitted by a User to the System.

[0016] "Means for receiving" refers to a function or module of the system for receiving a query sent from a user terminal.

[0017] The "analysis means" is a system function or module that uses natural language processing technology to interpret the content of the received inquiry and extract the necessary information.

[0018] A "database" is a storage device or system within a system that stores related information such as project information, employee information, and department information, and keeps it in a searchable state.

[0019] A "searching means" is a function or module of the system for searching for relevant information in a database based on the analyzed query content.

[0020] A "generative AI model" is a model of a system that uses artificial intelligence technology to automatically generate optimal solutions based on the content of inquiries and search results.

[0021] A "solution" is a specific response method or information that the generative AI model presents in response to a user's inquiry.

[0022] A "delivery means" is a system function or module for communicating generated solutions and related information to users.

[0023] "Employee information" refers to detailed information about employees who have expertise in specific technologies or projects.

[0024] The "appropriate employee" is the employee identified as having the most useful knowledge and experience regarding the user's inquiry. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] The present invention is a system that provides appropriate solutions to unclear points and technical issues in corporate projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0047] System Overview

[0048] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes it. Based on the analysis results, it searches the database for relevant information and uses a generative AI model to generate the optimal solution. Furthermore, by providing the user with appropriate employee information as needed, the system allows the user to quickly address the problem.

[0049] Program processing and its explanation

[0050] 1. Receiving Inquiries

[0051] Subject: Device

[0052] The terminal receives inquiries from users and sends them to the server. For example, if a user types into the terminal, "I don't understand the requirements for XX in project A," this information is sent to the server.

[0053] 2. Analysis of inquiry content

[0054] Subject: Server

[0055] The server analyzes the received inquiry and uses natural language processing (NLP) technology to extract important keywords from the inquiry and understand the intent of the inquiry. For example, it extracts information about "Project A," "Requirements," and "XX."

[0056] 3. Search for related information

[0057] Subject: Server

[0058] The server searches the database based on the analysis results, and retrieves relevant information from the database, which contains project information, employee information, and department information.

[0059] 4. Applying generative AI models

[0060] Subject: Server

[0061] Based on the acquired information, the server inputs the generative AI model to generate a solution. The generative AI model considers the query and related information to create a specific and feasible solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[0062] 5. Providing solutions and employee information

[0063] Subject: Server

[0064] The server sends the generated solution to the user's terminal and also provides appropriate employee information based on the solution (e.g., "Employee Y is knowledgeable about this problem"), allowing the user to know not only the solution but also who to contact for further information.

[0065] 6. User Display

[0066] Subject: Device

[0067] The terminal displays the solution and employee information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "The specific XX requirement for Project A is.... If you have any further questions, please contact employee Y."

[0068] Specific examples

[0069] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[0070] 1. The device receives the query and sends it to the server.

[0071] 2. The server analyzes the inquiry and extracts important keywords.

[0072] 3. The server searches and retrieves the relevant information from the database.

[0073] 4. The generative AI model generates a solution based on the acquired information.

[0074] 5. The server generates a solution and sends the relevant employee information to the terminal.

[0075] 6. The device displays the solution and employee information to the user, for example, "The information about XX is... Contact employee Y for more information."

[0076] As described above, the system of the present invention responds to user inquiries quickly and accurately, provides appropriate solutions, and, if necessary, connects users to appropriate staff members, thereby improving the efficiency of problem solving.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] Subject: User

[0080] The user inputs a query to the system from a terminal. For example, the user inputs "I would like to know about XX requirements for Project A" and presses the send button.

[0081] Step 2:

[0082] Subject: Terminal

[0083] The terminal receives the user's query and transmits the query data to the server, specifically, the terminal captures the user's input text and transfers it to the server via the network.

[0084] Step 3:

[0085] Subject: Server

[0086] The server receives the query sent from the terminal, stores the received text data, and prepares for the next analysis process.

[0087] Step 4:

[0088] Subject: Server

[0089] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text and extract important keywords, such as "Project A," "Requirements," and "XX."

[0090] Step 5:

[0091] Subject: Server

[0092] The server searches the database for relevant information based on the analyzed keywords, generates SQL queries to query the database, and retrieves the required project, employee, and department information.

[0093] Step 6:

[0094] Subject: Server

[0095] The server applies a generative AI model to generate a solution based on the information retrieved from the database. The query and related information are input to the generative AI model, which then outputs the optimal solution.

[0096] Step 7:

[0097] Subject: Server

[0098] The server transmits the generated solution and associated employee information to the terminal to provide the information to the user, including, for example, the solution text and contact information for the appropriate employee.

[0099] Step 8:

[0100] Subject: Terminal

[0101] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[0102] Step 9:

[0103] Subject: User

[0104] The user checks the solution displayed on the terminal and, if necessary, makes further inquiries using the provided employee information, taking an action such as "send an email to employee Y."

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] When technical questions or issues arise in corporate projects, there is a lack of systems that can provide solutions quickly and accurately. It is particularly difficult to quickly find the right information for complex issues involving multiple projects or departments. As a result, problem resolution can take a long time, leading to reduced work efficiency.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes a means for analyzing the received inquiry, a means for searching a database for related information based on the analysis results, and a means for generating a solution by applying a generative AI model using the search results, thereby enabling the server to provide a quick and accurate solution to the user's technical questions and problems, and also provide appropriate contact information as needed.

[0110] A "user terminal" is an electronic device that allows a user to input and send an inquiry.

[0111] An "inquiry" is a technical question or problem that a user needs to solve.

[0112] A "server" is a central processing unit for analyzing queries received from user terminals and generating and providing solutions.

[0113] "Analysis" is the process of using natural language processing technology to understand the content of the received inquiry and extract important keywords and phrases.

[0114] A "database" is a collection of information that holds related information such as project information, person information, department information, etc., and stores it in a searchable manner.

[0115] A "generative AI model" is a set of algorithms or programs that generate solutions based on the query and related information.

[0116] A "solution" is a specific, actionable answer or instruction to a user's technical question or problem.

[0117] "Contact information" refers to contact and related information for employees who are knowledgeable about specific technical questions or issues.

[0118] "Display" is the process of outputting information sent from the server to the terminal in a format that the user can visually confirm.

[0119] This invention is a system that provides appropriate solutions to technical questions and issues in in-house projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0120] When a user has a technical question or concern, they use their device to send the inquiry to the system, for example, "I don't understand the requirements for XX in Project A." The device converts this information into the appropriate format and sends it to the server using an HTTPS request.

[0121] The server analyzes the received query. First, it parses the received data to extract text data and then uses natural language processing (NLP) technology. Specifically, it uses an NLP engine such as SpaCy or NLTK to linguistically analyze the query content and extract important keywords and phrases. For example, keywords such as "Project A," "Requirements," and "XX" are extracted.

[0122] After the analysis is complete, the server searches the database based on the analysis results. The database stores project information, person in charge information, department information, etc. The server generates an SQL query to search for entries that match the analysis results and retrieve the relevant information.

[0123] Next, the server inputs the acquired information into a generative AI model (e.g., GPT-4) to generate a solution. The input prompt for the generative AI model is constructed based on the analysis results and information acquired from the database. For example, a prompt in the form "Please tell me about the XX requirement for Project A" is created. The generative AI model then generates a specific solution such as "The XX requirement for Project A is...".

[0124] The generated solution is sent to the user's device by the server. The solution is provided along with the analysis results and appropriate person-in-charge information obtained from the database. For example, information such as "Person in charge Y is knowledgeable about the requirements of XX" may be added.

[0125] The terminal displays the solution and contact information sent from the server to the user. It parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it displays, "The XX requirement for Project A is.... For detailed questions, please contact Contact Y."

[0126] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process it as follows:

[0127] 1. The device receives the query and sends it to the server.

[0128] 2. The server analyzes the query and extracts important keywords.

[0129] 3. The server searches and retrieves the relevant information from the database.

[0130] 4. The generative AI model generates a solution based on the acquired information.

[0131] 5. The server sends the generated solution and related person information to the terminal.

[0132] 6. The device displays the solution and contact information to the user. For example, it displays "Information about XX is... Contact contact Y for more information."

[0133] As described above, this system responds quickly and accurately to users' technical questions and issues, providing appropriate solutions. It also aims to improve the efficiency of problem-solving by connecting users to the appropriate person in charge as needed.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1: Receiving an inquiry

[0136] Input: User-supplied query text

[0137] Output: Query data sent to the server

[0138] Specific operation: A user types "I don't understand the requirements for XX in Project A" into the terminal. The terminal receives this input, structures the data including the query text, and sends it to the server as an HTTPS request. At this time, metadata such as the user ID and timestamp are also sent.

[0139] Step 2: Analyzing the inquiry

[0140] Input: Inquiry data sent from the terminal

[0141] Output: Parsed keywords and query intent

[0142] How it works: The server parses the received query data and extracts the text. It then uses an NLP engine (such as SpaCy or NLTK) to analyze the text and extract important keywords and phrases. The extracted keywords are "Project A," "Requirements," and "XX."

[0143] Step 3: Find related information

[0144] Input: Parsed keywords and query intent

[0145] Output: Relevant information retrieved from the database

[0146] Specific operation: Based on the analysis results, the server generates an SQL query to the database to search for relevant information. The database stores project information, person in charge information, and department information. For example, the database is searched using the condition "XX requirements for project A" to retrieve the relevant information.

[0147] Step 4: Applying the generative AI model

[0148] Input: relevant information retrieved from the database and the query intent

[0149] Output: The solution generated by the generative AI model

[0150] Specific operation: Based on the acquired information, the server constructs an input prompt for the generative AI model (e.g., GPT-4). For example, it creates a prompt in the form of "Please tell me about the XX requirement of Project A." The generative AI model generates a solution to this prompt and outputs a detailed answer such as "The XX requirement of Project A is..."

[0151] Step 5: Provide solutions and contact information

[0152] Input: Solutions generated by the generative AI model and appropriate personnel information

[0153] Output: Solution and contact information sent to the terminal

[0154] Specific operation: The server receives the generated solution and adds appropriate person information based on it. For example, it adds information such as "Person Y is knowledgeable about the requirements of XX." This information is sent to the user's terminal as an HTTPS response.

[0155] Step 6: Display to the user

[0156] Input: Solution and contact information sent from the server

[0157] Output: Solution and contact information displayed to the user

[0158] Specific operation: The device parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it might display, "Specifically, the XX requirement for Project A is.... If you have any further questions, please contact Person Y." The user can then confirm the displayed information and begin solving the problem.

[0159] Through these steps, users can quickly and accurately obtain specific solutions to technical questions and issues, as well as information on the appropriate person in charge.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] In modern factories, when complex machine troubles or technical questions arise, they need to be resolved quickly. However, it is difficult for on-site workers to immediately access the appropriate information and solutions. Furthermore, the time required to search for relevant information leads to reduced productivity. Another issue is the difficulty of quickly obtaining information on the appropriate engineers.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis result, means for applying a generative AI model using the search result to generate a solution, means for providing the generated solution to the user, means for providing appropriate staff information based on the solution, and means including smart glasses for converting the received inquiry from voice input into text data. This enables on-site workers to easily make inquiries by voice input and quickly obtain appropriate solutions and staff information.

[0165] A "user terminal" is a device used by a user, including a smartphone, tablet, PC, etc.

[0166] An "inquiry" is data that a user sends to the system regarding technical questions or concerns.

[0167] "Analysis" is the process of understanding the content of the received inquiry and extracting important keywords and their intent.

[0168] A "database" is a system for systematically storing information and making it searchable.

[0169] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions based on input information.

[0170] A "solution" is a specific response or answer provided to a user's inquiry.

[0171] "Contact Information" is contact information for employees or technicians with detailed knowledge of specific technical issues.

[0172] "Smart glasses" are wearable devices that use AR (augmented reality) technology to allow users to obtain information visually.

[0173] "Voice input" is an input method in which a device recognizes words spoken by a user and processes them as data.

[0174] "Text data" refers to data that has been converted from voice input into text information.

[0175] This invention is a system that allows users with technical questions or concerns to obtain solutions in real time using smart glasses. The system is basically composed of a user terminal (including smart glasses), a server, and a network infrastructure that links them.

[0176] System Overview

[0177] 1. Receiving Inquiries

[0178] The terminal, i.e., smart glasses, receives voice input from the user and converts the voice into text data. For example, if the user says, "The robot arm on line 2 is not working," the voice data is converted into text data.

[0179] 2. Analysis of inquiry content

[0180] The server analyzes the received text data. It uses natural language processing (NLP) techniques, such as spaCy or NLTK, to extract important keywords from the query and understand its intent. In this example, keywords such as "line 2," "robot arm," and "not working" are extracted.

[0181] 3. Search for related information

[0182] The server searches a database based on the analysis results, which contains machine manuals, error logs, and personnel information, and retrieves relevant information from these.

[0183] 4. Applying generative AI models

[0184] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific, actionable solution. For example, it generates a detailed explanation such as, "If the robot arm on Line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge."

[0185] 5. Providing solutions and contact information

[0186] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the solution, such as "Person Y is knowledgeable about this problem."

[0187] 6. User Display

[0188] The terminal, i.e., the smart glasses, displays the solution and contact information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact contact Y."

[0189] Specific examples

[0190] For example, if a user queries "The robot arm on line 2 is not working," the system will process it as follows:

[0191] 1. The device receives voice input, converts it into text data, and sends it to the server.

[0192] 2. The server analyzes the received text data and extracts important keywords.

[0193] 3. The server searches and retrieves the relevant information from the database.

[0194] 4. The generative AI model generates a solution based on the acquired information.

[0195] 5. The server generates a solution and sends it to the terminal along with relevant contact information.

[0196] 6. The terminal will display the solution and contact information to the user, for example, "If the robot arm on line 2 does not work, please check the power connection. Contact contact Y for more information."

[0197] Example prompt sentence:

[0198] User enquiry:

[0199] "What should I do if the robot arm on Line 2 doesn't work?"

[0200] Related information:

[0201] Line 2 Manual

[0202] Error Log

[0203] Contact Information

[0204] Output of the generative AI model:

[0205] "If the robot arm on line 2 doesn't work, first check the power connection. If that doesn't fix it, contact person Y."

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Step 1:

[0208] The smart glasses receive voice input and convert the voice data into text data. The voice input is converted into text format using the Google Speech-to-Text API. For example, if a user says, "The robot arm on line 2 is not working," the voice data is converted into text data that reads, "The robot arm on line 2 is not working." The input is voice data, and the output is text data.

[0209] Step 2:

[0210] The text data is sent to the server. The smart glasses send the text data to the server through the factory network. The text data is input to the server, and the received text data is used for the next analysis process. The input is text data, and the output is transmission to the server.

[0211] Step 3:

[0212] The server analyzes the received text data. Using a natural language processing (NLP) library (such as spaCy or NLTK), it extracts important keywords from the query and understands its intent. Specifically, keywords such as "line 2," "robot arm," and "not working" are extracted. The input is the text data, and the output is the important keywords.

[0213] Step 4:

[0214] The server searches the database based on the analysis results. The database stores machine manuals, error logs, and staff information, and searches these to obtain relevant information. For example, the error log and staff information for the "robot arm on line 2" are extracted from the database. The input is important keywords, and the output is related information.

[0215] Step 5:

[0216] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific and actionable solution. For example, it might generate a solution such as, "If the robot arm on line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge." The input is the related information, and the output is the solution.

[0217] Step 6:

[0218] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the generated solution. For example, it provides information such as "Person Y is knowledgeable about this problem." The input is the solution and person information, and the output is transmission completion.

[0219] Step 7:

[0220] The terminal, i.e., smart glasses, displays the solution and person in charge information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it displays, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact person Y." The input is the solution and person in charge information, and the output is the display to the user.

[0221] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0222] This invention relates to a system that provides appropriate solutions to unclear points and technical issues in corporate projects, and also combines it with an emotion engine that recognizes the user's emotions. The system is basically composed of a user terminal, a server, an emotion engine, and a network infrastructure that links them together.

[0223] System Overview

[0224] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and uses a generative AI model to generate the optimal solution. It also uses an emotion engine to analyze the user's emotions, adjusts the response based on that emotional state, and provides appropriate employee information, allowing the user to quickly address the problem.

[0225] Program processing and its explanation

[0226] 1. Receiving Inquiries

[0227] Subject: Device

[0228] The terminal receives inquiries from the user and sends them to the server. For example, if the user enters "I would like to know about XX requirements for project A," this information is sent to the server.

[0229] 2. Analysis of inquiry content

[0230] Subject: Server

[0231] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "Project A," "Requirements," and "XX."

[0232] 3. Emotion analysis using an emotion engine

[0233] Subject: Server

[0234] The server uses an emotion engine to analyze the user's emotion from the content of the inquiry. The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the user's emotional state at that time, such as "the user is feeling anxious."

[0235] 4. Searching for related information

[0236] Subject: Server

[0237] The server searches the database based on the analysis results (keywords and sentiment information), generates an SQL query to query the database, and retrieves the necessary project, employee, and department information.

[0238] 5. Applying generative AI models

[0239] Subject: Server

[0240] Based on the information acquired by the server, a generative AI model is applied to generate a solution. The query content, related information, and emotional information are input to the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[0241] 6. Providing solutions and employee information

[0242] Subject: Server

[0243] The server sends the generated solution and relevant employee information to the terminal to provide the user with the solution, including the solution text and the contact information of the appropriate employee. The server also adjusts the priority and response method of the inquiry based on the emotion information.

[0244] 7. User Display

[0245] Subject: Device

[0246] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[0247] Specific examples

[0248] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[0249] 1. The device receives the query and sends it to the server.

[0250] 2. The server analyzes the inquiry and extracts important keywords.

[0251] 3. The server uses an emotion engine to analyze the user's emotions and identify an emotion such as "anxiety."

[0252] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[0253] 5. The generative AI model generates a solution based on the acquired information and adjusts it based on emotional information.

[0254] 6. The server sends the generated solution and related employee information to the terminal.

[0255] 7. The device displays the solution and employee information to the user, for example, "The information about XX is... If you have further questions, please contact employee Y. If you have any concerns, we are happy to provide additional support."

[0256] As described above, the system of the present invention not only responds quickly and accurately to user inquiries, but also increases user satisfaction by providing tailored solutions that take into account the user's feelings and appropriate employee information.

[0257] The processing flow will be explained below.

[0258] Step 1:

[0259] Subject: User

[0260] The user inputs a query into the system from a terminal. For example, "I would like to know more about the XX requirement for Project A," and presses the send button.

[0261] Step 2:

[0262] Subject: Terminal

[0263] The terminal receives the user's inquiry and transmits the inquiry data to the server. Specifically, the terminal captures the input text data and transfers it to the server via the network.

[0264] Step 3:

[0265] Subject: Server

[0266] The server receives the query sent from the terminal, stores the query content, and prepares to pass it to the next processing step.

[0267] Step 4:

[0268] Subject: Server

[0269] The server analyzes the query received using natural language processing (NLP) technology. Specifically, it tokenizes the text and extracts important keywords (e.g., "Project A," "Requirements," "XX"), and also performs semantic analysis to understand the intent of the query.

[0270] Step 5:

[0271] Subject: Server

[0272] The server uses the results of the NLP analysis to analyze the user's emotions using an emotion engine. Specifically, it detects emotions such as "anxiety," "confusion," and "urgency" from the inquiry content and stores the results in a database.

[0273] Step 6:

[0274] Subject: Server

[0275] The server searches the database based on the analysis results and sentiment information. Specifically, it generates SQL queries to retrieve project information, employee information, and department information from the database. For example, it searches for materials related to "Project A" or "XX."

[0276] Step 7:

[0277] Subject: Server

[0278] The server applies a generative AI model based on the information it obtains to generate a solution. The generative AI model takes the inquiry, related information, and emotional information as input and outputs the optimal solution. For example, it generates a specific solution such as "The XX requirement for Project A is..."

[0279] Step 8:

[0280] Subject: Server

[0281] The server selects the appropriate employee information based on the generated solution and sentiment information. Based on the analysis results, it extracts the contact information of the most suitable employee and sets the priority of the inquiry.

[0282] Step 9:

[0283] Subject: Server

[0284] The server finally sends the generated solution and the selected employee information to the user terminal. For example, the solution text and "Employee Y's contact information" are packaged and sent.

[0285] Step 10:

[0286] Subject: Terminal

[0287] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can see the solution and providing information to contact an employee if necessary.

[0288] Step 11:

[0289] Subject: User

[0290] The user checks the solution displayed on the device and, if necessary, contacts the designated employee. For example, they send an email to employee Y using the provided contact information.

[0291] The above is a specific flow of processing in the system of the present invention. By performing detailed operations at each step, the user can receive a quick and accurate solution. Furthermore, by taking emotional information into consideration, the response to the user can be more appropriate and satisfying.

[0292] Example 2

[0293] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0294] Conventional technical support systems have not only difficulty in providing appropriate solutions to users' technical questions and uncertainties quickly and accurately, but also have the problem of not being able to respond in a way that takes into account the user's emotional state. In particular, if a user feels anxious or confused, ignoring their emotions can reduce the efficiency of problem-solving and lead to low user satisfaction.

[0295] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0296] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative AI model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate employee information based on the solution, means including an emotion engine for analyzing the user's emotions, and means for adjusting the solution and employee information based on the emotion information. This makes it possible to not only quickly and accurately respond to the user's technical questions and uncertainties, but also to provide adjusted solutions and appropriate employee information that take the user's emotional state into consideration.

[0297] A "user terminal" is an information input device that allows a user to input technical questions or uncertainties.

[0298] An "inquiry" is information about technical questions or uncertainties sent from a user terminal.

[0299] The "receiving means" is a function that allows the server to receive an inquiry sent from a user terminal.

[0300] "Means of analysis" refers to the function of analyzing the content of the received inquiry using natural language processing technology and extracting important keywords and intent.

[0301] "Natural language processing technology" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[0302] A "database" is a searchable collection of related information such as project information, employee information, and department information.

[0303] "Search means" is a function for retrieving related information from a database based on the analysis results.

[0304] A "generative AI model" is a machine learning model that generates optimal solutions based on input information.

[0305] The "means for generating" is the function for applying a generative AI model to generate a solution to a query.

[0306] A "solution" is a specific answer or countermeasure to a user's technical questions or uncertainties.

[0307] "Means of providing" refers to the function for delivering the generated solution to the user.

[0308] "Employee Information" means contact information and expertise of appropriate employees who can respond to inquiries.

[0309] An "emotion engine" is a technology for analyzing a user's emotional state from the content of their inquiry.

[0310] "Emotion information" is the result of the user's emotional state analyzed by the emotion engine.

[0311] "Adjustment means" is a function for optimizing solutions and employee information based on emotional information.

[0312] This is a system that provides quick and accurate solutions to technical issues and uncertainties users have in technical projects carried out within a company. The system consists of a user terminal, a server, an emotion engine, and a network infrastructure that links these elements.

[0313] A user terminal is a device that allows a user to input an inquiry, such as a computer or smartphone. A user inputs technical questions or concerns and submits them as an inquiry.

[0314] The server is a computer system that performs the following processes:

[0315] 1. Receiving means: Receives a query from a user terminal as an HTTP request.

[0316] 2. Analysis: The received query content is analyzed using natural language processing techniques, such as using a Python NLP library (NLTK or spaCy) to tokenize the text and extract important keywords.

[0317] 3. Search method: Based on the analysis results, search the database for relevant information. Generate SQL queries to retrieve project information, employee information, department information, etc.

[0318] 4. Generation method: The acquired information is input into a generative AI model (e.g., GPT-3) to generate an optimal solution. The prompt text is something like, "Please explain in detail the XX requirement for Project A."

[0319] 5. Means of provision: The generated solution and related employee information are sent to the user terminal as an HTTP response.

[0320] 6. Emotion Engine: Analyzes emotions from the user's inquiry. For example, it uses a Python emotion analysis library (TextBlob or VADER) to identify the user's emotion of "anxiety."

[0321] 7. Coordination: Optimize and provide solutions and employee information based on emotional information.

[0322] Here are some examples:

[0323] A user sends a query from their device, such as "I want to know about the XX requirement for Project A." The device sends this query as an HTTP request to the server. The server analyzes the received text and extracts keywords such as "Project A," "XX," and "requirements." It then uses an emotion engine to determine that the user's emotion is "anxiety."

[0324] The server searches the database based on keywords and sentiment information to retrieve relevant project information and the contact information of the responsible employee. Based on this information, the generative AI model is prompted with a prompt such as "Please explain in detail the XX requirement for Project A," and a detailed solution is generated.

[0325] Finally, the server sends the generated solution and related employee information to the user terminal as an HTTP response, and the user terminal displays this on its user interface. For example, information such as "The information about XX is.... If you have any further questions, please contact employee Y. If you have any concerns, we will provide additional support" is displayed on the screen.

[0326] As described above, the system of the present invention not only responds quickly and accurately to users' technical questions and uncertainties, but also increases user satisfaction by providing tailored solutions that take into account the user's emotional state and appropriate employee information.

[0327] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0328] Step 1:

[0329] Input: The user enters technical questions or concerns into the user's terminal and submits them as an inquiry.

[0330] Specific operation: The user enters "I would like to know about the XX requirements for Project A" into the device's input screen and presses the send button.

[0331] Output: The device sends the query to the server as an HTTP request.

[0332] Step 2:

[0333] Input: The server receives the HTTP request sent from the device.

[0334] Specific operation: The server analyzes the HTTP request and obtains the inquiry content.

[0335] Output: The query is sent to the server as text.

[0336] Step 3:

[0337] Input: The server analyzes the query received using natural language processing (NLP) techniques.

[0338] What it does: The server uses a Python NLP library (e.g., NLTK, spaCy) to tokenize the text and extract important keywords (e.g., "Project A," "Requirements," "XX").

[0339] Output: The extracted keywords are passed to the next processing step.

[0340] Step 4:

[0341] Input: The server passes the extracted keywords to the emotion engine.

[0342] Specific operation: The server uses an emotion analysis library (e.g., TextBlob, VADER) to analyze the user's emotional state from the query content and identify the emotion, for example, "anxiety."

[0343] Output: The sentiment analysis results are returned to the server and stored in a database.

[0344] Step 5:

[0345] Input: The server searches the database based on keywords and sentiment information.

[0346] Specific operation: The server generates SQL queries and queries the database for project information, employee information, department information, etc.

[0347] Output: The necessary information (project description, contact information of the responsible employee, etc.) is passed to the server.

[0348] Step 6:

[0349] Input: The information acquired by the server is input into the generative AI model.

[0350] Specific operation: The server inputs a prompt sentence to the generative AI model (e.g., GPT-3) such as "Please explain in detail the XX requirement for project A," and the generative AI model generates the optimal solution.

[0351] Output: The generated solution is returned to the server.

[0352] Step 7:

[0353] Input: The server compiles the generated solution and related employee information.

[0354] What happens: The server creates a single HTTP response with the solution text and appropriate employee information (e.g., contact details), and adjusts the priority and response based on the sentiment information.

[0355] Output: An HTTP response is formed and sent to the user's device.

[0356] Step 8:

[0357] Input: The device receives the HTTP response sent by the server.

[0358] Specific behavior: The device displays the solution and employee information in the user interface.

[0359] Output: The user can see the solution and employee information displayed on the screen.

[0360] Specifically, the device displays information such as "Information about XX is... If you have any detailed questions, please contact employee Y. If you have any concerns, we will provide additional support" on the screen, and the user confirms it.

[0361] (Application example 2)

[0362] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0363] There is a need for a means to quickly and accurately resolve the technical questions and uncertainties that drivers of autonomous vehicles face while driving, as well as the emotional stress that accompanies them. In particular, it is important to improve driver satisfaction and safety by providing optimal solutions according to the driver's emotional state.

[0364] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving an inquiry from a user device, means for analyzing the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative artificial intelligence model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate personal information based on the solution, means for receiving an inquiry via voice input or text input, means for applying a generative artificial intelligence model using the analysis results and emotion analysis to generate a solution, means for providing the generated solution and related information on a display device and by voice, and means for analyzing the driver's tone of voice and facial expression to identify the driver's emotional state. This makes it possible to quickly provide optimal solutions to technical questions and problems that the driver has while driving, based on their emotional state.

[0365] "User equipment" refers to devices such as smartphones or on-board displays inside or outside an autonomous vehicle, and is used for inquiries from and information provision to the driver.

[0366] A "generative artificial intelligence model" is a generative model that uses artificial intelligence to analyze the content of an inquiry and related information and generate an appropriate solution.

[0367] "Means for analyzing the content of an inquiry" refers to the process of tokenizing the content of an inquiry received from a user using natural language processing technology, extracting keywords, and understanding the intent.

[0368] A "database" is a collection of information that the system uses to search for relevant information, and includes road information, traffic information, location information, and the like.

[0369] "Emotion analysis" refers to technology for identifying a driver's emotional state from their tone of voice and facial expressions, and is the process of analyzing the anxiety and stress the driver is experiencing.

[0370] A "display device" is a device for providing generated solutions and related information to a user, including an in-car display or a smartphone screen.

[0371] "Voice input" refers to a means by which a user can make queries to a system through voice, and includes technology that converts received voice data into text.

[0372] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning, and is used here for analyzing inquiry content and preprocessing for generative AI models.

[0373] "Means of generating a solution" refers to the process of using the generated AI model to generate the optimal answer based on the information obtained from the database and the query content.

[0374] "Personal information" refers to information such as contact details and job titles of individuals appropriate to address specific challenges or questions, including information provided by the system to the driver.

[0375] System Overview

[0376] This invention is a system whose basic components are a user device, a server, an emotion engine, and a network infrastructure that links these together. When a driver has a technical question or concern, they use the user device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and generates an optimal solution using a generative artificial intelligence model. Furthermore, it uses the emotion engine to analyze the driver's emotions, adjusts countermeasures based on that emotional state, and provides appropriate personal information, allowing the driver to quickly address the problem.

[0377] Program processing and its explanation

[0378] 1. Receiving Inquiries

[0379] The user device (smartphone or in-car display) receives queries from the driver and sends them to the server using a voice recognition library (Google Speech-to-Text API) or a text input field. For example, the driver may ask by voice, "How should I get through this intersection?" and this information is sent to the server.

[0380] 2. Analysis of inquiry content

[0381] The server analyzes the query it receives, using natural language processing techniques (spaCy, NLTK) to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "intersection" and "road."

[0382] 3. Emotion analysis

[0383] The server uses an emotion engine to analyze the driver's emotions from their tone of voice and facial expressions. This involves using a face recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the driver's emotional state as "anxious."

[0384] 4. Searching for related information

[0385] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information), generates an SQL query to query the database, and obtains the necessary road, traffic, and location information.

[0386] 5. Solution generation and delivery

[0387] Based on the information acquired by the server, a generative artificial intelligence model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The query content, related information, and emotional information are input into the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "It is safe to turn right at this intersection." The generated solution and related information are displayed on the in-car display or smartphone, and are also provided to the driver using voice synthesis (Google Text-to-Speech API).

[0388] Specific examples

[0389] When a driver makes a voice inquiry such as "I want to know how to get through this intersection on this road," the specific processing steps are as follows:

[0390] 1. The user device receives the query and sends it to the server.

[0391] 2. The server analyzes the inquiry and extracts important keywords.

[0392] 3. The server uses an emotion engine to analyze the driver's emotions from their voice and facial expressions, and identifies an emotion such as "anxiety."

[0393] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[0394] 5. The generative AI model generates a solution based on the acquired information and adjusts the solution with reference to emotional information.

[0395] 6. The server sends the generated solution and related information to the user device.

[0396] 7. The user device displays the solution and related information to the driver and provides voice guidance: "It is safe to turn right at this intersection. Also, if you turn left at the next traffic light, you will reach your destination."

[0397] Example prompts to input to the generative AI model

[0398] Prompt: "Generate an appropriate response to a driver who is worried about how to get through this intersection. The driver's current emotion is anxiety."

[0399] Example response: "It's safe to turn right at this intersection, and turning left at the next light will get you to your destination."

[0400] This makes it possible to quickly provide optimal solutions to technical questions or problems that drivers may have while driving, depending on their emotional state.

[0401] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0402] Step 1:

[0403] The user inputs a query by voice or text (voice input or text input). The user's device (smartphone or in-car display) receives this query and sends it to the server. The Google Speech-to-Text API is used for input. For example, if the user voices a query such as "I want to know how to get through this intersection," this voice is converted into text and sent to the server.

[0404] Step 2:

[0405] The server analyzes the query it receives. It uses natural language processing techniques (spaCy, NLTK) to tokenize the text and extract important keywords. The input is the query text, and the output is the extracted keywords. For example, it identifies keywords such as "intersection" and "road." Specifically, the analysis engine analyzes the query content and identifies the main keywords.

[0406] Step 3:

[0407] The server uses an emotion engine to analyze the driver's emotions. This uses a facial recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The input is the driver's tone of voice and facial expression data, and the output is the driver's emotional state. Specifically, the server analyzes the driver's tone of voice and facial expression to identify emotions such as "anxiety."

[0408] Step 4:

[0409] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information). It generates an SQL query and queries the database. The input is keywords and emotion information, and the output is related road information, traffic information, and location information. Specifically, the server retrieves the required data from the database.

[0410] Step 5:

[0411] Based on the information acquired by the server, a generative AI model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The input is the inquiry content, related information, and emotional information, and the output is the optimal solution. Specifically, the generative AI model generates a solution such as "It is safe to turn right at this intersection."

[0412] Step 6:

[0413] The server sends the generated solution and related information to the user device, which displays it and provides it audibly. The output is the solution text and speech synthesis data. Specifically, the generated solution is displayed on the in-vehicle display, and a voice guides the user, saying, "It is safe to turn right at this intersection."

[0414] In this way, the system processes the data input at each step, performing appropriate data processing and calculations to provide the user with the optimal solution.

[0415] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0416] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0417] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0418] [Second embodiment]

[0419] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0420] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0421] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0422] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0423] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0424] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0425] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0426] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0427] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0428] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0429] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0430] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0431] The present invention is a system that provides appropriate solutions to unclear points and technical issues in corporate projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0432] System Overview

[0433] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes it. Based on the analysis results, it searches the database for relevant information and uses a generative AI model to generate the optimal solution. Furthermore, by providing the user with appropriate employee information as needed, the system allows the user to quickly address the problem.

[0434] Program processing and its explanation

[0435] 1. Receiving Inquiries

[0436] Subject: Device

[0437] The terminal receives inquiries from users and sends them to the server. For example, if a user types into the terminal, "I don't understand the requirements for XX in project A," this information is sent to the server.

[0438] 2. Analysis of inquiry content

[0439] Subject: Server

[0440] The server analyzes the received inquiry and uses natural language processing (NLP) technology to extract important keywords from the inquiry and understand the intent of the inquiry. For example, it extracts information about "Project A," "Requirements," and "XX."

[0441] 3. Search for related information

[0442] Subject: Server

[0443] The server searches the database based on the analysis results, and retrieves relevant information from the database, which contains project information, employee information, and department information.

[0444] 4. Applying generative AI models

[0445] Subject: Server

[0446] Based on the acquired information, the server inputs the generative AI model to generate a solution. The generative AI model considers the query and related information to create a specific and feasible solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[0447] 5. Providing solutions and employee information

[0448] Subject: Server

[0449] The server sends the generated solution to the user's terminal and also provides appropriate employee information based on the solution (e.g., "Employee Y is knowledgeable about this problem"), allowing the user to know not only the solution but also who to contact for further information.

[0450] 6. User Display

[0451] Subject: Device

[0452] The terminal displays the solution and employee information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "The specific XX requirement for Project A is.... If you have any further questions, please contact employee Y."

[0453] Specific examples

[0454] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[0455] 1. The device receives the query and sends it to the server.

[0456] 2. The server analyzes the inquiry and extracts important keywords.

[0457] 3. The server searches and retrieves the relevant information from the database.

[0458] 4. The generative AI model generates a solution based on the acquired information.

[0459] 5. The server generates a solution and sends the relevant employee information to the terminal.

[0460] 6. The device displays the solution and employee information to the user, for example, "The information about XX is... Contact employee Y for more information."

[0461] As described above, the system of the present invention responds to user inquiries quickly and accurately, provides appropriate solutions, and, if necessary, connects users to appropriate staff members, thereby improving the efficiency of problem solving.

[0462] The processing flow will be explained below.

[0463] Step 1:

[0464] Subject: User

[0465] The user inputs a query to the system from a terminal. For example, the user inputs "I would like to know about XX requirements for Project A" and presses the send button.

[0466] Step 2:

[0467] Subject: Terminal

[0468] The terminal receives the user's query and transmits the query data to the server, specifically, the terminal captures the user's input text and transfers it to the server via the network.

[0469] Step 3:

[0470] Subject: Server

[0471] The server receives the query sent from the terminal, stores the received text data, and prepares for the next analysis process.

[0472] Step 4:

[0473] Subject: Server

[0474] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text and extract important keywords, such as "Project A," "Requirements," and "XX."

[0475] Step 5:

[0476] Subject: Server

[0477] The server searches the database for relevant information based on the analyzed keywords, generates SQL queries to query the database, and retrieves the required project, employee, and department information.

[0478] Step 6:

[0479] Subject: Server

[0480] The server applies a generative AI model to generate a solution based on the information retrieved from the database. The query and related information are input to the generative AI model, which then outputs the optimal solution.

[0481] Step 7:

[0482] Subject: Server

[0483] The server transmits the generated solution and associated employee information to the terminal to provide the information to the user, including, for example, the solution text and contact information for the appropriate employee.

[0484] Step 8:

[0485] Subject: Terminal

[0486] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[0487] Step 9:

[0488] Subject: User

[0489] The user checks the solution displayed on the terminal and, if necessary, makes further inquiries using the provided employee information, taking an action such as "send an email to employee Y."

[0490] Example 1

[0491] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0492] When technical questions or issues arise in corporate projects, there is a lack of systems that can provide solutions quickly and accurately. It is particularly difficult to quickly find the right information for complex issues involving multiple projects or departments. As a result, problem resolution can take a long time, leading to reduced work efficiency.

[0493] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0494] In this invention, the server includes a means for analyzing the received inquiry, a means for searching a database for related information based on the analysis results, and a means for generating a solution by applying a generative AI model using the search results, thereby enabling the server to provide a quick and accurate solution to the user's technical questions and problems, and also provide appropriate contact information as needed.

[0495] A "user terminal" is an electronic device that allows a user to input and send an inquiry.

[0496] An "inquiry" is a technical question or problem that a user needs to solve.

[0497] A "server" is a central processing unit for analyzing queries received from user terminals and generating and providing solutions.

[0498] "Analysis" is the process of using natural language processing technology to understand the content of the received inquiry and extract important keywords and phrases.

[0499] A "database" is a collection of information that holds related information such as project information, person information, department information, etc., and stores it in a searchable manner.

[0500] A "generative AI model" is a set of algorithms or programs that generate solutions based on the query and related information.

[0501] A "solution" is a specific, actionable answer or instruction to a user's technical question or problem.

[0502] "Contact information" refers to contact and related information for employees who are knowledgeable about specific technical questions or issues.

[0503] "Display" is the process of outputting information sent from the server to the terminal in a format that the user can visually confirm.

[0504] This invention is a system that provides appropriate solutions to technical questions and issues in in-house projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0505] When a user has a technical question or concern, they use their device to send the inquiry to the system, for example, "I don't understand the requirements for XX in Project A." The device converts this information into the appropriate format and sends it to the server using an HTTPS request.

[0506] The server analyzes the received query. First, it parses the received data to extract text data and then uses natural language processing (NLP) technology. Specifically, it uses an NLP engine such as SpaCy or NLTK to linguistically analyze the query content and extract important keywords and phrases. For example, keywords such as "Project A," "Requirements," and "XX" are extracted.

[0507] After the analysis is complete, the server searches the database based on the analysis results. The database stores project information, person in charge information, department information, etc. The server generates an SQL query to search for entries that match the analysis results and retrieve the relevant information.

[0508] Next, the server inputs the acquired information into a generative AI model (e.g., GPT-4) to generate a solution. The input prompt for the generative AI model is constructed based on the analysis results and information acquired from the database. For example, a prompt in the form "Please tell me about the XX requirement for Project A" is created. The generative AI model then generates a specific solution such as "The XX requirement for Project A is...".

[0509] The generated solution is sent to the user's device by the server. The solution is provided along with the analysis results and appropriate person-in-charge information obtained from the database. For example, information such as "Person in charge Y is knowledgeable about the requirements of XX" may be added.

[0510] The terminal displays the solution and contact information sent from the server to the user. It parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it displays, "The XX requirement for Project A is.... For detailed questions, please contact Contact Y."

[0511] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process it as follows:

[0512] 1. The device receives the query and sends it to the server.

[0513] 2. The server analyzes the query and extracts important keywords.

[0514] 3. The server searches and retrieves the relevant information from the database.

[0515] 4. The generative AI model generates a solution based on the acquired information.

[0516] 5. The server sends the generated solution and related person information to the terminal.

[0517] 6. The device displays the solution and contact information to the user. For example, it displays "Information about XX is... Contact contact Y for more information."

[0518] As described above, this system responds quickly and accurately to users' technical questions and issues, providing appropriate solutions. It also aims to improve the efficiency of problem-solving by connecting users to the appropriate person in charge as needed.

[0519] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0520] Step 1: Receiving an inquiry

[0521] Input: User-supplied query text

[0522] Output: Query data sent to the server

[0523] Specific operation: A user types "I don't understand the requirements for XX in Project A" into the terminal. The terminal receives this input, structures the data including the query text, and sends it to the server as an HTTPS request. At this time, metadata such as the user ID and timestamp are also sent.

[0524] Step 2: Analyzing the inquiry

[0525] Input: Inquiry data sent from the terminal

[0526] Output: Parsed keywords and query intent

[0527] How it works: The server parses the received query data and extracts the text. It then uses an NLP engine (such as SpaCy or NLTK) to analyze the text and extract important keywords and phrases. The extracted keywords are "Project A," "Requirements," and "XX."

[0528] Step 3: Find related information

[0529] Input: Parsed keywords and query intent

[0530] Output: Relevant information retrieved from the database

[0531] Specific operation: Based on the analysis results, the server generates an SQL query to the database to search for relevant information. The database stores project information, person in charge information, and department information. For example, the database is searched using the condition "XX requirements for project A" to retrieve the relevant information.

[0532] Step 4: Applying the generative AI model

[0533] Input: relevant information retrieved from the database and the query intent

[0534] Output: The solution generated by the generative AI model

[0535] Specific operation: Based on the acquired information, the server constructs an input prompt for the generative AI model (e.g., GPT-4). For example, it creates a prompt in the form of "Please tell me about the XX requirement of Project A." The generative AI model generates a solution to this prompt and outputs a detailed answer such as "The XX requirement of Project A is..."

[0536] Step 5: Provide solutions and contact information

[0537] Input: Solutions generated by the generative AI model and appropriate personnel information

[0538] Output: Solution and contact information sent to the terminal

[0539] Specific operation: The server receives the generated solution and adds appropriate person information based on it. For example, it adds information such as "Person Y is knowledgeable about the requirements of XX." This information is sent to the user's terminal as an HTTPS response.

[0540] Step 6: Display to the user

[0541] Input: Solution and contact information sent from the server

[0542] Output: Solution and contact information displayed to the user

[0543] Specific operation: The device parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it might display, "Specifically, the XX requirement for Project A is.... If you have any further questions, please contact Person Y." The user can then confirm the displayed information and begin solving the problem.

[0544] Through these steps, users can quickly and accurately obtain specific solutions to technical questions and issues, as well as information on the appropriate person in charge.

[0545] (Application example 1)

[0546] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] In modern factories, when complex machine troubles or technical questions arise, they need to be resolved quickly. However, it is difficult for on-site workers to immediately access the appropriate information and solutions. Furthermore, the time required to search for relevant information leads to reduced productivity. Another issue is the difficulty of quickly obtaining information on the appropriate engineers.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0549] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis result, means for applying a generative AI model using the search result to generate a solution, means for providing the generated solution to the user, means for providing appropriate staff information based on the solution, and means including smart glasses for converting the received inquiry from voice input into text data. This enables on-site workers to easily make inquiries by voice input and quickly obtain appropriate solutions and staff information.

[0550] A "user terminal" is a device used by a user, including a smartphone, tablet, PC, etc.

[0551] An "inquiry" is data that a user sends to the system regarding technical questions or concerns.

[0552] "Analysis" is the process of understanding the content of the received inquiry and extracting important keywords and their intent.

[0553] A "database" is a system for systematically storing information and making it searchable.

[0554] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions based on input information.

[0555] A "solution" is a specific response or answer provided to a user's inquiry.

[0556] "Contact Information" is contact information for employees or technicians with detailed knowledge of specific technical issues.

[0557] "Smart glasses" are wearable devices that use AR (augmented reality) technology to allow users to obtain information visually.

[0558] "Voice input" is an input method in which a device recognizes words spoken by a user and processes them as data.

[0559] "Text data" refers to data that has been converted from voice input into text information.

[0560] This invention is a system that allows users with technical questions or concerns to obtain solutions in real time using smart glasses. The system is basically composed of a user terminal (including smart glasses), a server, and a network infrastructure that links them.

[0561] System Overview

[0562] 1. Receiving Inquiries

[0563] The terminal, i.e., smart glasses, receives voice input from the user and converts the voice into text data. For example, if the user says, "The robot arm on line 2 is not working," the voice data is converted into text data.

[0564] 2. Analysis of inquiry content

[0565] The server analyzes the received text data. It uses natural language processing (NLP) techniques, such as spaCy or NLTK, to extract important keywords from the query and understand its intent. In this example, keywords such as "line 2," "robot arm," and "not working" are extracted.

[0566] 3. Search for related information

[0567] The server searches a database based on the analysis results, which contains machine manuals, error logs, and personnel information, and retrieves relevant information from these.

[0568] 4. Applying generative AI models

[0569] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific, actionable solution. For example, it generates a detailed explanation such as, "If the robot arm on Line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge."

[0570] 5. Providing solutions and contact information

[0571] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the solution, such as "Person Y is knowledgeable about this problem."

[0572] 6. User Display

[0573] The terminal, i.e., the smart glasses, displays the solution and contact information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact contact Y."

[0574] Specific examples

[0575] For example, if a user queries "The robot arm on line 2 is not working," the system will process it as follows:

[0576] 1. The device receives voice input, converts it into text data, and sends it to the server.

[0577] 2. The server analyzes the received text data and extracts important keywords.

[0578] 3. The server searches and retrieves the relevant information from the database.

[0579] 4. The generative AI model generates a solution based on the acquired information.

[0580] 5. The server generates a solution and sends it to the terminal along with relevant contact information.

[0581] 6. The terminal will display the solution and contact information to the user, for example, "If the robot arm on line 2 does not work, please check the power connection. Contact contact Y for more information."

[0582] Example prompt sentence:

[0583] User enquiry:

[0584] "What should I do if the robot arm on Line 2 doesn't work?"

[0585] Related information:

[0586] Line 2 Manual

[0587] Error Log

[0588] Contact Information

[0589] Output of the generative AI model:

[0590] "If the robot arm on line 2 doesn't work, first check the power connection. If that doesn't fix it, contact person Y."

[0591] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0592] Step 1:

[0593] The smart glasses receive voice input and convert the voice data into text data. The voice input is converted into text format using the Google Speech-to-Text API. For example, if a user says, "The robot arm on line 2 is not working," the voice data is converted into text data that reads, "The robot arm on line 2 is not working." The input is voice data, and the output is text data.

[0594] Step 2:

[0595] The text data is sent to the server. The smart glasses send the text data to the server through the factory network. The text data is input to the server, and the received text data is used for the next analysis process. The input is text data, and the output is transmission to the server.

[0596] Step 3:

[0597] The server analyzes the received text data. Using a natural language processing (NLP) library (such as spaCy or NLTK), it extracts important keywords from the query and understands its intent. Specifically, keywords such as "line 2," "robot arm," and "not working" are extracted. The input is the text data, and the output is the important keywords.

[0598] Step 4:

[0599] The server searches the database based on the analysis results. The database stores machine manuals, error logs, and staff information, and searches these to obtain relevant information. For example, the error log and staff information for the "robot arm on line 2" are extracted from the database. The input is important keywords, and the output is related information.

[0600] Step 5:

[0601] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific and actionable solution. For example, it might generate a solution such as, "If the robot arm on line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge." The input is the related information, and the output is the solution.

[0602] Step 6:

[0603] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the generated solution. For example, it provides information such as "Person Y is knowledgeable about this problem." The input is the solution and person information, and the output is transmission completion.

[0604] Step 7:

[0605] The terminal, i.e., smart glasses, displays the solution and person in charge information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it displays, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact person Y." The input is the solution and person in charge information, and the output is the display to the user.

[0606] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0607] This invention relates to a system that provides appropriate solutions to unclear points and technical issues in corporate projects, and also combines it with an emotion engine that recognizes the user's emotions. The system is basically composed of a user terminal, a server, an emotion engine, and a network infrastructure that links them together.

[0608] System Overview

[0609] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and uses a generative AI model to generate the optimal solution. It also uses an emotion engine to analyze the user's emotions, adjusts the response based on that emotional state, and provides appropriate employee information, allowing the user to quickly address the problem.

[0610] Program processing and its explanation

[0611] 1. Receiving Inquiries

[0612] Subject: Device

[0613] The terminal receives inquiries from the user and sends them to the server. For example, if the user enters "I would like to know about XX requirements for project A," this information is sent to the server.

[0614] 2. Analysis of inquiry content

[0615] Subject: Server

[0616] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "Project A," "Requirements," and "XX."

[0617] 3. Emotion analysis using an emotion engine

[0618] Subject: Server

[0619] The server uses an emotion engine to analyze the user's emotion from the content of the inquiry. The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the user's emotional state at that time, such as "the user is feeling anxious."

[0620] 4. Searching for related information

[0621] Subject: Server

[0622] The server searches the database based on the analysis results (keywords and sentiment information), generates an SQL query to query the database, and retrieves the necessary project, employee, and department information.

[0623] 5. Applying generative AI models

[0624] Subject: Server

[0625] Based on the information acquired by the server, a generative AI model is applied to generate a solution. The query content, related information, and emotional information are input to the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[0626] 6. Providing solutions and employee information

[0627] Subject: Server

[0628] The server sends the generated solution and relevant employee information to the terminal to provide the user with the solution, including the solution text and the contact information of the appropriate employee. The server also adjusts the priority and response method of the inquiry based on the emotion information.

[0629] 7. User Display

[0630] Subject: Device

[0631] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[0632] Specific examples

[0633] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[0634] 1. The device receives the query and sends it to the server.

[0635] 2. The server analyzes the inquiry and extracts important keywords.

[0636] 3. The server uses an emotion engine to analyze the user's emotions and identify an emotion such as "anxiety."

[0637] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[0638] 5. The generative AI model generates a solution based on the acquired information and adjusts it based on emotional information.

[0639] 6. The server sends the generated solution and related employee information to the terminal.

[0640] 7. The device displays the solution and employee information to the user, for example, "The information about XX is... If you have further questions, please contact employee Y. If you have any concerns, we are happy to provide additional support."

[0641] As described above, the system of the present invention not only responds quickly and accurately to user inquiries, but also increases user satisfaction by providing tailored solutions that take into account the user's feelings and appropriate employee information.

[0642] The processing flow will be explained below.

[0643] Step 1:

[0644] Subject: User

[0645] The user inputs a query into the system from a terminal. For example, "I would like to know more about the XX requirement for Project A," and presses the send button.

[0646] Step 2:

[0647] Subject: Terminal

[0648] The terminal receives the user's inquiry and transmits the inquiry data to the server. Specifically, the terminal captures the input text data and transfers it to the server via the network.

[0649] Step 3:

[0650] Subject: Server

[0651] The server receives the query sent from the terminal, stores the query content, and prepares to pass it to the next processing step.

[0652] Step 4:

[0653] Subject: Server

[0654] The server analyzes the query received using natural language processing (NLP) technology. Specifically, it tokenizes the text and extracts important keywords (e.g., "Project A," "Requirements," "XX"), and also performs semantic analysis to understand the intent of the query.

[0655] Step 5:

[0656] Subject: Server

[0657] The server uses the results of the NLP analysis to analyze the user's emotions using an emotion engine. Specifically, it detects emotions such as "anxiety," "confusion," and "urgency" from the inquiry content and stores the results in a database.

[0658] Step 6:

[0659] Subject: Server

[0660] The server searches the database based on the analysis results and sentiment information. Specifically, it generates SQL queries to retrieve project information, employee information, and department information from the database. For example, it searches for materials related to "Project A" or "XX."

[0661] Step 7:

[0662] Subject: Server

[0663] The server applies a generative AI model based on the information it obtains to generate a solution. The generative AI model takes the inquiry, related information, and emotional information as input and outputs the optimal solution. For example, it generates a specific solution such as "The XX requirement for Project A is..."

[0664] Step 8:

[0665] Subject: Server

[0666] The server selects the appropriate employee information based on the generated solution and sentiment information. Based on the analysis results, it extracts the contact information of the most suitable employee and sets the priority of the inquiry.

[0667] Step 9:

[0668] Subject: Server

[0669] The server finally sends the generated solution and the selected employee information to the user terminal. For example, the solution text and "Employee Y's contact information" are packaged and sent.

[0670] Step 10:

[0671] Subject: Terminal

[0672] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can see the solution and providing information to contact an employee if necessary.

[0673] Step 11:

[0674] Subject: User

[0675] The user checks the solution displayed on the device and, if necessary, contacts the designated employee. For example, they send an email to employee Y using the provided contact information.

[0676] The above is a specific flow of processing in the system of the present invention. By performing detailed operations at each step, the user can receive a quick and accurate solution. Furthermore, by taking emotional information into consideration, the response to the user can be more appropriate and satisfying.

[0677] Example 2

[0678] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0679] Conventional technical support systems have not only difficulty in providing appropriate solutions to users' technical questions and uncertainties quickly and accurately, but also have the problem of not being able to respond in a way that takes into account the user's emotional state. In particular, if a user feels anxious or confused, ignoring their emotions can reduce the efficiency of problem-solving and lead to low user satisfaction.

[0680] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0681] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative AI model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate employee information based on the solution, means including an emotion engine for analyzing the user's emotions, and means for adjusting the solution and employee information based on the emotion information. This makes it possible to not only quickly and accurately respond to the user's technical questions and uncertainties, but also to provide adjusted solutions and appropriate employee information that take the user's emotional state into consideration.

[0682] A "user terminal" is an information input device that allows a user to input technical questions or uncertainties.

[0683] An "inquiry" is information about technical questions or uncertainties sent from a user terminal.

[0684] The "receiving means" is a function that allows the server to receive an inquiry sent from a user terminal.

[0685] "Means of analysis" refers to the function of analyzing the content of the received inquiry using natural language processing technology and extracting important keywords and intent.

[0686] "Natural language processing technology" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[0687] A "database" is a searchable collection of related information such as project information, employee information, and department information.

[0688] "Search means" is a function for retrieving related information from a database based on the analysis results.

[0689] A "generative AI model" is a machine learning model that generates optimal solutions based on input information.

[0690] The "means for generating" is the function for applying a generative AI model to generate a solution to a query.

[0691] A "solution" is a specific answer or countermeasure to a user's technical questions or uncertainties.

[0692] "Means of providing" refers to the function for delivering the generated solution to the user.

[0693] "Employee Information" means contact information and expertise of appropriate employees who can respond to inquiries.

[0694] An "emotion engine" is a technology for analyzing a user's emotional state from the content of their inquiry.

[0695] "Emotion information" is the result of the user's emotional state analyzed by the emotion engine.

[0696] "Adjustment means" is a function for optimizing solutions and employee information based on emotional information.

[0697] This is a system that provides quick and accurate solutions to technical issues and uncertainties users have in technical projects carried out within a company. The system consists of a user terminal, a server, an emotion engine, and a network infrastructure that links these elements.

[0698] A user terminal is a device that allows a user to input an inquiry, such as a computer or smartphone. A user inputs technical questions or concerns and submits them as an inquiry.

[0699] The server is a computer system that performs the following processes:

[0700] 1. Receiving means: Receives a query from a user terminal as an HTTP request.

[0701] 2. Analysis: The received query content is analyzed using natural language processing techniques, such as using a Python NLP library (NLTK or spaCy) to tokenize the text and extract important keywords.

[0702] 3. Search method: Based on the analysis results, search the database for relevant information. Generate SQL queries to retrieve project information, employee information, department information, etc.

[0703] 4. Generation method: The acquired information is input into a generative AI model (e.g., GPT-3) to generate an optimal solution. The prompt text is something like, "Please explain in detail the XX requirement for Project A."

[0704] 5. Means of provision: The generated solution and related employee information are sent to the user terminal as an HTTP response.

[0705] 6. Emotion Engine: Analyzes emotions from the user's inquiry. For example, it uses a Python emotion analysis library (TextBlob or VADER) to identify the user's emotion of "anxiety."

[0706] 7. Coordination: Optimize and provide solutions and employee information based on emotional information.

[0707] Here are some examples:

[0708] A user sends a query from their device, such as "I want to know about the XX requirement for Project A." The device sends this query as an HTTP request to the server. The server analyzes the received text and extracts keywords such as "Project A," "XX," and "requirements." It then uses an emotion engine to determine that the user's emotion is "anxiety."

[0709] The server searches the database based on keywords and sentiment information to retrieve relevant project information and the contact information of the responsible employee. Based on this information, the generative AI model is prompted with a prompt such as "Please explain in detail the XX requirement for Project A," and a detailed solution is generated.

[0710] Finally, the server sends the generated solution and related employee information to the user terminal as an HTTP response, and the user terminal displays this on its user interface. For example, information such as "The information about XX is.... If you have any further questions, please contact employee Y. If you have any concerns, we will provide additional support" is displayed on the screen.

[0711] As described above, the system of the present invention not only responds quickly and accurately to users' technical questions and uncertainties, but also increases user satisfaction by providing tailored solutions that take into account the user's emotional state and appropriate employee information.

[0712] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0713] Step 1:

[0714] Input: The user enters technical questions or concerns into the user's terminal and submits them as an inquiry.

[0715] Specific operation: The user enters "I would like to know about the XX requirements for Project A" into the device's input screen and presses the send button.

[0716] Output: The device sends the query to the server as an HTTP request.

[0717] Step 2:

[0718] Input: The server receives the HTTP request sent from the device.

[0719] Specific operation: The server analyzes the HTTP request and obtains the inquiry content.

[0720] Output: The query is sent to the server as text.

[0721] Step 3:

[0722] Input: The server analyzes the query received using natural language processing (NLP) techniques.

[0723] What it does: The server uses a Python NLP library (e.g., NLTK, spaCy) to tokenize the text and extract important keywords (e.g., "Project A," "Requirements," "XX").

[0724] Output: The extracted keywords are passed to the next processing step.

[0725] Step 4:

[0726] Input: The server passes the extracted keywords to the emotion engine.

[0727] Specific operation: The server uses an emotion analysis library (e.g., TextBlob, VADER) to analyze the user's emotional state from the query content and identify the emotion, for example, "anxiety."

[0728] Output: The sentiment analysis results are returned to the server and stored in a database.

[0729] Step 5:

[0730] Input: The server searches the database based on keywords and sentiment information.

[0731] Specific operation: The server generates SQL queries and queries the database for project information, employee information, department information, etc.

[0732] Output: The necessary information (project description, contact information of the responsible employee, etc.) is passed to the server.

[0733] Step 6:

[0734] Input: The information acquired by the server is input into the generative AI model.

[0735] Specific operation: The server inputs a prompt sentence to the generative AI model (e.g., GPT-3) such as "Please explain in detail the XX requirement for project A," and the generative AI model generates the optimal solution.

[0736] Output: The generated solution is returned to the server.

[0737] Step 7:

[0738] Input: The server compiles the generated solution and related employee information.

[0739] What happens: The server creates a single HTTP response with the solution text and appropriate employee information (e.g., contact details), and adjusts the priority and response based on the sentiment information.

[0740] Output: An HTTP response is formed and sent to the user's device.

[0741] Step 8:

[0742] Input: The device receives the HTTP response sent by the server.

[0743] Specific behavior: The device displays the solution and employee information in the user interface.

[0744] Output: The user can see the solution and employee information displayed on the screen.

[0745] Specifically, the device displays information such as "Information about XX is... If you have any detailed questions, please contact employee Y. If you have any concerns, we will provide additional support" on the screen, and the user confirms it.

[0746] (Application example 2)

[0747] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0748] There is a need for a means to quickly and accurately resolve the technical questions and uncertainties that drivers of autonomous vehicles face while driving, as well as the emotional stress that accompanies them. In particular, it is important to improve driver satisfaction and safety by providing optimal solutions according to the driver's emotional state.

[0749] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving an inquiry from a user device, means for analyzing the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative artificial intelligence model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate personal information based on the solution, means for receiving an inquiry via voice input or text input, means for applying a generative artificial intelligence model using the analysis results and emotion analysis to generate a solution, means for providing the generated solution and related information on a display device and by voice, and means for analyzing the driver's tone of voice and facial expression to identify the driver's emotional state. This makes it possible to quickly provide optimal solutions to technical questions and problems that the driver has while driving, based on their emotional state.

[0750] "User equipment" refers to devices such as smartphones or on-board displays inside or outside an autonomous vehicle, and is used for inquiries from and information provision to the driver.

[0751] A "generative artificial intelligence model" is a generative model that uses artificial intelligence to analyze the content of an inquiry and related information and generate an appropriate solution.

[0752] "Means for analyzing the content of an inquiry" refers to the process of tokenizing the content of an inquiry received from a user using natural language processing technology, extracting keywords, and understanding the intent.

[0753] A "database" is a collection of information that the system uses to search for relevant information, and includes road information, traffic information, location information, and the like.

[0754] "Emotion analysis" refers to technology for identifying a driver's emotional state from their tone of voice and facial expressions, and is the process of analyzing the anxiety and stress the driver is experiencing.

[0755] A "display device" is a device for providing generated solutions and related information to a user, including an in-car display or a smartphone screen.

[0756] "Voice input" refers to a means by which a user can make queries to a system through voice, and includes technology that converts received voice data into text.

[0757] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning, and is used here for analyzing inquiry content and preprocessing for generative AI models.

[0758] "Means of generating a solution" refers to the process of using the generated AI model to generate the optimal answer based on the information obtained from the database and the query content.

[0759] "Personal information" refers to information such as contact details and job titles of individuals appropriate to address specific challenges or questions, including information provided by the system to the driver.

[0760] System Overview

[0761] This invention is a system whose basic components are a user device, a server, an emotion engine, and a network infrastructure that links these together. When a driver has a technical question or concern, they use the user device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and generates an optimal solution using a generative artificial intelligence model. Furthermore, it uses the emotion engine to analyze the driver's emotions, adjusts countermeasures based on that emotional state, and provides appropriate personal information, allowing the driver to quickly address the problem.

[0762] Program processing and its explanation

[0763] 1. Receiving Inquiries

[0764] The user device (smartphone or in-car display) receives queries from the driver and sends them to the server using a voice recognition library (Google Speech-to-Text API) or a text input field. For example, the driver may ask by voice, "How should I get through this intersection?" and this information is sent to the server.

[0765] 2. Analysis of inquiry content

[0766] The server analyzes the query it receives, using natural language processing techniques (spaCy, NLTK) to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "intersection" and "road."

[0767] 3. Emotion analysis

[0768] The server uses an emotion engine to analyze the driver's emotions from their tone of voice and facial expressions. This involves using a face recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the driver's emotional state as "anxious."

[0769] 4. Searching for related information

[0770] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information), generates an SQL query to query the database, and obtains the necessary road, traffic, and location information.

[0771] 5. Solution generation and delivery

[0772] Based on the information acquired by the server, a generative artificial intelligence model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The query content, related information, and emotional information are input into the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "It is safe to turn right at this intersection." The generated solution and related information are displayed on the in-car display or smartphone, and are also provided to the driver using voice synthesis (Google Text-to-Speech API).

[0773] Specific examples

[0774] When a driver makes a voice inquiry such as "I want to know how to get through this intersection on this road," the specific processing steps are as follows:

[0775] 1. The user device receives the query and sends it to the server.

[0776] 2. The server analyzes the inquiry and extracts important keywords.

[0777] 3. The server uses an emotion engine to analyze the driver's emotions from their voice and facial expressions, and identifies an emotion such as "anxiety."

[0778] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[0779] 5. The generative AI model generates a solution based on the acquired information and adjusts the solution with reference to emotional information.

[0780] 6. The server sends the generated solution and related information to the user device.

[0781] 7. The user device displays the solution and related information to the driver and provides voice guidance: "It is safe to turn right at this intersection. Also, if you turn left at the next traffic light, you will reach your destination."

[0782] Example prompts to input to the generative AI model

[0783] Prompt: "Generate an appropriate response to a driver who is worried about how to get through this intersection. The driver's current emotion is anxiety."

[0784] Example response: "It's safe to turn right at this intersection, and turning left at the next light will get you to your destination."

[0785] This makes it possible to quickly provide optimal solutions to technical questions or problems that drivers may have while driving, depending on their emotional state.

[0786] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0787] Step 1:

[0788] The user inputs a query by voice or text (voice input or text input). The user's device (smartphone or in-car display) receives this query and sends it to the server. The Google Speech-to-Text API is used for input. For example, if the user voices a query such as "I want to know how to get through this intersection," this voice is converted into text and sent to the server.

[0789] Step 2:

[0790] The server analyzes the query it receives. It uses natural language processing techniques (spaCy, NLTK) to tokenize the text and extract important keywords. The input is the query text, and the output is the extracted keywords. For example, it identifies keywords such as "intersection" and "road." Specifically, the analysis engine analyzes the query content and identifies the main keywords.

[0791] Step 3:

[0792] The server uses an emotion engine to analyze the driver's emotions. This uses a facial recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The input is the driver's tone of voice and facial expression data, and the output is the driver's emotional state. Specifically, the server analyzes the driver's tone of voice and facial expression to identify emotions such as "anxiety."

[0793] Step 4:

[0794] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information). It generates an SQL query and queries the database. The input is keywords and emotion information, and the output is related road information, traffic information, and location information. Specifically, the server retrieves the required data from the database.

[0795] Step 5:

[0796] Based on the information acquired by the server, a generative AI model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The input is the inquiry content, related information, and emotional information, and the output is the optimal solution. Specifically, the generative AI model generates a solution such as "It is safe to turn right at this intersection."

[0797] Step 6:

[0798] The server sends the generated solution and related information to the user device, which displays it and provides it audibly. The output is the solution text and speech synthesis data. Specifically, the generated solution is displayed on the in-vehicle display, and a voice guides the user, saying, "It is safe to turn right at this intersection."

[0799] In this way, the system processes the data input at each step, performing appropriate data processing and calculations to provide the user with the optimal solution.

[0800] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0801] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0802] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0803] [Third embodiment]

[0804] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0805] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0806] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0807] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0808] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0809] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0810] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0811] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0812] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0813] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0814] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0815] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0816] The present invention is a system that provides appropriate solutions to unclear points and technical issues in corporate projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0817] System Overview

[0818] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes it. Based on the analysis results, it searches the database for relevant information and uses a generative AI model to generate the optimal solution. Furthermore, by providing the user with appropriate employee information as needed, the system allows the user to quickly address the problem.

[0819] Program processing and its explanation

[0820] 1. Receiving Inquiries

[0821] Subject: Device

[0822] The terminal receives inquiries from users and sends them to the server. For example, if a user types into the terminal, "I don't understand the requirements for XX in project A," this information is sent to the server.

[0823] 2. Analysis of inquiry content

[0824] Subject: Server

[0825] The server analyzes the received inquiry and uses natural language processing (NLP) technology to extract important keywords from the inquiry and understand the intent of the inquiry. For example, it extracts information about "Project A," "Requirements," and "XX."

[0826] 3. Search for related information

[0827] Subject: Server

[0828] The server searches the database based on the analysis results, and retrieves relevant information from the database, which contains project information, employee information, and department information.

[0829] 4. Applying generative AI models

[0830] Subject: Server

[0831] Based on the acquired information, the server inputs the generative AI model to generate a solution. The generative AI model considers the query and related information to create a specific and feasible solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[0832] 5. Providing solutions and employee information

[0833] Subject: Server

[0834] The server sends the generated solution to the user's terminal and also provides appropriate employee information based on the solution (e.g., "Employee Y is knowledgeable about this problem"), allowing the user to know not only the solution but also who to contact for further information.

[0835] 6. User Display

[0836] Subject: Device

[0837] The terminal displays the solution and employee information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "The specific XX requirement for Project A is.... If you have any further questions, please contact employee Y."

[0838] Specific examples

[0839] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[0840] 1. The device receives the query and sends it to the server.

[0841] 2. The server analyzes the inquiry and extracts important keywords.

[0842] 3. The server searches and retrieves the relevant information from the database.

[0843] 4. The generative AI model generates a solution based on the acquired information.

[0844] 5. The server generates a solution and sends the relevant employee information to the terminal.

[0845] 6. The device displays the solution and employee information to the user, for example, "The information about XX is... Contact employee Y for more information."

[0846] As described above, the system of the present invention responds to user inquiries quickly and accurately, provides appropriate solutions, and, if necessary, connects users to appropriate staff members, thereby improving the efficiency of problem solving.

[0847] The processing flow will be explained below.

[0848] Step 1:

[0849] Subject: User

[0850] The user inputs a query to the system from a terminal. For example, the user inputs "I would like to know about XX requirements for Project A" and presses the send button.

[0851] Step 2:

[0852] Subject: Terminal

[0853] The terminal receives the user's query and transmits the query data to the server, specifically, the terminal captures the user's input text and transfers it to the server via the network.

[0854] Step 3:

[0855] Subject: Server

[0856] The server receives the query sent from the terminal, stores the received text data, and prepares for the next analysis process.

[0857] Step 4:

[0858] Subject: Server

[0859] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text and extract important keywords, such as "Project A," "Requirements," and "XX."

[0860] Step 5:

[0861] Subject: Server

[0862] The server searches the database for relevant information based on the analyzed keywords, generates SQL queries to query the database, and retrieves the required project, employee, and department information.

[0863] Step 6:

[0864] Subject: Server

[0865] The server applies a generative AI model to generate a solution based on the information retrieved from the database. The query and related information are input to the generative AI model, which then outputs the optimal solution.

[0866] Step 7:

[0867] Subject: Server

[0868] The server transmits the generated solution and associated employee information to the terminal to provide the information to the user, including, for example, the solution text and contact information for the appropriate employee.

[0869] Step 8:

[0870] Subject: Terminal

[0871] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[0872] Step 9:

[0873] Subject: User

[0874] The user checks the solution displayed on the terminal and, if necessary, makes further inquiries using the provided employee information, taking an action such as "send an email to employee Y."

[0875] Example 1

[0876] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0877] When technical questions or issues arise in corporate projects, there is a lack of systems that can provide solutions quickly and accurately. It is particularly difficult to quickly find the right information for complex issues involving multiple projects or departments. As a result, problem resolution can take a long time, leading to reduced work efficiency.

[0878] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0879] In this invention, the server includes a means for analyzing the received inquiry, a means for searching a database for related information based on the analysis results, and a means for generating a solution by applying a generative AI model using the search results, thereby enabling the server to provide a quick and accurate solution to the user's technical questions and problems, and also provide appropriate contact information as needed.

[0880] A "user terminal" is an electronic device that allows a user to input and send an inquiry.

[0881] An "inquiry" is a technical question or problem that a user needs to solve.

[0882] A "server" is a central processing unit for analyzing queries received from user terminals and generating and providing solutions.

[0883] "Analysis" is the process of using natural language processing technology to understand the content of the received inquiry and extract important keywords and phrases.

[0884] A "database" is a collection of information that holds related information such as project information, person information, department information, etc., and stores it in a searchable manner.

[0885] A "generative AI model" is a set of algorithms or programs that generate solutions based on the query and related information.

[0886] A "solution" is a specific, actionable answer or instruction to a user's technical question or problem.

[0887] "Contact information" refers to contact and related information for employees who are knowledgeable about specific technical questions or issues.

[0888] "Display" is the process of outputting information sent from the server to the terminal in a format that the user can visually confirm.

[0889] This invention is a system that provides appropriate solutions to technical questions and issues in in-house projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[0890] When a user has a technical question or concern, they use their device to send the inquiry to the system, for example, "I don't understand the requirements for XX in Project A." The device converts this information into the appropriate format and sends it to the server using an HTTPS request.

[0891] The server analyzes the received query. First, it parses the received data to extract text data and then uses natural language processing (NLP) technology. Specifically, it uses an NLP engine such as SpaCy or NLTK to linguistically analyze the query content and extract important keywords and phrases. For example, keywords such as "Project A," "Requirements," and "XX" are extracted.

[0892] After the analysis is complete, the server searches the database based on the analysis results. The database stores project information, person in charge information, department information, etc. The server generates an SQL query to search for entries that match the analysis results and retrieve the relevant information.

[0893] Next, the server inputs the acquired information into a generative AI model (e.g., GPT-4) to generate a solution. The input prompt for the generative AI model is constructed based on the analysis results and information acquired from the database. For example, a prompt in the form "Please tell me about the XX requirement for Project A" is created. The generative AI model then generates a specific solution such as "The XX requirement for Project A is...".

[0894] The generated solution is sent to the user's device by the server. The solution is provided along with the analysis results and appropriate person-in-charge information obtained from the database. For example, information such as "Person in charge Y is knowledgeable about the requirements of XX" may be added.

[0895] The terminal displays the solution and contact information sent from the server to the user. It parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it displays, "The XX requirement for Project A is.... For detailed questions, please contact Contact Y."

[0896] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process it as follows:

[0897] 1. The device receives the query and sends it to the server.

[0898] 2. The server analyzes the query and extracts important keywords.

[0899] 3. The server searches and retrieves the relevant information from the database.

[0900] 4. The generative AI model generates a solution based on the acquired information.

[0901] 5. The server sends the generated solution and related person information to the terminal.

[0902] 6. The device displays the solution and contact information to the user. For example, it displays "Information about XX is... Contact contact Y for more information."

[0903] As described above, this system responds quickly and accurately to users' technical questions and issues, providing appropriate solutions. It also aims to improve the efficiency of problem-solving by connecting users to the appropriate person in charge as needed.

[0904] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0905] Step 1: Receiving an inquiry

[0906] Input: User-supplied query text

[0907] Output: Query data sent to the server

[0908] Specific operation: A user types "I don't understand the requirements for XX in Project A" into the terminal. The terminal receives this input, structures the data including the query text, and sends it to the server as an HTTPS request. At this time, metadata such as the user ID and timestamp are also sent.

[0909] Step 2: Analyzing the inquiry

[0910] Input: Inquiry data sent from the terminal

[0911] Output: Parsed keywords and query intent

[0912] How it works: The server parses the received query data and extracts the text. It then uses an NLP engine (such as SpaCy or NLTK) to analyze the text and extract important keywords and phrases. The extracted keywords are "Project A," "Requirements," and "XX."

[0913] Step 3: Find related information

[0914] Input: Parsed keywords and query intent

[0915] Output: Relevant information retrieved from the database

[0916] Specific operation: Based on the analysis results, the server generates an SQL query to the database to search for relevant information. The database stores project information, person in charge information, and department information. For example, the database is searched using the condition "XX requirements for project A" to retrieve the relevant information.

[0917] Step 4: Applying the generative AI model

[0918] Input: relevant information retrieved from the database and the query intent

[0919] Output: The solution generated by the generative AI model

[0920] Specific operation: Based on the acquired information, the server constructs an input prompt for the generative AI model (e.g., GPT-4). For example, it creates a prompt in the form of "Please tell me about the XX requirement of Project A." The generative AI model generates a solution to this prompt and outputs a detailed answer such as "The XX requirement of Project A is..."

[0921] Step 5: Provide solutions and contact information

[0922] Input: Solutions generated by the generative AI model and appropriate personnel information

[0923] Output: Solution and contact information sent to the terminal

[0924] Specific operation: The server receives the generated solution and adds appropriate person information based on it. For example, it adds information such as "Person Y is knowledgeable about the requirements of XX." This information is sent to the user's terminal as an HTTPS response.

[0925] Step 6: Display to the user

[0926] Input: Solution and contact information sent from the server

[0927] Output: Solution and contact information displayed to the user

[0928] Specific operation: The device parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it might display, "Specifically, the XX requirement for Project A is.... If you have any further questions, please contact Person Y." The user can then confirm the displayed information and begin solving the problem.

[0929] Through these steps, users can quickly and accurately obtain specific solutions to technical questions and issues, as well as information on the appropriate person in charge.

[0930] (Application example 1)

[0931] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0932] In modern factories, when complex machine troubles or technical questions arise, they need to be resolved quickly. However, it is difficult for on-site workers to immediately access the appropriate information and solutions. Furthermore, the time required to search for relevant information leads to reduced productivity. Another issue is the difficulty of quickly obtaining information on the appropriate engineers.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0934] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis result, means for applying a generative AI model using the search result to generate a solution, means for providing the generated solution to the user, means for providing appropriate staff information based on the solution, and means including smart glasses for converting the received inquiry from voice input into text data. This enables on-site workers to easily make inquiries by voice input and quickly obtain appropriate solutions and staff information.

[0935] A "user terminal" is a device used by a user, including a smartphone, tablet, PC, etc.

[0936] An "inquiry" is data that a user sends to the system regarding technical questions or concerns.

[0937] "Analysis" is the process of understanding the content of the received inquiry and extracting important keywords and their intent.

[0938] A "database" is a system for systematically storing information and making it searchable.

[0939] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions based on input information.

[0940] A "solution" is a specific response or answer provided to a user's inquiry.

[0941] "Contact Information" is contact information for employees or technicians with detailed knowledge of specific technical issues.

[0942] "Smart glasses" are wearable devices that use AR (augmented reality) technology to allow users to obtain information visually.

[0943] "Voice input" is an input method in which a device recognizes words spoken by a user and processes them as data.

[0944] "Text data" refers to data that has been converted from voice input into text information.

[0945] This invention is a system that allows users with technical questions or concerns to obtain solutions in real time using smart glasses. The system is basically composed of a user terminal (including smart glasses), a server, and a network infrastructure that links them.

[0946] System Overview

[0947] 1. Receiving Inquiries

[0948] The terminal, i.e., smart glasses, receives voice input from the user and converts the voice into text data. For example, if the user says, "The robot arm on line 2 is not working," the voice data is converted into text data.

[0949] 2. Analysis of inquiry content

[0950] The server analyzes the received text data. It uses natural language processing (NLP) techniques, such as spaCy or NLTK, to extract important keywords from the query and understand its intent. In this example, keywords such as "line 2," "robot arm," and "not working" are extracted.

[0951] 3. Search for related information

[0952] The server searches a database based on the analysis results, which contains machine manuals, error logs, and personnel information, and retrieves relevant information from these.

[0953] 4. Applying generative AI models

[0954] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific, actionable solution. For example, it generates a detailed explanation such as, "If the robot arm on Line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge."

[0955] 5. Providing solutions and contact information

[0956] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the solution, such as "Person Y is knowledgeable about this problem."

[0957] 6. User Display

[0958] The terminal, i.e., the smart glasses, displays the solution and contact information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact contact Y."

[0959] Specific examples

[0960] For example, if a user queries "The robot arm on line 2 is not working," the system will process it as follows:

[0961] 1. The device receives voice input, converts it into text data, and sends it to the server.

[0962] 2. The server analyzes the received text data and extracts important keywords.

[0963] 3. The server searches and retrieves the relevant information from the database.

[0964] 4. The generative AI model generates a solution based on the acquired information.

[0965] 5. The server generates a solution and sends it to the terminal along with relevant contact information.

[0966] 6. The terminal will display the solution and contact information to the user, for example, "If the robot arm on line 2 does not work, please check the power connection. Contact contact Y for more information."

[0967] Example prompt sentence:

[0968] User enquiry:

[0969] "What should I do if the robot arm on Line 2 doesn't work?"

[0970] Related information:

[0971] Line 2 Manual

[0972] Error Log

[0973] Contact Information

[0974] Output of the generative AI model:

[0975] "If the robot arm on line 2 doesn't work, first check the power connection. If that doesn't fix it, contact person Y."

[0976] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0977] Step 1:

[0978] The smart glasses receive voice input and convert the voice data into text data. The voice input is converted into text format using the Google Speech-to-Text API. For example, if a user says, "The robot arm on line 2 is not working," the voice data is converted into text data that reads, "The robot arm on line 2 is not working." The input is voice data, and the output is text data.

[0979] Step 2:

[0980] The text data is sent to the server. The smart glasses send the text data to the server through the factory network. The text data is input to the server, and the received text data is used for the next analysis process. The input is text data, and the output is transmission to the server.

[0981] Step 3:

[0982] The server analyzes the received text data. Using a natural language processing (NLP) library (such as spaCy or NLTK), it extracts important keywords from the query and understands its intent. Specifically, keywords such as "line 2," "robot arm," and "not working" are extracted. The input is the text data, and the output is the important keywords.

[0983] Step 4:

[0984] The server searches the database based on the analysis results. The database stores machine manuals, error logs, and staff information, and searches these to obtain relevant information. For example, the error log and staff information for the "robot arm on line 2" are extracted from the database. The input is important keywords, and the output is related information.

[0985] Step 5:

[0986] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific and actionable solution. For example, it might generate a solution such as, "If the robot arm on line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge." The input is the related information, and the output is the solution.

[0987] Step 6:

[0988] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the generated solution. For example, it provides information such as "Person Y is knowledgeable about this problem." The input is the solution and person information, and the output is transmission completion.

[0989] Step 7:

[0990] The terminal, i.e., smart glasses, displays the solution and person in charge information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it displays, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact person Y." The input is the solution and person in charge information, and the output is the display to the user.

[0991] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0992] This invention relates to a system that provides appropriate solutions to unclear points and technical issues in corporate projects, and also combines it with an emotion engine that recognizes the user's emotions. The system is basically composed of a user terminal, a server, an emotion engine, and a network infrastructure that links them together.

[0993] System Overview

[0994] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and uses a generative AI model to generate the optimal solution. It also uses an emotion engine to analyze the user's emotions, adjusts the response based on that emotional state, and provides appropriate employee information, allowing the user to quickly address the problem.

[0995] Program processing and its explanation

[0996] 1. Receiving Inquiries

[0997] Subject: Device

[0998] The terminal receives inquiries from the user and sends them to the server. For example, if the user enters "I would like to know about XX requirements for project A," this information is sent to the server.

[0999] 2. Analysis of inquiry content

[1000] Subject: Server

[1001] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "Project A," "Requirements," and "XX."

[1002] 3. Emotion analysis using an emotion engine

[1003] Subject: Server

[1004] The server uses an emotion engine to analyze the user's emotion from the content of the inquiry. The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the user's emotional state at that time, such as "the user is feeling anxious."

[1005] 4. Searching for related information

[1006] Subject: Server

[1007] The server searches the database based on the analysis results (keywords and sentiment information), generates an SQL query to query the database, and retrieves the necessary project, employee, and department information.

[1008] 5. Applying generative AI models

[1009] Subject: Server

[1010] Based on the information acquired by the server, a generative AI model is applied to generate a solution. The query content, related information, and emotional information are input to the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[1011] 6. Providing solutions and employee information

[1012] Subject: Server

[1013] The server sends the generated solution and relevant employee information to the terminal to provide the user with the solution, including the solution text and the contact information of the appropriate employee. The server also adjusts the priority and response method of the inquiry based on the emotion information.

[1014] 7. User Display

[1015] Subject: Device

[1016] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[1017] Specific examples

[1018] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[1019] 1. The device receives the query and sends it to the server.

[1020] 2. The server analyzes the inquiry and extracts important keywords.

[1021] 3. The server uses an emotion engine to analyze the user's emotions and identify an emotion such as "anxiety."

[1022] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[1023] 5. The generative AI model generates a solution based on the acquired information and adjusts it based on emotional information.

[1024] 6. The server sends the generated solution and related employee information to the terminal.

[1025] 7. The device displays the solution and employee information to the user, for example, "The information about XX is... If you have further questions, please contact employee Y. If you have any concerns, we are happy to provide additional support."

[1026] As described above, the system of the present invention not only responds quickly and accurately to user inquiries, but also increases user satisfaction by providing tailored solutions that take into account the user's feelings and appropriate employee information.

[1027] The processing flow will be explained below.

[1028] Step 1:

[1029] Subject: User

[1030] The user inputs a query into the system from a terminal. For example, "I would like to know more about the XX requirement for Project A," and presses the send button.

[1031] Step 2:

[1032] Subject: Terminal

[1033] The terminal receives the user's inquiry and transmits the inquiry data to the server. Specifically, the terminal captures the input text data and transfers it to the server via the network.

[1034] Step 3:

[1035] Subject: Server

[1036] The server receives the query sent from the terminal, stores the query content, and prepares to pass it to the next processing step.

[1037] Step 4:

[1038] Subject: Server

[1039] The server analyzes the query received using natural language processing (NLP) technology. Specifically, it tokenizes the text and extracts important keywords (e.g., "Project A," "Requirements," "XX"), and also performs semantic analysis to understand the intent of the query.

[1040] Step 5:

[1041] Subject: Server

[1042] The server uses the results of the NLP analysis to analyze the user's emotions using an emotion engine. Specifically, it detects emotions such as "anxiety," "confusion," and "urgency" from the inquiry content and stores the results in a database.

[1043] Step 6:

[1044] Subject: Server

[1045] The server searches the database based on the analysis results and sentiment information. Specifically, it generates SQL queries to retrieve project information, employee information, and department information from the database. For example, it searches for materials related to "Project A" or "XX."

[1046] Step 7:

[1047] Subject: Server

[1048] The server applies a generative AI model based on the information it obtains to generate a solution. The generative AI model takes the inquiry, related information, and emotional information as input and outputs the optimal solution. For example, it generates a specific solution such as "The XX requirement for Project A is..."

[1049] Step 8:

[1050] Subject: Server

[1051] The server selects the appropriate employee information based on the generated solution and sentiment information. Based on the analysis results, it extracts the contact information of the most suitable employee and sets the priority of the inquiry.

[1052] Step 9:

[1053] Subject: Server

[1054] The server finally sends the generated solution and the selected employee information to the user terminal. For example, the solution text and "Employee Y's contact information" are packaged and sent.

[1055] Step 10:

[1056] Subject: Terminal

[1057] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can see the solution and providing information to contact an employee if necessary.

[1058] Step 11:

[1059] Subject: User

[1060] The user checks the solution displayed on the device and, if necessary, contacts the designated employee. For example, they send an email to employee Y using the provided contact information.

[1061] The above is a specific flow of processing in the system of the present invention. By performing detailed operations at each step, the user can receive a quick and accurate solution. Furthermore, by taking emotional information into consideration, the response to the user can be more appropriate and satisfying.

[1062] Example 2

[1063] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1064] Conventional technical support systems have not only difficulty in providing appropriate solutions to users' technical questions and uncertainties quickly and accurately, but also have the problem of not being able to respond in a way that takes into account the user's emotional state. In particular, if a user feels anxious or confused, ignoring their emotions can reduce the efficiency of problem-solving and lead to low user satisfaction.

[1065] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1066] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative AI model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate employee information based on the solution, means including an emotion engine for analyzing the user's emotions, and means for adjusting the solution and employee information based on the emotion information. This makes it possible to not only quickly and accurately respond to the user's technical questions and uncertainties, but also to provide adjusted solutions and appropriate employee information that take the user's emotional state into consideration.

[1067] A "user terminal" is an information input device that allows a user to input technical questions or uncertainties.

[1068] An "inquiry" is information about technical questions or uncertainties sent from a user terminal.

[1069] The "receiving means" is a function that allows the server to receive an inquiry sent from a user terminal.

[1070] "Means of analysis" refers to the function of analyzing the content of the received inquiry using natural language processing technology and extracting important keywords and intent.

[1071] "Natural language processing technology" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[1072] A "database" is a searchable collection of related information such as project information, employee information, and department information.

[1073] "Search means" is a function for retrieving related information from a database based on the analysis results.

[1074] A "generative AI model" is a machine learning model that generates optimal solutions based on input information.

[1075] The "means for generating" is the function for applying a generative AI model to generate a solution to a query.

[1076] A "solution" is a specific answer or countermeasure to a user's technical questions or uncertainties.

[1077] "Means of providing" refers to the function for delivering the generated solution to the user.

[1078] "Employee Information" means contact information and expertise of appropriate employees who can respond to inquiries.

[1079] An "emotion engine" is a technology for analyzing a user's emotional state from the content of their inquiry.

[1080] "Emotion information" is the result of the user's emotional state analyzed by the emotion engine.

[1081] "Adjustment means" is a function for optimizing solutions and employee information based on emotional information.

[1082] This is a system that provides quick and accurate solutions to technical issues and uncertainties users have in technical projects carried out within a company. The system consists of a user terminal, a server, an emotion engine, and a network infrastructure that links these elements.

[1083] A user terminal is a device that allows a user to input an inquiry, such as a computer or smartphone. A user inputs technical questions or concerns and submits them as an inquiry.

[1084] The server is a computer system that performs the following processes:

[1085] 1. Receiving means: Receives a query from a user terminal as an HTTP request.

[1086] 2. Analysis: The received query content is analyzed using natural language processing techniques, such as using a Python NLP library (NLTK or spaCy) to tokenize the text and extract important keywords.

[1087] 3. Search method: Based on the analysis results, search the database for relevant information. Generate SQL queries to retrieve project information, employee information, department information, etc.

[1088] 4. Generation method: The acquired information is input into a generative AI model (e.g., GPT-3) to generate an optimal solution. The prompt text is something like, "Please explain in detail the XX requirement for Project A."

[1089] 5. Means of provision: The generated solution and related employee information are sent to the user terminal as an HTTP response.

[1090] 6. Emotion Engine: Analyzes emotions from the user's inquiry. For example, it uses a Python emotion analysis library (TextBlob or VADER) to identify the user's emotion of "anxiety."

[1091] 7. Coordination: Optimize and provide solutions and employee information based on emotional information.

[1092] Here are some examples:

[1093] A user sends a query from their device, such as "I want to know about the XX requirement for Project A." The device sends this query as an HTTP request to the server. The server analyzes the received text and extracts keywords such as "Project A," "XX," and "requirements." It then uses an emotion engine to determine that the user's emotion is "anxiety."

[1094] The server searches the database based on keywords and sentiment information to retrieve relevant project information and the contact information of the responsible employee. Based on this information, the generative AI model is prompted with a prompt such as "Please explain in detail the XX requirement for Project A," and a detailed solution is generated.

[1095] Finally, the server sends the generated solution and related employee information to the user terminal as an HTTP response, and the user terminal displays this on its user interface. For example, information such as "The information about XX is.... If you have any further questions, please contact employee Y. If you have any concerns, we will provide additional support" is displayed on the screen.

[1096] As described above, the system of the present invention not only responds quickly and accurately to users' technical questions and uncertainties, but also increases user satisfaction by providing tailored solutions that take into account the user's emotional state and appropriate employee information.

[1097] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1098] Step 1:

[1099] Input: The user enters technical questions or concerns into the user's terminal and submits them as an inquiry.

[1100] Specific operation: The user enters "I would like to know about the XX requirements for Project A" into the device's input screen and presses the send button.

[1101] Output: The device sends the query to the server as an HTTP request.

[1102] Step 2:

[1103] Input: The server receives the HTTP request sent from the device.

[1104] Specific operation: The server analyzes the HTTP request and obtains the inquiry content.

[1105] Output: The query is sent to the server as text.

[1106] Step 3:

[1107] Input: The server analyzes the query received using natural language processing (NLP) techniques.

[1108] What it does: The server uses a Python NLP library (e.g., NLTK, spaCy) to tokenize the text and extract important keywords (e.g., "Project A," "Requirements," "XX").

[1109] Output: The extracted keywords are passed to the next processing step.

[1110] Step 4:

[1111] Input: The server passes the extracted keywords to the emotion engine.

[1112] Specific operation: The server uses an emotion analysis library (e.g., TextBlob, VADER) to analyze the user's emotional state from the query content and identify the emotion, for example, "anxiety."

[1113] Output: The sentiment analysis results are returned to the server and stored in a database.

[1114] Step 5:

[1115] Input: The server searches the database based on keywords and sentiment information.

[1116] Specific operation: The server generates SQL queries and queries the database for project information, employee information, department information, etc.

[1117] Output: The necessary information (project description, contact information of the responsible employee, etc.) is passed to the server.

[1118] Step 6:

[1119] Input: The information acquired by the server is input into the generative AI model.

[1120] Specific operation: The server inputs a prompt sentence to the generative AI model (e.g., GPT-3) such as "Please explain in detail the XX requirement for project A," and the generative AI model generates the optimal solution.

[1121] Output: The generated solution is returned to the server.

[1122] Step 7:

[1123] Input: The server compiles the generated solution and related employee information.

[1124] What happens: The server creates a single HTTP response with the solution text and appropriate employee information (e.g., contact details), and adjusts the priority and response based on the sentiment information.

[1125] Output: An HTTP response is formed and sent to the user's device.

[1126] Step 8:

[1127] Input: The device receives the HTTP response sent by the server.

[1128] Specific behavior: The device displays the solution and employee information in the user interface.

[1129] Output: The user can see the solution and employee information displayed on the screen.

[1130] Specifically, the device displays information such as "Information about XX is... If you have any detailed questions, please contact employee Y. If you have any concerns, we will provide additional support" on the screen, and the user confirms it.

[1131] (Application example 2)

[1132] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] There is a need for a means to quickly and accurately resolve the technical questions and uncertainties that drivers of autonomous vehicles face while driving, as well as the emotional stress that accompanies them. In particular, it is important to improve driver satisfaction and safety by providing optimal solutions according to the driver's emotional state.

[1134] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving an inquiry from a user device, means for analyzing the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative artificial intelligence model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate personal information based on the solution, means for receiving an inquiry via voice input or text input, means for applying a generative artificial intelligence model using the analysis results and emotion analysis to generate a solution, means for providing the generated solution and related information on a display device and by voice, and means for analyzing the driver's tone of voice and facial expression to identify the driver's emotional state. This makes it possible to quickly provide optimal solutions to technical questions and problems that the driver has while driving, based on their emotional state.

[1135] "User equipment" refers to devices such as smartphones or on-board displays inside or outside an autonomous vehicle, and is used for inquiries from and information provision to the driver.

[1136] A "generative artificial intelligence model" is a generative model that uses artificial intelligence to analyze the content of an inquiry and related information and generate an appropriate solution.

[1137] "Means for analyzing the content of an inquiry" refers to the process of tokenizing the content of an inquiry received from a user using natural language processing technology, extracting keywords, and understanding the intent.

[1138] A "database" is a collection of information that the system uses to search for relevant information, and includes road information, traffic information, location information, and the like.

[1139] "Emotion analysis" refers to technology for identifying a driver's emotional state from their tone of voice and facial expressions, and is the process of analyzing the anxiety and stress the driver is experiencing.

[1140] A "display device" is a device for providing generated solutions and related information to a user, including an in-car display or a smartphone screen.

[1141] "Voice input" refers to a means by which a user can make queries to a system through voice, and includes technology that converts received voice data into text.

[1142] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning, and is used here for analyzing inquiry content and preprocessing for generative AI models.

[1143] "Means of generating a solution" refers to the process of using the generated AI model to generate the optimal answer based on the information obtained from the database and the query content.

[1144] "Personal information" refers to information such as contact details and job titles of individuals appropriate to address specific challenges or questions, including information provided by the system to the driver.

[1145] System Overview

[1146] This invention is a system whose basic components are a user device, a server, an emotion engine, and a network infrastructure that links these together. When a driver has a technical question or concern, they use the user device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and generates an optimal solution using a generative artificial intelligence model. Furthermore, it uses the emotion engine to analyze the driver's emotions, adjusts countermeasures based on that emotional state, and provides appropriate personal information, allowing the driver to quickly address the problem.

[1147] Program processing and its explanation

[1148] 1. Receiving Inquiries

[1149] The user device (smartphone or in-car display) receives queries from the driver and sends them to the server using a voice recognition library (Google Speech-to-Text API) or a text input field. For example, the driver may ask by voice, "How should I get through this intersection?" and this information is sent to the server.

[1150] 2. Analysis of inquiry content

[1151] The server analyzes the query it receives, using natural language processing techniques (spaCy, NLTK) to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "intersection" and "road."

[1152] 3. Emotion analysis

[1153] The server uses an emotion engine to analyze the driver's emotions from their tone of voice and facial expressions. This involves using a face recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the driver's emotional state as "anxious."

[1154] 4. Searching for related information

[1155] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information), generates an SQL query to query the database, and obtains the necessary road, traffic, and location information.

[1156] 5. Solution generation and delivery

[1157] Based on the information acquired by the server, a generative artificial intelligence model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The query content, related information, and emotional information are input into the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "It is safe to turn right at this intersection." The generated solution and related information are displayed on the in-car display or smartphone, and are also provided to the driver using voice synthesis (Google Text-to-Speech API).

[1158] Specific examples

[1159] When a driver makes a voice inquiry such as "I want to know how to get through this intersection on this road," the specific processing steps are as follows:

[1160] 1. The user device receives the query and sends it to the server.

[1161] 2. The server analyzes the inquiry and extracts important keywords.

[1162] 3. The server uses an emotion engine to analyze the driver's emotions from their voice and facial expressions, and identifies an emotion such as "anxiety."

[1163] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[1164] 5. The generative AI model generates a solution based on the acquired information and adjusts the solution with reference to emotional information.

[1165] 6. The server sends the generated solution and related information to the user device.

[1166] 7. The user device displays the solution and related information to the driver and provides voice guidance: "It is safe to turn right at this intersection. Also, if you turn left at the next traffic light, you will reach your destination."

[1167] Example prompts to input to the generative AI model

[1168] Prompt: "Generate an appropriate response to a driver who is worried about how to get through this intersection. The driver's current emotion is anxiety."

[1169] Example response: "It's safe to turn right at this intersection, and turning left at the next light will get you to your destination."

[1170] This makes it possible to quickly provide optimal solutions to technical questions or problems that drivers may have while driving, depending on their emotional state.

[1171] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1172] Step 1:

[1173] The user inputs a query by voice or text (voice input or text input). The user's device (smartphone or in-car display) receives this query and sends it to the server. The Google Speech-to-Text API is used for input. For example, if the user voices a query such as "I want to know how to get through this intersection," this voice is converted into text and sent to the server.

[1174] Step 2:

[1175] The server analyzes the query it receives. It uses natural language processing techniques (spaCy, NLTK) to tokenize the text and extract important keywords. The input is the query text, and the output is the extracted keywords. For example, it identifies keywords such as "intersection" and "road." Specifically, the analysis engine analyzes the query content and identifies the main keywords.

[1176] Step 3:

[1177] The server uses an emotion engine to analyze the driver's emotions. This uses a facial recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The input is the driver's tone of voice and facial expression data, and the output is the driver's emotional state. Specifically, the server analyzes the driver's tone of voice and facial expression to identify emotions such as "anxiety."

[1178] Step 4:

[1179] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information). It generates an SQL query and queries the database. The input is keywords and emotion information, and the output is related road information, traffic information, and location information. Specifically, the server retrieves the required data from the database.

[1180] Step 5:

[1181] Based on the information acquired by the server, a generative AI model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The input is the inquiry content, related information, and emotional information, and the output is the optimal solution. Specifically, the generative AI model generates a solution such as "It is safe to turn right at this intersection."

[1182] Step 6:

[1183] The server sends the generated solution and related information to the user device, which displays it and provides it audibly. The output is the solution text and speech synthesis data. Specifically, the generated solution is displayed on the in-vehicle display, and a voice guides the user, saying, "It is safe to turn right at this intersection."

[1184] In this way, the system processes the data input at each step, performing appropriate data processing and calculations to provide the user with the optimal solution.

[1185] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1186] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1187] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1188] [Fourth embodiment]

[1189] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1190] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1191] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1192] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1193] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1194] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1195] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1196] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1197] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1198] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1199] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1200] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1201] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1202] The present invention is a system that provides appropriate solutions to unclear points and technical issues in corporate projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[1203] System Overview

[1204] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes it. Based on the analysis results, it searches the database for relevant information and uses a generative AI model to generate the optimal solution. Furthermore, by providing the user with appropriate employee information as needed, the system allows the user to quickly address the problem.

[1205] Program processing and its explanation

[1206] 1. Receiving Inquiries

[1207] Subject: Device

[1208] The terminal receives inquiries from users and sends them to the server. For example, if a user types into the terminal, "I don't understand the requirements for XX in project A," this information is sent to the server.

[1209] 2. Analysis of inquiry content

[1210] Subject: Server

[1211] The server analyzes the received inquiry and uses natural language processing (NLP) technology to extract important keywords from the inquiry and understand the intent of the inquiry. For example, it extracts information about "Project A," "Requirements," and "XX."

[1212] 3. Search for related information

[1213] Subject: Server

[1214] The server searches the database based on the analysis results, and retrieves relevant information from the database, which contains project information, employee information, and department information.

[1215] 4. Applying generative AI models

[1216] Subject: Server

[1217] Based on the acquired information, the server inputs the generative AI model to generate a solution. The generative AI model considers the query and related information to create a specific and feasible solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[1218] 5. Providing solutions and employee information

[1219] Subject: Server

[1220] The server sends the generated solution to the user's terminal and also provides appropriate employee information based on the solution (e.g., "Employee Y is knowledgeable about this problem"), allowing the user to know not only the solution but also who to contact for further information.

[1221] 6. User Display

[1222] Subject: Device

[1223] The terminal displays the solution and employee information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "The specific XX requirement for Project A is.... If you have any further questions, please contact employee Y."

[1224] Specific examples

[1225] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[1226] 1. The device receives the query and sends it to the server.

[1227] 2. The server analyzes the inquiry and extracts important keywords.

[1228] 3. The server searches and retrieves the relevant information from the database.

[1229] 4. The generative AI model generates a solution based on the acquired information.

[1230] 5. The server generates a solution and sends the relevant employee information to the terminal.

[1231] 6. The device displays the solution and employee information to the user, for example, "The information about XX is... Contact employee Y for more information."

[1232] As described above, the system of the present invention responds to user inquiries quickly and accurately, provides appropriate solutions, and, if necessary, connects users to appropriate staff members, thereby improving the efficiency of problem solving.

[1233] The processing flow will be explained below.

[1234] Step 1:

[1235] Subject: User

[1236] The user inputs a query to the system from a terminal. For example, the user inputs "I would like to know about XX requirements for Project A" and presses the send button.

[1237] Step 2:

[1238] Subject: Terminal

[1239] The terminal receives the user's query and transmits the query data to the server, specifically, the terminal captures the user's input text and transfers it to the server via the network.

[1240] Step 3:

[1241] Subject: Server

[1242] The server receives the query sent from the terminal, stores the received text data, and prepares for the next analysis process.

[1243] Step 4:

[1244] Subject: Server

[1245] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text and extract important keywords, such as "Project A," "Requirements," and "XX."

[1246] Step 5:

[1247] Subject: Server

[1248] The server searches the database for relevant information based on the analyzed keywords, generates SQL queries to query the database, and retrieves the required project, employee, and department information.

[1249] Step 6:

[1250] Subject: Server

[1251] The server applies a generative AI model to generate a solution based on the information retrieved from the database. The query and related information are input to the generative AI model, which then outputs the optimal solution.

[1252] Step 7:

[1253] Subject: Server

[1254] The server transmits the generated solution and associated employee information to the terminal to provide the information to the user, including, for example, the solution text and contact information for the appropriate employee.

[1255] Step 8:

[1256] Subject: Terminal

[1257] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[1258] Step 9:

[1259] Subject: User

[1260] The user checks the solution displayed on the terminal and, if necessary, makes further inquiries using the provided employee information, taking an action such as "send an email to employee Y."

[1261] Example 1

[1262] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1263] When technical questions or issues arise in corporate projects, there is a lack of systems that can provide solutions quickly and accurately. It is particularly difficult to quickly find the right information for complex issues involving multiple projects or departments. As a result, problem resolution can take a long time, leading to reduced work efficiency.

[1264] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1265] In this invention, the server includes a means for analyzing the received inquiry, a means for searching a database for related information based on the analysis results, and a means for generating a solution by applying a generative AI model using the search results, thereby enabling the server to provide a quick and accurate solution to the user's technical questions and problems, and also provide appropriate contact information as needed.

[1266] A "user terminal" is an electronic device that allows a user to input and send an inquiry.

[1267] An "inquiry" is a technical question or problem that a user needs to solve.

[1268] A "server" is a central processing unit for analyzing queries received from user terminals and generating and providing solutions.

[1269] "Analysis" is the process of using natural language processing technology to understand the content of the received inquiry and extract important keywords and phrases.

[1270] A "database" is a collection of information that holds related information such as project information, person information, department information, etc., and stores it in a searchable manner.

[1271] A "generative AI model" is a set of algorithms or programs that generate solutions based on the query and related information.

[1272] A "solution" is a specific, actionable answer or instruction to a user's technical question or problem.

[1273] "Contact information" refers to contact and related information for employees who are knowledgeable about specific technical questions or issues.

[1274] "Display" is the process of outputting information sent from the server to the terminal in a format that the user can visually confirm.

[1275] This invention is a system that provides appropriate solutions to technical questions and issues in in-house projects. This system is basically composed of user terminals, servers, and a network infrastructure that links them together.

[1276] When a user has a technical question or concern, they use their device to send the inquiry to the system, for example, "I don't understand the requirements for XX in Project A." The device converts this information into the appropriate format and sends it to the server using an HTTPS request.

[1277] The server analyzes the received query. First, it parses the received data to extract text data and then uses natural language processing (NLP) technology. Specifically, it uses an NLP engine such as SpaCy or NLTK to linguistically analyze the query content and extract important keywords and phrases. For example, keywords such as "Project A," "Requirements," and "XX" are extracted.

[1278] After the analysis is complete, the server searches the database based on the analysis results. The database stores project information, person in charge information, department information, etc. The server generates an SQL query to search for entries that match the analysis results and retrieve the relevant information.

[1279] Next, the server inputs the acquired information into a generative AI model (e.g., GPT-4) to generate a solution. The input prompt for the generative AI model is constructed based on the analysis results and information acquired from the database. For example, a prompt in the form "Please tell me about the XX requirement for Project A" is created. The generative AI model then generates a specific solution such as "The XX requirement for Project A is...".

[1280] The generated solution is sent to the user's device by the server. The solution is provided along with the analysis results and appropriate person-in-charge information obtained from the database. For example, information such as "Person in charge Y is knowledgeable about the requirements of XX" may be added.

[1281] The terminal displays the solution and contact information sent from the server to the user. It parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it displays, "The XX requirement for Project A is.... For detailed questions, please contact Contact Y."

[1282] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process it as follows:

[1283] 1. The device receives the query and sends it to the server.

[1284] 2. The server analyzes the query and extracts important keywords.

[1285] 3. The server searches and retrieves the relevant information from the database.

[1286] 4. The generative AI model generates a solution based on the acquired information.

[1287] 5. The server sends the generated solution and related person information to the terminal.

[1288] 6. The device displays the solution and contact information to the user. For example, it displays "Information about XX is... Contact contact Y for more information."

[1289] As described above, this system responds quickly and accurately to users' technical questions and issues, providing appropriate solutions. It also aims to improve the efficiency of problem-solving by connecting users to the appropriate person in charge as needed.

[1290] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1291] Step 1: Receiving an inquiry

[1292] Input: User-supplied query text

[1293] Output: Query data sent to the server

[1294] Specific operation: A user types "I don't understand the requirements for XX in Project A" into the terminal. The terminal receives this input, structures the data including the query text, and sends it to the server as an HTTPS request. At this time, metadata such as the user ID and timestamp are also sent.

[1295] Step 2: Analyzing the inquiry

[1296] Input: Inquiry data sent from the terminal

[1297] Output: Parsed keywords and query intent

[1298] How it works: The server parses the received query data and extracts the text. It then uses an NLP engine (such as SpaCy or NLTK) to analyze the text and extract important keywords and phrases. The extracted keywords are "Project A," "Requirements," and "XX."

[1299] Step 3: Find related information

[1300] Input: Parsed keywords and query intent

[1301] Output: Relevant information retrieved from the database

[1302] Specific operation: Based on the analysis results, the server generates an SQL query to the database to search for relevant information. The database stores project information, person in charge information, and department information. For example, the database is searched using the condition "XX requirements for project A" to retrieve the relevant information.

[1303] Step 4: Applying the generative AI model

[1304] Input: relevant information retrieved from the database and the query intent

[1305] Output: The solution generated by the generative AI model

[1306] Specific operation: Based on the acquired information, the server constructs an input prompt for the generative AI model (e.g., GPT-4). For example, it creates a prompt in the form of "Please tell me about the XX requirement of Project A." The generative AI model generates a solution to this prompt and outputs a detailed answer such as "The XX requirement of Project A is..."

[1307] Step 5: Provide solutions and contact information

[1308] Input: Solutions generated by the generative AI model and appropriate personnel information

[1309] Output: Solution and contact information sent to the terminal

[1310] Specific operation: The server receives the generated solution and adds appropriate person information based on it. For example, it adds information such as "Person Y is knowledgeable about the requirements of XX." This information is sent to the user's terminal as an HTTPS response.

[1311] Step 6: Display to the user

[1312] Input: Solution and contact information sent from the server

[1313] Output: Solution and contact information displayed to the user

[1314] Specific operation: The device parses the received data, converts it into a format that is easy for the user to understand, and displays it on the screen. For example, it might display, "Specifically, the XX requirement for Project A is.... If you have any further questions, please contact Person Y." The user can then confirm the displayed information and begin solving the problem.

[1315] Through these steps, users can quickly and accurately obtain specific solutions to technical questions and issues, as well as information on the appropriate person in charge.

[1316] (Application example 1)

[1317] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1318] In modern factories, when complex machine troubles or technical questions arise, they need to be resolved quickly. However, it is difficult for on-site workers to immediately access the appropriate information and solutions. Furthermore, the time required to search for relevant information leads to reduced productivity. Another issue is the difficulty of quickly obtaining information on the appropriate engineers.

[1319] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1320] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis result, means for applying a generative AI model using the search result to generate a solution, means for providing the generated solution to the user, means for providing appropriate staff information based on the solution, and means including smart glasses for converting the received inquiry from voice input into text data. This enables on-site workers to easily make inquiries by voice input and quickly obtain appropriate solutions and staff information.

[1321] A "user terminal" is a device used by a user, including a smartphone, tablet, PC, etc.

[1322] An "inquiry" is data that a user sends to the system regarding technical questions or concerns.

[1323] "Analysis" is the process of understanding the content of the received inquiry and extracting important keywords and their intent.

[1324] A "database" is a system for systematically storing information and making it searchable.

[1325] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions based on input information.

[1326] A "solution" is a specific response or answer provided to a user's inquiry.

[1327] "Contact Information" is contact information for employees or technicians with detailed knowledge of specific technical issues.

[1328] "Smart glasses" are wearable devices that use AR (augmented reality) technology to allow users to obtain information visually.

[1329] "Voice input" is an input method in which a device recognizes words spoken by a user and processes them as data.

[1330] "Text data" refers to data that has been converted from voice input into text information.

[1331] This invention is a system that allows users with technical questions or concerns to obtain solutions in real time using smart glasses. The system is basically composed of a user terminal (including smart glasses), a server, and a network infrastructure that links them.

[1332] System Overview

[1333] 1. Receiving Inquiries

[1334] The terminal, i.e., smart glasses, receives voice input from the user and converts the voice into text data. For example, if the user says, "The robot arm on line 2 is not working," the voice data is converted into text data.

[1335] 2. Analysis of inquiry content

[1336] The server analyzes the received text data. It uses natural language processing (NLP) techniques, such as spaCy or NLTK, to extract important keywords from the query and understand its intent. In this example, keywords such as "line 2," "robot arm," and "not working" are extracted.

[1337] 3. Search for related information

[1338] The server searches a database based on the analysis results, which contains machine manuals, error logs, and personnel information, and retrieves relevant information from these.

[1339] 4. Applying generative AI models

[1340] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific, actionable solution. For example, it generates a detailed explanation such as, "If the robot arm on Line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge."

[1341] 5. Providing solutions and contact information

[1342] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the solution, such as "Person Y is knowledgeable about this problem."

[1343] 6. User Display

[1344] The terminal, i.e., the smart glasses, displays the solution and contact information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it might say, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact contact Y."

[1345] Specific examples

[1346] For example, if a user queries "The robot arm on line 2 is not working," the system will process it as follows:

[1347] 1. The device receives voice input, converts it into text data, and sends it to the server.

[1348] 2. The server analyzes the received text data and extracts important keywords.

[1349] 3. The server searches and retrieves the relevant information from the database.

[1350] 4. The generative AI model generates a solution based on the acquired information.

[1351] 5. The server generates a solution and sends it to the terminal along with relevant contact information.

[1352] 6. The terminal will display the solution and contact information to the user, for example, "If the robot arm on line 2 does not work, please check the power connection. Contact contact Y for more information."

[1353] Example prompt sentence:

[1354] User enquiry:

[1355] "What should I do if the robot arm on Line 2 doesn't work?"

[1356] Related information:

[1357] Line 2 Manual

[1358] Error Log

[1359] Contact Information

[1360] Output of the generative AI model:

[1361] "If the robot arm on line 2 doesn't work, first check the power connection. If that doesn't fix it, contact person Y."

[1362] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1363] Step 1:

[1364] The smart glasses receive voice input and convert the voice data into text data. The voice input is converted into text format using the Google Speech-to-Text API. For example, if a user says, "The robot arm on line 2 is not working," the voice data is converted into text data that reads, "The robot arm on line 2 is not working." The input is voice data, and the output is text data.

[1365] Step 2:

[1366] The text data is sent to the server. The smart glasses send the text data to the server through the factory network. The text data is input to the server, and the received text data is used for the next analysis process. The input is text data, and the output is transmission to the server.

[1367] Step 3:

[1368] The server analyzes the received text data. Using a natural language processing (NLP) library (such as spaCy or NLTK), it extracts important keywords from the query and understands its intent. Specifically, keywords such as "line 2," "robot arm," and "not working" are extracted. The input is the text data, and the output is the important keywords.

[1369] Step 4:

[1370] The server searches the database based on the analysis results. The database stores machine manuals, error logs, and staff information, and searches these to obtain relevant information. For example, the error log and staff information for the "robot arm on line 2" are extracted from the database. The input is important keywords, and the output is related information.

[1371] Step 5:

[1372] Based on the acquired information, the server inputs the information into a generative AI model (such as OpenAI's GPT-3) to generate a solution. The generative AI model takes into account the query and related information to create a specific and actionable solution. For example, it might generate a solution such as, "If the robot arm on line 2 is not working, first check the power connection. If that doesn't work, contact the person in charge." The input is the related information, and the output is the solution.

[1373] Step 6:

[1374] The server sends the generated solution to the user's device, i.e., the smart glasses. It also provides appropriate person information based on the generated solution. For example, it provides information such as "Person Y is knowledgeable about this problem." The input is the solution and person information, and the output is transmission completion.

[1375] Step 7:

[1376] The terminal, i.e., smart glasses, displays the solution and person in charge information sent from the server to the user. The user can then begin solving the problem based on the displayed information. For example, it displays, "If the robot arm on line 2 does not work, please check the power connection. For more detailed questions, please contact person Y." The input is the solution and person in charge information, and the output is the display to the user.

[1377] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1378] This invention relates to a system that provides appropriate solutions to unclear points and technical issues in corporate projects, and also combines it with an emotion engine that recognizes the user's emotions. The system is basically composed of a user terminal, a server, an emotion engine, and a network infrastructure that links them together.

[1379] System Overview

[1380] When a user has a technical question or concern, they use their device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and uses a generative AI model to generate the optimal solution. It also uses an emotion engine to analyze the user's emotions, adjusts the response based on that emotional state, and provides appropriate employee information, allowing the user to quickly address the problem.

[1381] Program processing and its explanation

[1382] 1. Receiving Inquiries

[1383] Subject: Device

[1384] The terminal receives inquiries from the user and sends them to the server. For example, if the user enters "I would like to know about XX requirements for project A," this information is sent to the server.

[1385] 2. Analysis of inquiry content

[1386] Subject: Server

[1387] The server analyzes the received query and uses natural language processing (NLP) techniques to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "Project A," "Requirements," and "XX."

[1388] 3. Emotion analysis using an emotion engine

[1389] Subject: Server

[1390] The server uses an emotion engine to analyze the user's emotion from the content of the inquiry. The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the user's emotional state at that time, such as "the user is feeling anxious."

[1391] 4. Searching for related information

[1392] Subject: Server

[1393] The server searches the database based on the analysis results (keywords and sentiment information), generates an SQL query to query the database, and retrieves the necessary project, employee, and department information.

[1394] 5. Applying generative AI models

[1395] Subject: Server

[1396] Based on the information acquired by the server, a generative AI model is applied to generate a solution. The query content, related information, and emotional information are input to the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "The XX requirement for Project A is..."

[1397] 6. Providing solutions and employee information

[1398] Subject: Server

[1399] The server sends the generated solution and relevant employee information to the terminal to provide the user with the solution, including the solution text and the contact information of the appropriate employee. The server also adjusts the priority and response method of the inquiry based on the emotion information.

[1400] 7. User Display

[1401] Subject: Device

[1402] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can confirm the solution and providing information to contact an employee if necessary.

[1403] Specific examples

[1404] For example, if a user makes a query such as "I would like more information about XX in Project A," the system will process the query as follows:

[1405] 1. The device receives the query and sends it to the server.

[1406] 2. The server analyzes the inquiry and extracts important keywords.

[1407] 3. The server uses an emotion engine to analyze the user's emotions and identify an emotion such as "anxiety."

[1408] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[1409] 5. The generative AI model generates a solution based on the acquired information and adjusts it based on emotional information.

[1410] 6. The server sends the generated solution and related employee information to the terminal.

[1411] 7. The device displays the solution and employee information to the user, for example, "The information about XX is... If you have further questions, please contact employee Y. If you have any concerns, we are happy to provide additional support."

[1412] As described above, the system of the present invention not only responds quickly and accurately to user inquiries, but also increases user satisfaction by providing tailored solutions that take into account the user's feelings and appropriate employee information.

[1413] The processing flow will be explained below.

[1414] Step 1:

[1415] Subject: User

[1416] The user inputs a query into the system from a terminal. For example, "I would like to know more about the XX requirement for Project A," and presses the send button.

[1417] Step 2:

[1418] Subject: Terminal

[1419] The terminal receives the user's inquiry and transmits the inquiry data to the server. Specifically, the terminal captures the input text data and transfers it to the server via the network.

[1420] Step 3:

[1421] Subject: Server

[1422] The server receives the query sent from the terminal, stores the query content, and prepares to pass it to the next processing step.

[1423] Step 4:

[1424] Subject: Server

[1425] The server analyzes the query received using natural language processing (NLP) technology. Specifically, it tokenizes the text and extracts important keywords (e.g., "Project A," "Requirements," "XX"), and also performs semantic analysis to understand the intent of the query.

[1426] Step 5:

[1427] Subject: Server

[1428] The server uses the results of the NLP analysis to analyze the user's emotions using an emotion engine. Specifically, it detects emotions such as "anxiety," "confusion," and "urgency" from the inquiry content and stores the results in a database.

[1429] Step 6:

[1430] Subject: Server

[1431] The server searches the database based on the analysis results and sentiment information. Specifically, it generates SQL queries to retrieve project information, employee information, and department information from the database. For example, it searches for materials related to "Project A" or "XX."

[1432] Step 7:

[1433] Subject: Server

[1434] The server applies a generative AI model based on the information it obtains to generate a solution. The generative AI model takes the inquiry, related information, and emotional information as input and outputs the optimal solution. For example, it generates a specific solution such as "The XX requirement for Project A is..."

[1435] Step 8:

[1436] Subject: Server

[1437] The server selects the appropriate employee information based on the generated solution and sentiment information. Based on the analysis results, it extracts the contact information of the most suitable employee and sets the priority of the inquiry.

[1438] Step 9:

[1439] Subject: Server

[1440] The server finally sends the generated solution and the selected employee information to the user terminal. For example, the solution text and "Employee Y's contact information" are packaged and sent.

[1441] Step 10:

[1442] Subject: Terminal

[1443] The device receives the information sent from the server and displays it in a user interface, displaying text on the screen so the user can see the solution and providing information to contact an employee if necessary.

[1444] Step 11:

[1445] Subject: User

[1446] The user checks the solution displayed on the device and, if necessary, contacts the designated employee. For example, they send an email to employee Y using the provided contact information.

[1447] The above is a specific flow of processing in the system of the present invention. By performing detailed operations at each step, the user can receive a quick and accurate solution. Furthermore, by taking emotional information into consideration, the response to the user can be more appropriate and satisfying.

[1448] Example 2

[1449] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1450] Conventional technical support systems have not only difficulty in providing appropriate solutions to users' technical questions and uncertainties quickly and accurately, but also have the problem of not being able to respond in a way that takes into account the user's emotional state. In particular, if a user feels anxious or confused, ignoring their emotions can reduce the efficiency of problem-solving and lead to low user satisfaction.

[1451] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1452] In this invention, the server includes means for receiving an inquiry from a user terminal, means for analyzing the content of the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative AI model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate employee information based on the solution, means including an emotion engine for analyzing the user's emotions, and means for adjusting the solution and employee information based on the emotion information. This makes it possible to not only quickly and accurately respond to the user's technical questions and uncertainties, but also to provide adjusted solutions and appropriate employee information that take the user's emotional state into consideration.

[1453] A "user terminal" is an information input device that allows a user to input technical questions or uncertainties.

[1454] An "inquiry" is information about technical questions or uncertainties sent from a user terminal.

[1455] The "receiving means" is a function that allows the server to receive an inquiry sent from a user terminal.

[1456] "Means of analysis" refers to the function of analyzing the content of the received inquiry using natural language processing technology and extracting important keywords and intent.

[1457] "Natural language processing technology" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[1458] A "database" is a searchable collection of related information such as project information, employee information, and department information.

[1459] "Search means" is a function for retrieving related information from a database based on the analysis results.

[1460] A "generative AI model" is a machine learning model that generates optimal solutions based on input information.

[1461] The "means for generating" is the function for applying a generative AI model to generate a solution to a query.

[1462] A "solution" is a specific answer or countermeasure to a user's technical questions or uncertainties.

[1463] "Means of providing" refers to the function for delivering the generated solution to the user.

[1464] "Employee Information" means contact information and expertise of appropriate employees who can respond to inquiries.

[1465] An "emotion engine" is a technology for analyzing a user's emotional state from the content of their inquiry.

[1466] "Emotion information" is the result of the user's emotional state analyzed by the emotion engine.

[1467] "Adjustment means" is a function for optimizing solutions and employee information based on emotional information.

[1468] This is a system that provides quick and accurate solutions to technical issues and uncertainties users have in technical projects carried out within a company. The system consists of a user terminal, a server, an emotion engine, and a network infrastructure that links these elements.

[1469] A user terminal is a device that allows a user to input an inquiry, such as a computer or smartphone. A user inputs technical questions or concerns and submits them as an inquiry.

[1470] The server is a computer system that performs the following processes:

[1471] 1. Receiving means: Receives a query from a user terminal as an HTTP request.

[1472] 2. Analysis: The received query content is analyzed using natural language processing techniques, such as using a Python NLP library (NLTK or spaCy) to tokenize the text and extract important keywords.

[1473] 3. Search method: Based on the analysis results, search the database for relevant information. Generate SQL queries to retrieve project information, employee information, department information, etc.

[1474] 4. Generation method: The acquired information is input into a generative AI model (e.g., GPT-3) to generate an optimal solution. The prompt text is something like, "Please explain in detail the XX requirement for Project A."

[1475] 5. Means of provision: The generated solution and related employee information are sent to the user terminal as an HTTP response.

[1476] 6. Emotion Engine: Analyzes emotions from the user's inquiry. For example, it uses a Python emotion analysis library (TextBlob or VADER) to identify the user's emotion of "anxiety."

[1477] 7. Coordination: Optimize and provide solutions and employee information based on emotional information.

[1478] Here are some examples:

[1479] A user sends a query from their device, such as "I want to know about the XX requirement for Project A." The device sends this query as an HTTP request to the server. The server analyzes the received text and extracts keywords such as "Project A," "XX," and "requirements." It then uses an emotion engine to determine that the user's emotion is "anxiety."

[1480] The server searches the database based on keywords and sentiment information to retrieve relevant project information and the contact information of the responsible employee. Based on this information, the generative AI model is prompted with a prompt such as "Please explain in detail the XX requirement for Project A," and a detailed solution is generated.

[1481] Finally, the server sends the generated solution and related employee information to the user terminal as an HTTP response, and the user terminal displays this on its user interface. For example, information such as "The information about XX is.... If you have any further questions, please contact employee Y. If you have any concerns, we will provide additional support" is displayed on the screen.

[1482] As described above, the system of the present invention not only responds quickly and accurately to users' technical questions and uncertainties, but also increases user satisfaction by providing tailored solutions that take into account the user's emotional state and appropriate employee information.

[1483] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1484] Step 1:

[1485] Input: The user enters technical questions or concerns into the user's terminal and submits them as an inquiry.

[1486] Specific operation: The user enters "I would like to know about the XX requirements for Project A" into the device's input screen and presses the send button.

[1487] Output: The device sends the query to the server as an HTTP request.

[1488] Step 2:

[1489] Input: The server receives the HTTP request sent from the device.

[1490] Specific operation: The server analyzes the HTTP request and obtains the inquiry content.

[1491] Output: The query is sent to the server as text.

[1492] Step 3:

[1493] Input: The server analyzes the query received using natural language processing (NLP) techniques.

[1494] What it does: The server uses a Python NLP library (e.g., NLTK, spaCy) to tokenize the text and extract important keywords (e.g., "Project A," "Requirements," "XX").

[1495] Output: The extracted keywords are passed to the next processing step.

[1496] Step 4:

[1497] Input: The server passes the extracted keywords to the emotion engine.

[1498] Specific operation: The server uses an emotion analysis library (e.g., TextBlob, VADER) to analyze the user's emotional state from the query content and identify the emotion, for example, "anxiety."

[1499] Output: The sentiment analysis results are returned to the server and stored in a database.

[1500] Step 5:

[1501] Input: The server searches the database based on keywords and sentiment information.

[1502] Specific operation: The server generates SQL queries and queries the database for project information, employee information, department information, etc.

[1503] Output: The necessary information (project description, contact information of the responsible employee, etc.) is passed to the server.

[1504] Step 6:

[1505] Input: The information acquired by the server is input into the generative AI model.

[1506] Specific operation: The server inputs a prompt sentence to the generative AI model (e.g., GPT-3) such as "Please explain in detail the XX requirement for project A," and the generative AI model generates the optimal solution.

[1507] Output: The generated solution is returned to the server.

[1508] Step 7:

[1509] Input: The server compiles the generated solution and related employee information.

[1510] What happens: The server creates a single HTTP response with the solution text and appropriate employee information (e.g., contact details), and adjusts the priority and response based on the sentiment information.

[1511] Output: An HTTP response is formed and sent to the user's device.

[1512] Step 8:

[1513] Input: The device receives the HTTP response sent by the server.

[1514] Specific behavior: The device displays the solution and employee information in the user interface.

[1515] Output: The user can see the solution and employee information displayed on the screen.

[1516] Specifically, the device displays information such as "Information about XX is... If you have any detailed questions, please contact employee Y. If you have any concerns, we will provide additional support" on the screen, and the user confirms it.

[1517] (Application example 2)

[1518] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1519] There is a need for a means to quickly and accurately resolve the technical questions and uncertainties that drivers of autonomous vehicles face while driving, as well as the emotional stress that accompanies them. In particular, it is important to improve driver satisfaction and safety by providing optimal solutions according to the driver's emotional state.

[1520] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving an inquiry from a user device, means for analyzing the received inquiry, means for searching a database for related information based on the analysis results, means for applying a generative artificial intelligence model using the search results to generate a solution, means for providing the generated solution to the user, means for providing appropriate personal information based on the solution, means for receiving an inquiry via voice input or text input, means for applying a generative artificial intelligence model using the analysis results and emotion analysis to generate a solution, means for providing the generated solution and related information on a display device and by voice, and means for analyzing the driver's tone of voice and facial expression to identify the driver's emotional state. This makes it possible to quickly provide optimal solutions to technical questions and problems that the driver has while driving, based on their emotional state.

[1521] "User equipment" refers to devices such as smartphones or on-board displays inside or outside an autonomous vehicle, and is used for inquiries from and information provision to the driver.

[1522] A "generative artificial intelligence model" is a generative model that uses artificial intelligence to analyze the content of an inquiry and related information and generate an appropriate solution.

[1523] "Means for analyzing the content of an inquiry" refers to the process of tokenizing the content of an inquiry received from a user using natural language processing technology, extracting keywords, and understanding the intent.

[1524] A "database" is a collection of information that the system uses to search for relevant information, and includes road information, traffic information, location information, and the like.

[1525] "Emotion analysis" refers to technology for identifying a driver's emotional state from their tone of voice and facial expressions, and is the process of analyzing the anxiety and stress the driver is experiencing.

[1526] A "display device" is a device for providing generated solutions and related information to a user, including an in-car display or a smartphone screen.

[1527] "Voice input" refers to a means by which a user can make queries to a system through voice, and includes technology that converts received voice data into text.

[1528] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning, and is used here for analyzing inquiry content and preprocessing for generative AI models.

[1529] "Means of generating a solution" refers to the process of using the generated AI model to generate the optimal answer based on the information obtained from the database and the query content.

[1530] "Personal information" refers to information such as contact details and job titles of individuals appropriate to address specific challenges or questions, including information provided by the system to the driver.

[1531] System Overview

[1532] This invention is a system whose basic components are a user device, a server, an emotion engine, and a network infrastructure that links these together. When a driver has a technical question or concern, they use the user device to send an inquiry to the system. The system receives the inquiry on the server and analyzes its contents. Based on the analysis results, it searches a database for relevant information and generates an optimal solution using a generative artificial intelligence model. Furthermore, it uses the emotion engine to analyze the driver's emotions, adjusts countermeasures based on that emotional state, and provides appropriate personal information, allowing the driver to quickly address the problem.

[1533] Program processing and its explanation

[1534] 1. Receiving Inquiries

[1535] The user device (smartphone or in-car display) receives queries from the driver and sends them to the server using a voice recognition library (Google Speech-to-Text API) or a text input field. For example, the driver may ask by voice, "How should I get through this intersection?" and this information is sent to the server.

[1536] 2. Analysis of inquiry content

[1537] The server analyzes the query it receives, using natural language processing techniques (spaCy, NLTK) to tokenize the text, extract important keywords, and understand the intent of the query. For example, it identifies keywords such as "intersection" and "road."

[1538] 3. Emotion analysis

[1539] The server uses an emotion engine to analyze the driver's emotions from their tone of voice and facial expressions. This involves using a face recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The analysis results are stored in a database and used as a reference for generating solutions. For example, the server determines the driver's emotional state as "anxious."

[1540] 4. Searching for related information

[1541] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information), generates an SQL query to query the database, and obtains the necessary road, traffic, and location information.

[1542] 5. Solution generation and delivery

[1543] Based on the information acquired by the server, a generative artificial intelligence model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The query content, related information, and emotional information are input into the generative AI model, and the model outputs the optimal solution. For example, it generates a detailed explanation such as "It is safe to turn right at this intersection." The generated solution and related information are displayed on the in-car display or smartphone, and are also provided to the driver using voice synthesis (Google Text-to-Speech API).

[1544] Specific examples

[1545] When a driver makes a voice inquiry such as "I want to know how to get through this intersection on this road," the specific processing steps are as follows:

[1546] 1. The user device receives the query and sends it to the server.

[1547] 2. The server analyzes the inquiry and extracts important keywords.

[1548] 3. The server uses an emotion engine to analyze the driver's emotions from their voice and facial expressions, and identifies an emotion such as "anxiety."

[1549] 4. The server searches the database based on keywords and emotion information to obtain relevant information.

[1550] 5. The generative AI model generates a solution based on the acquired information and adjusts the solution with reference to emotional information.

[1551] 6. The server sends the generated solution and related information to the user device.

[1552] 7. The user device displays the solution and related information to the driver and provides voice guidance: "It is safe to turn right at this intersection. Also, if you turn left at the next traffic light, you will reach your destination."

[1553] Example prompts to input to the generative AI model

[1554] Prompt: "Generate an appropriate response to a driver who is worried about how to get through this intersection. The driver's current emotion is anxiety."

[1555] Example response: "It's safe to turn right at this intersection, and turning left at the next light will get you to your destination."

[1556] This makes it possible to quickly provide optimal solutions to technical questions or problems that drivers may have while driving, depending on their emotional state.

[1557] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1558] Step 1:

[1559] The user inputs a query by voice or text (voice input or text input). The user's device (smartphone or in-car display) receives this query and sends it to the server. The Google Speech-to-Text API is used for input. For example, if the user voices a query such as "I want to know how to get through this intersection," this voice is converted into text and sent to the server.

[1560] Step 2:

[1561] The server analyzes the query it receives. It uses natural language processing techniques (spaCy, NLTK) to tokenize the text and extract important keywords. The input is the query text, and the output is the extracted keywords. For example, it identifies keywords such as "intersection" and "road." Specifically, the analysis engine analyzes the query content and identifies the main keywords.

[1562] Step 3:

[1563] The server uses an emotion engine to analyze the driver's emotions. This uses a facial recognition library (OpenCV + Dlib) and an emotion recognition model (emotion analysis with the BERT model using the Transformers library). The input is the driver's tone of voice and facial expression data, and the output is the driver's emotional state. Specifically, the server analyzes the driver's tone of voice and facial expression to identify emotions such as "anxiety."

[1564] Step 4:

[1565] The server searches a database (PostgreSQL) based on the analysis results (keywords and emotion information). It generates an SQL query and queries the database. The input is keywords and emotion information, and the output is related road information, traffic information, and location information. Specifically, the server retrieves the required data from the database.

[1566] Step 5:

[1567] Based on the information acquired by the server, a generative AI model (large-scale language model such as GPT-4, OpenAI API) is applied to generate a solution. The input is the inquiry content, related information, and emotional information, and the output is the optimal solution. Specifically, the generative AI model generates a solution such as "It is safe to turn right at this intersection."

[1568] Step 6:

[1569] The server sends the generated solution and related information to the user device, which displays it and provides it audibly. The output is the solution text and speech synthesis data. Specifically, the generated solution is displayed on the in-vehicle display, and a voice guides the user, saying, "It is safe to turn right at this intersection."

[1570] In this way, the system processes the data input at each step, performing appropriate data processing and calculations to provide the user with the optimal solution.

[1571] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1572] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1573] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1574] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1575] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1576] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1577] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1578] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1579] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1580] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1581] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1582] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1583] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1584] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1585] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1586] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1587] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1588] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1589] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1590] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1591] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1592] The following is further disclosed regarding the above embodiment.

[1593] (Claim 1)

[1594] means for receiving an inquiry from a user terminal;

[1595] means for analyzing the received inquiry;

[1596] A means for searching a database for related information based on the analysis results;

[1597] a means for applying a generative AI model using the search results to generate a solution; and

[1598] a means for providing the generated solution to a user;

[1599] a means for providing appropriate employee information based on said solution;

[1600] A system including:

[1601] (Claim 2)

[1602] 10. The system of claim 1, wherein the generative AI model includes means for analyzing query content and generating solutions using natural language processing techniques.

[1603] (Claim 3)

[1604] 10. The system of claim 1, wherein the database includes means for maintaining and retrieving information including project information, employee information, and department information.

[1605] "Example 1"

[1606] (Claim 1)

[1607] means for receiving an inquiry from a user terminal;

[1608] means for analyzing the received inquiry;

[1609] A means for searching a database for related information based on the analysis results;

[1610] a means for applying a generative AI model using the search results to generate a solution; and

[1611] a means for providing the generated solution to a user;

[1612] a means for providing appropriate contact information based on the solution;

[1613] means for displaying solution and contact information to a user;

[1614] A system including:

[1615] (Claim 2)

[1616] 10. The system of claim 1, wherein the generative AI model includes means for analyzing query content and generating solutions using natural language processing techniques.

[1617] (Claim 3)

[1618] 10. The system of claim 1, wherein the database includes means for holding and searching information including project information, person information, and department information.

[1619] "Application Example 1"

[1620] (Claim 1)

[1621] means for receiving an inquiry from a user terminal;

[1622] means for analyzing the received inquiry;

[1623] A means for searching a database for related information based on the analysis results;

[1624] a means for applying a generative AI model using the search results to generate a solution; and

[1625] a means for providing the generated solution to a user;

[1626] a means for providing appropriate contact information based on the solution;

[1627] a means including smart glasses for converting a received query from voice input to text data;

[1628] A system including:

[1629] (Claim 2)

[1630] 10. The system of claim 1, wherein the generative AI model includes means for analyzing query content and generating solutions using natural language processing techniques.

[1631] (Claim 3)

[1632] 10. The system of claim 1, wherein the database includes means for maintaining and retrieving information including machine manuals, error logs, and personnel information.

[1633] "Example 2: Combining Emotion Engines"

[1634] (Claim 1)

[1635] means for receiving an inquiry from a user terminal;

[1636] means for analyzing the received inquiry;

[1637] A means for searching a database for related information based on the analysis results;

[1638] a means for applying a generative AI model using the search results to generate a solution; and

[1639] a means for providing the generated solution to a user;

[1640] a means for providing appropriate employee information based on said solution;

[1641] means including an emotion engine for analyzing the emotion of a user;

[1642] a means of adjusting solutions and employee information based on emotional information;

[1643] A system including:

[1644] (Claim 2)

[1645] 10. The system of claim 1, wherein the generative AI model includes means for analyzing query content and generating solutions using natural language processing techniques.

[1646] (Claim 3)

[1647] 10. The system of claim 1, wherein the database includes means for maintaining and retrieving information including project information, employee information, and department information.

[1648] "Application example 2 when combining emotion engines"

[1649] (Claim 1)

[1650] means for receiving a query from a user equipment;

[1651] means for analyzing the received inquiry;

[1652] A means for searching a database for related information based on the analysis results;

[1653] a means for applying a generative artificial intelligence model using the search results to generate a solution; and

[1654] a means for providing the generated solution to a user;

[1655] a means for providing appropriate person information based on the solution;

[1656] means for receiving a voice or text query;

[1657] a means for applying a generative artificial intelligence model using the analysis results and sentiment analysis to generate a solution;

[1658] means for providing the generated solution and related information on a display and by voice;

[1659] A means for analyzing the driver's tone of voice and facial expression to identify the driver's emotional state;

[1660] A system including:

[1661] (Claim 2)

[1662] 10. The system of claim 1, wherein the generative artificial intelligence model uses natural language processing techniques to analyze query content and generate solutions, and further includes means for generating tailored solutions based on sentiment analysis.

[1663] (Claim 3)

[1664] 10. The system of claim 1, wherein the database includes means for holding and retrieving information including road information, traffic information, and location information. [Explanation of symbols]

[1665] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving an inquiry from a user terminal; means for analyzing the received inquiry; A means for searching a database for related information based on the analysis results; a means for applying a generative AI model using the search results to generate a solution; and a means for providing the generated solution to a user; a means for providing appropriate employee information based on said solution; A system including:

2. The system of claim 1 , wherein the generative AI model includes means for analyzing query content and generating solutions using natural language processing techniques.

3. 2. The system of claim 1, wherein the database includes means for holding and retrieving information including project information, employee information, and department information.

Images