system

Abstract

We provide the system. [Solution] A means for receiving search queries entered by the user, A means for searching the database based on the aforementioned search query, A means for organizing the aforementioned search results and extracting contact information and related system information, A system including means for displaying the extracted information to the user in a list.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

[0003] [[ID=?]] [[ID=?]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, there are many terms and systems, and it is difficult for users to quickly find the inquiry destination, business system, and request system related to a specific term. As a result, the efficiency of business operations decreases, and there is a possibility of business delays and errors. The present invention aims to solve these problems and provide a system that allows users to easily obtain relevant information.

Means for Solving the Problems

[0005] It seems there are some tags with "?" in the original that need to be double-checked for their correct form or intended meaning. If they are just misformatted in the input, the above translation should be adjusted accordingly.The present invention solves the above problem with a system that includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for organizing the search results and extracting contact information and related system information, and means for displaying the extracted information to the user in a list. Furthermore, by including a function for sending the search query to a server in HTTP request format and a function for formatting the search results in JSON or XML format, the system realizes rapid and efficient information provision.

[0006] "User" refers to a person or entity that uses a system.

[0007] A "search query" refers to the input data that a user sends to the system for searching.

[0008] "Means" refers to the hardware or software implementation that a system uses to achieve its function.

[0009] A "database" refers to a system that organizes and stores information to facilitate searching and manipulation.

[0010] A "server" refers to a computer or program that provides services in response to requests from clients.

[0011] An "HTTP request" is a communication protocol used over the internet, and refers to a message sent by a client to a server requesting information.

[0012] JSON format is a lightweight data exchange format that allows humans and machines to easily read and write data.

[0013] XML is a markup language for structuring, storing, and exchanging data.

[0014] "Contact point" refers to the person to whom questions or requests for information regarding specific terms or issues should be addressed.

[0015] The "business system" refers to a computer system or software for managing and operating business processes and tasks.

[0016] The "request system" refers to a computer system or software for processing specific requests and demands.

[0017] "Sorting" refers to sorting the acquired information into a meaningful form and processing it for providing to the user.

[0018] "List display" refers to displaying the acquired information on the screen in a list format.

Brief Description of Drawings

[0019] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings. <000010​​​​​​​​​​​​​ In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] System Overview

[0041] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0042] Specific implementations of the system

[0043] 1. Initialization process

[0044] The server initializes the database. The database stores various business terms and related contact information, business systems, and request systems. This prepares the server for search processing.

[0045] 2. User actions

[0046] The terminal is set to wait for user input. The user operates the terminal and enters a specific business term as a search query. For example, they enter the term "login" and click the search button.

[0047] 3. Submitting a search query

[0048] The terminal receives the search query entered by the user and sends it to the server in HTTP request format. This allows the server to search the database based on that search query.

[0049] 4. Database search and result retrieval

[0050] The server searches the database based on the search query it receives. The server extracts information from the database about contacts, business systems, and request systems related to the relevant terms. The search results are formatted in JSON or XML and sent to the terminal.

[0051] 5. Summarizing and displaying the results

[0052] The terminal analyzes the search results received from the server and displays them to the user in a list. For example, it may display contact information related to "login" (e.g., support desk), business systems (e.g., user management system), request systems (e.g., password reset request system), etc.

[0053] 6. User Verification and Use

[0054] Users review the displayed list, contact the necessary contacts, and use relevant systems to carry out their tasks. This allows users to quickly and efficiently obtain the necessary information and perform their tasks smoothly.

[0055] Specific example

[0056] Example 1: Searching for the term "login"

[0057] 1. The user enters "login" into their device and clicks the search button.

[0058] 2. The terminal sends the input to the server in the form of an HTTP request.

[0059] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0060] 4. The server sends that information to the terminal in JSON format.

[0061] 5. Display a list of information received by the device.

[0062] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0063] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0064] The following describes the processing flow.

[0065] Step 1:

[0066] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0067] Step 2:

[0068] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction.

[0069] Step 3:

[0070] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button.

[0071] Step 4:

[0072] The terminal receives user input and sends that input as a search query in the form of an HTTP request to the server.

[0073] Step 5:

[0074] The server receives the HTTP request and parses the submitted search query. It then verifies that the parsed search query contains specific business terminology.

[0075] Step 6:

[0076] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[0077] Step 7:

[0078] The server extracts information, organizes it, and converts it into a format that is easy for users to understand. For example, it can be formatted in JSON or XML format.

[0079] Step 8:

[0080] The server sends organized and formatted information to the terminal as an HTTP response.

[0081] Step 9:

[0082] The system analyzes the response received by the terminal and displays it on the user interface. Specifically, information on contact points, business systems, and request systems is displayed in a list format.

[0083] Step 10:

[0084] Users review the information displayed on their devices, contact the necessary contacts, and perform their tasks using relevant systems.

[0085] By following these steps, users can quickly obtain information related to specific business terminology and improve their work efficiency.

[0086] (Example 1)

[0087] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] Conventional business search systems have a problem in that users have to spend a lot of time and effort searching for specific terms. In particular, obtaining contact information and related system information requires manual operation, which results in a decrease in work efficiency.

[0089] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0090] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for formatting the search results in JSON or XML format, means for transmitting the formatted data to a terminal, means for the terminal to analyze the received data and visually display it to the user in a list, means for the user to confirm the displayed information and contact a relevant contact or system as necessary, means for transmitting the search query to the server in HTTP request format, and means for initializing the database at startup. As a result, the user can quickly and easily obtain relevant information and significantly improve the efficiency of their work.

[0091] A "user" refers to someone who uses the system and is responsible for searching for specific terms to retrieve information.

[0092] A "search query" is a word or phrase that a user enters into a system to search for something.

[0093] A "server" is a computer system that receives search queries, searches a database, formats the results, and sends them to a terminal.

[0094] A "terminal" is a device operated by a user, where they input search queries and display information sent from a server.

[0095] A "database" is an information aggregation system that stores business terminology and related contact information, business systems, and request systems.

[0096] An "HTTP request" is a communication protocol used by web browsers and devices to request information from a server.

[0097] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation, and is a format for representing data in a structured manner.

[0098] "XML format" is an abbreviation for eXtensible Markup Language, and it is a format for representing data hierarchically.

[0099] "List view" refers to displaying acquired information in a way that allows the user to visually confirm it.

[0100] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0101] System Overview

[0102] The basic operation of this system is as follows: The user operates a terminal and enters specific business terms as a search query. The terminal sends the query to the server, which searches the database, formats the results, and sends them back to the terminal. The terminal parses the received data and displays it visually to the user.

[0103] Hardware and software used

[0104] 1. Server: Use a Linux®-based web server (e.g., Apache®, Nginx). This server is responsible for receiving search queries, searching the database, formatting the results, and sending them.

[0105] 2. Database: A database management system such as MySQL (registered trademark) or PostgreSQL will be used. The database will store information on various terms and related query contacts, business systems, and request systems.

[0106] 3. Terminal: Use a PC or tablet device running Windows or Mac OS. The terminal receives user input, sends it to the server, and displays the results.

[0107] 4. Communication protocol: Use HTTP / HTTPS. HTTP requests are used to send and receive search queries and results.

[0108] Specific example

[0109] Next, we will show a concrete example based on the sequence of operations of this system.

[0110] Example 1: Searching for the term "login"

[0111] 1. The user enters "login" into their device and clicks the search button.

[0112] 2. The terminal sends the input to the server in the form of an HTTP request.

[0113] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0114] 4. The server sends that information to the terminal in JSON format.

[0115] 5. Display a list of information received by the device.

[0116] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0117] Examples of prompts to input into a generative AI model

[0118] "Please describe the overall process for when a user searches for the business term 'login'."

[0119] "Please explain in detail the data processing that occurs when searching for specific business terms, from the perspectives of the server, terminal, and user."

[0120] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0121] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0122] Step 1:

[0123] The server initializes the database. At startup, it calls the `init_database()` function to clear existing data and then populate it with new data, such as business terminology, contact information, and system information. This prepares the system to handle search queries.

[0124] Input: None (at server startup)

[0125] Output: Initialized database

[0126] Specific operation: Initialize the database table using an SQL query and insert new data.

[0127] Step 2:

[0128] The terminal is set to wait for user input. The user enters a specific business term into the terminal's search box. For example, they might type "login" into the search box and click the search button.

[0129] Input: The term entered by the user in the search box.

[0130] Output: Preparation of search queries by terminal

[0131] Specific actions: Search box ( <input type="text" id="search_term"> The user enters information in the field.

[0132] Step 3:

[0133] The terminal receives the search query entered by the user and sends it to the server in the form of an HTTP request. The request is sent using the JavaScript fetch function.

[0134] Input: Search query entered by the user

[0135] Output: Query submission in HTTP request format

[0136] Specific operation: The fetch function sends a request to the server as follows:

[0137] JavaScript

[0138] fetch(' / search', {

[0139] method: 'POST',

[0140] headers: {

[0141] 'Content-Type': 'application / json'

[0142] },

[0143] body: JSON.stringify({ term: 'Login'})

[0144] });

[0145] Step 4:

[0146] The server parses the received search query and searches the database using SQL or other query languages. The server extracts information related to the relevant terms and formats it in JSON or XML format.

[0147] Input: HTTP request sent from the terminal

[0148] Output: Formatted search results

[0149] Specific operation: Execute the following SQL statement and convert the result to JSON format.

[0150] sql

[0151] SELECT contact, system, request_system FROM terms WHERE term = 'Login';

[0152] Step 5:

[0153] The server formats the search results in JSON format and sends them to the terminal as an HTTP response. This returns the data from the server to the terminal.

[0154] Input: Search results from database

[0155] Output: Response data in JSON format

[0156] Specific operation: The search results are structured in JSON format and sent to the terminal as an HTTP response.

[0157] Step 6:

[0158] The terminal parses the received JSON data and displays it to the user in a list. It parses the data and displays it in a visually easy-to-understand format.

[0159] Input: Search results in JSON format from the server

[0160] Output: Information list displayed to the user

[0161] Specific operation: Use JavaScript to display the results on HTML.

[0162] JavaScript

[0163] const results = await response.json();

[0164] document.getElementById('results').innerHTML =

[0165] Contact us at: ${results.contact}

[0166] Business system: ${results.system}

[0167] Request system: ${results.request_system}

[0168] ;

[0169] Step 7:

[0170] Users review the displayed information, contact the necessary contacts, and use relevant systems. This allows users to quickly and efficiently obtain the necessary information and proceed with their work.

[0171] Input: Search results displayed on the device

[0172] Output: User actions (inquiries, system usage)

[0173] Specific action: Based on the information displayed to the user, for example, they might call the support desk.

[0174] This concludes the detailed explanation of the system's program processing steps.

[0175] (Application Example 1)

[0176] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0177] In conventional factories, workers spent considerable time and effort quickly searching for specific work terms and obtaining related contact information and system details. Furthermore, the lack of an efficient search system utilizing voice commands resulted in reduced work efficiency.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0179] In this invention, the server includes means for recognizing voice instructions input by a user, means for converting the search query from voice to text, means for receiving the search query, means for searching a database based on the search query, means for organizing the search results and extracting contact information, related system information, and instruction manuals, and means for displaying the extracted information to the user in a list. This enables workers to quickly obtain necessary information using voice instructions and to proceed with their work efficiently.

[0180] "Voice commands" refer to instructions given by the user to the system using voice.

[0181] A "search query" is a term or phrase entered by the user to be searched for.

[0182] "Means of converting to text" refers to technologies and devices for converting spoken instructions into written information.

[0183] A "database" is a system that stores data such as various business terms and related contact information, system information, and instruction manuals.

[0184] "Means of searching a database" refers to the technologies and functions used to retrieve relevant information within a database based on a search query.

[0185] A "search result" is a set of relevant information extracted from a database.

[0186] "Contact information" refers to contact details related to specific business terminology.

[0187] "System information" refers to information about systems and services related to specific business terminology.

[0188] An "instruction manual" is a document that describes the operating procedures and usage methods related to specific business terminology.

[0189] "Means of displaying in a list" refers to technologies or devices that display search results in a format that users can view.

[0190] System Overview

[0191] This system allows users to input business-related voice commands and quickly and easily obtain contact information, system information, and instruction manuals related to those voice commands. The system primarily consists of three components: a server, terminals, and users.

[0192] Initialization process

[0193] The server first initializes the database. The database contains information about various business terms. This prepares the server to perform search operations based on voice commands.

[0194] User actions

[0195] The user inputs the search term or phrase into the device using voice commands. The device uses a voice recognition system (e.g., Google® Cloud Speech API) to convert the voice into text. For example, if the user says "parts replacement procedure" by voice, the system will convert it into text format.

[0196] Submit a search query

[0197] The terminal sends the converted text-based search query to the server as an HTTP request. This allows the server to search the database based on that search query.

[0198] Database search and result retrieval

[0199] The server searches the database based on the received search query and extracts information such as contact information, system information, and instruction manuals related to the relevant terms. The search results are formatted in JSON or XML format and sent to the terminal.

[0200] Sorting and displaying results

[0201] The terminal analyzes the search results received from the server and displays them in a list. This allows the user to see the necessary information at a glance. For example, it displays contact information related to "parts replacement procedures" (e.g., technical support desk), system information (e.g., parts management system), and instruction manuals (e.g., parts replacement manual).

[0202] User verification and usage

[0203] Users review the displayed list and proceed with their tasks by contacting the necessary contacts or using relevant systems. This allows users to quickly and efficiently obtain the necessary information and carry out their tasks smoothly.

[0204] Specific example

[0205] For example, if the user enters the voice command "parts replacement procedure," this application will operate as follows:

[0206] Example of a prompt:

[0207] Please enter "parts replacement procedure".

[0208] result:

[0209] Terminology: Parts replacement procedure

[0210] Contact: Technical Support Desk

[0211] System: Parts Management System

[0212] Manual: Parts Replacement Manual v1.2

[0213] The main hardware and software used include a computer built into the factory robot, a speech recognition microphone, a display, Python, the Google Cloud Speech API, and the Requests library.

[0214] This allows workers to quickly obtain necessary information using voice commands and carry out their tasks efficiently.

[0215] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0216] Step 1:

[0217] The user inputs voice instructions. Specifically, the user speaks business terms such as "parts replacement procedure" into the terminal within the factory. Voice data is generated as input.

[0218] Step 2:

[0219] The device receives voice commands and passes them to the speech recognition system. The speech recognition system (e.g., Google Cloud Speech API) analyzes the voice data and converts it into text data. Specifically, this step converts the voice data into a text-based search query. The output is text data titled "Parts Replacement Procedure".

[0220] Step 3:

[0221] The terminal sends the converted text data to the server as a search query in the form of an HTTP request. Specifically, it formats the text data into an HTTP request format and sends it to the specified endpoint on the server. The request sent as input contains the text data "parts replacement procedure".

[0222] Step 4:

[0223] The server searches the database based on the search query it receives. Specifically, the server executes an SQL query against the database to extract contact information, system information, and instruction manuals related to the relevant business term. Given a text query "parts replacement procedure" as input, the output will be contact information, system information, and instruction manuals.

[0224] Step 5:

[0225] The server formats the retrieved search results in JSON or XML format. Specifically, it converts the extracted information into the specified data format. The search results data is given as input, and the output is data in JSON or XML format.

[0226] Step 6:

[0227] The server sends formatted search results to the terminal. Specifically, the server sends data to the terminal as an HTTP response. JSON or XML formatted data is given as input, and an HTTP response is generated as output.

[0228] Step 7:

[0229] The terminal analyzes the search results received from the server and displays them to the user in a list. Specifically, the terminal analyzes the received data and formats it so that it can be displayed on the screen. JSON or XML format data is provided as input, and the content of the list is generated as output.

[0230] Step 8:

[0231] Users review the displayed information and contact the necessary contacts or use related systems. Specifically, users view the displayed information and take the required actions. The displayed information is provided as input, and the user's actions are the output.

[0232] The steps described above allow users to efficiently obtain necessary information using voice commands and carry out their work smoothly.

[0233] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0234] System Overview

[0235] This invention enhances the user experience by combining a system that allows users to search for business terms and quickly and easily obtain information on related contacts, business systems, and request systems with an emotion engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion engine.

[0236] Specific implementations of the system

[0237] 1. Initialization process

[0238] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0239] 2. User actions

[0240] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction. At the same time, the emotion engine is activated, analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[0241] 3. User emotion recognition

[0242] The emotion engine analyzes the user's emotions in real time as they enter their search queries and retrieves emotional information. For example, if the user is irritated, it will recognize that.

[0243] 4. Submitting a search query

[0244] The terminal sends the user-inputted search query to the server in HTTP request format, along with the recognized sentiment information.

[0245] 5. Database search and result retrieval

[0246] The server receives an HTTP request and searches the database based on the submitted search query. The server extracts information from the database regarding contacts, business systems, and request systems related to the relevant term.

[0247] 6. Organizing results based on emotional information

[0248] The server organizes the acquired information based on the emotional information recognized by the emotion engine. If the user is frustrated, it prioritizes displaying information that can resolve the issue more simply and quickly.

[0249] 7. Submitting and displaying results

[0250] The server sends organized and formatted information to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. It displays a list of contact information, business systems, and request systems according to a specific layout.

[0251] 8. User Verification and Use

[0252] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to carry out their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and carry out their tasks efficiently.

[0253] Specific example

[0254] Example 1: Searching for the term "login" and sentiment recognition

[0255] 1. The user enters "login" into their device and clicks the search button.

[0256] 2. The emotion engine recognizes that the user is frustrated.

[0257] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[0258] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[0259] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[0260] 6. The server sends that information to the terminal in JSON format.

[0261] 7. Display a list of the information received by the device.

[0262] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0263] This system allows users to receive information related to the terms they enter, tailored to their emotions, and as a result, significantly improves work efficiency.

[0264] The following describes the processing flow.

[0265] Step 1:

[0266] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0267] Step 2:

[0268] The terminal enters a state of waiting for user input. A search text box and search button appear on the terminal, and it is ready for user interaction. Simultaneously, the emotion engine starts up and begins analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[0269] Step 3:

[0270] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button. The emotion engine retrieves emotional information from the user's facial expressions and voice, and recognizes, for example, that the user is irritated.

[0271] Step 4:

[0272] The device sends the user's input as a search query in the form of an HTTP request to the server. Simultaneously, it also sends the sentiment information acquired by the sentiment engine.

[0273] Step 5:

[0274] The server receives the HTTP request and analyzes the submitted search query and sentiment information. It then verifies that the analyzed search query is specific business terminology.

[0275] Step 6:

[0276] The server searches the database. Specifically, it looks for terms in the database that match the sent search query and extracts information on the contact destinations, business systems, and request systems related to those terms.

[0277] Step 7:

[0278] The server organizes the extracted information. Based on the sentiment information recognized by the sentiment engine, when the user is frustrated, it preferentially organizes information that can be resolved more simply and quickly. For example, it highlights high-priority contact destinations and links to immediately usable systems.

[0279] Step 8:

[0280] The server sends the organized and formatted information to the terminal as an HTTP response. The formatting is done in a user-friendly format such as JSON or XML.

[0281] Step 9:

[0282] The terminal analyzes the received response and displays it on the user interface. According to the specific layout, information on contact destinations, business systems, and request systems is displayed in a list format.

[0283] Step 10:

[0284] The user checks the information displayed on the terminal, contacts the necessary contact destinations, or uses the relevant systems to perform the business. Through the information display based on sentiment information, the user can quickly obtain the necessary information and efficiently proceed with the business.

[0285] Through the above steps, the user is provided with information related to specific business terms according to sentiment, and as a result, the efficiency of the business can be significantly improved.

[0286] (Example 2)

[0287] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0288] In a conventional business information search system, although relevant information is presented for a search query input by a user, the user experience may deteriorate because uniform information is displayed without considering the user's emotions. In particular, for users with negative emotions such as frustration and anxiety, quick and efficient information provision has been demanded. However, conventional systems have lacked the function of recognizing the user's emotions in real time and reorganizing information based on them. Therefore, the problem to be solved is to improve the user experience by providing flexible information according to the user's emotions.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0290] In this invention, the server includes means for receiving a search query input from a user, means for searching a database based on the search query, means for obtaining the search result and recognizing the user's emotion using an emotion analysis engine, means for reorganizing the obtained search result based on the recognized emotion information and setting a priority, and means for displaying the reorganized information to the user in a list. Thereby, quick and efficient information provision according to the user's emotion becomes possible.

[0291] A "search query" is a term or phrase of a search target input by a user to the system.

[0292] A "database" is an aggregate of information structured so that data such as business terms, inquiry destinations, and related system information can be efficiently managed and searched.

[0293] An "emotion analysis engine" is a system that analyzes a user's emotional state in real time based on information obtained from the user's facial expressions, voice, and other interfaces.

[0294] An "HTTP request" is a type of data format sent from a user's device to a server, and is particularly used to send and receive information using the Internet communication protocol.

[0295] "JSON format" is a format for representing data as text, structuring it, and making it exchangeable. It is an abbreviation for JavaScript Object Notation.

[0296] XML format is a markup language used to structure and represent data, and it is an abbreviation for Extensible Markup Language.

[0297] "Setting priorities" means selecting the most useful information for the user from the search results based on the analyzed sentiment information and determining the display order.

[0298] "List display" refers to organizing acquired information and visually presenting it so that users can view it at a glance.

[0299] An "HTTP response" is response data sent from a server to a terminal, and it includes the results and processing details of a request.

[0300] This invention enhances the user experience by combining a system that allows users to quickly and easily search for business terms and obtain information on related contacts, business systems, and request systems with an emotion analysis engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion analysis engine.

[0301] The hardware used includes a standard computer server (e.g., an x86 architecture server). The software includes Apache HTTP Server and a MySQL database for the server, a web browser (e.g., Chrome, Firefox, etc.) for the terminal, and IBM Watson® or Microsoft® Azure® Emotion API for the sentiment analysis engine.

[0302] When the server starts up, the Apache HTTP Server and MySQL database are initialized, and pre-prepared business terminology, contact information, business systems, and request systems are stored in the database. This database is then ready for search processing.

[0303] The user initiates a search using their device. A search text box and search button appear in the device's web browser, where the user enters the search terms. The sentiment analysis engine runs in the background, ready to analyze the user's facial expressions and voice in real time.

[0304] When a user enters a search query and clicks the search button, the device sends the search query and the recognized sentiment information to the server in the form of an HTTP request. The server receives this request and queries a MySQL database to extract relevant information. Next, the retrieved information is reorganized based on the user's sentiment information recognized by the sentiment analysis engine.

[0305] For example, if a user is frustrated, information that can resolve the issue quickly and easily will be prioritized and formatted. This prioritized information will be formatted in JSON or XML format and sent to the terminal as an HTTP response.

[0306] The terminal analyzes the received information and displays it on the user interface. The user can quickly solve problems without frustration by checking the displayed information, contacting the necessary inquiry destinations, or proceeding with business using related systems.

[0307] Specific Example

[0308] Example 1: Search and sentiment recognition of the term "login"

[0309] 1. The user enters "login" in the text box of the terminal and clicks the search button.

[0310] 2. The sentiment analysis engine recognizes frustration from the user's expression and voice.

[0311] 3. The terminal sends the input and sentiment information to the server in the form of an HTTP request.

[0312] 4. The server searches the database and obtains information on inquiry destinations, business systems, and request systems related to "login".

[0313] 5. The server organizes the obtained information based on the sentiment information and preferentially formats the information that can be quickly resolved.

[0314] 6. The server sends the information to the terminal in JSON format.

[0315] 7. The terminal displays the received information in a list.

[0316] 8. The user checks the displayed information and uses the necessary inquiry destinations or systems to quickly solve problems related to "login".

[0317] Through the above process, this invention enables the user to quickly and efficiently obtain information related to business terms, and further improve the user experience through sentiment analysis. The detailed procedures for realizing this system are as follows.

[0318] Example of a prompt

[0319] "A scenario where a user is experiencing login issues. The user types 'login'. The sentiment analysis engine detects frustration. The system then describes the steps involved in retrieving relevant information, organizing it based on emotion, and displaying the results."

[0320] This allows users to quickly obtain appropriate information tailored to their emotions, resulting in a significant improvement in work efficiency.

[0321] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0322] Step 1: Initialization process

[0323] The server starts the Apache HTTP Server and initializes the MySQL database. During the initialization process, business terminology, contact information, business systems, and request systems are stored in the database. This prepares the server for search operations. The input is the database initialization script, and the output is the initialized database. The specific actions involve the startup commands for Apache and MySQL, and the execution of the script.

[0324] Step 2: Wait for user response

[0325] The device enters a state where it waits for user input in a web browser. A search text box and search button appear on the screen, and the sentiment analysis engine starts running in the background. Input is the user interface initialization script, and output is an interactive search form. The specific operation involves building the UI using HTML and JavaScript, and starting the sentiment analysis engine.

[0326] Step 3: Enter search query and sentiment recognition

[0327] The user enters a search query into a text box and clicks the search button. The sentiment analysis engine analyzes the user's facial expressions and voice in real time to obtain emotional information. The input is the user's search query and emotional data, and the output is the obtained emotional state. Specifically, the operation includes data collection and analysis by the sentiment analysis engine.

[0328] Step 4: Submit your search query

[0329] The terminal converts the user's search query and recognized sentiment information into JSON format and sends it to the server as an HTTP request. The input is the search query and sentiment information, and the output is the HTTP request sent to the server. Specifically, the operation involves converting the data to JSON and generating the HTTP request using JavaScript.

[0330] Step 5: Database search and result retrieval

[0331] The server receives an HTTP request and searches a MySQL database based on the submitted search query. It extracts information about relevant contacts, business systems, and request systems related to the specified terms. The input is the search query included in the HTTP request, and the output is the retrieved search results. Specific operations include database queries and data extraction using PHP or Python scripts.

[0332] Step 6: Outcome organization based on emotional information

[0333] The server reorganizes the retrieved search results based on the sentiment information recognized by the sentiment analysis engine. If the user is frustrated, it prioritizes displaying information that can provide a quick and simple solution. The input is the recognized sentiment information and search results, and the output is the reorganized search results. The specific operation involves applying sentiment information analysis algorithms and restructuring the data.

[0334] Step 7: Submit and view results

[0335] The server formats the reorganized information in JSON or XML format and sends it to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. The input is the organized search results, and the output is the information displayed on the UI. Specific operations include generating the HTTP response, creating the JSON / XML format, and updating the UI using JavaScript.

[0336] Step 8: User Verification and Use

[0337] Users review information displayed on their devices, contact necessary contacts, and use relevant systems to carry out their tasks. Input is the displayed information, and output is the user's actions (e.g., contacting a contact). Specific actions include the user clicking or selecting information and then performing specific actions based on those clicks.

[0338] (Application Example 2)

[0339] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0340] In previous factory troubleshooting processes, operators had to spend time and effort searching for information without considering their own emotions. This made it difficult to quickly access the right information, especially when feeling anxious or frustrated, leading to decreased work efficiency.

[0341] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving a search query input by a user, means for searching a database based on the search query, means for organizing the search results and extracting the contact information and related system information, means for organizing the extracted information based on emotion recognition and presenting it with priority according to the user's emotions, means for displaying the extracted information to the user in a list, emotion engine means for recognizing the user's emotions in real time, and means for attaching emotion information obtained from the emotion engine when sending a search query. As a result, even when the operator is feeling frustrated or anxious, it becomes possible to quickly and efficiently access appropriate information.

[0342] A "user" refers to a person who operates this system and enters a search query.

[0343] A "search query" refers to the keywords or phrases that a user enters to identify the information they are looking for.

[0344] A "database" refers to an information aggregation system that stores business-related data such as contact information and related system information.

[0345] An "emotion engine" refers to a system that recognizes and analyzes a user's emotions in real time and acquires that information.

[0346] An "HTTP request" refers to a data communication method used to send a user's search queries and sentiment information to a server.

[0347] "Organization" refers to the act of prioritizing and presenting search results according to the user's emotional state.

[0348] A "server" refers to a computer system that receives search queries and sentiment information, searches a database, and organizes the results.

[0349] "JSON format" is a lightweight data description format for data exchange, and is an abbreviation for JavaScript Object Notation.

[0350] XML is a highly readable markup language widely used for data exchange, and its name is an abbreviation for Extensible Markup Language.

[0351] "Troubleshooting" refers to the procedures and processes used to diagnose and resolve malfunctions and problems in machines and systems.

[0352] This invention is a system that can be applied to applications that support troubleshooting within a factory. Specifically, it provides a system that allows a user to operate a factory robot and efficiently obtain the information necessary for troubleshooting. Because this system integrates search query processing and sentiment recognition functions, it can present optimal information according to the user's emotional state.

[0353] Hardware and software to be used

[0354] Hardware:

[0355] User interface devices such as tablet devices and smart glasses

[0356] Factory robots and central control systems

[0357] software:

[0358] Emotion engine (e.g., Microsoft Azure Cognitive Services)

[0359] SQL database (e.g., MySQL)

[0360] HTTP server (e.g., Apache)

[0361] Explanation of the process

[0362] 1. Server role:

[0363] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information.

[0364] 2. The role of the terminal:

[0365] Users input details of their problem using a tablet or smart glasses. The device displays a search text box and a search button for user interaction. Additionally, an emotion engine analyzes the user's facial expressions, voice, and typing speed in real time to acquire emotional information.

[0366] 3. The role of the emotional engine:

[0367] The emotion engine analyzes the user's emotions in real time, retrieves that information, and attaches it when the search query is submitted. This emotion information is used in subsequent processing to organize the results.

[0368] 4. Sending search queries to the server:

[0369] The device sends search queries and sentiment information from the user to the server in the form of an HTTP request.

[0370] 5. Search the server's database and retrieve the results:

[0371] The server searches the database based on the submitted search query and extracts relevant information. The search results are formatted in JSON or XML and provided to the user.

[0372] 6. Organizing search results:

[0373] The server organizes the emotional information obtained by the emotion engine and displays it with priority according to the user's emotional state.

[0374] Specific example

[0375] For example, when a user is performing a "fault diagnosis" on a machine in a factory, if the operator is in a hurry, it would be good to input a prompt message like the following into the AI ​​model:

[0376] Example of a prompt:

[0377] An operator is diagnosing a machine malfunction in the factory. They enter "Error Code 123" into the device, and the emotion engine recognizes an emotion of frustration. Generate clear instructions for the operator, including steps and contact information for a quick and easy resolution.

[0378] By following the steps outlined above, the generative AI model can provide optimal troubleshooting information based on the search queries and sentiment information entered by the user. This system allows users to efficiently and quickly resolve manufacturing issues.

[0379] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0380] Step 1:

[0381] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information. This information is used to prepare subsequent search processes. Initial database data is supplied to the server as input, and an initialized database is generated as output.

[0382] Step 2:

[0383] Users input details of their problem using a tablet or smart glasses. The device displays a text box and a search button for user interaction. The input is a search query entered by the user, and the output is the search query displayed on the user interface.

[0384] Step 3:

[0385] The device's emotion engine analyzes the user's facial expressions, voice, typing speed, etc., in real time to acquire emotional information. The input is biometric data indicating the user's emotions, which the emotion engine analyzes and outputs as emotional information.

[0386] Step 4:

[0387] The terminal sends search queries and sentiment information from the user to the server in the form of an HTTP request. The input consists of search queries and sentiment information, which are sent to the server in JSON or XML format. The server receives this data as output.

[0388] Step 5:

[0389] The server searches the database based on the submitted search query. It uses SQL to extract relevant information. The input is the received search query, and the output is the relevant information as search results. The server organizes this data and provides it in the format best suited to the user's request.

[0390] Step 6:

[0391] The server organizes information based on emotional data acquired by the emotion engine. Relevant information is prioritized and displayed according to the user's emotional state. Input consists of emotional data and search results; data processing is performed based on the emotional data, and prioritized information is generated as output.

[0392] Step 7:

[0393] The server sends organized information to the terminal in JSON or XML format. The input is prioritized information, which is then formatted and sent to the terminal. The terminal receives this information as output.

[0394] Step 8:

[0395] The terminal analyzes the information it receives and displays it on the user interface. The input is information sent from the server and displayed in an appropriate format on the user interface. The output is a display of information that the user can actually view and use.

[0396] Step 9:

[0397] The user reviews the information displayed on the device and performs troubleshooting steps. The input is the information displayed on the device, which the user uses to contact support or perform steps to resolve the problem. The output is the result of the resolved work.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0399] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0401] [Second Embodiment]

[0402] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0403] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0404] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0406] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0408] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0409] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0410] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0413] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0414] System Overview

[0415] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0416] Specific implementations of the system

[0417] 1. Initialization process

[0418] The server initializes the database. The database stores various business terms and related contact information, business systems, and request systems. This prepares the server for search processing.

[0419] 2. User actions

[0420] The terminal is set to wait for user input. The user operates the terminal and enters a specific business term as a search query. For example, they enter the term "login" and click the search button.

[0421] 3. Submitting a search query

[0422] The terminal receives the search query entered by the user and sends it to the server in HTTP request format. This allows the server to search the database based on that search query.

[0423] 4. Database search and result retrieval

[0424] The server searches the database based on the search query it receives. The server extracts information from the database about contacts, business systems, and request systems related to the relevant terms. The search results are formatted in JSON or XML and sent to the terminal.

[0425] 5. Summarizing and displaying the results

[0426] The terminal analyzes the search results received from the server and displays them to the user in a list. For example, it may display contact information related to "login" (e.g., support desk), business systems (e.g., user management system), request systems (e.g., password reset request system), etc.

[0427] 6. User Verification and Use

[0428] Users review the displayed list, contact the necessary contacts, and use relevant systems to carry out their tasks. This allows users to quickly and efficiently obtain the necessary information and perform their tasks smoothly.

[0429] Specific example

[0430] Example 1: Searching for the term "login"

[0431] 1. The user enters "login" into their device and clicks the search button.

[0432] 2. The terminal sends the input to the server in the form of an HTTP request.

[0433] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0434] 4. The server sends that information to the terminal in JSON format.

[0435] 5. Display a list of information received by the device.

[0436] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0437] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0438] The following describes the processing flow.

[0439] Step 1:

[0440] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0441] Step 2:

[0442] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction.

[0443] Step 3:

[0444] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button.

[0445] Step 4:

[0446] The terminal receives user input and sends that input as a search query in the form of an HTTP request to the server.

[0447] Step 5:

[0448] The server receives the HTTP request and parses the submitted search query. It then verifies that the parsed search query contains specific business terminology.

[0449] Step 6:

[0450] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[0451] Step 7:

[0452] The server extracts information, organizes it, and converts it into a format that is easy for users to understand. For example, it can be formatted in JSON or XML format.

[0453] Step 8:

[0454] The server sends organized and formatted information to the terminal as an HTTP response.

[0455] Step 9:

[0456] The system analyzes the response received by the terminal and displays it on the user interface. Specifically, information on contact points, business systems, and request systems is displayed in a list format.

[0457] Step 10:

[0458] Users review the information displayed on their devices, contact the necessary contacts, and perform their tasks using relevant systems.

[0459] By following these steps, users can quickly obtain information related to specific business terminology and improve their work efficiency.

[0460] (Example 1)

[0461] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0462] Conventional business search systems have a problem in that users have to spend a lot of time and effort searching for specific terms. In particular, obtaining contact information and related system information requires manual operation, which results in a decrease in work efficiency.

[0463] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0464] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for formatting the search results in JSON or XML format, means for transmitting the formatted data to a terminal, means for the terminal to analyze the received data and visually display it to the user in a list, means for the user to confirm the displayed information and contact a relevant contact or system as necessary, means for transmitting the search query to the server in HTTP request format, and means for initializing the database at startup. As a result, the user can quickly and easily obtain relevant information and significantly improve the efficiency of their work.

[0465] A "user" refers to someone who uses the system and is responsible for searching for specific terms to retrieve information.

[0466] A "search query" is a word or phrase that a user enters into a system to search for something.

[0467] A "server" is a computer system that receives search queries, searches a database, formats the results, and sends them to a terminal.

[0468] A "terminal" is a device operated by a user, where they input search queries and display information sent from a server.

[0469] A "database" is an information aggregation system that stores business terminology and related contact information, business systems, and request systems.

[0470] An "HTTP request" is a communication protocol used by web browsers and devices to request information from a server.

[0471] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[0472] "XML format" is an abbreviation for eXtensible Markup Language, and it is a format for representing data hierarchically.

[0473] "List view" refers to displaying acquired information in a way that allows the user to visually confirm it.

[0474] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0475] System Overview

[0476] The basic operation of this system is as follows: The user operates a terminal and enters specific business terms as a search query. The terminal sends the query to the server, which searches the database, formats the results, and sends them back to the terminal. The terminal parses the received data and displays it visually to the user.

[0477] Hardware and software used

[0478] 1. Server: Use a Linux-based web server (e.g., Apache, Nginx). This server will receive search queries, search the database, format the results, and send them.

[0479] 2. Database: A database management system such as MySQL or PostgreSQL will be used. The database will store information on various terms and related query targets, business systems, and request systems.

[0480] 3. Terminal: Use a PC or tablet device running Windows or Mac OS. The terminal receives user input, sends it to the server, and displays the results.

[0481] 4. Communication protocol: Use HTTP / HTTPS. HTTP requests are used to send and receive search queries and results.

[0482] Specific example

[0483] Next, we will show a concrete example based on the sequence of operations of this system.

[0484] Example 1: Searching for the term "login"

[0485] 1. The user enters "login" into their device and clicks the search button.

[0486] 2. The terminal sends the input to the server in the form of an HTTP request.

[0487] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0488] 4. The server sends that information to the terminal in JSON format.

[0489] 5. Display a list of information received by the device.

[0490] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0491] Examples of prompts to input into a generative AI model

[0492] "Please describe the overall process for when a user searches for the business term 'login'."

[0493] "Please explain in detail the data processing that occurs when searching for specific business terms, from the perspectives of the server, terminal, and user."

[0494] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0495] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0496] Step 1:

[0497] The server initializes the database. At startup, it calls the `init_database()` function to clear existing data and then populate it with new data, such as business terminology, contact information, and system information. This prepares the system to handle search queries.

[0498] Input: None (at server startup)

[0499] Output: Initialized database

[0500] Specific operation: Initialize the database table using an SQL query and insert new data.

[0501] Step 2:

[0502] The terminal is set to wait for user input. The user enters a specific business term into the terminal's search box. For example, they might type "login" into the search box and click the search button.

[0503] Input: The term entered by the user in the search box.

[0504] Output: Preparation of search queries by terminal

[0505] Specific actions: Search box ( <input type="text" id="search_term"> The user enters information in the field.

[0506] Step 3:

[0507] The terminal receives the search query entered by the user and sends it to the server in the form of an HTTP request. The request is sent using the JavaScript fetch function.

[0508] Input: Search query entered by the user

[0509] Output: Query submission in HTTP request format

[0510] Specific operation: The fetch function sends a request to the server as follows:

[0511] JavaScript

[0512] fetch(' / search', {

[0513] method: 'POST',

[0514] headers: {

[0515] 'Content-Type': 'application / json'

[0516] },

[0517] body: JSON.stringify({ term: 'Login'})

[0518] });

[0519] Step 4:

[0520] The server parses the received search query and searches the database using SQL or other query languages. The server extracts information related to the relevant terms and formats it in JSON or XML format.

[0521] Input: HTTP request sent from the terminal

[0522] Output: Formatted search results

[0523] Specific operation: Execute the following SQL statement and convert the result to JSON format.

[0524] sql

[0525] SELECT contact, system, request_system FROM terms WHERE term = 'Login';

[0526] Step 5:

[0527] The server formats the search results in JSON format and sends them to the terminal as an HTTP response. This returns the data from the server to the terminal.

[0528] Input: Search results from database

[0529] Output: Response data in JSON format

[0530] Specific operation: The search results are structured in JSON format and sent to the terminal as an HTTP response.

[0531] Step 6:

[0532] The terminal parses the received JSON data and displays it to the user in a list. It parses the data and displays it in a visually easy-to-understand format.

[0533] Input: Search results in JSON format from the server

[0534] Output: Information list displayed to the user

[0535] Specific operation: Use JavaScript to display the results on HTML.

[0536] JavaScript

[0537] const results = await response.json();

[0538] document.getElementById('results').innerHTML =

[0539] Contact us at: ${results.contact}

[0540] Business system: ${results.system}

[0541] Request system: ${results.request_system}

[0542] ;

[0543] Step 7:

[0544] Users review the displayed information, contact the necessary contacts, and use relevant systems. This allows users to quickly and efficiently obtain the necessary information and proceed with their work.

[0545] Input: Search results displayed on the device

[0546] Output: User actions (inquiries, system usage)

[0547] Specific action: Based on the information displayed to the user, for example, they might call the support desk.

[0548] This concludes the detailed explanation of the system's program processing steps.

[0549] (Application Example 1)

[0550] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0551] In conventional factories, workers spent considerable time and effort quickly searching for specific work terms and obtaining related contact information and system details. Furthermore, the lack of an efficient search system utilizing voice commands resulted in reduced work efficiency.

[0552] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0553] In this invention, the server includes means for recognizing voice instructions input by a user, means for converting the search query from voice to text, means for receiving the search query, means for searching a database based on the search query, means for organizing the search results and extracting contact information, related system information, and instruction manuals, and means for displaying the extracted information to the user in a list. This enables workers to quickly obtain necessary information using voice instructions and to proceed with their work efficiently.

[0554] "Voice commands" refer to instructions given by the user to the system using voice.

[0555] A "search query" is a term or phrase entered by the user to be searched for.

[0556] "Means of converting to text" refers to technologies and devices for converting spoken instructions into written information.

[0557] A "database" is a system that stores data such as various business terms and related contact information, system information, and instruction manuals.

[0558] "Means of searching a database" refers to the technologies and functions used to retrieve relevant information within a database based on a search query.

[0559] A "search result" is a set of relevant information extracted from a database.

[0560] "Contact information" refers to contact details related to specific business terminology.

[0561] "System information" refers to information about systems and services related to specific business terminology.

[0562] An "instruction manual" is a document that describes the operating procedures and usage methods related to specific business terminology.

[0563] "Means of displaying in a list" refers to technologies or devices that display search results in a format that users can view.

[0564] System Overview

[0565] This system allows users to input business-related voice commands and quickly and easily obtain contact information, system information, and instruction manuals related to those voice commands. The system primarily consists of three components: a server, terminals, and users.

[0566] Initialization process

[0567] The server first initializes the database. The database contains information about various business terms. This prepares the server to perform search operations based on voice commands.

[0568] User actions

[0569] The user inputs the search term or phrase into the device using voice commands. The device then uses a speech recognition system (e.g., Google Cloud Speech API) to convert the voice into text. For example, if the user says "parts replacement procedure" by voice, the system will convert it into text.

[0570] Submit a search query

[0571] The terminal sends the converted text-based search query to the server as an HTTP request. This allows the server to search the database based on that search query.

[0572] Database search and result retrieval

[0573] The server searches the database based on the received search query and extracts information such as contact information, system information, and instruction manuals related to the relevant terms. The search results are formatted in JSON or XML format and sent to the terminal.

[0574] Sorting and displaying results

[0575] The terminal analyzes the search results received from the server and displays them in a list. This allows the user to see the necessary information at a glance. For example, it displays contact information related to "parts replacement procedures" (e.g., technical support desk), system information (e.g., parts management system), and instruction manuals (e.g., parts replacement manual).

[0576] User verification and usage

[0577] Users review the displayed list and proceed with their tasks by contacting the necessary contacts or using relevant systems. This allows users to quickly and efficiently obtain the necessary information and carry out their tasks smoothly.

[0578] Specific example

[0579] For example, if the user enters the voice command "parts replacement procedure," this application will operate as follows:

[0580] Example of a prompt:

[0581] Please enter "parts replacement procedure".

[0582] result:

[0583] Terminology: Parts replacement procedure

[0584] Contact: Technical Support Desk

[0585] System: Parts Management System

[0586] Manual: Parts Replacement Manual v1.2

[0587] The main hardware and software used include a computer built into the factory robot, a speech recognition microphone, a display, Python, the Google Cloud Speech API, and the Requests library.

[0588] This allows workers to quickly obtain necessary information using voice commands and carry out their tasks efficiently.

[0589] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0590] Step 1:

[0591] The user inputs voice instructions. Specifically, the user speaks business terms such as "parts replacement procedure" into the terminal within the factory. Voice data is generated as input.

[0592] Step 2:

[0593] The device receives voice commands and passes them to the speech recognition system. The speech recognition system (e.g., Google Cloud Speech API) analyzes the voice data and converts it into text data. Specifically, this step converts the voice data into a text-based search query. The output is text data titled "Parts Replacement Procedure".

[0594] Step 3:

[0595] The terminal sends the converted text data to the server as a search query in the form of an HTTP request. Specifically, it formats the text data into an HTTP request format and sends it to the specified endpoint on the server. The request sent as input contains the text data "parts replacement procedure".

[0596] Step 4:

[0597] The server searches the database based on the search query it receives. Specifically, the server executes an SQL query against the database to extract contact information, system information, and instruction manuals related to the relevant business term. Given a text query "parts replacement procedure" as input, the output will be contact information, system information, and instruction manuals.

[0598] Step 5:

[0599] The server formats the retrieved search results in JSON or XML format. Specifically, it converts the extracted information into the specified data format. The search results data is given as input, and the output is data in JSON or XML format.

[0600] Step 6:

[0601] The server sends formatted search results to the terminal. Specifically, the server sends data to the terminal as an HTTP response. JSON or XML formatted data is given as input, and an HTTP response is generated as output.

[0602] Step 7:

[0603] The terminal analyzes the search results received from the server and displays them to the user in a list. Specifically, the terminal analyzes the received data and formats it so that it can be displayed on the screen. JSON or XML format data is provided as input, and the content of the list is generated as output.

[0604] Step 8:

[0605] Users review the displayed information and contact the necessary contacts or use related systems. Specifically, users view the displayed information and take the required actions. The displayed information is provided as input, and the user's actions are the output.

[0606] The steps described above allow users to efficiently obtain necessary information using voice commands and carry out their work smoothly.

[0607] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0608] System Overview

[0609] This invention enhances the user experience by combining a system that allows users to search for business terms and quickly and easily obtain information on related contacts, business systems, and request systems with an emotion engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion engine.

[0610] Specific implementations of the system

[0611] 1. Initialization process

[0612] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0613] 2. User actions

[0614] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction. At the same time, the emotion engine is activated, analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[0615] 3. User emotion recognition

[0616] The emotion engine analyzes the user's emotions in real time as they enter their search queries and retrieves emotional information. For example, if the user is irritated, it will recognize that.

[0617] 4. Submitting a search query

[0618] The terminal sends the user-inputted search query to the server in HTTP request format, along with the recognized sentiment information.

[0619] 5. Database search and result retrieval

[0620] The server receives an HTTP request and searches the database based on the submitted search query. The server extracts information from the database regarding contacts, business systems, and request systems related to the relevant term.

[0621] 6. Organizing results based on emotional information

[0622] The server organizes the acquired information based on the emotional information recognized by the emotion engine. If the user is frustrated, it prioritizes displaying information that can resolve the issue more simply and quickly.

[0623] 7. Submitting and displaying results

[0624] The server sends organized and formatted information to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. It displays a list of contact information, business systems, and request systems according to a specific layout.

[0625] 8. User Verification and Use

[0626] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to carry out their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and carry out their tasks efficiently.

[0627] Specific example

[0628] Example 1: Searching for the term "login" and sentiment recognition

[0629] 1. The user enters "login" into their device and clicks the search button.

[0630] 2. The emotion engine recognizes that the user is frustrated.

[0631] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[0632] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[0633] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[0634] 6. The server sends that information to the terminal in JSON format.

[0635] 7. Display a list of the information received by the device.

[0636] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0637] This system allows users to receive information related to the terms they enter, tailored to their emotions, and as a result, significantly improves work efficiency.

[0638] The following describes the processing flow.

[0639] Step 1:

[0640] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0641] Step 2:

[0642] The terminal enters a state of waiting for user input. A search text box and search button appear on the terminal, and it is ready for user interaction. Simultaneously, the emotion engine starts up and begins analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[0643] Step 3:

[0644] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button. The emotion engine retrieves emotional information from the user's facial expressions and voice, and recognizes, for example, that the user is irritated.

[0645] Step 4:

[0646] The device sends the user's input as a search query in the form of an HTTP request to the server. Simultaneously, it also sends the sentiment information acquired by the sentiment engine.

[0647] Step 5:

[0648] The server receives the HTTP request and analyzes the submitted search query and sentiment information. It then verifies that the analyzed search query is specific business terminology.

[0649] Step 6:

[0650] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[0651] Step 7:

[0652] The server organizes the extracted information. Based on the sentiment information recognized by the sentiment engine, if the user is frustrated, it prioritizes and organizes information that can be resolved more simply and quickly. For example, it highlights high-priority contact information and links to readily available systems.

[0653] Step 8:

[0654] The server sends organized and formatted information to the terminal as an HTTP response. The format should be user-friendly, such as JSON or XML.

[0655] Step 9:

[0656] The system analyzes the response received by the terminal and displays it on the user interface. Information on contact points, business systems, and request systems is displayed in a list format according to a specific layout.

[0657] Step 10:

[0658] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to perform their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and perform their tasks efficiently.

[0659] Through these steps, users can be provided with information related to specific business terminology in an emotionally resonant manner, resulting in a significant improvement in work efficiency.

[0660] (Example 2)

[0661] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0662] Conventional business information retrieval systems, while presenting relevant information in response to user-entered search queries, often displayed uniform information without considering the user's emotions, resulting in a poor user experience. In particular, users experiencing negative emotions such as frustration or impatience required rapid and efficient information delivery. However, existing systems lacked the ability to recognize user emotions in real time and reorganize information based on those emotions. Therefore, the challenge to be addressed is improving the user experience through flexible information delivery that responds to the user's emotions.

[0663] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0664] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for obtaining the search results and recognizing the user's emotions using an emotion analysis engine, means for reorganizing the obtained search results based on the recognized emotion information and setting priorities, and means for displaying the reorganized information to the user in a list. This enables the rapid and efficient provision of information in accordance with the user's emotions.

[0665] A "search query" is a term or phrase that a user enters into the system to search for.

[0666] A "database" is a collection of structured information that allows for the efficient management and retrieval of business-related terminology, contact information, and related system information.

[0667] An "emotion analysis engine" is a system that analyzes a user's emotional state in real time based on information obtained from the user's facial expressions, voice, and other interfaces.

[0668] An "HTTP request" is a type of data format sent from a user's device to a server, and is particularly used to send and receive information using the Internet communication protocol.

[0669] "JSON format" is a format for representing data as text, structuring it, and making it exchangeable. It is an abbreviation for JavaScript Object Notation.

[0670] XML format is a markup language used to structure and represent data, and it is an abbreviation for Extensible Markup Language.

[0671] "Setting priorities" means selecting the most useful information for the user from the search results based on the analyzed sentiment information and determining the display order.

[0672] "List display" refers to organizing acquired information and visually presenting it so that users can view it at a glance.

[0673] An "HTTP response" is response data sent from a server to a terminal, and it includes the results and processing details of a request.

[0674] This invention enhances the user experience by combining a system that allows users to quickly and easily search for business terms and obtain information on related contacts, business systems, and request systems with an emotion analysis engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion analysis engine.

[0675] The hardware used includes a standard computer server (e.g., an x86 architecture server). The software includes Apache HTTP Server and a MySQL database for the server, a web browser (e.g., Chrome, Firefox, etc.) for the terminal, and IBM Watson or Microsoft Azure Emotion API for the sentiment analysis engine.

[0676] When the server starts up, the Apache HTTP Server and MySQL database are initialized, and pre-prepared business terminology, contact information, business systems, and request systems are stored in the database. This database is then ready for search processing.

[0677] The user initiates a search using their device. A search text box and search button appear in the device's web browser, where the user enters the search terms. The sentiment analysis engine runs in the background, ready to analyze the user's facial expressions and voice in real time.

[0678] When a user enters a search query and clicks the search button, the device sends the search query and the recognized sentiment information to the server in the form of an HTTP request. The server receives this request and queries a MySQL database to extract relevant information. Next, the retrieved information is reorganized based on the user's sentiment information recognized by the sentiment analysis engine.

[0679] For example, if a user is frustrated, information that can resolve the issue quickly and easily will be prioritized and formatted. This prioritized information will be formatted in JSON or XML format and sent to the terminal as an HTTP response.

[0680] The terminal analyzes the received information and displays it on the user interface. Users can then review the displayed information, contact the necessary contacts, or use relevant systems to proceed with their work, enabling them to resolve problems quickly and without frustration.

[0681] Specific example

[0682] Example 1: Searching for the term "login" and sentiment recognition

[0683] 1. The user enters "login" into the text box on their device and clicks the search button.

[0684] 2. The emotion analysis engine recognizes irritation from the user's facial expressions and voice.

[0685] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[0686] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[0687] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[0688] 6. The server sends that information to the terminal in JSON format.

[0689] 7. Display a list of the information received by the device.

[0690] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0691] Following the above process, this invention enables users to quickly and efficiently acquire information related to business terminology and further enhance the user experience through sentiment analysis. The detailed steps for realizing this system are as follows.

[0692] Example of a prompt

[0693] "A scenario where a user is experiencing login issues. The user types 'login'. The sentiment analysis engine detects frustration. The system then describes the steps involved in retrieving relevant information, organizing it based on emotion, and displaying the results."

[0694] This allows users to quickly obtain appropriate information tailored to their emotions, resulting in a significant improvement in work efficiency.

[0695] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0696] Step 1: Initialization process

[0697] The server starts the Apache HTTP Server and initializes the MySQL database. During the initialization process, business terminology, contact information, business systems, and request systems are stored in the database. This prepares the server for search operations. The input is the database initialization script, and the output is the initialized database. The specific actions involve the startup commands for Apache and MySQL, and the execution of the script.

[0698] Step 2: Wait for user response

[0699] The device enters a state where it waits for user input in a web browser. A search text box and search button appear on the screen, and the sentiment analysis engine starts running in the background. Input is the user interface initialization script, and output is an interactive search form. The specific operation involves building the UI using HTML and JavaScript, and starting the sentiment analysis engine.

[0700] Step 3: Enter search query and sentiment recognition

[0701] The user enters a search query into a text box and clicks the search button. The sentiment analysis engine analyzes the user's facial expressions and voice in real time to obtain emotional information. The input is the user's search query and emotional data, and the output is the obtained emotional state. Specifically, the operation includes data collection and analysis by the sentiment analysis engine.

[0702] Step 4: Submit your search query

[0703] The terminal converts the user's search query and recognized sentiment information into JSON format and sends it to the server as an HTTP request. The input is the search query and sentiment information, and the output is the HTTP request sent to the server. Specifically, the operation involves converting the data to JSON and generating the HTTP request using JavaScript.

[0704] Step 5: Database search and result retrieval

[0705] The server receives an HTTP request and searches a MySQL database based on the submitted search query. It extracts information about relevant contacts, business systems, and request systems related to the specified terms. The input is the search query included in the HTTP request, and the output is the retrieved search results. Specific operations include database queries and data extraction using PHP or Python scripts.

[0706] Step 6: Outcome organization based on emotional information

[0707] The server reorganizes the retrieved search results based on the sentiment information recognized by the sentiment analysis engine. If the user is frustrated, it prioritizes displaying information that can provide a quick and simple solution. The input is the recognized sentiment information and search results, and the output is the reorganized search results. The specific operation involves applying sentiment information analysis algorithms and restructuring the data.

[0708] Step 7: Submit and view results

[0709] The server formats the reorganized information in JSON or XML format and sends it to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. The input is the organized search results, and the output is the information displayed on the UI. Specific operations include generating the HTTP response, creating the JSON / XML format, and updating the UI using JavaScript.

[0710] Step 8: User Verification and Use

[0711] Users review information displayed on their devices, contact necessary contacts, and use relevant systems to carry out their tasks. Input is the displayed information, and output is the user's actions (e.g., contacting a contact). Specific actions include the user clicking or selecting information and then performing specific actions based on those clicks.

[0712] (Application Example 2)

[0713] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0714] In previous factory troubleshooting processes, operators had to spend time and effort searching for information without considering their own emotions. This made it difficult to quickly access the right information, especially when feeling anxious or frustrated, leading to decreased work efficiency.

[0715] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving a search query input by a user, means for searching a database based on the search query, means for organizing the search results and extracting the contact information and related system information, means for organizing the extracted information based on emotion recognition and presenting it with priority according to the user's emotions, means for displaying the extracted information to the user in a list, emotion engine means for recognizing the user's emotions in real time, and means for attaching emotion information obtained from the emotion engine when sending a search query. As a result, even when the operator is feeling frustrated or anxious, it becomes possible to quickly and efficiently access appropriate information.

[0716] A "user" refers to a person who operates this system and enters a search query.

[0717] A "search query" refers to the keywords or phrases that a user enters to identify the information they are looking for.

[0718] A "database" refers to an information aggregation system that stores business-related data such as contact information and related system information.

[0719] An "emotion engine" refers to a system that recognizes and analyzes a user's emotions in real time and acquires that information.

[0720] An "HTTP request" refers to a data communication method used to send a user's search queries and sentiment information to a server.

[0721] "Organization" refers to the act of prioritizing and presenting search results according to the user's emotional state.

[0722] A "server" refers to a computer system that receives search queries and sentiment information, searches a database, and organizes the results.

[0723] "JSON format" is a lightweight data description format for data exchange, and is an abbreviation for JavaScript Object Notation.

[0724] XML is a highly readable markup language widely used for data exchange, and its name is an abbreviation for Extensible Markup Language.

[0725] "Troubleshooting" refers to the procedures and processes used to diagnose and resolve malfunctions and problems in machines and systems.

[0726] This invention is a system that can be applied to applications that support troubleshooting within a factory. Specifically, it provides a system that allows a user to operate a factory robot and efficiently obtain the information necessary for troubleshooting. Because this system integrates search query processing and sentiment recognition functions, it can present optimal information according to the user's emotional state.

[0727] Hardware and software to be used

[0728] Hardware:

[0729] User interface devices such as tablet devices and smart glasses

[0730] Factory robots and central control systems

[0731] software:

[0732] Emotion engine (e.g., Microsoft Azure Cognitive Services)

[0733] SQL database (e.g., MySQL)

[0734] HTTP server (e.g., Apache)

[0735] Explanation of the process

[0736] 1. Server role:

[0737] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information.

[0738] 2. The role of the terminal:

[0739] Users input details of their problem using a tablet or smart glasses. The device displays a search text box and a search button for user interaction. Additionally, an emotion engine analyzes the user's facial expressions, voice, and typing speed in real time to acquire emotional information.

[0740] 3. The role of the emotional engine:

[0741] The emotion engine analyzes the user's emotions in real time, retrieves that information, and attaches it when the search query is submitted. This emotion information is used in subsequent processing to organize the results.

[0742] 4. Sending search queries to the server:

[0743] The device sends search queries and sentiment information from the user to the server in the form of an HTTP request.

[0744] 5. Search the server's database and retrieve the results:

[0745] The server searches the database based on the submitted search query and extracts relevant information. The search results are formatted in JSON or XML and provided to the user.

[0746] 6. Organizing search results:

[0747] The server organizes the emotional information obtained by the emotion engine and displays it with priority according to the user's emotional state.

[0748] Specific example

[0749] For example, when a user is performing a "fault diagnosis" on a machine in a factory, if the operator is in a hurry, it would be good to input a prompt message like the following into the AI ​​model:

[0750] Example of a prompt:

[0751] An operator is diagnosing a machine malfunction in the factory. They enter "Error Code 123" into the device, and the emotion engine recognizes an emotion of frustration. Generate clear instructions for the operator, including steps and contact information for a quick and easy resolution.

[0752] By following the steps outlined above, the generative AI model can provide optimal troubleshooting information based on the search queries and sentiment information entered by the user. This system allows users to efficiently and quickly resolve manufacturing issues.

[0753] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0754] Step 1:

[0755] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information. This information is used to prepare subsequent search processes. Initial database data is supplied to the server as input, and an initialized database is generated as output.

[0756] Step 2:

[0757] Users input details of their problem using a tablet or smart glasses. The device displays a text box and a search button for user interaction. The input is a search query entered by the user, and the output is the search query displayed on the user interface.

[0758] Step 3:

[0759] The device's emotion engine analyzes the user's facial expressions, voice, typing speed, etc., in real time to acquire emotional information. The input is biometric data indicating the user's emotions, which the emotion engine analyzes and outputs as emotional information.

[0760] Step 4:

[0761] The terminal sends search queries and sentiment information from the user to the server in the form of an HTTP request. The input consists of search queries and sentiment information, which are sent to the server in JSON or XML format. The server receives this data as output.

[0762] Step 5:

[0763] The server searches the database based on the submitted search query. It uses SQL to extract relevant information. The input is the received search query, and the output is the relevant information as search results. The server organizes this data and provides it in the format best suited to the user's request.

[0764] Step 6:

[0765] The server organizes information based on emotional data acquired by the emotion engine. Relevant information is prioritized and displayed according to the user's emotional state. Input consists of emotional data and search results; data processing is performed based on the emotional data, and prioritized information is generated as output.

[0766] Step 7:

[0767] The server sends organized information to the terminal in JSON or XML format. The input is prioritized information, which is then formatted and sent to the terminal. The terminal receives this information as output.

[0768] Step 8:

[0769] The terminal analyzes the information it receives and displays it on the user interface. The input is information sent from the server and displayed in an appropriate format on the user interface. The output is a display of information that the user can actually view and use.

[0770] Step 9:

[0771] The user reviews the information displayed on the device and performs troubleshooting steps. The input is the information displayed on the device, which the user uses to contact support or perform steps to resolve the problem. The output is the result of the resolved work.

[0772] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0773] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0775] [Third Embodiment]

[0776] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0777] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0778] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0779] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0780] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0781] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0782] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0783] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0784] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0785] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0786] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0787] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0788] System Overview

[0789] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0790] Specific implementations of the system

[0791] 1. Initialization process

[0792] The server initializes the database. The database stores various business terms and related contact information, business systems, and request systems. This prepares the server for search processing.

[0793] 2. User actions

[0794] The terminal is set to wait for user input. The user operates the terminal and enters a specific business term as a search query. For example, they enter the term "login" and click the search button.

[0795] 3. Submitting a search query

[0796] The terminal receives the search query entered by the user and sends it to the server in HTTP request format. This allows the server to search the database based on that search query.

[0797] 4. Database search and result retrieval

[0798] The server searches the database based on the search query it receives. The server extracts information from the database about contacts, business systems, and request systems related to the relevant terms. The search results are formatted in JSON or XML and sent to the terminal.

[0799] 5. Summarizing and displaying the results

[0800] The terminal analyzes the search results received from the server and displays them to the user in a list. For example, it may display contact information related to "login" (e.g., support desk), business systems (e.g., user management system), request systems (e.g., password reset request system), etc.

[0801] 6. User Verification and Use

[0802] Users review the displayed list, contact the necessary contacts, and use relevant systems to carry out their tasks. This allows users to quickly and efficiently obtain the necessary information and perform their tasks smoothly.

[0803] Specific example

[0804] Example 1: Searching for the term "login"

[0805] 1. The user enters "login" into their device and clicks the search button.

[0806] 2. The terminal sends the input to the server in the form of an HTTP request.

[0807] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0808] 4. The server sends that information to the terminal in JSON format.

[0809] 5. Display a list of information received by the device.

[0810] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0811] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0812] The following describes the processing flow.

[0813] Step 1:

[0814] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0815] Step 2:

[0816] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction.

[0817] Step 3:

[0818] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button.

[0819] Step 4:

[0820] The terminal receives user input and sends that input as a search query in the form of an HTTP request to the server.

[0821] Step 5:

[0822] The server receives the HTTP request and parses the submitted search query. It then verifies that the parsed search query contains specific business terminology.

[0823] Step 6:

[0824] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[0825] Step 7:

[0826] The server extracts information, organizes it, and converts it into a format that is easy for users to understand. For example, it can be formatted in JSON or XML format.

[0827] Step 8:

[0828] The server sends organized and formatted information to the terminal as an HTTP response.

[0829] Step 9:

[0830] The system analyzes the response received by the terminal and displays it on the user interface. Specifically, information on contact points, business systems, and request systems is displayed in a list format.

[0831] Step 10:

[0832] Users review the information displayed on their devices, contact the necessary contacts, and perform their tasks using relevant systems.

[0833] By following these steps, users can quickly obtain information related to specific business terminology and improve their work efficiency.

[0834] (Example 1)

[0835] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0836] Conventional business search systems have a problem in that users have to spend a lot of time and effort searching for specific terms. In particular, obtaining contact information and related system information requires manual operation, which results in a decrease in work efficiency.

[0837] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0838] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for formatting the search results in JSON or XML format, means for transmitting the formatted data to a terminal, means for the terminal to analyze the received data and visually display it to the user in a list, means for the user to confirm the displayed information and contact a relevant contact or system as necessary, means for transmitting the search query to the server in HTTP request format, and means for initializing the database at startup. As a result, the user can quickly and easily obtain relevant information and significantly improve the efficiency of their work.

[0839] A "user" refers to someone who uses the system and is responsible for searching for specific terms to retrieve information.

[0840] A "search query" is a word or phrase that a user enters into a system to search for something.

[0841] A "server" is a computer system that receives search queries, searches a database, formats the results, and sends them to a terminal.

[0842] A "terminal" is a device operated by a user, where they input search queries and display information sent from a server.

[0843] A "database" is an information aggregation system that stores business terminology and related contact information, business systems, and request systems.

[0844] An "HTTP request" is a communication protocol used by web browsers and devices to request information from a server.

[0845] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[0846] "XML format" is an abbreviation for eXtensible Markup Language, and it is a format for representing data hierarchically.

[0847] "List view" refers to displaying acquired information in a way that allows the user to visually confirm it.

[0848] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[0849] System Overview

[0850] The basic operation of this system is as follows: The user operates a terminal and enters specific business terms as a search query. The terminal sends the query to the server, which searches the database, formats the results, and sends them back to the terminal. The terminal parses the received data and displays it visually to the user.

[0851] Hardware and software used

[0852] 1. Server: Use a Linux-based web server (e.g., Apache, Nginx). This server will receive search queries, search the database, format the results, and send them.

[0853] 2. Database: A database management system such as MySQL or PostgreSQL will be used. The database will store information on various terms and related query targets, business systems, and request systems.

[0854] 3. Terminal: Use a PC or tablet device running Windows or Mac OS. The terminal receives user input, sends it to the server, and displays the results.

[0855] 4. Communication protocol: Use HTTP / HTTPS. HTTP requests are used to send and receive search queries and results.

[0856] Specific example

[0857] Next, we will show a concrete example based on the sequence of operations of this system.

[0858] Example 1: Searching for the term "login"

[0859] 1. The user enters "login" into their device and clicks the search button.

[0860] 2. The terminal sends the input to the server in the form of an HTTP request.

[0861] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[0862] 4. The server sends that information to the terminal in JSON format.

[0863] 5. Display a list of information received by the device.

[0864] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[0865] Examples of prompts to input into a generative AI model

[0866] "Please describe the overall process for when a user searches for the business term 'login'."

[0867] "Please explain in detail the data processing that occurs when searching for specific business terms, from the perspectives of the server, terminal, and user."

[0868] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[0869] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0870] Step 1:

[0871] The server initializes the database. At startup, it calls the `init_database()` function to clear existing data and then populate it with new data, such as business terminology, contact information, and system information. This prepares the system to handle search queries.

[0872] Input: None (at server startup)

[0873] Output: Initialized database

[0874] Specific operation: Initialize the database table using an SQL query and insert new data.

[0875] Step 2:

[0876] The terminal is set to wait for user input. The user enters a specific business term into the terminal's search box. For example, they might type "login" into the search box and click the search button.

[0877] Input: The term entered by the user in the search box.

[0878] Output: Preparation of search queries by terminal

[0879] Specific actions: Search box ( <input type="text" id="search_term"> The user enters information in the field.

[0880] Step 3:

[0881] The terminal receives the search query entered by the user and sends it to the server in the form of an HTTP request. The request is sent using the JavaScript fetch function.

[0882] Input: Search query entered by the user

[0883] Output: Query submission in HTTP request format

[0884] Specific operation: The fetch function sends a request to the server as follows:

[0885] JavaScript

[0886] fetch(' / search', {

[0887] method: 'POST',

[0888] headers: {

[0889] 'Content-Type': 'application / json'

[0890] },

[0891] body: JSON.stringify({ term: 'Login'})

[0892] });

[0893] Step 4:

[0894] The server parses the received search query and searches the database using SQL or other query languages. The server extracts information related to the relevant terms and formats it in JSON or XML format.

[0895] Input: HTTP request sent from the terminal

[0896] Output: Formatted search results

[0897] Specific operation: Execute the following SQL statement and convert the result to JSON format.

[0898] sql

[0899] SELECT contact, system, request_system FROM terms WHERE term = 'Login';

[0900] Step 5:

[0901] The server formats the search results in JSON format and sends them to the terminal as an HTTP response. This returns the data from the server to the terminal.

[0902] Input: Search results from database

[0903] Output: Response data in JSON format

[0904] Specific operation: The search results are structured in JSON format and sent to the terminal as an HTTP response.

[0905] Step 6:

[0906] The terminal parses the received JSON data and displays it to the user in a list. It parses the data and displays it in a visually easy-to-understand format.

[0907] Input: Search results in JSON format from the server

[0908] Output: Information list displayed to the user

[0909] Specific operation: Use JavaScript to display the results on HTML.

[0910] JavaScript

[0911] const results = await response.json();

[0912] document.getElementById('results').innerHTML =

[0913] Contact us at: ${results.contact}

[0914] Business system: ${results.system}

[0915] Request system: ${results.request_system}

[0916] ;

[0917] Step 7:

[0918] Users review the displayed information, contact the necessary contacts, and use relevant systems. This allows users to quickly and efficiently obtain the necessary information and proceed with their work.

[0919] Input: Search results displayed on the device

[0920] Output: User actions (inquiries, system usage)

[0921] Specific action: Based on the information displayed to the user, for example, they might call the support desk.

[0922] This concludes the detailed explanation of the system's program processing steps.

[0923] (Application Example 1)

[0924] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0925] In conventional factories, workers spent considerable time and effort quickly searching for specific work terms and obtaining related contact information and system details. Furthermore, the lack of an efficient search system utilizing voice commands resulted in reduced work efficiency.

[0926] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0927] In this invention, the server includes means for recognizing voice instructions input by a user, means for converting the search query from voice to text, means for receiving the search query, means for searching a database based on the search query, means for organizing the search results and extracting contact information, related system information, and instruction manuals, and means for displaying the extracted information to the user in a list. This enables workers to quickly obtain necessary information using voice instructions and to proceed with their work efficiently.

[0928] "Voice commands" refer to instructions given by the user to the system using voice.

[0929] A "search query" is a term or phrase entered by the user to be searched for.

[0930] "Means of converting to text" refers to technologies and devices for converting spoken instructions into written information.

[0931] A "database" is a system that stores data such as various business terms and related contact information, system information, and instruction manuals.

[0932] "Means of searching a database" refers to the technologies and functions used to retrieve relevant information within a database based on a search query.

[0933] A "search result" is a set of relevant information extracted from a database.

[0934] "Contact information" refers to contact details related to specific business terminology.

[0935] "System information" refers to information about systems and services related to specific business terminology.

[0936] An "instruction manual" is a document that describes the operating procedures and usage methods related to specific business terminology.

[0937] "Means of displaying in a list" refers to technologies or devices that display search results in a format that users can view.

[0938] System Overview

[0939] This system allows users to input business-related voice commands and quickly and easily obtain contact information, system information, and instruction manuals related to those voice commands. The system primarily consists of three components: a server, terminals, and users.

[0940] Initialization process

[0941] The server first initializes the database. The database contains information about various business terms. This prepares the server to perform search operations based on voice commands.

[0942] User actions

[0943] The user inputs the search term or phrase into the device using voice commands. The device then uses a speech recognition system (e.g., Google Cloud Speech API) to convert the voice into text. For example, if the user says "parts replacement procedure" by voice, the system will convert it into text.

[0944] Submit a search query

[0945] The terminal sends the converted text-based search query to the server as an HTTP request. This allows the server to search the database based on that search query.

[0946] Database search and result retrieval

[0947] The server searches the database based on the received search query and extracts information such as contact information, system information, and instruction manuals related to the relevant terms. The search results are formatted in JSON or XML format and sent to the terminal.

[0948] Sorting and displaying results

[0949] The terminal analyzes the search results received from the server and displays them in a list. This allows the user to see the necessary information at a glance. For example, it displays contact information related to "parts replacement procedures" (e.g., technical support desk), system information (e.g., parts management system), and instruction manuals (e.g., parts replacement manual).

[0950] User verification and usage

[0951] Users review the displayed list and proceed with their tasks by contacting the necessary contacts or using relevant systems. This allows users to quickly and efficiently obtain the necessary information and carry out their tasks smoothly.

[0952] Specific example

[0953] For example, if the user enters the voice command "parts replacement procedure," this application will operate as follows:

[0954] Example of a prompt:

[0955] Please enter "parts replacement procedure".

[0956] result:

[0957] Terminology: Parts replacement procedure

[0958] Contact: Technical Support Desk

[0959] System: Parts Management System

[0960] Manual: Parts Replacement Manual v1.2

[0961] The main hardware and software used include a computer built into the factory robot, a speech recognition microphone, a display, Python, the Google Cloud Speech API, and the Requests library.

[0962] This allows workers to quickly obtain necessary information using voice commands and carry out their tasks efficiently.

[0963] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0964] Step 1:

[0965] The user inputs voice instructions. Specifically, the user speaks business terms such as "parts replacement procedure" into the terminal within the factory. Voice data is generated as input.

[0966] Step 2:

[0967] The device receives voice commands and passes them to the speech recognition system. The speech recognition system (e.g., Google Cloud Speech API) analyzes the voice data and converts it into text data. Specifically, this step converts the voice data into a text-based search query. The output is text data titled "Parts Replacement Procedure".

[0968] Step 3:

[0969] The terminal sends the converted text data to the server as a search query in the form of an HTTP request. Specifically, it formats the text data into an HTTP request format and sends it to the specified endpoint on the server. The request sent as input contains the text data "parts replacement procedure".

[0970] Step 4:

[0971] The server searches the database based on the search query it receives. Specifically, the server executes an SQL query against the database to extract contact information, system information, and instruction manuals related to the relevant business term. Given a text query "parts replacement procedure" as input, the output will be contact information, system information, and instruction manuals.

[0972] Step 5:

[0973] The server formats the retrieved search results in JSON or XML format. Specifically, it converts the extracted information into the specified data format. The search results data is given as input, and the output is data in JSON or XML format.

[0974] Step 6:

[0975] The server sends formatted search results to the terminal. Specifically, the server sends data to the terminal as an HTTP response. JSON or XML formatted data is given as input, and an HTTP response is generated as output.

[0976] Step 7:

[0977] The terminal analyzes the search results received from the server and displays them to the user in a list. Specifically, the terminal analyzes the received data and formats it so that it can be displayed on the screen. JSON or XML format data is provided as input, and the content of the list is generated as output.

[0978] Step 8:

[0979] Users review the displayed information and contact the necessary contacts or use related systems. Specifically, users view the displayed information and take the required actions. The displayed information is provided as input, and the user's actions are the output.

[0980] The steps described above allow users to efficiently obtain necessary information using voice commands and carry out their work smoothly.

[0981] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0982] System Overview

[0983] This invention enhances the user experience by combining a system that allows users to search for business terms and quickly and easily obtain information on related contacts, business systems, and request systems with an emotion engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion engine.

[0984] Specific implementations of the system

[0985] 1. Initialization process

[0986] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[0987] 2. User actions

[0988] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction. At the same time, the emotion engine is activated, analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[0989] 3. User emotion recognition

[0990] The emotion engine analyzes the user's emotions in real time as they enter their search queries and retrieves emotional information. For example, if the user is irritated, it will recognize that.

[0991] 4. Submitting a search query

[0992] The terminal sends the user-inputted search query to the server in HTTP request format, along with the recognized sentiment information.

[0993] 5. Database search and result retrieval

[0994] The server receives an HTTP request and searches the database based on the submitted search query. The server extracts information from the database regarding contacts, business systems, and request systems related to the relevant term.

[0995] 6. Organizing results based on emotional information

[0996] The server organizes the acquired information based on the emotional information recognized by the emotion engine. If the user is frustrated, it prioritizes displaying information that can resolve the issue more simply and quickly.

[0997] 7. Submitting and displaying results

[0998] The server sends organized and formatted information to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. It displays a list of contact information, business systems, and request systems according to a specific layout.

[0999] 8. User Verification and Use

[1000] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to carry out their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and carry out their tasks efficiently.

[1001] Specific example

[1002] Example 1: Searching for the term "login" and sentiment recognition

[1003] 1. The user enters "login" into their device and clicks the search button.

[1004] 2. The emotion engine recognizes that the user is frustrated.

[1005] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[1006] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[1007] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[1008] 6. The server sends that information to the terminal in JSON format.

[1009] 7. Display a list of the information received by the device.

[1010] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1011] This system allows users to receive information related to the terms they enter, tailored to their emotions, and as a result, significantly improves work efficiency.

[1012] The following describes the processing flow.

[1013] Step 1:

[1014] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[1015] Step 2:

[1016] The terminal enters a state of waiting for user input. A search text box and search button appear on the terminal, and it is ready for user interaction. Simultaneously, the emotion engine starts up and begins analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[1017] Step 3:

[1018] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button. The emotion engine retrieves emotional information from the user's facial expressions and voice, and recognizes, for example, that the user is irritated.

[1019] Step 4:

[1020] The device sends the user's input as a search query in the form of an HTTP request to the server. Simultaneously, it also sends the sentiment information acquired by the sentiment engine.

[1021] Step 5:

[1022] The server receives the HTTP request and analyzes the submitted search query and sentiment information. It then verifies that the analyzed search query is specific business terminology.

[1023] Step 6:

[1024] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[1025] Step 7:

[1026] The server organizes the extracted information. Based on the sentiment information recognized by the sentiment engine, if the user is frustrated, it prioritizes and organizes information that can be resolved more simply and quickly. For example, it highlights high-priority contact information and links to readily available systems.

[1027] Step 8:

[1028] The server sends organized and formatted information to the terminal as an HTTP response. The format should be user-friendly, such as JSON or XML.

[1029] Step 9:

[1030] The system analyzes the response received by the terminal and displays it on the user interface. Information on contact points, business systems, and request systems is displayed in a list format according to a specific layout.

[1031] Step 10:

[1032] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to perform their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and perform their tasks efficiently.

[1033] Through these steps, users can be provided with information related to specific business terminology in an emotionally resonant manner, resulting in a significant improvement in work efficiency.

[1034] (Example 2)

[1035] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1036] Conventional business information retrieval systems, while presenting relevant information in response to user-entered search queries, often displayed uniform information without considering the user's emotions, resulting in a poor user experience. In particular, users experiencing negative emotions such as frustration or impatience required rapid and efficient information delivery. However, existing systems lacked the ability to recognize user emotions in real time and reorganize information based on those emotions. Therefore, the challenge to be addressed is improving the user experience through flexible information delivery that responds to the user's emotions.

[1037] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1038] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for obtaining the search results and recognizing the user's emotions using an emotion analysis engine, means for reorganizing the obtained search results based on the recognized emotion information and setting priorities, and means for displaying the reorganized information to the user in a list. This enables the rapid and efficient provision of information in accordance with the user's emotions.

[1039] A "search query" is a term or phrase that a user enters into the system to search for.

[1040] A "database" is a collection of structured information that allows for the efficient management and retrieval of business-related terminology, contact information, and related system information.

[1041] An "emotion analysis engine" is a system that analyzes a user's emotional state in real time based on information obtained from the user's facial expressions, voice, and other interfaces.

[1042] An "HTTP request" is a type of data format sent from a user's device to a server, and is particularly used to send and receive information using the Internet communication protocol.

[1043] "JSON format" is a format for representing data as text, structuring it, and making it exchangeable. It is an abbreviation for JavaScript Object Notation.

[1044] XML format is a markup language used to structure and represent data, and it is an abbreviation for Extensible Markup Language.

[1045] "Setting priorities" means selecting the most useful information for the user from the search results based on the analyzed sentiment information and determining the display order.

[1046] "List display" refers to organizing acquired information and visually presenting it so that users can view it at a glance.

[1047] An "HTTP response" is response data sent from a server to a terminal, and it includes the results and processing details of a request.

[1048] This invention enhances the user experience by combining a system that allows users to quickly and easily search for business terms and obtain information on related contacts, business systems, and request systems with an emotion analysis engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion analysis engine.

[1049] The hardware used includes a standard computer server (e.g., an x86 architecture server). The software includes Apache HTTP Server and a MySQL database for the server, a web browser (e.g., Chrome, Firefox, etc.) for the terminal, and IBM Watson or Microsoft Azure Emotion API for the sentiment analysis engine.

[1050] When the server starts up, the Apache HTTP Server and MySQL database are initialized, and pre-prepared business terminology, contact information, business systems, and request systems are stored in the database. This database is then ready for search processing.

[1051] The user initiates a search using their device. A search text box and search button appear in the device's web browser, where the user enters the search terms. The sentiment analysis engine runs in the background, ready to analyze the user's facial expressions and voice in real time.

[1052] When a user enters a search query and clicks the search button, the device sends the search query and the recognized sentiment information to the server in the form of an HTTP request. The server receives this request and queries a MySQL database to extract relevant information. Next, the retrieved information is reorganized based on the user's sentiment information recognized by the sentiment analysis engine.

[1053] For example, if a user is frustrated, information that can resolve the issue quickly and easily will be prioritized and formatted. This prioritized information will be formatted in JSON or XML format and sent to the terminal as an HTTP response.

[1054] The terminal analyzes the received information and displays it on the user interface. Users can then review the displayed information, contact the necessary contacts, or use relevant systems to proceed with their work, enabling them to resolve problems quickly and without frustration.

[1055] Specific example

[1056] Example 1: Searching for the term "login" and sentiment recognition

[1057] 1. The user enters "login" into the text box on their device and clicks the search button.

[1058] 2. The emotion analysis engine recognizes irritation from the user's facial expressions and voice.

[1059] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[1060] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[1061] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[1062] 6. The server sends that information to the terminal in JSON format.

[1063] 7. Display a list of the information received by the device.

[1064] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1065] Following the above process, this invention enables users to quickly and efficiently acquire information related to business terminology and further enhance the user experience through sentiment analysis. The detailed steps for realizing this system are as follows.

[1066] Example of a prompt

[1067] "A scenario where a user is experiencing login issues. The user types 'login'. The sentiment analysis engine detects frustration. The system then describes the steps involved in retrieving relevant information, organizing it based on emotion, and displaying the results."

[1068] This allows users to quickly obtain appropriate information tailored to their emotions, resulting in a significant improvement in work efficiency.

[1069] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1070] Step 1: Initialization process

[1071] The server starts the Apache HTTP Server and initializes the MySQL database. During the initialization process, business terminology, contact information, business systems, and request systems are stored in the database. This prepares the server for search operations. The input is the database initialization script, and the output is the initialized database. The specific actions involve the startup commands for Apache and MySQL, and the execution of the script.

[1072] Step 2: Wait for user response

[1073] The device enters a state where it waits for user input in a web browser. A search text box and search button appear on the screen, and the sentiment analysis engine starts running in the background. Input is the user interface initialization script, and output is an interactive search form. The specific operation involves building the UI using HTML and JavaScript, and starting the sentiment analysis engine.

[1074] Step 3: Enter search query and sentiment recognition

[1075] The user enters a search query into a text box and clicks the search button. The sentiment analysis engine analyzes the user's facial expressions and voice in real time to obtain emotional information. The input is the user's search query and emotional data, and the output is the obtained emotional state. Specifically, the operation includes data collection and analysis by the sentiment analysis engine.

[1076] Step 4: Submit your search query

[1077] The terminal converts the user's search query and recognized sentiment information into JSON format and sends it to the server as an HTTP request. The input is the search query and sentiment information, and the output is the HTTP request sent to the server. Specifically, the operation involves converting the data to JSON and generating the HTTP request using JavaScript.

[1078] Step 5: Database search and result retrieval

[1079] The server receives an HTTP request and searches a MySQL database based on the submitted search query. It extracts information about relevant contacts, business systems, and request systems related to the specified terms. The input is the search query included in the HTTP request, and the output is the retrieved search results. Specific operations include database queries and data extraction using PHP or Python scripts.

[1080] Step 6: Outcome organization based on emotional information

[1081] The server reorganizes the retrieved search results based on the sentiment information recognized by the sentiment analysis engine. If the user is frustrated, it prioritizes displaying information that can provide a quick and simple solution. The input is the recognized sentiment information and search results, and the output is the reorganized search results. The specific operation involves applying sentiment information analysis algorithms and restructuring the data.

[1082] Step 7: Submit and view results

[1083] The server formats the reorganized information in JSON or XML format and sends it to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. The input is the organized search results, and the output is the information displayed on the UI. Specific operations include generating the HTTP response, creating the JSON / XML format, and updating the UI using JavaScript.

[1084] Step 8: User Verification and Use

[1085] Users review information displayed on their devices, contact necessary contacts, and use relevant systems to carry out their tasks. Input is the displayed information, and output is the user's actions (e.g., contacting a contact). Specific actions include the user clicking or selecting information and then performing specific actions based on those clicks.

[1086] (Application Example 2)

[1087] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1088] In previous factory troubleshooting processes, operators had to spend time and effort searching for information without considering their own emotions. This made it difficult to quickly access the right information, especially when feeling anxious or frustrated, leading to decreased work efficiency.

[1089] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving a search query input by a user, means for searching a database based on the search query, means for organizing the search results and extracting the contact information and related system information, means for organizing the extracted information based on emotion recognition and presenting it with priority according to the user's emotions, means for displaying the extracted information to the user in a list, emotion engine means for recognizing the user's emotions in real time, and means for attaching emotion information obtained from the emotion engine when sending a search query. As a result, even when the operator is feeling frustrated or anxious, it becomes possible to quickly and efficiently access appropriate information.

[1090] A "user" refers to a person who operates this system and enters a search query.

[1091] A "search query" refers to the keywords or phrases that a user enters to identify the information they are looking for.

[1092] A "database" refers to an information aggregation system that stores business-related data such as contact information and related system information.

[1093] An "emotion engine" refers to a system that recognizes and analyzes a user's emotions in real time and acquires that information.

[1094] An "HTTP request" refers to a data communication method used to send a user's search queries and sentiment information to a server.

[1095] "Organization" refers to the act of prioritizing and presenting search results according to the user's emotional state.

[1096] A "server" refers to a computer system that receives search queries and sentiment information, searches a database, and organizes the results.

[1097] "JSON format" is a lightweight data description format for data exchange, and is an abbreviation for JavaScript Object Notation.

[1098] XML is a highly readable markup language widely used for data exchange, and its name is an abbreviation for Extensible Markup Language.

[1099] "Troubleshooting" refers to the procedures and processes used to diagnose and resolve malfunctions and problems in machines and systems.

[1100] This invention is a system that can be applied to applications that support troubleshooting within a factory. Specifically, it provides a system that allows a user to operate a factory robot and efficiently obtain the information necessary for troubleshooting. Because this system integrates search query processing and sentiment recognition functions, it can present optimal information according to the user's emotional state.

[1101] Hardware and software to be used

[1102] Hardware:

[1103] User interface devices such as tablet devices and smart glasses

[1104] Factory robots and central control systems

[1105] software:

[1106] Emotion engine (e.g., Microsoft Azure Cognitive Services)

[1107] SQL database (e.g., MySQL)

[1108] HTTP server (e.g., Apache)

[1109] Explanation of the process

[1110] 1. Server role:

[1111] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information.

[1112] 2. The role of the terminal:

[1113] Users input details of their problem using a tablet or smart glasses. The device displays a search text box and a search button for user interaction. Additionally, an emotion engine analyzes the user's facial expressions, voice, and typing speed in real time to acquire emotional information.

[1114] 3. The role of the emotional engine:

[1115] The emotion engine analyzes the user's emotions in real time, retrieves that information, and attaches it when the search query is submitted. This emotion information is used in subsequent processing to organize the results.

[1116] 4. Sending search queries to the server:

[1117] The device sends search queries and sentiment information from the user to the server in the form of an HTTP request.

[1118] 5. Search the server's database and retrieve the results:

[1119] The server searches the database based on the submitted search query and extracts relevant information. The search results are formatted in JSON or XML and provided to the user.

[1120] 6. Organizing search results:

[1121] The server organizes the emotional information obtained by the emotion engine and displays it with priority according to the user's emotional state.

[1122] Specific example

[1123] For example, when a user is performing a "fault diagnosis" on a machine in a factory, if the operator is in a hurry, it would be good to input a prompt message like the following into the AI ​​model:

[1124] Example of a prompt:

[1125] An operator is diagnosing a machine malfunction in the factory. They enter "Error Code 123" into the device, and the emotion engine recognizes an emotion of frustration. Generate clear instructions for the operator, including steps and contact information for a quick and easy resolution.

[1126] By following the steps outlined above, the generative AI model can provide optimal troubleshooting information based on the search queries and sentiment information entered by the user. This system allows users to efficiently and quickly resolve manufacturing issues.

[1127] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1128] Step 1:

[1129] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information. This information is used to prepare subsequent search processes. Initial database data is supplied to the server as input, and an initialized database is generated as output.

[1130] Step 2:

[1131] Users input details of their problem using a tablet or smart glasses. The device displays a text box and a search button for user interaction. The input is a search query entered by the user, and the output is the search query displayed on the user interface.

[1132] Step 3:

[1133] The device's emotion engine analyzes the user's facial expressions, voice, typing speed, etc., in real time to acquire emotional information. The input is biometric data indicating the user's emotions, which the emotion engine analyzes and outputs as emotional information.

[1134] Step 4:

[1135] The terminal sends search queries and sentiment information from the user to the server in the form of an HTTP request. The input consists of search queries and sentiment information, which are sent to the server in JSON or XML format. The server receives this data as output.

[1136] Step 5:

[1137] The server searches the database based on the submitted search query. It uses SQL to extract relevant information. The input is the received search query, and the output is the relevant information as search results. The server organizes this data and provides it in the format best suited to the user's request.

[1138] Step 6:

[1139] The server organizes information based on emotional data acquired by the emotion engine. Relevant information is prioritized and displayed according to the user's emotional state. Input consists of emotional data and search results; data processing is performed based on the emotional data, and prioritized information is generated as output.

[1140] Step 7:

[1141] The server sends organized information to the terminal in JSON or XML format. The input is prioritized information, which is then formatted and sent to the terminal. The terminal receives this information as output.

[1142] Step 8:

[1143] The terminal analyzes the information it receives and displays it on the user interface. The input is information sent from the server and displayed in an appropriate format on the user interface. The output is a display of information that the user can actually view and use.

[1144] Step 9:

[1145] The user reviews the information displayed on the device and performs troubleshooting steps. The input is the information displayed on the device, which the user uses to contact support or perform steps to resolve the problem. The output is the result of the resolved work.

[1146] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1147] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1148] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1149] [Fourth Embodiment]

[1150] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1151] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1152] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1153] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1154] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1155] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1156] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1157] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1158] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1159] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1160] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1161] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1162] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1163] System Overview

[1164] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[1165] Specific implementations of the system

[1166] 1. Initialization process

[1167] The server initializes the database. The database stores various business terms and related contact information, business systems, and request systems. This prepares the server for search processing.

[1168] 2. User actions

[1169] The terminal is set to wait for user input. The user operates the terminal and enters a specific business term as a search query. For example, they enter the term "login" and click the search button.

[1170] 3. Submitting a search query

[1171] The terminal receives the search query entered by the user and sends it to the server in HTTP request format. This allows the server to search the database based on that search query.

[1172] 4. Database search and result retrieval

[1173] The server searches the database based on the search query it receives. The server extracts information from the database about contacts, business systems, and request systems related to the relevant terms. The search results are formatted in JSON or XML and sent to the terminal.

[1174] 5. Summarizing and displaying the results

[1175] The terminal analyzes the search results received from the server and displays them to the user in a list. For example, it may display contact information related to "login" (e.g., support desk), business systems (e.g., user management system), request systems (e.g., password reset request system), etc.

[1176] 6. User Verification and Use

[1177] Users review the displayed list, contact the necessary contacts, and use relevant systems to carry out their tasks. This allows users to quickly and efficiently obtain the necessary information and perform their tasks smoothly.

[1178] Specific example

[1179] Example 1: Searching for the term "login"

[1180] 1. The user enters "login" into their device and clicks the search button.

[1181] 2. The terminal sends the input to the server in the form of an HTTP request.

[1182] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[1183] 4. The server sends that information to the terminal in JSON format.

[1184] 5. Display a list of information received by the device.

[1185] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1186] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[1187] The following describes the processing flow.

[1188] Step 1:

[1189] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[1190] Step 2:

[1191] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction.

[1192] Step 3:

[1193] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button.

[1194] Step 4:

[1195] The terminal receives user input and sends that input as a search query in the form of an HTTP request to the server.

[1196] Step 5:

[1197] The server receives the HTTP request and parses the submitted search query. It then verifies that the parsed search query contains specific business terminology.

[1198] Step 6:

[1199] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[1200] Step 7:

[1201] The server extracts information, organizes it, and converts it into a format that is easy for users to understand. For example, it can be formatted in JSON or XML format.

[1202] Step 8:

[1203] The server sends organized and formatted information to the terminal as an HTTP response.

[1204] Step 9:

[1205] The system analyzes the response received by the terminal and displays it on the user interface. Specifically, information on contact points, business systems, and request systems is displayed in a list format.

[1206] Step 10:

[1207] Users review the information displayed on their devices, contact the necessary contacts, and perform their tasks using relevant systems.

[1208] By following these steps, users can quickly obtain information related to specific business terminology and improve their work efficiency.

[1209] (Example 1)

[1210] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1211] Conventional business search systems have a problem in that users have to spend a lot of time and effort searching for specific terms. In particular, obtaining contact information and related system information requires manual operation, which results in a decrease in work efficiency.

[1212] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1213] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for formatting the search results in JSON or XML format, means for transmitting the formatted data to a terminal, means for the terminal to analyze the received data and visually display it to the user in a list, means for the user to confirm the displayed information and contact a relevant contact or system as necessary, means for transmitting the search query to the server in HTTP request format, and means for initializing the database at startup. As a result, the user can quickly and easily obtain relevant information and significantly improve the efficiency of their work.

[1214] A "user" refers to someone who uses the system and is responsible for searching for specific terms to retrieve information.

[1215] A "search query" is a word or phrase that a user enters into a system to search for something.

[1216] A "server" is a computer system that receives search queries, searches a database, formats the results, and sends them to a terminal.

[1217] A "terminal" is a device operated by a user, where they input search queries and display information sent from a server.

[1218] A "database" is an information aggregation system that stores business terminology and related contact information, business systems, and request systems.

[1219] An "HTTP request" is a communication protocol used by web browsers and devices to request information from a server.

[1220] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[1221] "XML format" is an abbreviation for eXtensible Markup Language, and it is a format for representing data hierarchically.

[1222] "List view" refers to displaying acquired information in a way that allows the user to visually confirm it.

[1223] This invention relates to a system that allows users to search for business terms and quickly and easily obtain information on contact points, business systems, and request systems related to those terms. This system consists of three main components: a server, a terminal, and a user.

[1224] System Overview

[1225] The basic operation of this system is as follows: The user operates a terminal and enters specific business terms as a search query. The terminal sends the query to the server, which searches the database, formats the results, and sends them back to the terminal. The terminal parses the received data and displays it visually to the user.

[1226] Hardware and software used

[1227] 1. Server: Use a Linux-based web server (e.g., Apache, Nginx). This server will receive search queries, search the database, format the results, and send them.

[1228] 2. Database: A database management system such as MySQL or PostgreSQL will be used. The database will store information on various terms and related query targets, business systems, and request systems.

[1229] 3. Terminal: Use a PC or tablet device running Windows or Mac OS. The terminal receives user input, sends it to the server, and displays the results.

[1230] 4. Communication protocol: Use HTTP / HTTPS. HTTP requests are used to send and receive search queries and results.

[1231] Specific example

[1232] Next, we will show a concrete example based on the sequence of operations of this system.

[1233] Example 1: Searching for the term "login"

[1234] 1. The user enters "login" into their device and clicks the search button.

[1235] 2. The terminal sends the input to the server in the form of an HTTP request.

[1236] 3. The server searches the database and retrieves information about contact points, business systems, and request systems related to "login".

[1237] 4. The server sends that information to the terminal in JSON format.

[1238] 5. Display a list of information received by the device.

[1239] 6. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1240] Examples of prompts to input into a generative AI model

[1241] "Please describe the overall process for when a user searches for the business term 'login'."

[1242] "Please explain in detail the data processing that occurs when searching for specific business terms, from the perspectives of the server, terminal, and user."

[1243] This system allows users to easily obtain information related to the terms they enter, thereby significantly improving work efficiency.

[1244] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1245] Step 1:

[1246] The server initializes the database. At startup, it calls the `init_database()` function to clear existing data and then populate it with new data, such as business terminology, contact information, and system information. This prepares the system to handle search queries.

[1247] Input: None (at server startup)

[1248] Output: Initialized database

[1249] Specific operation: Initialize the database table using an SQL query and insert new data.

[1250] Step 2:

[1251] The terminal is set to wait for user input. The user enters a specific business term into the terminal's search box. For example, they might type "login" into the search box and click the search button.

[1252] Input: The term entered by the user in the search box.

[1253] Output: Preparation of search queries by terminal

[1254] Specific actions: Search box ( <input type="text" id="search_term"> The user enters information in the field.

[1255] Step 3:

[1256] The terminal receives the search query entered by the user and sends it to the server in the form of an HTTP request. The request is sent using the JavaScript fetch function.

[1257] Input: Search query entered by the user

[1258] Output: Query submission in HTTP request format

[1259] Specific operation: The fetch function sends a request to the server as follows:

[1260] JavaScript

[1261] fetch(' / search', {

[1262] method: 'POST',

[1263] headers: {

[1264] 'Content-Type': 'application / json'

[1265] },

[1266] body: JSON.stringify({ term: 'Login'})

[1267] });

[1268] Step 4:

[1269] The server parses the received search query and searches the database using SQL or other query languages. The server extracts information related to the relevant terms and formats it in JSON or XML format.

[1270] Input: HTTP request sent from the terminal

[1271] Output: Formatted search results

[1272] Specific operation: Execute the following SQL statement and convert the result to JSON format.

[1273] sql

[1274] SELECT contact, system, request_system FROM terms WHERE term = 'Login';

[1275] Step 5:

[1276] The server formats the search results in JSON format and sends them to the terminal as an HTTP response. This returns the data from the server to the terminal.

[1277] Input: Search results from database

[1278] Output: Response data in JSON format

[1279] Specific operation: The search results are structured in JSON format and sent to the terminal as an HTTP response.

[1280] Step 6:

[1281] The terminal parses the received JSON data and displays it to the user in a list. It parses the data and displays it in a visually easy-to-understand format.

[1282] Input: Search results in JSON format from the server

[1283] Output: Information list displayed to the user

[1284] Specific operation: Use JavaScript to display the results on HTML.

[1285] JavaScript

[1286] const results = await response.json();

[1287] document.getElementById('results').innerHTML =

[1288] Contact us at: ${results.contact}

[1289] Business system: ${results.system}

[1290] Request system: ${results.request_system}

[1291] ;

[1292] Step 7:

[1293] Users review the displayed information, contact the necessary contacts, and use relevant systems. This allows users to quickly and efficiently obtain the necessary information and proceed with their work.

[1294] Input: Search results displayed on the device

[1295] Output: User actions (inquiries, system usage)

[1296] Specific action: Based on the information displayed to the user, for example, they might call the support desk.

[1297] This concludes the detailed explanation of the system's program processing steps.

[1298] (Application Example 1)

[1299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1300] In conventional factories, workers spent considerable time and effort quickly searching for specific work terms and obtaining related contact information and system details. Furthermore, the lack of an efficient search system utilizing voice commands resulted in reduced work efficiency.

[1301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1302] In this invention, the server includes means for recognizing voice instructions input by a user, means for converting the search query from voice to text, means for receiving the search query, means for searching a database based on the search query, means for organizing the search results and extracting contact information, related system information, and instruction manuals, and means for displaying the extracted information to the user in a list. This enables workers to quickly obtain necessary information using voice instructions and to proceed with their work efficiently.

[1303] "Voice commands" refer to instructions given by the user to the system using voice.

[1304] A "search query" is a term or phrase entered by the user to be searched for.

[1305] "Means of converting to text" refers to technologies and devices for converting spoken instructions into written information.

[1306] A "database" is a system that stores data such as various business terms and related contact information, system information, and instruction manuals.

[1307] "Means of searching a database" refers to the technologies and functions used to retrieve relevant information within a database based on a search query.

[1308] A "search result" is a set of relevant information extracted from a database.

[1309] "Contact information" refers to contact details related to specific business terminology.

[1310] "System information" refers to information about systems and services related to specific business terminology.

[1311] An "instruction manual" is a document that describes the operating procedures and usage methods related to specific business terminology.

[1312] "Means of displaying in a list" refers to technologies or devices that display search results in a format that users can view.

[1313] System Overview

[1314] This system allows users to input business-related voice commands and quickly and easily obtain contact information, system information, and instruction manuals related to those voice commands. The system primarily consists of three components: a server, terminals, and users.

[1315] Initialization process

[1316] The server first initializes the database. The database contains information about various business terms. This prepares the server to perform search operations based on voice commands.

[1317] User actions

[1318] The user inputs the search term or phrase into the device using voice commands. The device then uses a speech recognition system (e.g., Google Cloud Speech API) to convert the voice into text. For example, if the user says "parts replacement procedure" by voice, the system will convert it into text.

[1319] Submit a search query

[1320] The terminal sends the converted text-based search query to the server as an HTTP request. This allows the server to search the database based on that search query.

[1321] Database search and result retrieval

[1322] The server searches the database based on the received search query and extracts information such as contact information, system information, and instruction manuals related to the relevant terms. The search results are formatted in JSON or XML format and sent to the terminal.

[1323] Sorting and displaying results

[1324] The terminal analyzes the search results received from the server and displays them in a list. This allows the user to see the necessary information at a glance. For example, it displays contact information related to "parts replacement procedures" (e.g., technical support desk), system information (e.g., parts management system), and instruction manuals (e.g., parts replacement manual).

[1325] User verification and usage

[1326] Users review the displayed list and proceed with their tasks by contacting the necessary contacts or using relevant systems. This allows users to quickly and efficiently obtain the necessary information and carry out their tasks smoothly.

[1327] Specific example

[1328] For example, if the user enters the voice command "parts replacement procedure," this application will operate as follows:

[1329] Example of a prompt:

[1330] Please enter "parts replacement procedure".

[1331] result:

[1332] Terminology: Parts replacement procedure

[1333] Contact: Technical Support Desk

[1334] System: Parts Management System

[1335] Manual: Parts Replacement Manual v1.2

[1336] The main hardware and software used include a computer built into the factory robot, a speech recognition microphone, a display, Python, the Google Cloud Speech API, and the Requests library.

[1337] This allows workers to quickly obtain necessary information using voice commands and carry out their tasks efficiently.

[1338] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1339] Step 1:

[1340] The user inputs voice instructions. Specifically, the user speaks business terms such as "parts replacement procedure" into the terminal within the factory. Voice data is generated as input.

[1341] Step 2:

[1342] The device receives voice commands and passes them to the speech recognition system. The speech recognition system (e.g., Google Cloud Speech API) analyzes the voice data and converts it into text data. Specifically, this step converts the voice data into a text-based search query. The output is text data titled "Parts Replacement Procedure".

[1343] Step 3:

[1344] The terminal sends the converted text data to the server as a search query in the form of an HTTP request. Specifically, it formats the text data into an HTTP request format and sends it to the specified endpoint on the server. The request sent as input contains the text data "parts replacement procedure".

[1345] Step 4:

[1346] The server searches the database based on the search query it receives. Specifically, the server executes an SQL query against the database to extract contact information, system information, and instruction manuals related to the relevant business term. Given a text query "parts replacement procedure" as input, the output will be contact information, system information, and instruction manuals.

[1347] Step 5:

[1348] The server formats the retrieved search results in JSON or XML format. Specifically, it converts the extracted information into the specified data format. The search results data is given as input, and the output is data in JSON or XML format.

[1349] Step 6:

[1350] The server sends formatted search results to the terminal. Specifically, the server sends data to the terminal as an HTTP response. JSON or XML formatted data is given as input, and an HTTP response is generated as output.

[1351] Step 7:

[1352] The terminal analyzes the search results received from the server and displays them to the user in a list. Specifically, the terminal analyzes the received data and formats it so that it can be displayed on the screen. JSON or XML format data is provided as input, and the content of the list is generated as output.

[1353] Step 8:

[1354] Users review the displayed information and contact the necessary contacts or use related systems. Specifically, users view the displayed information and take the required actions. The displayed information is provided as input, and the user's actions are the output.

[1355] The steps described above allow users to efficiently obtain necessary information using voice commands and carry out their work smoothly.

[1356] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1357] System Overview

[1358] This invention enhances the user experience by combining a system that allows users to search for business terms and quickly and easily obtain information on related contacts, business systems, and request systems with an emotion engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion engine.

[1359] Specific implementations of the system

[1360] 1. Initialization process

[1361] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[1362] 2. User actions

[1363] The terminal enters a state of waiting for user input. A search text box and search button are displayed on the terminal, and it is ready for user interaction. At the same time, the emotion engine is activated, analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[1364] 3. User emotion recognition

[1365] The emotion engine analyzes the user's emotions in real time as they enter their search queries and retrieves emotional information. For example, if the user is irritated, it will recognize that.

[1366] 4. Submitting a search query

[1367] The terminal sends the user-inputted search query to the server in HTTP request format, along with the recognized sentiment information.

[1368] 5. Database search and result retrieval

[1369] The server receives an HTTP request and searches the database based on the submitted search query. The server extracts information from the database regarding contacts, business systems, and request systems related to the relevant term.

[1370] 6. Organizing results based on emotional information

[1371] The server organizes the acquired information based on the emotional information recognized by the emotion engine. If the user is frustrated, it prioritizes displaying information that can resolve the issue more simply and quickly.

[1372] 7. Submitting and displaying results

[1373] The server sends organized and formatted information to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. It displays a list of contact information, business systems, and request systems according to a specific layout.

[1374] 8. User Verification and Use

[1375] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to carry out their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and carry out their tasks efficiently.

[1376] Specific example

[1377] Example 1: Searching for the term "login" and sentiment recognition

[1378] 1. The user enters "login" into their device and clicks the search button.

[1379] 2. The emotion engine recognizes that the user is frustrated.

[1380] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[1381] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[1382] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[1383] 6. The server sends that information to the terminal in JSON format.

[1384] 7. Display a list of the information received by the device.

[1385] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1386] This system allows users to receive information related to the terms they enter, tailored to their emotions, and as a result, significantly improves work efficiency.

[1387] The following describes the processing flow.

[1388] Step 1:

[1389] The server starts up and initializes the database. The database stores information on business terminology, contacts, business systems, and request systems. This prepares the server for search processing.

[1390] Step 2:

[1391] The terminal enters a state of waiting for user input. A search text box and search button appear on the terminal, and it is ready for user interaction. Simultaneously, the emotion engine starts up and begins analyzing the user's facial expressions, voice, typing speed, and other factors in real time.

[1392] Step 3:

[1393] The user enters a specific business term into the terminal. For example, they might type "login" and click the search button. The emotion engine retrieves emotional information from the user's facial expressions and voice, and recognizes, for example, that the user is irritated.

[1394] Step 4:

[1395] The device sends the user's input as a search query in the form of an HTTP request to the server. Simultaneously, it also sends the sentiment information acquired by the sentiment engine.

[1396] Step 5:

[1397] The server receives the HTTP request and analyzes the submitted search query and sentiment information. It then verifies that the analyzed search query is specific business terminology.

[1398] Step 6:

[1399] The server performs a search on the database. Specifically, it searches the database for terms that match the submitted search query and extracts information about the contacts, business systems, and request systems associated with those terms.

[1400] Step 7:

[1401] The server organizes the extracted information. Based on the sentiment information recognized by the sentiment engine, if the user is frustrated, it prioritizes and organizes information that can be resolved more simply and quickly. For example, it highlights high-priority contact information and links to readily available systems.

[1402] Step 8:

[1403] The server sends organized and formatted information to the terminal as an HTTP response. The format should be user-friendly, such as JSON or XML.

[1404] Step 9:

[1405] The system analyzes the response received by the terminal and displays it on the user interface. Information on contact points, business systems, and request systems is displayed in a list format according to a specific layout.

[1406] Step 10:

[1407] Users review the information displayed on their devices, contact the necessary contacts, and use relevant systems to perform their tasks. Information display based on emotional information allows users to quickly obtain the necessary information and perform their tasks efficiently.

[1408] Through these steps, users can be provided with information related to specific business terminology in an emotionally resonant manner, resulting in a significant improvement in work efficiency.

[1409] (Example 2)

[1410] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1411] Conventional business information retrieval systems, while presenting relevant information in response to user-entered search queries, often displayed uniform information without considering the user's emotions, resulting in a poor user experience. In particular, users experiencing negative emotions such as frustration or impatience required rapid and efficient information delivery. However, existing systems lacked the ability to recognize user emotions in real time and reorganize information based on those emotions. Therefore, the challenge to be addressed is improving the user experience through flexible information delivery that responds to the user's emotions.

[1412] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1413] In this invention, the server includes means for receiving a search query entered by a user, means for searching a database based on the search query, means for obtaining the search results and recognizing the user's emotions using an emotion analysis engine, means for reorganizing the obtained search results based on the recognized emotion information and setting priorities, and means for displaying the reorganized information to the user in a list. This enables the rapid and efficient provision of information in accordance with the user's emotions.

[1414] A "search query" is a term or phrase that a user enters into the system to search for.

[1415] A "database" is a collection of structured information that allows for the efficient management and retrieval of business-related terminology, contact information, and related system information.

[1416] An "emotion analysis engine" is a system that analyzes a user's emotional state in real time based on information obtained from the user's facial expressions, voice, and other interfaces.

[1417] An "HTTP request" is a type of data format sent from a user's device to a server, and is particularly used to send and receive information using the Internet communication protocol.

[1418] "JSON format" is a format for representing data as text, structuring it, and making it exchangeable. It is an abbreviation for JavaScript Object Notation.

[1419] XML format is a markup language used to structure and represent data, and it is an abbreviation for Extensible Markup Language.

[1420] "Setting priorities" means selecting the most useful information for the user from the search results based on the analyzed sentiment information and determining the display order.

[1421] "List display" refers to organizing acquired information and visually presenting it so that users can view it at a glance.

[1422] An "HTTP response" is response data sent from a server to a terminal, and it includes the results and processing details of a request.

[1423] This invention enhances the user experience by combining a system that allows users to quickly and easily search for business terms and obtain information on related contacts, business systems, and request systems with an emotion analysis engine that recognizes the user's emotions. The system consists of four main components: a server, a terminal, a user, and an emotion analysis engine.

[1424] The hardware used includes a standard computer server (e.g., an x86 architecture server). The software includes Apache HTTP Server and a MySQL database for the server, a web browser (e.g., Chrome, Firefox, etc.) for the terminal, and IBM Watson or Microsoft Azure Emotion API for the sentiment analysis engine.

[1425] When the server starts up, the Apache HTTP Server and MySQL database are initialized, and pre-prepared business terminology, contact information, business systems, and request systems are stored in the database. This database is then ready for search processing.

[1426] The user initiates a search using their device. A search text box and search button appear in the device's web browser, where the user enters the search terms. The sentiment analysis engine runs in the background, ready to analyze the user's facial expressions and voice in real time.

[1427] When a user enters a search query and clicks the search button, the device sends the search query and the recognized sentiment information to the server in the form of an HTTP request. The server receives this request and queries a MySQL database to extract relevant information. Next, the retrieved information is reorganized based on the user's sentiment information recognized by the sentiment analysis engine.

[1428] For example, if a user is frustrated, information that can resolve the issue quickly and easily will be prioritized and formatted. This prioritized information will be formatted in JSON or XML format and sent to the terminal as an HTTP response.

[1429] The terminal analyzes the received information and displays it on the user interface. Users can then review the displayed information, contact the necessary contacts, or use relevant systems to proceed with their work, enabling them to resolve problems quickly and without frustration.

[1430] Specific example

[1431] Example 1: Searching for the term "login" and sentiment recognition

[1432] 1. The user enters "login" into the text box on their device and clicks the search button.

[1433] 2. The emotion analysis engine recognizes irritation from the user's facial expressions and voice.

[1434] 3. The device sends its input and emotional information to the server in the form of an HTTP request.

[1435] 4. The server searches the database and retrieves information about contacts, business systems, and request systems related to "login".

[1436] 5. The server organizes the acquired information based on emotional information and prioritizes formatting information that can be resolved quickly.

[1437] 6. The server sends that information to the terminal in JSON format.

[1438] 7. Display a list of the information received by the device.

[1439] 8. Users should review the displayed information and use the necessary contact points or systems to quickly resolve login-related issues.

[1440] Following the above process, this invention enables users to quickly and efficiently acquire information related to business terminology and further enhance the user experience through sentiment analysis. The detailed steps for realizing this system are as follows.

[1441] Example of a prompt

[1442] "A scenario where a user is experiencing login issues. The user types 'login'. The sentiment analysis engine detects frustration. The system then describes the steps involved in retrieving relevant information, organizing it based on emotion, and displaying the results."

[1443] This allows users to quickly obtain appropriate information tailored to their emotions, resulting in a significant improvement in work efficiency.

[1444] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1445] Step 1: Initialization process

[1446] The server starts the Apache HTTP Server and initializes the MySQL database. During the initialization process, business terminology, contact information, business systems, and request systems are stored in the database. This prepares the server for search operations. The input is the database initialization script, and the output is the initialized database. The specific actions involve the startup commands for Apache and MySQL, and the execution of the script.

[1447] Step 2: Wait for user response

[1448] The device enters a state where it waits for user input in a web browser. A search text box and search button appear on the screen, and the sentiment analysis engine starts running in the background. Input is the user interface initialization script, and output is an interactive search form. The specific operation involves building the UI using HTML and JavaScript, and starting the sentiment analysis engine.

[1449] Step 3: Enter search query and sentiment recognition

[1450] The user enters a search query into a text box and clicks the search button. The sentiment analysis engine analyzes the user's facial expressions and voice in real time to obtain emotional information. The input is the user's search query and emotional data, and the output is the obtained emotional state. Specifically, the operation includes data collection and analysis by the sentiment analysis engine.

[1451] Step 4: Submit your search query

[1452] The terminal converts the user's search query and recognized sentiment information into JSON format and sends it to the server as an HTTP request. The input is the search query and sentiment information, and the output is the HTTP request sent to the server. Specifically, the operation involves converting the data to JSON and generating the HTTP request using JavaScript.

[1453] Step 5: Database search and result retrieval

[1454] The server receives an HTTP request and searches a MySQL database based on the submitted search query. It extracts information about relevant contacts, business systems, and request systems related to the specified terms. The input is the search query included in the HTTP request, and the output is the retrieved search results. Specific operations include database queries and data extraction using PHP or Python scripts.

[1455] Step 6: Outcome organization based on emotional information

[1456] The server reorganizes the retrieved search results based on the sentiment information recognized by the sentiment analysis engine. If the user is frustrated, it prioritizes displaying information that can provide a quick and simple solution. The input is the recognized sentiment information and search results, and the output is the reorganized search results. The specific operation involves applying sentiment information analysis algorithms and restructuring the data.

[1457] Step 7: Submit and view results

[1458] The server formats the reorganized information in JSON or XML format and sends it to the terminal as an HTTP response. The terminal parses the received information and displays it in the user interface. The input is the organized search results, and the output is the information displayed on the UI. Specific operations include generating the HTTP response, creating the JSON / XML format, and updating the UI using JavaScript.

[1459] Step 8: User Verification and Use

[1460] Users review information displayed on their devices, contact necessary contacts, and use relevant systems to carry out their tasks. Input is the displayed information, and output is the user's actions (e.g., contacting a contact). Specific actions include the user clicking or selecting information and then performing specific actions based on those clicks.

[1461] (Application Example 2)

[1462] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1463] In previous factory troubleshooting processes, operators had to spend time and effort searching for information without considering their own emotions. This made it difficult to quickly access the right information, especially when feeling anxious or frustrated, leading to decreased work efficiency.

[1464] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving a search query input by a user, means for searching a database based on the search query, means for organizing the search results and extracting the contact information and related system information, means for organizing the extracted information based on emotion recognition and presenting it with priority according to the user's emotions, means for displaying the extracted information to the user in a list, emotion engine means for recognizing the user's emotions in real time, and means for attaching emotion information obtained from the emotion engine when sending a search query. As a result, even when the operator is feeling frustrated or anxious, it becomes possible to quickly and efficiently access appropriate information.

[1465] A "user" refers to a person who operates this system and enters a search query.

[1466] A "search query" refers to the keywords or phrases that a user enters to identify the information they are looking for.

[1467] A "database" refers to an information aggregation system that stores business-related data such as contact information and related system information.

[1468] An "emotion engine" refers to a system that recognizes and analyzes a user's emotions in real time and acquires that information.

[1469] An "HTTP request" refers to a data communication method used to send a user's search queries and sentiment information to a server.

[1470] "Organization" refers to the act of prioritizing and presenting search results according to the user's emotional state.

[1471] A "server" refers to a computer system that receives search queries and sentiment information, searches a database, and organizes the results.

[1472] "JSON format" is a lightweight data description format for data exchange, and is an abbreviation for JavaScript Object Notation.

[1473] XML is a highly readable markup language widely used for data exchange, and its name is an abbreviation for Extensible Markup Language.

[1474] "Troubleshooting" refers to the procedures and processes used to diagnose and resolve malfunctions and problems in machines and systems.

[1475] This invention is a system that can be applied to applications that support troubleshooting within a factory. Specifically, it provides a system that allows a user to operate a factory robot and efficiently obtain the information necessary for troubleshooting. Because this system integrates search query processing and sentiment recognition functions, it can present optimal information according to the user's emotional state.

[1476] Hardware and software to be used

[1477] Hardware:

[1478] User interface devices such as tablet devices and smart glasses

[1479] Factory robots and central control systems

[1480] software:

[1481] Emotion engine (e.g., Microsoft Azure Cognitive Services)

[1482] SQL database (e.g., MySQL)

[1483] HTTP server (e.g., Apache)

[1484] Explanation of the process

[1485] 1. Server role:

[1486] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information.

[1487] 2. The role of the terminal:

[1488] Users input details of their problem using a tablet or smart glasses. The device displays a search text box and a search button for user interaction. Additionally, an emotion engine analyzes the user's facial expressions, voice, and typing speed in real time to acquire emotional information.

[1489] 3. The role of the emotional engine:

[1490] The emotion engine analyzes the user's emotions in real time, retrieves that information, and attaches it when the search query is submitted. This emotion information is used in subsequent processing to organize the results.

[1491] 4. Sending search queries to the server:

[1492] The device sends search queries and sentiment information from the user to the server in the form of an HTTP request.

[1493] 5. Search the server's database and retrieve the results:

[1494] The server searches the database based on the submitted search query and extracts relevant information. The search results are formatted in JSON or XML and provided to the user.

[1495] 6. Organizing search results:

[1496] The server organizes the emotional information obtained by the emotion engine and displays it with priority according to the user's emotional state.

[1497] Specific example

[1498] For example, when a user is performing a "fault diagnosis" on a machine in a factory, if the operator is in a hurry, it would be good to input a prompt message like the following into the AI ​​model:

[1499] Example of a prompt:

[1500] An operator is diagnosing a machine malfunction in the factory. They enter "Error Code 123" into the device, and the emotion engine recognizes an emotion of frustration. Generate clear instructions for the operator, including steps and contact information for a quick and easy resolution.

[1501] By following the steps outlined above, the generative AI model can provide optimal troubleshooting information based on the search queries and sentiment information entered by the user. This system allows users to efficiently and quickly resolve manufacturing issues.

[1502] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1503] Step 1:

[1504] The server starts up and initializes the database. The database stores information such as technical terms used for troubleshooting, operation manuals, and contact information. This information is used to prepare subsequent search processes. Initial database data is supplied to the server as input, and an initialized database is generated as output.

[1505] Step 2:

[1506] Users input details of their problem using a tablet or smart glasses. The device displays a text box and a search button for user interaction. The input is a search query entered by the user, and the output is the search query displayed on the user interface.

[1507] Step 3:

[1508] The device's emotion engine analyzes the user's facial expressions, voice, typing speed, etc., in real time to acquire emotional information. The input is biometric data indicating the user's emotions, which the emotion engine analyzes and outputs as emotional information.

[1509] Step 4:

[1510] The terminal sends search queries and sentiment information from the user to the server in the form of an HTTP request. The input consists of search queries and sentiment information, which are sent to the server in JSON or XML format. The server receives this data as output.

[1511] Step 5:

[1512] The server searches the database based on the submitted search query. It uses SQL to extract relevant information. The input is the received search query, and the output is the relevant information as search results. The server organizes this data and provides it in the format best suited to the user's request.

[1513] Step 6:

[1514] The server organizes information based on emotional data acquired by the emotion engine. Relevant information is prioritized and displayed according to the user's emotional state. Input consists of emotional data and search results; data processing is performed based on the emotional data, and prioritized information is generated as output.

[1515] Step 7:

[1516] The server sends organized information to the terminal in JSON or XML format. The input is prioritized information, which is then formatted and sent to the terminal. The terminal receives this information as output.

[1517] Step 8:

[1518] The terminal analyzes the information it receives and displays it on the user interface. The input is information sent from the server and displayed in an appropriate format on the user interface. The output is a display of information that the user can actually view and use.

[1519] Step 9:

[1520] The user reviews the information displayed on the device and performs troubleshooting steps. The input is the information displayed on the device, which the user uses to contact support or perform steps to resolve the problem. The output is the result of the resolved work.

[1521] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1522] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1523] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1524] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1525] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1526] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1527] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1528] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1529] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1530] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1531] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1532] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1533] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1534] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1535] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1536] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1537] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1538] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1539] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1540] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1541] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1542] The following is further disclosed regarding the embodiments described above.

[1543] (Claim 1)

[1544] A means of receiving search queries entered by the user,

[1545] A means for searching the database based on the aforementioned search query,

[1546] A means for organizing the aforementioned search results and extracting contact information and related system information,

[1547] A system including means for displaying the extracted information to the user in a list.

[1548] (Claim 2)

[1549] The system according to claim 1, wherein the search query is sent to the server in HTTP request format.

[1550] (Claim 3)

[1551] The system according to claim 1, which formats the search results in JSON format or XML format.

[1552] "Example 1"

[1553] Claims

[1554] (Claim 1)

[1555] A means of receiving search queries entered by the user,

[1556] A means for searching the database based on the aforementioned search query,

[1557] A means for formatting the search results in JSON or XML format,

[1558] Means for transmitting the formatted data to a terminal,

[1559] The aforementioned terminal analyzes the data it receives and displays it visually to the user in a list,

[1560] A system that includes means for the user to review the displayed information and, if necessary, contact the relevant contact person or system.

[1561] (Claim 2)

[1562] The system according to claim 1, wherein the search query is sent to the server in HTTP request format.

[1563] (Claim 3)

[1564] The system according to claim 1, further comprising means for initializing the database at startup.

[1565] "Application Example 1"

[1566] (Claim 1)

[1567] A means of recognizing voice commands entered by the user,

[1568] A means for converting the aforementioned search query from speech to text,

[1569] The means for receiving the aforementioned search query,

[1570] A means for searching the database based on the aforementioned search query,

[1571] A means for organizing the aforementioned search results and extracting contact information, related system information, and instruction manuals,

[1572] A system including means for displaying the extracted information to the user in a list.

[1573] (Claim 2)

[1574] The system according to claim 1, wherein the search query is sent to the server in HTTP request format.

[1575] (Claim 3)

[1576] The system according to claim 1, which formats the search results in JSON format or XML format.

[1577] "Example 2 of combining an emotion engine"

[1578] (Claim 1)

[1579] A means of receiving search queries entered by the user,

[1580] A means for searching the database based on the aforementioned search query,

[1581] A means for obtaining the aforementioned search results and recognizing the user's emotions using an emotion analysis engine,

[1582] A means for reorganizing the search results obtained based on the recognized emotional information and setting priorities,

[1583] A system including means for displaying the aforementioned reorganized information to the user in a list.

[1584] (Claim 2)

[1585] The system according to claim 1, wherein the search query is sent to the server in HTTP request format and sent together with the analysis results of the sentiment analysis engine.

[1586] (Claim 3)

[1587] The system according to claim 1, which formats the reorganized search results in JSON format or XML format.

[1588] "Application example 2 when combining with an emotional engine"

[1589] (Claim 1)

[1590] A means of receiving search queries entered by the user,

[1591] A means for searching the database based on the aforementioned search query,

[1592] A means for organizing the aforementioned search results and extracting contact information and related system information,

[1593] A method for organizing extracted information based on emotion recognition and presenting it with priority according to the user's emotions,

[1594] A means for displaying the extracted information to the user in a list,

[1595] An emotion engine that recognizes user emotions in real time,

[1596] A method for attaching emotional information obtained from an emotional engine when submitting a search query.

[1597] A system that includes this.

[1598] (Claim 2)

[1599] The system according to claim 1, wherein the search query and sentiment information are sent to a server in HTTP request format.

[1600] (Claim 3)

[1601] The system according to claim 1, which formats the search results in JSON or XML format and provides the organized results to the user. [Explanation of Symbols]

[1602] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving search queries entered by the user, A means for searching the database based on the aforementioned search query, A means for organizing the aforementioned search results and extracting contact information and related system information, A system including means for displaying the extracted information to the user in a list.

2. The system according to claim 1, wherein the search query is sent to the server in HTTP request format.

3. The system according to claim 1, which formats the search results in JSON format or XML format.

Images