system

Abstract

We provide the system. [Solution] An input terminal that receives user input, A server that uses natural language processing to analyze user input and identify the language, A means of searching for relevant information from a knowledge base based on user input, Means for generating an appropriate response based on analysis results and search results, A means of translating the generated response into the user's native language, A system including means for sending and displaying a translated response on an input terminal.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Foreign technical trainees often cannot appropriately solve legal questions and problems related to the workplace environment due to language barriers and cultural differences in the Japanese workplace. For this reason, they often feel stress and anxiety, which has become an obstacle to leading a healthy work life. However, the current situation where there is no system in place to always receive support is an issue.

Means for Solving the Problems

[0005] This invention provides a 24-hour multilingual chatbot system for foreign technical trainees. This system achieves multilingual support by analyzing user input using natural language processing and automatically identifying the language. Furthermore, it searches a knowledge base for relevant legal information and mental health resources based on the input content, providing the user with the most appropriate information. In addition, it analyzes the sentiment of the input text and provides additional support as needed, enabling it to address the challenges faced by users.

[0006] "Users" refer to individuals, such as foreign technical trainees, who use the system.

[0007] An "input terminal" refers to a device used by a user to input text and communicate with the system through an interface.

[0008] "Natural language processing" is a computer processing technology that analyzes user input to understand its intent and content.

[0009] A "server" refers to a computer within a central management system that receives data transmitted from input terminals, processes it, and generates appropriate responses.

[0010] "Identifying the language" refers to the process of determining the language used in the text entered by the user.

[0011] A "knowledge base" refers to a database that stores information for generating responses to users, such as legal information and mental health support resources.

[0012] "Generating a response" refers to the process by which the system creates an appropriate answer based on the analyzed user input.

[0013] "Translating" refers to the process of converting the generated response into the user's native language.

[0014] "Analyzing emotions" refers to the process of determining the emotional state and urgency of a user based on their input.

[0015] "Mental health resources" refer to information and support measures designed to help users maintain their mental well-being.

[0016] "Emergency support information" refers to specific information and means provided to address urgent problems that users face. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the language used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is embodied as a multilingual, 24-hour chatbot system for foreign technical trainees. Through user interaction, this system accepts consultations regarding legal issues and mental health, and provides appropriate information and support. A specific embodiment of this system is described below.

[0039] User Interface

[0040] Device: Users access the chatbot via a web browser or smartphone app. The interface is multilingual, allowing users to operate it in their chosen language.

[0041] User: Start communicating with the chatbot by typing text about problems or legal questions you are actually facing at work. For example, you can enter a specific question such as, "How should I deal with harassment from my boss?"

[0042] Data processing and analysis

[0043] Server: Receives text sent from the user and analyzes it using a natural language processing engine. This analysis identifies the language of the input text and understands the content and intent of the question.

[0044] Server: Based on the analysis results, it consults the knowledge base and searches for relevant information. If the question is legal, it plans to retrieve explanations regarding labor standards law and working conditions and provide them to the user.

[0045] Response generation

[0046] Server: Based on information retrieved from the knowledge base, it generates appropriate responses to user inquiries. These responses also take into account the user's emotional state and the urgency of their input. If the sentiment analysis indicates that the user is experiencing high levels of stress, it also provides mental health resources.

[0047] Translate and send

[0048] Server: Automatically translates the generated response into the user's native language, ensuring accurate translation without changing the meaning.

[0049] Terminal: Displays the translated response to the user, completing the information provision from the chatbot. This allows the user to receive specific advice on their issue and guidance on the next steps.

[0050] This implementation allows foreign technical trainees to receive appropriate support for problems they face in the workplace, regardless of language barriers, creating a secure working environment. Furthermore, users can access advice and information anytime, anywhere through the chatbot, enabling appropriate responses tailored to their individual needs.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] User: Using a smartphone or PC, open the chatbot window, type your question or concern as text, and press the send button.

[0054] Step 2:

[0055] Terminal: Prepares and sends the text data entered by the user to the server via a secure communication protocol.

[0056] Step 3:

[0057] Server: To analyze the received user data, the server initiates text analysis via a natural language processing engine. This involves identifying the language and inferring intent and context from the input.

[0058] Step 4:

[0059] Server: Determines the language of the input and, if translation into the system's internal processing language is necessary, performs automatic translation through the translation module.

[0060] Step 5:

[0061] Server: Based on the analyzed intent and content, it searches the knowledge base for relevant information and guidelines and retrieves the relevant information. If it is legal information, it extracts the key points of labor-related laws and regulations.

[0062] Step 6:

[0063] Server: Analyzes the emotions contained in the user's text, detects keywords and expressions indicating stress and anxiety, and identifies necessary mental health resources.

[0064] Step 7:

[0065] Server: Generates responses to resolve user questions based on knowledge base information and sentiment analysis results. Include additional relevant support information as needed.

[0066] Step 8:

[0067] Server: Translates the generated response into the user's native language, verifies the translation accuracy, and then prepares the formatted content for output.

[0068] Step 9:

[0069] Terminal: Receives translated response data sent from the server and displays it to the user in an appropriate format. The user can then review the information and decide on the next steps or actions to take.

[0070] (Example 1)

[0071] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0072] Foreign technical trainees face challenges in seeking help for workplace problems due to language barriers and cultural differences. In particular, they often struggle to obtain prompt and accurate support regarding legal issues and mental health. This can jeopardize the safety and security of these trainees. Therefore, it is necessary to develop a multilingual, 24-hour support system to provide appropriate information and serve as a consultation service.

[0073] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0074] In this invention, the server includes a device for receiving information from a user, an information processing device that analyzes the user's information and identifies the language using natural language processing, means for retrieving relevant information from knowledge resources based on the user's information content, means for generating natural and human-like responses tailored to the user's inquiry using a generative AI model, and means for evaluating the user's emotions and the urgency of the input and adjusting the response content based on the results. This makes it possible to provide appropriate and prompt support in multiple languages ​​24 hours a day to address the problems faced by foreign technical trainees.

[0075] An "information terminal" is an electronic device used by users to input or receive information, and is connected to a network.

[0076] "Natural language processing" refers to the technology that allows computers to recognize and analyze natural language used by humans in everyday life.

[0077] An "information processing device" refers to a computer system that analyzes received data and performs necessary processing.

[0078] "Knowledge resources" refer to databases and information repositories that manage specific information and allow it to be retrieved as needed.

[0079] A "generative AI model" refers to software that uses artificial intelligence technology to automatically generate appropriate responses based on user input.

[0080] "Sentiment assessment" is a process for inferring a user's emotional state from their input and context, and determining the necessary response.

[0081] "Response content adjustment" refers to the operation of appropriately changing the content of information and advice provided based on the user's emotional state and input.

[0082] This invention is a multilingual chatbot system designed to support foreign technical trainees facing various challenges. Specific embodiments of this system are described below.

[0083] Terminal: Users access the system using a web browser or smartphone application. This provides a mechanism for easily sending questions and inquiries via text input. To enhance user convenience, the interface is multilingual.

[0084] Server: The server receives text data from users and analyzes the text using natural language processing technology. Specific software used includes natural language processing engines and text analysis APIs (e.g., the cloud service provider's reference API). Based on the analysis results, the server references internal knowledge resources (databases) to retrieve relevant information.

[0085] Knowledge resources: These contain information tailored to the user's needs, such as legal information and mental health-related information. The server accesses this information to extract the necessary details.

[0086] Generative AI Model: The server utilizes a generative AI model to generate highly natural and human-like responses to user input. These responses are designed to take into account the user's emotional state and urgency. Furthermore, automatic translation technology (e.g., online translation APIs) is used as needed to accurately translate the responses into the user's native language.

[0087] (Specific example)

[0088] For example, if a user asks, "How should I deal with harassment from my boss?", the server will provide appropriate legal information based on its knowledge of labor standards law, and if necessary, it will also provide mental health resources. An example of a prompt might be, "Generate an appropriate response to provide information in an easy-to-understand format to a foreign technical intern who seeks legal advice regarding their work environment," instructing the generation AI model in this way.

[0089] The distinguishing feature of this invention is the provision of appropriate and prompt support in multiple languages ​​for the diverse challenges that users face, thereby ensuring the accuracy and timeliness of information and providing peace of mind to users.

[0090] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0091] Step 1:

[0092] Terminal: Users access the system using a web browser or smartphone app. Users use the input interface to enter their concerns and questions in text format. The information entered consists of specific worries and problems, such as "working conditions at my workplace are not clearly stated."

[0093] Step 2:

[0094] Terminal: Sends text entered by the user to the information processing device. The data sent is text data entered by the user. Transmission is performed using a secure communication protocol.

[0095] Step 3:

[0096] Server: The server starts the natural language processing engine to analyze the received text data. The input data is a text message from the user. The analysis engine identifies the language of the text and processes the data to extract its content and intent. As a result, the subject and focus of the question in the text are obtained.

[0097] Step 4:

[0098] Server: Based on the analysis results, it references knowledge resources. The input is the output data of the analysis engine, and the server uses this to search the database for relevant legal and supporting information. This search process retrieves relevant legal provisions, FAQs, and case study data.

[0099] Step 5:

[0100] Server: Based on the acquired information, it generates a response using a generative AI model. The input data consists of information extracted from knowledge resources and the user's original question. The generative AI model utilizes this information to automatically construct the most appropriate response for the user. This response also takes the user's emotional state into consideration and adds mental health information as needed.

[0101] Step 6:

[0102] Server: Translates the generated response into the user's native language. The input is the response text generated by the generation AI model. A translation API is used for translation, maintaining semantic accuracy while converting to each language.

[0103] Step 7:

[0104] Server: Sends the translated response back to the terminal. The output is a text message ready to be sent to the user.

[0105] Step 8:

[0106] Terminal: The translated response is displayed on the user's terminal. The user can obtain specific advice regarding their problem based on the information provided. This information serves as a guide for their next actions.

[0107] (Application Example 1)

[0108] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0109] The legal issues and mental stress faced by foreign workers in the workplace are often not resolved quickly due to their insufficient understanding of Japanese or lack of access to appropriate resources for consultation, which increases anxiety in the work environment. Furthermore, the lack of systems capable of providing immediate support in multiple languages ​​makes early detection and response to problems difficult.

[0110] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0111] In this invention, the server includes means for using a terminal to receive input from a user, means for using a computer that analyzes the user's input using natural language processing technology and identifies the language, and means for retrieving relevant information from a knowledge base based on the user's input. This makes it possible for foreign workers to immediately consult about legal issues in the workplace and receive appropriate support and mental health assistance.

[0112] A "terminal" refers to a device used by a user to input and receive information, such as a smartphone or computer.

[0113] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, including language identification and semantic understanding of input text.

[0114] A "computer" is a device that has a central processing unit used to analyze user input and obtain necessary information, and generally functions as a server.

[0115] A "knowledge base" is a collection of information and data related to a specific domain, and it is a source of information that is referenced when generating answers to user questions.

[0116] "Sentiment analysis" is a data analysis technique that evaluates the emotions contained in user input and determines psychological states such as stress and anxiety.

[0117] "Multilingual support" refers to a feature that provides information in multiple languages, enabling users to solve problems in their own language.

[0118] "Legal issues" refer to problems and questions related to labor law, contract law, etc., and include legal consultations that foreign workers may face in the course of their work.

[0119] "Workplace" refers to the physical or virtual environment in which an individual performs work, and includes places such as shops and offices.

[0120] "Additional support" refers to supplementary information and resources provided to alleviate users' anxiety and stress, including guidance on professional consultation services.

[0121] This invention relates to a multilingual chatbot system for resolving legal issues and mental stress faced by foreign workers in the workplace. The system consists of a user terminal, a server, and a knowledge base.

[0122] The primary hardware used by users is either a smartphone or a computer. The device receives text input from the user and transmits it to the server via a communication line. The server analyzes the input text using natural language processing technology to identify the user's language and understand legally relevant questions.

[0123] On the server, the input data is processed by a computer, and relevant information is retrieved from a knowledge base. This knowledge base includes legal information such as labor law and contract law, and is used to generate appropriate responses to user questions. Furthermore, sentiment analysis technology is used to analyze the emotions contained in the user's input and assess stress and anxiety.

[0124] The generated responses are automatically translated into the user's native language and sent to their device for display. This allows users to quickly obtain specific advice relevant to their issues. Additional support information is also provided to help reduce stress, depending on the user's emotional state. For example, if a user seeks advice regarding harassment in the workplace, legal advice and links to specialists will be provided.

[0125] Users can instantly receive this information through the application, enabling them to confidently address problems despite language barriers. An example of a prompt message is, "Please explain the legal procedures regarding unpaid wages for foreign technical trainees." In this way, the overall working environment for foreign workers is improved, and their social stability is enhanced.

[0126] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0127] Step 1:

[0128] The terminal receives text input from the user. This input includes questions about legal issues and workplace anxieties. The terminal prepares this text data for transmission to the server and sends it to the server over the network.

[0129] Step 2:

[0130] The server receives text data sent from the terminal. The server uses natural language processing techniques to identify the language of the input text. At the same time, it analyzes the overall context and intent of the text and determines the type of question based on this. This process utilizes a generative AI model to deepen the meaning of the text. The output is the category of the analyzed question and the determined language.

[0131] Step 3:

[0132] The server searches its knowledge base for relevant information based on the analyzed question. This search is performed based on specified legal categories and user needs. For example, if it is determined that information related to labor law is needed, the server retrieves the relevant data. As a result, the user receives specific legal information tailored to their needs.

[0133] Step 4:

[0134] The server performs sentiment analysis, evaluating the emotions contained in the user's input. This process analyzes the level of stress and anxiety the user is experiencing. The results of the sentiment analysis are used to determine whether additional support is needed. The output is the sentiment evaluation result.

[0135] Step 5:

[0136] The server combines the retrieved information with sentiment analysis to generate the optimal response. The generated response is then translated into the user's native language. This translation is carefully performed to preserve meaning and is implemented through a multilingual interface. The output is the translated text.

[0137] Step 6:

[0138] The server sends the generated translated text to the terminal. The terminal displays this text to the user. The user reviews the displayed information and receives guidance on specific legal advice and additional mental support. This process makes it easier to decide on the next steps toward resolving the problem.

[0139] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0140] This invention is a 24-hour multilingual chatbot system for foreign technical trainees. Its key feature is the incorporation of an emotion engine, which allows it to recognize the user's emotional state in real time and provide appropriate responses accordingly. This system analyzes emotions from user input and, when necessary, provides mental health resources to support improvements in the work environment.

[0141] User-level operations

[0142] Terminal: Users connect to the system via a dedicated application accessible from a smartphone or PC, or through a web browser. Users can freely input and submit questions and problems in text format.

[0143] Users can use the service by anonymously entering specific problems or questions, such as, "I've been feeling stressed at work lately. I don't know who to talk to about it."

[0144] Data analysis and response generation

[0145] Server: The server analyzes the received text data using a natural language processing engine to identify the language and understand the user's intent. This analysis helps determine which category the user's input belongs to.

[0146] Server: Based on the analysis results and emotional information recognized by the emotion engine, it searches for relevant information from the knowledge base. If stress or anxiety is detected, it can prioritize the selection of appropriate mental health resources.

[0147] Server: Based on this information, it generates an appropriate response to the user. Sentiment recognition ensures that the response is emotionally sensitive.

[0148] Adjusting and sending responses

[0149] Server: Automatically translates the generated response into the user's native language and formats it into natural-sounding, nuanced language.

[0150] Terminal: Displays the response sent from the server to the user. The user can read the provided information and decide what action to take next.

[0151] Recognition and utilization of emotions

[0152] Server: The emotion engine recognizes changes in emotions in real time based on user input and immediately notifies the user of additional support information if stress levels are high. It can also analyze past emotional trends and accumulate foundational information to provide personalized support.

[0153] This system will allow foreign technical trainees to receive appropriate advice and support tailored to their specific situation in real time, overcoming language barriers. The introduction of an emotion engine enables nuanced responses that respond to the user's emotions, allowing them to receive support to improve their work environment with peace of mind.

[0154] The following describes the processing flow.

[0155] Step 1:

[0156] User: Access the chatbot interface via their device, enter their question, concern, or anxiety in text format, and click the send button. Example: "Recently, I've been having trouble with relationships at work. How can I improve things?"

[0157] Step 2:

[0158] Terminal: Receives user input text, prepares it for transfer to the server via a secure communication channel, and sends the text data to the server.

[0159] Step 3:

[0160] Server: Passes the received text data to the natural language processing engine, which performs language identification and intent analysis of the text. This helps to understand the subject and category of the input content.

[0161] Step 4:

[0162] Server: Using an emotion engine, it extracts emotional components from user input and identifies emotions such as stress, anxiety, and joy. It also analyzes emotional changes in real time.

[0163] Step 5:

[0164] Server: Based on the results of language processing and sentiment analysis, it searches for relevant information from the knowledge base. Specifically, it integrates legal guidelines and mental health information to address users' concerns.

[0165] Step 6:

[0166] Server: After receiving information, it generates a response tailored to the user's emotional state. If high stress levels are detected, it prioritizes including mental health support information and uses warm, empathetic language.

[0167] Step 7:

[0168] Server: Translates the generated response into the user's chosen native language and adjusts it to produce natural-sounding text.

[0169] Step 8:

[0170] Terminal: Receives translated responses sent from the server and displays them to the user. The user can use this information to decide on their next action.

[0171] Step 9:

[0172] Server: Accumulates user emotional data and analyzes long-term emotional trends. Based on this data, responses in future interactions are personalized, and this data is used to develop individualized support plans.

[0173] (Example 2)

[0174] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0175] In today's work environment, providing effective mental support is challenging, especially for workers from diverse linguistic and cultural backgrounds. Many systems lack the ability to properly analyze users' emotions and respond in real time. Therefore, there is a need to provide individualized support that takes into account the user's psychological state.

[0176] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0177] In this invention, the server includes means for recognizing the user's emotions in real time and generating a response corresponding to that state, means for accumulating past emotional information and enabling personalized support, and means for generating an appropriate response based on analysis results and search results. This makes it possible to provide detailed support tailored to each user's situation in real time and in multiple languages.

[0178] A "user" refers to an individual who uses the system to input information and receive support.

[0179] "Data" refers to strings of characters or documents containing information that users input into the system.

[0180] "Device" refers to a combination of hardware and software used for receiving, displaying, converting, and transmitting information.

[0181] A "central processing unit" refers to a computer system that analyzes data received from users and processes it to generate appropriate responses.

[0182] A "database" refers to an information aggregation system that stores related information and manages it in a searchable format.

[0183] "Emotions" refer to the psychological state analyzed based on user input, and serve as the foundation for the system to provide support.

[0184] "Real-time" means that processing is performed immediately and responses are generated and provided without delay.

[0185] "Support information" refers to advice and resources provided according to the user's situation and needs.

[0186] This invention is a multilingual chatbot system that aims to provide real-time support that takes into account the user's psychological state. This system is particularly designed to support workers with diverse language and cultural backgrounds. An embodiment of this system is described below.

[0187] First, the device provides an interface with the user through applications or web browsers on smartphones or PCs. The user uses this device to send text data to the system. Specifically, they can input concerns such as, "I'm experiencing increased stress at work."

[0188] The server analyzes the received data using a natural language processing engine. This analysis includes language detection, intent identification, and sentiment analysis. Examples of natural language processing engines used include the open-source "SpaCy" and "NLTK".

[0189] To perform sentiment analysis, a pre-trained sentiment recognition model is utilized. The server uses this model to evaluate the user's emotions in real time and determine states such as stress and anxiety.

[0190] Based on the determined emotional state, the server searches its knowledge base for appropriate support information for the user. The knowledge base is a collection of mental health-related information sources and support services. Based on this information, it generates a helpful and emotionally sensitive response for the user. For example, it may include specific advice such as, "Try AA to reduce stress."

[0191] The generated response is translated into the user's native language by a translation system. This translation uses tools such as "DeepL" or "Google Translate." The translated response is then refined to sound natural and fluent.

[0192] The device receives response data sent from the server and displays it to the user. This allows the user to decide on their next course of action based on the advice provided. Furthermore, past emotional data is stored in a database and used as a foundation for personalized support.

[0193] As a concrete example, the prompt could read: "Analyze the user's input, identify their emotions, and generate a response that suggests the most appropriate mental health resource if their stress level is high."

[0194] This system allows users to receive appropriate support tailored to their emotional state, transcending language barriers. This is expected to improve the psychological stability and productivity of workers in cross-cultural environments.

[0195] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0196] Step 1:

[0197] Users connect to the system via a smartphone or PC application or web browser and input their questions or concerns in text format. This text is then sent to the server as input data.

[0198] Step 2:

[0199] The server inputs the received text data into a natural language processing engine. Here, language identification and intent analysis are performed. The data analysis determines the intent behind the user's statements from the input text, and this information is passed on to the next processing step. The output consists of the analyzed text and its intent information.

[0200] Step 3:

[0201] The server inputs the analyzed text data into an emotion recognition model. Here, the model evaluates the user's emotional state in real time. An emotion score is calculated from the input text data, determining whether the user is experiencing stress or anxiety. The output includes the emotion score and the evaluation result.

[0202] Step 4:

[0203] The server sends queries to the knowledge database based on the results of emotion recognition and intent analysis, and extracts resource information suitable for the user. Relevant information is searched from the knowledge database according to the input query, and appropriate support information is obtained. A list of support information is generated as output.

[0204] Step 5:

[0205] The server generates a response based on the acquired support information and emotional information. It utilizes a generative AI model to create text responses that take into account the user's state and emotions. For example, it can produce sentences such as, "We will introduce you to mental health resources that are suitable for your current situation." The generated response text is then output.

[0206] Step 6:

[0207] The server inputs the generated response into the translation engine and translates it into the user's native language. The input text is formatted to appropriate grammar and expression through the translation process. As output, a response text with natural expression is generated.

[0208] Step 7:

[0209] The device receives the translated response and displays it to the user. At this stage, the user can choose their next action based on what is displayed. The user uses this information to decide on the appropriate course of action based on the information provided.

[0210] Through this series of processes, the system provides real-time support based on the user's emotions and intentions.

[0211] (Application Example 2)

[0212] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0213] Foreign staff working in multilingual and cross-cultural work environments face communication barriers and increased workplace stress, which can negatively impact their mental health. There are also concerns about decreased work efficiency and reduced job satisfaction due to stress. A system that can resolve these issues in real time is needed.

[0214] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0215] This invention includes a server that uses emotion recognition functionality to acquire user emotional information and adjusts its response based on the results; a server that measures the user's stress level in the work environment and provides information on mental health as needed; and a server that proposes specific advice to support job performance in physical stores based on user input and emotion analysis results. This enables real-time support for communication and stress-related problems faced by foreign staff, improving the work environment and protecting their mental health.

[0216] "Users" refer to foreign staff members who use this system and are people who seek support regarding communication and mental health in the workplace environment.

[0217] An "input device" is a device used by a user to register information, and includes terminals equipped with communication functions, such as smartphones and personal computers.

[0218] "Natural language processing technology" is a technology that converts human language into a format that computers can understand and interpret, and is used to identify language and analyze the user's intent.

[0219] A "knowledge database" is a database that stores information related to user input and plays a role in providing the information necessary for generating responses.

[0220] "Emotion recognition functionality" is a technology that analyzes emotional information from user input to identify the user's emotional state at any given time.

[0221] "Mental health information" refers to advice and resources to alleviate users' stress and anxiety, and includes specific support measures for improving the workplace environment.

[0222] "Specific advice" refers to practical advice provided to support users in carrying out their work, and it indicates the optimal course of action for the situation the user is facing.

[0223] This invention provides a system to address the communication and mental health challenges that foreign staff experience while working in physical stores. This system is accessible via input devices such as the user's smartphone, and a server equipped with natural language processing technology and emotion recognition capabilities plays a crucial role.

[0224] The server first receives input from the user and analyzes its content and sentiment using natural language processing technology. This process may utilize Google Cloud Natural Language API or Azure Cognitive Services. Based on the analysis, the user's intentions and emotional state are understood, and relevant information is retrieved from the knowledge database. This knowledge database contains data necessary to support the user's work performance and mental health.

[0225] Based on the analysis results and information obtained from the knowledge database, the server generates an appropriate response. Using emotion recognition data, the response takes into account the user's emotional state. The generated response is translated into the user's native language and displayed on the user's input device.

[0226] As a concrete example of its use, if a user inputs "Today, no matter what I do during customer service, customers won't smile, and I'm feeling down," the server can categorize this as "Needs suggestions for stress relief" and return advice such as, "Take a few deep breaths and focus on the next customer. If it's difficult to concentrate, consider taking a short break." The prompt itself is "Please tell me how to relieve stress while serving customers."

[0227] Thus, this invention provides real-time support to staff working in physical stores, helping them to perform their duties smoothly.

[0228] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0229] Step 1:

[0230] The user's input device receives input from the user. The input is in text format and includes the user's questions and requests. This information is sent to the server.

[0231] Step 2:

[0232] The server analyzes the received text data using natural language processing techniques. Technologies such as the Google Cloud Natural Language API are used to identify the input language and recognize the user's intent and emotional state. The analysis yields data on key keywords and emotional states.

[0233] Step 3:

[0234] The server searches for relevant information from the knowledge database based on the analysis results. Based on the keywords obtained, it selects data from the database that is useful to the user. This ensures that the information best suited to the user's needs is selected.

[0235] Step 4:

[0236] The server generates an appropriate response based on information obtained from the knowledge database and sentiment analysis results. The response will be considerate of the user's emotional state and may include advice and information to reduce the user's stress.

[0237] Step 5:

[0238] The server translates the generated response into the user's native language. Translation software, such as the Google Translate API, is used to format the response into natural and fluent language.

[0239] Step 6:

[0240] The server sends the translated response to the user's input device. The terminal displays the response in a user-friendly format to guide the user in deciding their next action.

[0241] Step 7:

[0242] The server monitors user feedback and ongoing emotional changes, providing additional support information and counseling resources as needed. This ensures users receive continuous and reassuring support.

[0243] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0244] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0245] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0246] [Second Embodiment]

[0247] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0248] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0249] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0250] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0251] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0252] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0253] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0254] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0255] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0256] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0257] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0258] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0259] This invention is embodied as a multilingual, 24-hour chatbot system for foreign technical trainees. Through user interaction, this system accepts consultations regarding legal issues and mental health, and provides appropriate information and support. A specific embodiment of this system is described below.

[0260] User Interface

[0261] Device: Users access the chatbot via a web browser or smartphone app. The interface is multilingual, allowing users to operate it in their chosen language.

[0262] User: Start communicating with the chatbot by typing text about problems or legal questions you are actually facing at work. For example, you can enter a specific question such as, "How should I deal with harassment from my boss?"

[0263] Data processing and analysis

[0264] Server: Receives text sent from the user and analyzes it using a natural language processing engine. This analysis identifies the language of the input text and understands the content and intent of the question.

[0265] Server: Based on the analysis results, it consults the knowledge base and searches for relevant information. If the question is legal, it plans to retrieve explanations regarding labor standards law and working conditions and provide them to the user.

[0266] Response generation

[0267] Server: Based on information retrieved from the knowledge base, it generates appropriate responses to user inquiries. These responses also take into account the user's emotional state and the urgency of their input. If the sentiment analysis indicates that the user is experiencing high levels of stress, it also provides mental health resources.

[0268] Translate and send

[0269] Server: Automatically translates the generated response into the user's native language, ensuring accurate translation without changing the meaning.

[0270] Terminal: Displays the translated response to the user, completing the information provision from the chatbot. This allows the user to receive specific advice on their issue and guidance on the next steps.

[0271] This implementation allows foreign technical trainees to receive appropriate support for problems they face in the workplace, regardless of language barriers, creating a secure working environment. Furthermore, users can access advice and information anytime, anywhere through the chatbot, enabling appropriate responses tailored to their individual needs.

[0272] The following describes the processing flow.

[0273] Step 1:

[0274] User: Using a smartphone or PC, open the chatbot window, type your question or concern as text, and press the send button.

[0275] Step 2:

[0276] Terminal: Prepares and sends the text data entered by the user to the server via a secure communication protocol.

[0277] Step 3:

[0278] Server: To analyze the received user data, the server initiates text analysis via a natural language processing engine. This involves identifying the language and inferring intent and context from the input.

[0279] Step 4:

[0280] Server: Determines the language of the input and, if translation into the system's internal processing language is necessary, performs automatic translation through the translation module.

[0281] Step 5:

[0282] Server: Based on the analyzed intent and content, it searches the knowledge base for relevant information and guidelines and retrieves the relevant information. If it is legal information, it extracts the key points of labor-related laws and regulations.

[0283] Step 6:

[0284] Server: Analyze the emotions contained in the user's text, detect keywords and expressions indicating stress and anxiety, and identify the necessary mental health resources.

[0285] Step 7:

[0286] Server: Based on the information in the knowledge base and the results of the emotion analysis, generate a response to resolve the user's questions. Include additional information for related support if necessary.

[0287] Step 8:

[0288] Server: Translate the generated response into the user's native language, confirm that the translation accuracy is high, and prepare the formatted content for output.

[0289] Step 9:

[0290] Terminal: Receive the translated response data sent from the server and display it to the user in an appropriate format. The user can use the information to confirm and determine the next steps and actions.

[0291] (Example 1)

[0292] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0293] There is an issue that it is difficult for foreign technical trainees to consult workplace problems due to language barriers and cultural differences. In particular, in the case of legal issues and mental health, there is a situation where it is difficult to obtain prompt and accurate support. As a result, the safety and peace of mind of the technical trainees may be threatened. Therefore, it is necessary to develop a multilingual and 24-hour available support system to play the role of providing appropriate information and a consultation window.

[0294] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0295] In this invention, the server includes a device for receiving information from a user, an information processing device that analyzes the user's information and identifies the language using natural language processing, means for retrieving relevant information from knowledge resources based on the user's information content, means for generating natural and human-like responses tailored to the user's inquiry using a generative AI model, and means for evaluating the user's emotions and the urgency of the input and adjusting the response content based on the results. This makes it possible to provide appropriate and prompt support in multiple languages ​​24 hours a day to address the problems faced by foreign technical trainees.

[0296] An "information terminal" is an electronic device used by users to input or receive information, and is connected to a network.

[0297] "Natural language processing" refers to the technology that allows computers to recognize and analyze natural language used by humans in everyday life.

[0298] An "information processing device" refers to a computer system that analyzes received data and performs necessary processing.

[0299] "Knowledge resources" refer to databases and information repositories that manage specific information and allow it to be retrieved as needed.

[0300] A "generative AI model" refers to software that uses artificial intelligence technology to automatically generate appropriate responses based on user input.

[0301] "Sentiment assessment" is a process for inferring a user's emotional state from their input and context, and determining the necessary response.

[0302] "Response content adjustment" refers to the operation of appropriately changing the content of the information and advice provided based on the user's emotional state and input content.

[0303] This invention is a multilingual chatbot system designed to support the problems faced by foreign technical trainees. The following will explain the specific embodiments of this system.

[0304] Terminal: The user accesses the system using a web browser or a smartphone application. This provides a mechanism that allows questions and consultation content to be easily sent through text input. To enhance the user's convenience, the interface supports multiple languages.

[0305] Server: The server receives text data from the user and analyzes the text using natural language processing technology. As specific software, a natural language processing engine and a text analysis API (for example, the reference API of a cloud service provider) are used. Based on the analysis results, the server refers to the internal knowledge resources (database) to search for relevant information.

[0306] Knowledge resources: Information that meets the content required by the user, such as legal information and mental health-related information, is stored, and the server refers to this to extract the necessary information.

[0307] Generative AI model: The server utilizes a generative AI model to generate responses that are very natural and human-like to the user's input content. This response is designed to take into account the user's emotional state and urgency. Furthermore, if necessary, automatic translation technology (e.g., online translation API) is used to accurately translate it into the user's native language.

[0308] (Specific example)

[0309] For example, if a user asks, "How should I deal with harassment from my boss?", the server will provide appropriate legal information based on its knowledge of labor standards law, and if necessary, it will also provide mental health resources. An example of a prompt might be, "Generate an appropriate response to provide information in an easy-to-understand format to a foreign technical intern who seeks legal advice regarding their work environment," instructing the generation AI model in this way.

[0310] The distinguishing feature of this invention is the provision of appropriate and prompt support in multiple languages ​​for the diverse challenges that users face, thereby ensuring the accuracy and timeliness of information and providing peace of mind to users.

[0311] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0312] Step 1:

[0313] Terminal: Users access the system using a web browser or smartphone app. Users use the input interface to enter their concerns and questions in text format. The information entered consists of specific worries and problems, such as "working conditions at my workplace are not clearly stated."

[0314] Step 2:

[0315] Terminal: Sends text entered by the user to the information processing device. The data sent is text data entered by the user. Transmission is performed using a secure communication protocol.

[0316] Step 3:

[0317] Server: The server starts the natural language processing engine to analyze the received text data. The input data is a text message from the user. The analysis engine identifies the language of the text and processes the data to extract its content and intent. As a result, the subject and focus of the question in the text are obtained.

[0318] Step 4:

[0319] Server: Based on the analysis results, it references knowledge resources. The input is the output data of the analysis engine, and the server uses this to search the database for relevant legal and supporting information. This search process retrieves relevant legal provisions, FAQs, and case study data.

[0320] Step 5:

[0321] Server: Based on the acquired information, it generates a response using a generative AI model. The input data consists of information extracted from knowledge resources and the user's original question. The generative AI model utilizes this information to automatically construct the most appropriate response for the user. This response also takes the user's emotional state into consideration and adds mental health information as needed.

[0322] Step 6:

[0323] Server: Translates the generated response into the user's native language. The input is the response text generated by the generation AI model. A translation API is used for translation, maintaining semantic accuracy while converting to each language.

[0324] Step 7:

[0325] Server: Sends the translated response back to the terminal. The output is a text message ready to be sent to the user.

[0326] Step 8:

[0327] Terminal: The translated response is displayed on the user's terminal. The user can obtain specific advice regarding their problem based on the information provided. This information serves as a guide for their next actions.

[0328] (Application Example 1)

[0329] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0330] The legal issues and mental stress faced by foreign workers in the workplace are often not resolved quickly due to their insufficient understanding of Japanese or lack of access to appropriate resources for consultation, which increases anxiety in the work environment. Furthermore, the lack of systems capable of providing immediate support in multiple languages ​​makes early detection and response to problems difficult.

[0331] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0332] In this invention, the server includes means for using a terminal to receive input from a user, means for using a computer that analyzes the user's input using natural language processing technology and identifies the language, and means for retrieving relevant information from a knowledge base based on the user's input. This makes it possible for foreign workers to immediately consult about legal issues in the workplace and receive appropriate support and mental health assistance.

[0333] A "terminal" refers to a device used by a user to input and receive information, such as a smartphone or computer.

[0334] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, including language identification and semantic understanding of input text.

[0335] A "computer" is a device that has a central processing unit used to analyze user input and obtain necessary information, and generally functions as a server.

[0336] A "knowledge base" is a collection of information and data related to a specific domain, and it is a source of information that is referenced when generating answers to user questions.

[0337] "Sentiment analysis" is a data analysis technique that evaluates the emotions contained in user input and determines psychological states such as stress and anxiety.

[0338] "Multilingual support" refers to a feature that provides information in multiple languages, enabling users to solve problems in their own language.

[0339] "Legal issues" refer to problems and questions related to labor law, contract law, etc., and include legal consultations that foreign workers may face in the course of their work.

[0340] "Workplace" refers to the physical or virtual environment in which an individual performs work, and includes places such as shops and offices.

[0341] "Additional support" refers to supplementary information and resources provided to alleviate users' anxiety and stress, including guidance on professional consultation services.

[0342] This invention relates to a multilingual chatbot system for resolving legal issues and mental stress faced by foreign workers in the workplace. The system consists of a user terminal, a server, and a knowledge base.

[0343] The primary hardware used by users is either a smartphone or a computer. The device receives text input from the user and transmits it to the server via a communication line. The server analyzes the input text using natural language processing technology to identify the user's language and understand legally relevant questions.

[0344] On the server, the input data is processed by a computer, and relevant information is retrieved from a knowledge base. This knowledge base includes legal information such as labor law and contract law, and is used to generate appropriate responses to user questions. Furthermore, sentiment analysis technology is used to analyze the emotions contained in the user's input and assess stress and anxiety.

[0345] The generated responses are automatically translated into the user's native language and sent to their device for display. This allows users to quickly obtain specific advice relevant to their issues. Additional support information is also provided to help reduce stress, depending on the user's emotional state. For example, if a user seeks advice regarding harassment in the workplace, legal advice and links to specialists will be provided.

[0346] Users can instantly receive this information through the application, enabling them to confidently address problems despite language barriers. An example of a prompt message is, "Please explain the legal procedures regarding unpaid wages for foreign technical trainees." In this way, the overall working environment for foreign workers is improved, and their social stability is enhanced.

[0347] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0348] Step 1:

[0349] The terminal receives text input from the user. This input includes questions about legal issues and workplace anxieties. The terminal prepares this text data for transmission to the server and sends it to the server over the network.

[0350] Step 2:

[0351] The server receives text data sent from the terminal. The server uses natural language processing techniques to identify the language of the input text. At the same time, it analyzes the overall context and intent of the text and determines the type of question based on this. This process utilizes a generative AI model to deepen the meaning of the text. The output is the category of the analyzed question and the determined language.

[0352] Step 3:

[0353] The server searches its knowledge base for relevant information based on the analyzed question. This search is performed based on specified legal categories and user needs. For example, if it is determined that information related to labor law is needed, the server retrieves the relevant data. As a result, the user receives specific legal information tailored to their needs.

[0354] Step 4:

[0355] The server performs sentiment analysis, evaluating the emotions contained in the user's input. This process analyzes the level of stress and anxiety the user is experiencing. The results of the sentiment analysis are used to determine whether additional support is needed. The output is the sentiment evaluation result.

[0356] Step 5:

[0357] The server combines the retrieved information with sentiment analysis to generate the optimal response. The generated response is then translated into the user's native language. This translation is carefully performed to preserve meaning and is implemented through a multilingual interface. The output is the translated text.

[0358] Step 6:

[0359] The server sends the generated translated text to the terminal. The terminal displays this text to the user. The user reviews the displayed information and receives guidance on specific legal advice and additional mental support. This process makes it easier to decide on the next steps toward resolving the problem.

[0360] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0361] This invention is a 24-hour multilingual chatbot system for foreign technical trainees. Its key feature is the incorporation of an emotion engine, which allows it to recognize the user's emotional state in real time and provide appropriate responses accordingly. This system analyzes emotions from user input and, when necessary, provides mental health resources to support improvements in the work environment.

[0362] User-level operations

[0363] Terminal: Users connect to the system via a dedicated application accessible from a smartphone or PC, or through a web browser. Users can freely input and submit questions and problems in text format.

[0364] Users can use the service by anonymously entering specific problems or questions, such as, "I've been feeling stressed at work lately. I don't know who to talk to about it."

[0365] Data analysis and response generation

[0366] Server: The server analyzes the received text data using a natural language processing engine to identify the language and understand the user's intent. This analysis helps determine which category the user's input belongs to.

[0367] Server: Based on the analysis results and emotional information recognized by the emotion engine, it searches for relevant information from the knowledge base. If stress or anxiety is detected, it can prioritize the selection of appropriate mental health resources.

[0368] Server: Based on this information, it generates an appropriate response to the user. Sentiment recognition ensures that the response is emotionally sensitive.

[0369] Adjusting and sending responses

[0370] Server: Automatically translates the generated response into the user's native language and formats it into natural-sounding, nuanced language.

[0371] Terminal: Displays the response sent from the server to the user. The user can read the provided information and decide what action to take next.

[0372] Recognition and utilization of emotions

[0373] Server: The emotion engine recognizes changes in emotions in real time based on user input and immediately notifies the user of additional support information if stress levels are high. It can also analyze past emotional trends and accumulate foundational information to provide personalized support.

[0374] This system will allow foreign technical trainees to receive appropriate advice and support tailored to their specific situation in real time, overcoming language barriers. The introduction of an emotion engine enables nuanced responses that respond to the user's emotions, allowing them to receive support to improve their work environment with peace of mind.

[0375] The following describes the processing flow.

[0376] Step 1:

[0377] User: Access the chatbot interface via their device, enter their question, concern, or anxiety in text format, and click the send button. Example: "Recently, I've been having trouble with relationships at work. How can I improve things?"

[0378] Step 2:

[0379] Terminal: Receives user input text, prepares it for transfer to the server via a secure communication channel, and sends the text data to the server.

[0380] Step 3:

[0381] Server: Passes the received text data to the natural language processing engine, which performs language identification and intent analysis of the text. This helps to understand the subject and category of the input content.

[0382] Step 4:

[0383] Server: Using an emotion engine, it extracts emotional components from user input and identifies emotions such as stress, anxiety, and joy. It also analyzes emotional changes in real time.

[0384] Step 5:

[0385] Server: Based on the results of language processing and sentiment analysis, it searches for relevant information from the knowledge base. Specifically, it integrates legal guidelines and mental health information to address users' concerns.

[0386] Step 6:

[0387] Server: After receiving information, it generates a response tailored to the user's emotional state. If high stress levels are detected, it prioritizes including mental health support information and uses warm, empathetic language.

[0388] Step 7:

[0389] Server: Translates the generated response into the user's chosen native language and adjusts it to produce natural-sounding text.

[0390] Step 8:

[0391] Terminal: Receives translated responses sent from the server and displays them to the user. The user can use this information to decide on their next action.

[0392] Step 9:

[0393] Server: Accumulates user emotional data and analyzes long-term emotional trends. Based on this data, responses in future interactions are personalized, and this data is used to develop individualized support plans.

[0394] (Example 2)

[0395] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0396] In today's work environment, providing effective mental support is challenging, especially for workers from diverse linguistic and cultural backgrounds. Many systems lack the ability to properly analyze users' emotions and respond in real time. Therefore, there is a need to provide individualized support that takes into account the user's psychological state.

[0397] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0398] In this invention, the server includes means for recognizing the user's emotions in real time and generating a response corresponding to that state, means for accumulating past emotional information and enabling personalized support, and means for generating an appropriate response based on analysis results and search results. This makes it possible to provide detailed support tailored to each user's situation in real time and in multiple languages.

[0399] A "user" refers to an individual who uses the system to input information and receive support.

[0400] "Data" refers to strings of characters or documents containing information that users input into the system.

[0401] "Device" refers to a combination of hardware and software used for receiving, displaying, converting, and transmitting information.

[0402] A "central processing unit" refers to a computer system that analyzes data received from users and processes it to generate appropriate responses.

[0403] A "database" refers to an information aggregation system that stores related information and manages it in a searchable format.

[0404] "Emotions" refer to the psychological state analyzed based on user input, and serve as the foundation for the system to provide support.

[0405] "Real-time" means that processing is performed immediately and responses are generated and provided without delay.

[0406] "Support information" refers to advice and resources provided according to the user's situation and needs.

[0407] This invention is a multilingual chatbot system that aims to provide real-time support that takes into account the user's psychological state. This system is particularly designed to support workers with diverse language and cultural backgrounds. An embodiment of this system is described below.

[0408] First, the device provides an interface with the user through applications or web browsers on smartphones or PCs. The user uses this device to send text data to the system. Specifically, they can input concerns such as, "I'm experiencing increased stress at work."

[0409] The server analyzes the received data using a natural language processing engine. This analysis includes language detection, intent identification, and sentiment analysis. Examples of natural language processing engines used include the open-source "SpaCy" and "NLTK".

[0410] To perform sentiment analysis, a pre-trained sentiment recognition model is utilized. The server uses this model to evaluate the user's emotions in real time and determine states such as stress and anxiety.

[0411] Based on the determined emotional state, the server searches its knowledge base for appropriate support information for the user. The knowledge base is a collection of mental health-related information sources and support services. Based on this information, it generates a helpful and emotionally sensitive response for the user. For example, it may include specific advice such as, "Try AA to reduce stress."

[0412] The generated response is translated into the user's native language by a translation system. Examples of translation tools used include "DeepL" and "Google Translate." The translated response is then refined to sound natural and fluent.

[0413] The device receives response data sent from the server and displays it to the user. This allows the user to decide on their next course of action based on the advice provided. Furthermore, past emotional data is stored in a database and used as a foundation for personalized support.

[0414] As a concrete example, the prompt could read: "Analyze the user's input, identify their emotions, and generate a response that suggests the most appropriate mental health resource if their stress level is high."

[0415] This system allows users to receive appropriate support tailored to their emotional state, transcending language barriers. This is expected to improve the psychological stability and productivity of workers in cross-cultural environments.

[0416] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0417] Step 1:

[0418] Users connect to the system via a smartphone or PC application or web browser and input their questions or concerns in text format. This text is then sent to the server as input data.

[0419] Step 2:

[0420] The server inputs the received text data into a natural language processing engine. Here, language identification and intent analysis are performed. The data analysis determines the intent behind the user's statements from the input text, and this information is passed on to the next processing step. The output consists of the analyzed text and its intent information.

[0421] Step 3:

[0422] The server inputs the analyzed text data into an emotion recognition model. Here, the model evaluates the user's emotional state in real time. An emotion score is calculated from the input text data, determining whether the user is experiencing stress or anxiety. The output includes the emotion score and the evaluation result.

[0423] Step 4:

[0424] The server sends queries to the knowledge database based on the results of emotion recognition and intent analysis, and extracts resource information suitable for the user. Relevant information is searched from the knowledge database according to the input query, and appropriate support information is obtained. A list of support information is generated as output.

[0425] Step 5:

[0426] The server generates a response based on the acquired support information and emotional information. It utilizes a generative AI model to create text responses that take into account the user's state and emotions. For example, it can produce sentences such as, "We will introduce you to mental health resources that are suitable for your current situation." The generated response text is then output.

[0427] Step 6:

[0428] The server inputs the generated response into the translation engine and translates it into the user's native language. The input text is formatted to appropriate grammar and expression through the translation process. As output, a response text with natural expression is generated.

[0429] Step 7:

[0430] The device receives the translated response and displays it to the user. At this stage, the user can choose their next action based on what is displayed. The user uses this information to decide on the appropriate course of action based on the information provided.

[0431] Through this series of processes, the system provides real-time support based on the user's emotions and intentions.

[0432] (Application Example 2)

[0433] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0434] Foreign staff working in multilingual and cross-cultural work environments face communication barriers and increased workplace stress, which can negatively impact their mental health. There are also concerns about decreased work efficiency and reduced job satisfaction due to stress. A system that can resolve these issues in real time is needed.

[0435] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0436] This invention includes a server that uses emotion recognition functionality to acquire user emotional information and adjusts its response based on the results; a server that measures the user's stress level in the work environment and provides information on mental health as needed; and a server that proposes specific advice to support job performance in physical stores based on user input and emotion analysis results. This enables real-time support for communication and stress-related problems faced by foreign staff, improving the work environment and protecting their mental health.

[0437] "Users" refer to foreign staff members who use this system and are people who seek support regarding communication and mental health in the workplace environment.

[0438] An "input device" is a device used by a user to register information, and includes terminals equipped with communication functions, such as smartphones and personal computers.

[0439] "Natural language processing technology" is a technology that converts human language into a format that computers can understand and interpret, and is used to identify language and analyze the user's intent.

[0440] A "knowledge database" is a database that stores information related to user input and plays a role in providing the information necessary for generating responses.

[0441] "Emotion recognition functionality" is a technology that analyzes emotional information from user input to identify the user's emotional state at any given time.

[0442] "Mental health information" refers to advice and resources to alleviate users' stress and anxiety, and includes specific support measures for improving the workplace environment.

[0443] "Specific advice" refers to practical advice provided to support users in carrying out their work, and it indicates the optimal course of action for the situation the user is facing.

[0444] This invention provides a system to address the communication and mental health challenges that foreign staff experience while working in physical stores. This system is accessible via input devices such as the user's smartphone, and a server equipped with natural language processing technology and emotion recognition capabilities plays a crucial role.

[0445] The server first receives input from the user and analyzes its content and sentiment using natural language processing technology. This process may utilize Google Cloud Natural Language API or Azure Cognitive Services. Based on the analysis, the user's intentions and emotional state are understood, and relevant information is retrieved from a knowledge database. This knowledge database contains data necessary to support the user's work performance and mental health.

[0446] Based on the analysis results and information obtained from the knowledge database, the server generates an appropriate response. Using emotion recognition data, the response takes into account the user's emotional state. The generated response is translated into the user's native language and displayed on the user's input device.

[0447] As a concrete example of its use, if a user inputs "Today, no matter what I do during customer service, customers won't smile, and I'm feeling down," the server can categorize this as "Needs suggestions for stress relief" and return advice such as, "Take a few deep breaths and focus on the next customer. If it's difficult to concentrate, consider taking a short break." The prompt itself is "Please tell me how to relieve stress while serving customers."

[0448] Thus, this invention provides real-time support to staff working in physical stores, helping them to perform their duties smoothly.

[0449] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0450] Step 1:

[0451] The user's input device receives input from the user. The input is in text format and includes the user's questions and requests. This information is sent to the server.

[0452] Step 2:

[0453] The server analyzes the received text data using natural language processing techniques. Technologies such as the Google Cloud Natural Language API are used to identify the input language and recognize the user's intent and emotional state. The analysis yields data on key keywords and emotional states.

[0454] Step 3:

[0455] The server searches for relevant information from the knowledge database based on the analysis results. Based on the keywords obtained, it selects data from the database that is useful to the user. This ensures that the information best suited to the user's needs is selected.

[0456] Step 4:

[0457] The server generates an appropriate response based on information obtained from the knowledge database and sentiment analysis results. The response will be considerate of the user's emotional state and may include advice and information to reduce the user's stress.

[0458] Step 5:

[0459] The server translates the generated response into the user's native language. Translation software, such as the Google Translate API, is used to format the response into natural and fluent language.

[0460] Step 6:

[0461] The server sends the translated response to the user's input device. The terminal displays the response in a user-friendly format to guide the user in deciding their next action.

[0462] Step 7:

[0463] The server monitors user feedback and ongoing emotional changes, providing additional support information and counseling resources as needed. This ensures users receive continuous and reassuring support.

[0464] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0465] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0466] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0467] [Third Embodiment]

[0468] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0469] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0470] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0471] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0472] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0474] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0475] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0476] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0477] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0478] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0479] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0480] This invention is embodied as a multilingual, 24-hour chatbot system for foreign technical trainees. Through user interaction, this system accepts consultations regarding legal issues and mental health, and provides appropriate information and support. A specific embodiment of this system is described below.

[0481] User Interface

[0482] Device: Users access the chatbot via a web browser or smartphone app. The interface is multilingual, allowing users to operate it in their chosen language.

[0483] User: Start communicating with the chatbot by typing text about problems or legal questions you are actually facing at work. For example, you can enter a specific question such as, "How should I deal with harassment from my boss?"

[0484] Data processing and analysis

[0485] Server: Receives text sent from the user and analyzes it using a natural language processing engine. This analysis identifies the language of the input text and understands the content and intent of the question.

[0486] Server: Based on the analysis results, it consults the knowledge base and searches for relevant information. If the question is legal, it plans to retrieve explanations regarding labor standards law and working conditions and provide them to the user.

[0487] Response generation

[0488] Server: Based on information retrieved from the knowledge base, it generates appropriate responses to user inquiries. These responses also take into account the user's emotional state and the urgency of their input. If the sentiment analysis indicates that the user is experiencing high levels of stress, it also provides mental health resources.

[0489] Translate and send

[0490] Server: Automatically translates the generated response into the user's native language, ensuring accurate translation without changing the meaning.

[0491] Terminal: Displays the translated response to the user, completing the information provision from the chatbot. This allows the user to receive specific advice on their issue and guidance on the next steps.

[0492] This implementation allows foreign technical trainees to receive appropriate support for problems they face in the workplace, regardless of language barriers, creating a secure working environment. Furthermore, users can access advice and information anytime, anywhere through the chatbot, enabling appropriate responses tailored to their individual needs.

[0493] The following describes the processing flow.

[0494] Step 1:

[0495] User: Using a smartphone or PC, open the chatbot window, type your question or concern as text, and press the send button.

[0496] Step 2:

[0497] Terminal: Prepares and sends the text data entered by the user to the server via a secure communication protocol.

[0498] Step 3:

[0499] Server: To analyze the received user data, the server initiates text analysis via a natural language processing engine. This involves identifying the language and inferring intent and context from the input.

[0500] Step 4:

[0501] Server: Determines the language of the input and, if translation into the system's internal processing language is necessary, performs automatic translation through the translation module.

[0502] Step 5:

[0503] Server: Based on the analyzed intent and content, it searches the knowledge base for relevant information and guidelines and retrieves the relevant information. If it is legal information, it extracts the key points of labor-related laws and regulations.

[0504] Step 6:

[0505] Server: Analyzes the emotions contained in the user's text, detects keywords and expressions indicating stress and anxiety, and identifies necessary mental health resources.

[0506] Step 7:

[0507] Server: Generates responses to resolve user questions based on knowledge base information and sentiment analysis results. Include additional relevant support information as needed.

[0508] Step 8:

[0509] Server: Translates the generated response into the user's native language, verifies the translation accuracy, and then prepares the formatted content for output.

[0510] Step 9:

[0511] Terminal: Receives translated response data sent from the server and displays it to the user in an appropriate format. The user can then review the information and decide on the next steps or actions to take.

[0512] (Example 1)

[0513] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0514] Foreign technical trainees face challenges in seeking help for workplace problems due to language barriers and cultural differences. In particular, they often struggle to obtain prompt and accurate support regarding legal issues and mental health. This can jeopardize the safety and security of these trainees. Therefore, it is necessary to develop a multilingual, 24-hour support system to provide appropriate information and serve as a consultation service.

[0515] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0516] In this invention, the server includes a device for receiving information from a user, an information processing device that analyzes the user's information and identifies the language using natural language processing, means for retrieving relevant information from knowledge resources based on the user's information content, means for generating natural and human-like responses tailored to the user's inquiry using a generative AI model, and means for evaluating the user's emotions and the urgency of the input and adjusting the response content based on the results. This makes it possible to provide appropriate and prompt support in multiple languages ​​24 hours a day to address the problems faced by foreign technical trainees.

[0517] An "information terminal" is an electronic device used by users to input or receive information, and is connected to a network.

[0518] "Natural language processing" refers to the technology that allows computers to recognize and analyze natural language used by humans in everyday life.

[0519] An "information processing device" refers to a computer system that analyzes received data and performs necessary processing.

[0520] "Knowledge resources" refer to databases and information repositories that manage specific information and allow it to be retrieved as needed.

[0521] A "generative AI model" refers to software that uses artificial intelligence technology to automatically generate appropriate responses based on user input.

[0522] "Sentiment assessment" is a process for inferring a user's emotional state from their input and context, and determining the necessary response.

[0523] "Response content adjustment" refers to the operation of appropriately changing the content of information and advice provided based on the user's emotional state and input.

[0524] This invention is a multilingual chatbot system designed to support foreign technical trainees facing various challenges. Specific embodiments of this system are described below.

[0525] Terminal: Users access the system using a web browser or smartphone application. This provides a mechanism for easily sending questions and inquiries via text input. To enhance user convenience, the interface is multilingual.

[0526] Server: The server receives text data from users and analyzes the text using natural language processing technology. Specific software used includes natural language processing engines and text analysis APIs (e.g., the cloud service provider's reference API). Based on the analysis results, the server references internal knowledge resources (databases) to retrieve relevant information.

[0527] Knowledge resources: These contain information tailored to the user's needs, such as legal information and mental health-related information. The server accesses this information to extract the necessary details.

[0528] Generative AI Model: The server utilizes a generative AI model to generate highly natural and human-like responses to user input. These responses are designed to take into account the user's emotional state and urgency. Furthermore, automatic translation technology (e.g., online translation APIs) is used as needed to accurately translate the responses into the user's native language.

[0529] (Specific example)

[0530] For example, if a user asks, "How should I deal with harassment from my boss?", the server will provide appropriate legal information based on its knowledge of labor standards law, and if necessary, it will also provide mental health resources. An example of a prompt might be, "Generate an appropriate response to provide information in an easy-to-understand format to a foreign technical intern who seeks legal advice regarding their work environment," instructing the generation AI model in this way.

[0531] The distinguishing feature of this invention is the provision of appropriate and prompt support in multiple languages ​​for the diverse challenges that users face, thereby ensuring the accuracy and timeliness of information and providing peace of mind to users.

[0532] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0533] Step 1:

[0534] Terminal: Users access the system using a web browser or smartphone app. Users use the input interface to enter their concerns and questions in text format. The information entered consists of specific worries and problems, such as "working conditions at my workplace are not clearly stated."

[0535] Step 2:

[0536] Terminal: Sends text entered by the user to the information processing device. The data sent is text data entered by the user. Transmission is performed using a secure communication protocol.

[0537] Step 3:

[0538] Server: The server starts the natural language processing engine to analyze the received text data. The input data is a text message from the user. The analysis engine identifies the language of the text and processes the data to extract its content and intent. As a result, the subject and focus of the question in the text are obtained.

[0539] Step 4:

[0540] Server: Based on the analysis results, it references knowledge resources. The input is the output data of the analysis engine, and the server uses this to search the database for relevant legal and supporting information. This search process retrieves relevant legal provisions, FAQs, and case study data.

[0541] Step 5:

[0542] Server: Based on the acquired information, it generates a response using a generative AI model. The input data consists of information extracted from knowledge resources and the user's original question. The generative AI model utilizes this information to automatically construct the most appropriate response for the user. This response also takes the user's emotional state into consideration and adds mental health information as needed.

[0543] Step 6:

[0544] Server: Translates the generated response into the user's native language. The input is the response text generated by the generation AI model. A translation API is used for translation, maintaining semantic accuracy while converting to each language.

[0545] Step 7:

[0546] Server: Sends the translated response back to the terminal. The output is a text message ready to be sent to the user.

[0547] Step 8:

[0548] Terminal: The translated response is displayed on the user's terminal. The user can obtain specific advice regarding their problem based on the information provided. This information serves as a guide for their next actions.

[0549] (Application Example 1)

[0550] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0551] The legal issues and mental stress faced by foreign workers in the workplace are often not resolved quickly due to their insufficient understanding of Japanese or lack of access to appropriate resources for consultation, which increases anxiety in the work environment. Furthermore, the lack of systems capable of providing immediate support in multiple languages ​​makes early detection and response to problems difficult.

[0552] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0553] In this invention, the server includes means for using a terminal to receive input from a user, means for using a computer that analyzes the user's input using natural language processing technology and identifies the language, and means for retrieving relevant information from a knowledge base based on the user's input. This makes it possible for foreign workers to immediately consult about legal issues in the workplace and receive appropriate support and mental health assistance.

[0554] A "terminal" refers to a device used by a user to input and receive information, such as a smartphone or computer.

[0555] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, including language identification and semantic understanding of input text.

[0556] A "computer" is a device that has a central processing unit used to analyze user input and obtain necessary information, and generally functions as a server.

[0557] A "knowledge base" is a collection of information and data related to a specific domain, and it is a source of information that is referenced when generating answers to user questions.

[0558] "Sentiment analysis" is a data analysis technique that evaluates the emotions contained in user input and determines psychological states such as stress and anxiety.

[0559] "Multilingual support" refers to a feature that provides information in multiple languages, enabling users to solve problems in their own language.

[0560] "Legal issues" refer to problems and questions related to labor law, contract law, etc., and include legal consultations that foreign workers may face in the course of their work.

[0561] "Workplace" refers to the physical or virtual environment in which an individual performs work, and includes places such as shops and offices.

[0562] "Additional support" refers to supplementary information and resources provided to alleviate users' anxiety and stress, including guidance on professional consultation services.

[0563] This invention relates to a multilingual chatbot system for resolving legal issues and mental stress faced by foreign workers in the workplace. The system consists of a user terminal, a server, and a knowledge base.

[0564] The primary hardware used by users is either a smartphone or a computer. The device receives text input from the user and transmits it to the server via a communication line. The server analyzes the input text using natural language processing technology to identify the user's language and understand legally relevant questions.

[0565] On the server, the input data is processed by a computer, and relevant information is retrieved from a knowledge base. This knowledge base includes legal information such as labor law and contract law, and is used to generate appropriate responses to user questions. Furthermore, sentiment analysis technology is used to analyze the emotions contained in the user's input and assess stress and anxiety.

[0566] The generated responses are automatically translated into the user's native language and sent to their device for display. This allows users to quickly obtain specific advice relevant to their issues. Additional support information is also provided to help reduce stress, depending on the user's emotional state. For example, if a user seeks advice regarding harassment in the workplace, legal advice and links to specialists will be provided.

[0567] Users can instantly receive this information through the application, enabling them to confidently address problems despite language barriers. An example of a prompt message is, "Please explain the legal procedures regarding unpaid wages for foreign technical trainees." In this way, the overall working environment for foreign workers is improved, and their social stability is enhanced.

[0568] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0569] Step 1:

[0570] The terminal receives text input from the user. This input includes questions about legal issues and workplace anxieties. The terminal prepares this text data for transmission to the server and sends it to the server over the network.

[0571] Step 2:

[0572] The server receives text data sent from the terminal. The server uses natural language processing techniques to identify the language of the input text. At the same time, it analyzes the overall context and intent of the text and determines the type of question based on this. This process utilizes a generative AI model to deepen the meaning of the text. The output is the category of the analyzed question and the determined language.

[0573] Step 3:

[0574] The server searches its knowledge base for relevant information based on the analyzed question. This search is performed based on specified legal categories and user needs. For example, if it is determined that information related to labor law is needed, the server retrieves the relevant data. As a result, the user receives specific legal information tailored to their needs.

[0575] Step 4:

[0576] The server performs sentiment analysis, evaluating the emotions contained in the user's input. This process analyzes the level of stress and anxiety the user is experiencing. The results of the sentiment analysis are used to determine whether additional support is needed. The output is the sentiment evaluation result.

[0577] Step 5:

[0578] The server combines the retrieved information with sentiment analysis to generate the optimal response. The generated response is then translated into the user's native language. This translation is carefully performed to preserve meaning and is implemented through a multilingual interface. The output is the translated text.

[0579] Step 6:

[0580] The server sends the generated translated text to the terminal. The terminal displays this text to the user. The user reviews the displayed information and receives guidance on specific legal advice and additional mental support. This process makes it easier to decide on the next steps toward resolving the problem.

[0581] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0582] This invention is a 24-hour multilingual chatbot system for foreign technical trainees. Its key feature is the incorporation of an emotion engine, which allows it to recognize the user's emotional state in real time and provide appropriate responses accordingly. This system analyzes emotions from user input and, when necessary, provides mental health resources to support improvements in the work environment.

[0583] User-level operations

[0584] Terminal: Users connect to the system via a dedicated application accessible from a smartphone or PC, or through a web browser. Users can freely input and submit questions and problems in text format.

[0585] Users can use the service by anonymously entering specific problems or questions, such as, "I've been feeling stressed at work lately. I don't know who to talk to about it."

[0586] Data analysis and response generation

[0587] Server: The server analyzes the received text data using a natural language processing engine to identify the language and understand the user's intent. This analysis helps determine which category the user's input belongs to.

[0588] Server: Based on the analysis results and emotional information recognized by the emotion engine, it searches for relevant information from the knowledge base. If stress or anxiety is detected, it can prioritize the selection of appropriate mental health resources.

[0589] Server: Based on this information, it generates an appropriate response to the user. Sentiment recognition ensures that the response is emotionally sensitive.

[0590] Adjusting and sending responses

[0591] Server: Automatically translates the generated response into the user's native language and formats it into natural-sounding, nuanced language.

[0592] Terminal: Displays the response sent from the server to the user. The user can read the provided information and decide what action to take next.

[0593] Recognition and utilization of emotions

[0594] Server: The emotion engine recognizes changes in emotions in real time based on user input and immediately notifies the user of additional support information if stress levels are high. It can also analyze past emotional trends and accumulate foundational information to provide personalized support.

[0595] This system will allow foreign technical trainees to receive appropriate advice and support tailored to their specific situation in real time, overcoming language barriers. The introduction of an emotion engine enables nuanced responses that respond to the user's emotions, allowing them to receive support to improve their work environment with peace of mind.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] User: Access the chatbot interface via their device, enter their question, concern, or anxiety in text format, and click the send button. Example: "Recently, I've been having trouble with relationships at work. How can I improve things?"

[0599] Step 2:

[0600] Terminal: Receives user input text, prepares it for transfer to the server via a secure communication channel, and sends the text data to the server.

[0601] Step 3:

[0602] Server: Passes the received text data to the natural language processing engine, which performs language identification and intent analysis of the text. This helps to understand the subject and category of the input content.

[0603] Step 4:

[0604] Server: Using an emotion engine, it extracts emotional components from user input and identifies emotions such as stress, anxiety, and joy. It also analyzes emotional changes in real time.

[0605] Step 5:

[0606] Server: Based on the results of language processing and sentiment analysis, it searches for relevant information from the knowledge base. Specifically, it integrates legal guidelines and mental health information to address users' concerns.

[0607] Step 6:

[0608] Server: After receiving information, it generates a response tailored to the user's emotional state. If high stress levels are detected, it prioritizes including mental health support information and uses warm, empathetic language.

[0609] Step 7:

[0610] Server: Translates the generated response into the user's chosen native language and adjusts it to produce natural-sounding text.

[0611] Step 8:

[0612] Terminal: Receives translated responses sent from the server and displays them to the user. The user can use this information to decide on their next action.

[0613] Step 9:

[0614] Server: Accumulates user emotional data and analyzes long-term emotional trends. Based on this data, responses in future interactions are personalized, and this data is used to develop individualized support plans.

[0615] (Example 2)

[0616] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0617] In today's work environment, providing effective mental support is challenging, especially for workers from diverse linguistic and cultural backgrounds. Many systems lack the ability to properly analyze users' emotions and respond in real time. Therefore, there is a need to provide individualized support that takes into account the user's psychological state.

[0618] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0619] In this invention, the server includes means for recognizing the user's emotions in real time and generating a response corresponding to that state, means for accumulating past emotional information and enabling personalized support, and means for generating an appropriate response based on analysis results and search results. This makes it possible to provide detailed support tailored to each user's situation in real time and in multiple languages.

[0620] A "user" refers to an individual who uses the system to input information and receive support.

[0621] "Data" refers to strings of characters or documents containing information that users input into the system.

[0622] "Device" refers to a combination of hardware and software used for receiving, displaying, converting, and transmitting information.

[0623] A "central processing unit" refers to a computer system that analyzes data received from users and processes it to generate appropriate responses.

[0624] A "database" refers to an information aggregation system that stores related information and manages it in a searchable format.

[0625] "Emotions" refer to the psychological state analyzed based on user input, and serve as the foundation for the system to provide support.

[0626] "Real-time" means that processing is performed immediately and responses are generated and provided without delay.

[0627] "Support information" refers to advice and resources provided according to the user's situation and needs.

[0628] This invention is a multilingual chatbot system that aims to provide real-time support that takes into account the user's psychological state. This system is particularly designed to support workers with diverse language and cultural backgrounds. An embodiment of this system is described below.

[0629] First, the device provides an interface with the user through applications or web browsers on smartphones or PCs. The user uses this device to send text data to the system. Specifically, they can input concerns such as, "I'm experiencing increased stress at work."

[0630] The server analyzes the received data using a natural language processing engine. This analysis includes language detection, intent identification, and sentiment analysis. Examples of natural language processing engines used include the open-source "SpaCy" and "NLTK".

[0631] To perform sentiment analysis, a pre-trained sentiment recognition model is utilized. The server uses this model to evaluate the user's emotions in real time and determine states such as stress and anxiety.

[0632] Based on the determined emotional state, the server searches its knowledge base for appropriate support information for the user. The knowledge base is a collection of mental health-related information sources and support services. Based on this information, it generates a helpful and emotionally sensitive response for the user. For example, it may include specific advice such as, "Try AA to reduce stress."

[0633] The generated response is translated into the user's native language by a translation system. Examples of translation tools used include "DeepL" and "Google Translate." The translated response is then refined to sound natural and fluent.

[0634] The device receives response data sent from the server and displays it to the user. This allows the user to decide on their next course of action based on the advice provided. Furthermore, past emotional data is stored in a database and used as a foundation for personalized support.

[0635] As a concrete example, the prompt could read: "Analyze the user's input, identify their emotions, and generate a response that suggests the most appropriate mental health resource if their stress level is high."

[0636] This system allows users to receive appropriate support tailored to their emotional state, transcending language barriers. This is expected to improve the psychological stability and productivity of workers in cross-cultural environments.

[0637] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0638] Step 1:

[0639] Users connect to the system via a smartphone or PC application or web browser and input their questions or concerns in text format. This text is then sent to the server as input data.

[0640] Step 2:

[0641] The server inputs the received text data into a natural language processing engine. Here, language identification and intent analysis are performed. The data analysis determines the intent behind the user's statements from the input text, and this information is passed on to the next processing step. The output consists of the analyzed text and its intent information.

[0642] Step 3:

[0643] The server inputs the analyzed text data into an emotion recognition model. Here, the model evaluates the user's emotional state in real time. An emotion score is calculated from the input text data, determining whether the user is experiencing stress or anxiety. The output includes the emotion score and the evaluation result.

[0644] Step 4:

[0645] The server sends queries to the knowledge database based on the results of emotion recognition and intent analysis, and extracts resource information suitable for the user. Relevant information is searched from the knowledge database according to the input query, and appropriate support information is obtained. A list of support information is generated as output.

[0646] Step 5:

[0647] The server generates a response based on the acquired support information and emotional information. It utilizes a generative AI model to create text responses that take into account the user's state and emotions. For example, it can produce sentences such as, "We will introduce you to mental health resources that are suitable for your current situation." The generated response text is then output.

[0648] Step 6:

[0649] The server inputs the generated response into the translation engine and translates it into the user's native language. The input text is formatted to appropriate grammar and expression through the translation process. As output, a response text with natural expression is generated.

[0650] Step 7:

[0651] The device receives the translated response and displays it to the user. At this stage, the user can choose their next action based on what is displayed. The user uses this information to decide on the appropriate course of action based on the information provided.

[0652] Through this series of processes, the system provides real-time support based on the user's emotions and intentions.

[0653] (Application Example 2)

[0654] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0655] Foreign staff working in multilingual and cross-cultural work environments face communication barriers and increased workplace stress, which can negatively impact their mental health. There are also concerns about decreased work efficiency and reduced job satisfaction due to stress. A system that can resolve these issues in real time is needed.

[0656] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0657] This invention includes a server that uses emotion recognition functionality to acquire user emotional information and adjusts its response based on the results; a server that measures the user's stress level in the work environment and provides information on mental health as needed; and a server that proposes specific advice to support job performance in physical stores based on user input and emotion analysis results. This enables real-time support for communication and stress-related problems faced by foreign staff, improving the work environment and protecting their mental health.

[0658] "Users" refer to foreign staff members who use this system and are people who seek support regarding communication and mental health in the workplace environment.

[0659] An "input device" is a device used by a user to register information, and includes terminals equipped with communication functions, such as smartphones and personal computers.

[0660] "Natural language processing technology" is a technology that converts human language into a format that computers can understand and interpret, and is used to identify language and analyze the user's intent.

[0661] A "knowledge database" is a database that stores information related to user input and plays a role in providing the information necessary for generating responses.

[0662] "Emotion recognition functionality" is a technology that analyzes emotional information from user input to identify the user's emotional state at any given time.

[0663] "Mental health information" refers to advice and resources to alleviate users' stress and anxiety, and includes specific support measures for improving the workplace environment.

[0664] "Specific advice" refers to practical advice provided to support users in carrying out their work, and it indicates the optimal course of action for the situation the user is facing.

[0665] This invention provides a system to address the communication and mental health challenges that foreign staff experience while working in physical stores. This system is accessible via input devices such as the user's smartphone, and a server equipped with natural language processing technology and emotion recognition capabilities plays a crucial role.

[0666] The server first receives input from the user and analyzes its content and sentiment using natural language processing technology. This process may utilize Google Cloud Natural Language API or Azure Cognitive Services. Based on the analysis, the user's intentions and emotional state are understood, and relevant information is retrieved from a knowledge database. This knowledge database contains data necessary to support the user's work performance and mental health.

[0667] Based on the analysis results and information obtained from the knowledge database, the server generates an appropriate response. Using emotion recognition data, the response takes into account the user's emotional state. The generated response is translated into the user's native language and displayed on the user's input device.

[0668] As a concrete example of its use, if a user inputs "Today, no matter what I do during customer service, customers won't smile, and I'm feeling down," the server can categorize this as "Needs suggestions for stress relief" and return advice such as, "Take a few deep breaths and focus on the next customer. If it's difficult to concentrate, consider taking a short break." The prompt itself is "Please tell me how to relieve stress while serving customers."

[0669] Thus, this invention provides real-time support to staff working in physical stores, helping them to perform their duties smoothly.

[0670] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0671] Step 1:

[0672] The user's input device receives input from the user. The input is in text format and includes the user's questions and requests. This information is sent to the server.

[0673] Step 2:

[0674] The server analyzes the received text data using natural language processing techniques. Technologies such as the Google Cloud Natural Language API are used to identify the input language and recognize the user's intent and emotional state. The analysis yields data on key keywords and emotional states.

[0675] Step 3:

[0676] The server searches for relevant information from the knowledge database based on the analysis results. Based on the keywords obtained, it selects data from the database that is useful to the user. This ensures that the information best suited to the user's needs is selected.

[0677] Step 4:

[0678] The server generates an appropriate response based on information obtained from the knowledge database and sentiment analysis results. The response will be considerate of the user's emotional state and may include advice and information to reduce the user's stress.

[0679] Step 5:

[0680] The server translates the generated response into the user's native language. Translation software, such as the Google Translate API, is used to format the response into natural and fluent language.

[0681] Step 6:

[0682] The server sends the translated response to the user's input device. The terminal displays the response in a user-friendly format to guide the user in deciding their next action.

[0683] Step 7:

[0684] The server monitors user feedback and ongoing emotional changes, providing additional support information and counseling resources as needed. This ensures users receive continuous and reassuring support.

[0685] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0686] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0687] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0688] [Fourth Embodiment]

[0689] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0690] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0691] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0692] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0693] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0694] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0695] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0696] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0697] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0698] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0699] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0700] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0701] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0702] This invention is embodied as a multilingual, 24-hour chatbot system for foreign technical trainees. Through user interaction, this system accepts consultations regarding legal issues and mental health, and provides appropriate information and support. A specific embodiment of this system is described below.

[0703] User Interface

[0704] Device: Users access the chatbot via a web browser or smartphone app. The interface is multilingual, allowing users to operate it in their chosen language.

[0705] User: Start communicating with the chatbot by typing text about problems or legal questions you are actually facing at work. For example, you can enter a specific question such as, "How should I deal with harassment from my boss?"

[0706] Data processing and analysis

[0707] Server: Receives text sent from the user and analyzes it using a natural language processing engine. This analysis identifies the language of the input text and understands the content and intent of the question.

[0708] Server: Based on the analysis results, it consults the knowledge base and searches for relevant information. If the question is legal, it plans to retrieve explanations regarding labor standards law and working conditions and provide them to the user.

[0709] Response generation

[0710] Server: Based on information retrieved from the knowledge base, it generates appropriate responses to user inquiries. These responses also take into account the user's emotional state and the urgency of their input. If the sentiment analysis indicates that the user is experiencing high levels of stress, it also provides mental health resources.

[0711] Translate and send

[0712] Server: Automatically translates the generated response into the user's native language, ensuring accurate translation without changing the meaning.

[0713] Terminal: Displays the translated response to the user, completing the information provision from the chatbot. This allows the user to receive specific advice on their issue and guidance on the next steps.

[0714] This implementation allows foreign technical trainees to receive appropriate support for problems they face in the workplace, regardless of language barriers, creating a secure working environment. Furthermore, users can access advice and information anytime, anywhere through the chatbot, enabling appropriate responses tailored to their individual needs.

[0715] The following describes the processing flow.

[0716] Step 1:

[0717] User: Using a smartphone or PC, open the chatbot window, type your question or concern as text, and press the send button.

[0718] Step 2:

[0719] Terminal: Prepares and sends the text data entered by the user to the server via a secure communication protocol.

[0720] Step 3:

[0721] Server: To analyze the received user data, the server initiates text analysis via a natural language processing engine. This involves identifying the language and inferring intent and context from the input.

[0722] Step 4:

[0723] Server: Determines the language of the input and, if translation into the system's internal processing language is necessary, performs automatic translation through the translation module.

[0724] Step 5:

[0725] Server: Based on the analyzed intent and content, it searches the knowledge base for relevant information and guidelines and retrieves the relevant information. If it is legal information, it extracts the key points of labor-related laws and regulations.

[0726] Step 6:

[0727] Server: Analyzes the emotions contained in the user's text, detects keywords and expressions indicating stress and anxiety, and identifies necessary mental health resources.

[0728] Step 7:

[0729] Server: Generates responses to resolve user questions based on knowledge base information and sentiment analysis results. Include additional relevant support information as needed.

[0730] Step 8:

[0731] Server: Translates the generated response into the user's native language, verifies the translation accuracy, and then prepares the formatted content for output.

[0732] Step 9:

[0733] Terminal: Receives translated response data sent from the server and displays it to the user in an appropriate format. The user can then review the information and decide on the next steps or actions to take.

[0734] (Example 1)

[0735] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Foreign technical trainees face challenges in seeking help for workplace problems due to language barriers and cultural differences. In particular, they often struggle to obtain prompt and accurate support regarding legal issues and mental health. This can jeopardize the safety and security of these trainees. Therefore, it is necessary to develop a multilingual, 24-hour support system to provide appropriate information and serve as a consultation service.

[0737] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0738] In this invention, the server includes a device for receiving information from a user, an information processing device that analyzes the user's information and identifies the language using natural language processing, means for retrieving relevant information from knowledge resources based on the user's information content, means for generating natural and human-like responses tailored to the user's inquiry using a generative AI model, and means for evaluating the user's emotions and the urgency of the input and adjusting the response content based on the results. This makes it possible to provide appropriate and prompt support in multiple languages ​​24 hours a day to address the problems faced by foreign technical trainees.

[0739] An "information terminal" is an electronic device used by users to input or receive information, and is connected to a network.

[0740] "Natural language processing" refers to the technology that allows computers to recognize and analyze natural language used by humans in everyday life.

[0741] An "information processing device" refers to a computer system that analyzes received data and performs necessary processing.

[0742] "Knowledge resources" refer to databases and information repositories that manage specific information and allow it to be retrieved as needed.

[0743] A "generative AI model" refers to software that uses artificial intelligence technology to automatically generate appropriate responses based on user input.

[0744] "Sentiment assessment" is a process for inferring a user's emotional state from their input and context, and determining the necessary response.

[0745] "Response content adjustment" refers to the operation of appropriately changing the content of information and advice provided based on the user's emotional state and input.

[0746] This invention is a multilingual chatbot system designed to support foreign technical trainees facing various challenges. Specific embodiments of this system are described below.

[0747] Terminal: Users access the system using a web browser or smartphone application. This provides a mechanism for easily sending questions and inquiries via text input. To enhance user convenience, the interface is multilingual.

[0748] Server: The server receives text data from users and analyzes the text using natural language processing technology. Specific software used includes natural language processing engines and text analysis APIs (e.g., the cloud service provider's reference API). Based on the analysis results, the server references internal knowledge resources (databases) to retrieve relevant information.

[0749] Knowledge resources: These contain information tailored to the user's needs, such as legal information and mental health-related information. The server accesses this information to extract the necessary details.

[0750] Generative AI Model: The server utilizes a generative AI model to generate highly natural and human-like responses to user input. These responses are designed to take into account the user's emotional state and urgency. Furthermore, automatic translation technology (e.g., online translation APIs) is used as needed to accurately translate the responses into the user's native language.

[0751] (Specific example)

[0752] For example, if a user asks, "How should I deal with harassment from my boss?", the server will provide appropriate legal information based on its knowledge of labor standards law, and if necessary, it will also provide mental health resources. An example of a prompt might be, "Generate an appropriate response to provide information in an easy-to-understand format to a foreign technical intern who seeks legal advice regarding their work environment," instructing the generation AI model in this way.

[0753] The distinguishing feature of this invention is the provision of appropriate and prompt support in multiple languages ​​for the diverse challenges that users face, thereby ensuring the accuracy and timeliness of information and providing peace of mind to users.

[0754] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0755] Step 1:

[0756] Terminal: Users access the system using a web browser or smartphone app. Users use the input interface to enter their concerns and questions in text format. The information entered consists of specific worries and problems, such as "working conditions at my workplace are not clearly stated."

[0757] Step 2:

[0758] Terminal: Sends text entered by the user to the information processing device. The data sent is text data entered by the user. Transmission is performed using a secure communication protocol.

[0759] Step 3:

[0760] Server: The server starts the natural language processing engine to analyze the received text data. The input data is a text message from the user. The analysis engine identifies the language of the text and processes the data to extract its content and intent. As a result, the subject and focus of the question in the text are obtained.

[0761] Step 4:

[0762] Server: Based on the analysis results, it references knowledge resources. The input is the output data of the analysis engine, and the server uses this to search the database for relevant legal and supporting information. This search process retrieves relevant legal provisions, FAQs, and case study data.

[0763] Step 5:

[0764] Server: Based on the acquired information, it generates a response using a generative AI model. The input data consists of information extracted from knowledge resources and the user's original question. The generative AI model utilizes this information to automatically construct the most appropriate response for the user. This response also takes the user's emotional state into consideration and adds mental health information as needed.

[0765] Step 6:

[0766] Server: Translates the generated response into the user's native language. The input is the response text generated by the generation AI model. A translation API is used for translation, maintaining semantic accuracy while converting to each language.

[0767] Step 7:

[0768] Server: Sends the translated response back to the terminal. The output is a text message ready to be sent to the user.

[0769] Step 8:

[0770] Terminal: The translated response is displayed on the user's terminal. The user can obtain specific advice regarding their problem based on the information provided. This information serves as a guide for their next actions.

[0771] (Application Example 1)

[0772] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0773] The legal issues and mental stress faced by foreign workers in the workplace are often not resolved quickly due to their insufficient understanding of Japanese or lack of access to appropriate resources for consultation, which increases anxiety in the work environment. Furthermore, the lack of systems capable of providing immediate support in multiple languages ​​makes early detection and response to problems difficult.

[0774] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0775] In this invention, the server includes means for using a terminal to receive input from a user, means for using a computer that analyzes the user's input using natural language processing technology and identifies the language, and means for retrieving relevant information from a knowledge base based on the user's input. This makes it possible for foreign workers to immediately consult about legal issues in the workplace and receive appropriate support and mental health assistance.

[0776] A "terminal" refers to a device used by a user to input and receive information, such as a smartphone or computer.

[0777] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, including language identification and semantic understanding of input text.

[0778] A "computer" is a device that has a central processing unit used to analyze user input and obtain necessary information, and generally functions as a server.

[0779] A "knowledge base" is a collection of information and data related to a specific domain, and it is a source of information that is referenced when generating answers to user questions.

[0780] "Sentiment analysis" is a data analysis technique that evaluates the emotions contained in user input and determines psychological states such as stress and anxiety.

[0781] "Multilingual support" refers to a feature that provides information in multiple languages, enabling users to solve problems in their own language.

[0782] "Legal issues" refer to problems and questions related to labor law, contract law, etc., and include legal consultations that foreign workers may face in the course of their work.

[0783] "Workplace" refers to the physical or virtual environment in which an individual performs work, and includes places such as shops and offices.

[0784] "Additional support" refers to supplementary information and resources provided to alleviate users' anxiety and stress, including guidance on professional consultation services.

[0785] This invention relates to a multilingual chatbot system for resolving legal issues and mental stress faced by foreign workers in the workplace. The system consists of a user terminal, a server, and a knowledge base.

[0786] The primary hardware used by users is either a smartphone or a computer. The device receives text input from the user and transmits it to the server via a communication line. The server analyzes the input text using natural language processing technology to identify the user's language and understand legally relevant questions.

[0787] On the server, the input data is processed by a computer, and relevant information is retrieved from a knowledge base. This knowledge base includes legal information such as labor law and contract law, and is used to generate appropriate responses to user questions. Furthermore, sentiment analysis technology is used to analyze the emotions contained in the user's input and assess stress and anxiety.

[0788] The generated responses are automatically translated into the user's native language and sent to their device for display. This allows users to quickly obtain specific advice relevant to their issues. Additional support information is also provided to help reduce stress, depending on the user's emotional state. For example, if a user seeks advice regarding harassment in the workplace, legal advice and links to specialists will be provided.

[0789] Users can instantly receive this information through the application, enabling them to confidently address problems despite language barriers. An example of a prompt message is, "Please explain the legal procedures regarding unpaid wages for foreign technical trainees." In this way, the overall working environment for foreign workers is improved, and their social stability is enhanced.

[0790] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0791] Step 1:

[0792] The terminal receives text input from the user. This input includes questions about legal issues and workplace anxieties. The terminal prepares this text data for transmission to the server and sends it to the server over the network.

[0793] Step 2:

[0794] The server receives text data sent from the terminal. The server uses natural language processing techniques to identify the language of the input text. At the same time, it analyzes the overall context and intent of the text and determines the type of question based on this. This process utilizes a generative AI model to deepen the meaning of the text. The output is the category of the analyzed question and the determined language.

[0795] Step 3:

[0796] The server searches its knowledge base for relevant information based on the analyzed question. This search is performed based on specified legal categories and user needs. For example, if it is determined that information related to labor law is needed, the server retrieves the relevant data. As a result, the user receives specific legal information tailored to their needs.

[0797] Step 4:

[0798] The server performs sentiment analysis, evaluating the emotions contained in the user's input. This process analyzes the level of stress and anxiety the user is experiencing. The results of the sentiment analysis are used to determine whether additional support is needed. The output is the sentiment evaluation result.

[0799] Step 5:

[0800] The server combines the retrieved information with sentiment analysis to generate the optimal response. The generated response is then translated into the user's native language. This translation is carefully performed to preserve meaning and is implemented through a multilingual interface. The output is the translated text.

[0801] Step 6:

[0802] The server sends the generated translated text to the terminal. The terminal displays this text to the user. The user reviews the displayed information and receives guidance on specific legal advice and additional mental support. This process makes it easier to decide on the next steps toward resolving the problem.

[0803] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0804] This invention is a 24-hour multilingual chatbot system for foreign technical trainees. Its key feature is the incorporation of an emotion engine, which allows it to recognize the user's emotional state in real time and provide appropriate responses accordingly. This system analyzes emotions from user input and, when necessary, provides mental health resources to support improvements in the work environment.

[0805] User-level operations

[0806] Terminal: Users connect to the system via a dedicated application accessible from a smartphone or PC, or through a web browser. Users can freely input and submit questions and problems in text format.

[0807] Users can use the service by anonymously entering specific problems or questions, such as, "I've been feeling stressed at work lately. I don't know who to talk to about it."

[0808] Data analysis and response generation

[0809] Server: The server analyzes the received text data using a natural language processing engine to identify the language and understand the user's intent. This analysis helps determine which category the user's input belongs to.

[0810] Server: Based on the analysis results and emotional information recognized by the emotion engine, it searches for relevant information from the knowledge base. If stress or anxiety is detected, it can prioritize the selection of appropriate mental health resources.

[0811] Server: Based on this information, it generates an appropriate response to the user. Sentiment recognition ensures that the response is emotionally sensitive.

[0812] Adjusting and sending responses

[0813] Server: Automatically translates the generated response into the user's native language and formats it into natural-sounding, nuanced language.

[0814] Terminal: Displays the response sent from the server to the user. The user can read the provided information and decide what action to take next.

[0815] Recognition and utilization of emotions

[0816] Server: The emotion engine recognizes changes in emotions in real time based on user input and immediately notifies the user of additional support information if stress levels are high. It can also analyze past emotional trends and accumulate foundational information to provide personalized support.

[0817] This system will allow foreign technical trainees to receive appropriate advice and support tailored to their specific situation in real time, overcoming language barriers. The introduction of an emotion engine enables nuanced responses that respond to the user's emotions, allowing them to receive support to improve their work environment with peace of mind.

[0818] The following describes the processing flow.

[0819] Step 1:

[0820] User: Access the chatbot interface via their device, enter their question, concern, or anxiety in text format, and click the send button. Example: "Recently, I've been having trouble with relationships at work. How can I improve things?"

[0821] Step 2:

[0822] Terminal: Receives user input text, prepares it for transfer to the server via a secure communication channel, and sends the text data to the server.

[0823] Step 3:

[0824] Server: Passes the received text data to the natural language processing engine, which performs language identification and intent analysis of the text. This helps to understand the subject and category of the input content.

[0825] Step 4:

[0826] Server: Using an emotion engine, it extracts emotional components from user input and identifies emotions such as stress, anxiety, and joy. It also analyzes emotional changes in real time.

[0827] Step 5:

[0828] Server: Based on the results of language processing and sentiment analysis, it searches for relevant information from the knowledge base. Specifically, it integrates legal guidelines and mental health information to address users' concerns.

[0829] Step 6:

[0830] Server: After receiving information, it generates a response tailored to the user's emotional state. If high stress levels are detected, it prioritizes including mental health support information and uses warm, empathetic language.

[0831] Step 7:

[0832] Server: Translates the generated response into the user's chosen native language and adjusts it to produce natural-sounding text.

[0833] Step 8:

[0834] Terminal: Receives translated responses sent from the server and displays them to the user. The user can use this information to decide on their next action.

[0835] Step 9:

[0836] Server: Accumulates user emotional data and analyzes long-term emotional trends. Based on this data, responses in future interactions are personalized, and this data is used to develop individualized support plans.

[0837] (Example 2)

[0838] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0839] In today's work environment, providing effective mental support is challenging, especially for workers from diverse linguistic and cultural backgrounds. Many systems lack the ability to properly analyze users' emotions and respond in real time. Therefore, there is a need to provide individualized support that takes into account the user's psychological state.

[0840] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0841] In this invention, the server includes means for recognizing the user's emotions in real time and generating a response corresponding to that state, means for accumulating past emotional information and enabling personalized support, and means for generating an appropriate response based on analysis results and search results. This makes it possible to provide detailed support tailored to each user's situation in real time and in multiple languages.

[0842] A "user" refers to an individual who uses the system to input information and receive support.

[0843] "Data" refers to strings of characters or documents containing information that users input into the system.

[0844] "Device" refers to a combination of hardware and software used for receiving, displaying, converting, and transmitting information.

[0845] A "central processing unit" refers to a computer system that analyzes data received from users and processes it to generate appropriate responses.

[0846] A "database" refers to an information aggregation system that stores related information and manages it in a searchable format.

[0847] "Emotions" refer to the psychological state analyzed based on user input, and serve as the foundation for the system to provide support.

[0848] "Real-time" means that processing is performed immediately and responses are generated and provided without delay.

[0849] "Support information" refers to advice and resources provided according to the user's situation and needs.

[0850] This invention is a multilingual chatbot system that aims to provide real-time support that takes into account the user's psychological state. This system is particularly designed to support workers with diverse language and cultural backgrounds. An embodiment of this system is described below.

[0851] First, the device provides an interface with the user through applications or web browsers on smartphones or PCs. The user uses this device to send text data to the system. Specifically, they can input concerns such as, "I'm experiencing increased stress at work."

[0852] The server analyzes the received data using a natural language processing engine. This analysis includes language detection, intent identification, and sentiment analysis. Examples of natural language processing engines used include the open-source "SpaCy" and "NLTK".

[0853] To perform sentiment analysis, a pre-trained sentiment recognition model is utilized. The server uses this model to evaluate the user's emotions in real time and determine states such as stress and anxiety.

[0854] Based on the determined emotional state, the server searches its knowledge base for appropriate support information for the user. The knowledge base is a collection of mental health-related information sources and support services. Based on this information, it generates a helpful and emotionally sensitive response for the user. For example, it may include specific advice such as, "Try AA to reduce stress."

[0855] The generated response is translated into the user's native language by a translation system. Examples of translation tools used include "DeepL" and "Google Translate." The translated response is then refined to sound natural and fluent.

[0856] The device receives response data sent from the server and displays it to the user. This allows the user to decide on their next course of action based on the advice provided. Furthermore, past emotional data is stored in a database and used as a foundation for personalized support.

[0857] As a concrete example, the prompt could read: "Analyze the user's input, identify their emotions, and generate a response that suggests the most appropriate mental health resource if their stress level is high."

[0858] This system allows users to receive appropriate support tailored to their emotional state, transcending language barriers. This is expected to improve the psychological stability and productivity of workers in cross-cultural environments.

[0859] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0860] Step 1:

[0861] Users connect to the system via a smartphone or PC application or web browser and input their questions or concerns in text format. This text is then sent to the server as input data.

[0862] Step 2:

[0863] The server inputs the received text data into a natural language processing engine. Here, language identification and intent analysis are performed. The data analysis determines the intent behind the user's statements from the input text, and this information is passed on to the next processing step. The output consists of the analyzed text and its intent information.

[0864] Step 3:

[0865] The server inputs the analyzed text data into an emotion recognition model. Here, the model evaluates the user's emotional state in real time. An emotion score is calculated from the input text data, determining whether the user is experiencing stress or anxiety. The output includes the emotion score and the evaluation result.

[0866] Step 4:

[0867] The server sends queries to the knowledge database based on the results of emotion recognition and intent analysis, and extracts resource information suitable for the user. Relevant information is searched from the knowledge database according to the input query, and appropriate support information is obtained. A list of support information is generated as output.

[0868] Step 5:

[0869] The server generates a response based on the acquired support information and emotional information. It utilizes a generative AI model to create text responses that take into account the user's state and emotions. For example, it can produce sentences such as, "We will introduce you to mental health resources that are suitable for your current situation." The generated response text is then output.

[0870] Step 6:

[0871] The server inputs the generated response into the translation engine and translates it into the user's native language. The input text is formatted to appropriate grammar and expression through the translation process. As output, a response text with natural expression is generated.

[0872] Step 7:

[0873] The device receives the translated response and displays it to the user. At this stage, the user can choose their next action based on what is displayed. The user uses this information to decide on the appropriate course of action based on the information provided.

[0874] Through this series of processes, the system provides real-time support based on the user's emotions and intentions.

[0875] (Application Example 2)

[0876] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0877] Foreign staff working in multilingual and cross-cultural work environments face communication barriers and increased workplace stress, which can negatively impact their mental health. There are also concerns about decreased work efficiency and reduced job satisfaction due to stress. A system that can resolve these issues in real time is needed.

[0878] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0879] This invention includes a server that uses emotion recognition functionality to acquire user emotional information and adjusts its response based on the results; a server that measures the user's stress level in the work environment and provides information on mental health as needed; and a server that proposes specific advice to support job performance in physical stores based on user input and emotion analysis results. This enables real-time support for communication and stress-related problems faced by foreign staff, improving the work environment and protecting their mental health.

[0880] "Users" refer to foreign staff members who use this system and are people who seek support regarding communication and mental health in the workplace environment.

[0881] An "input device" is a device used by a user to register information, and includes terminals equipped with communication functions, such as smartphones and personal computers.

[0882] "Natural language processing technology" is a technology that converts human language into a format that computers can understand and interpret, and is used to identify language and analyze the user's intent.

[0883] A "knowledge database" is a database that stores information related to user input and plays a role in providing the information necessary for generating responses.

[0884] "Emotion recognition functionality" is a technology that analyzes emotional information from user input to identify the user's emotional state at any given time.

[0885] "Mental health information" refers to advice and resources to alleviate users' stress and anxiety, and includes specific support measures for improving the workplace environment.

[0886] "Specific advice" refers to practical advice provided to support users in carrying out their work, and it indicates the optimal course of action for the situation the user is facing.

[0887] This invention provides a system to address the communication and mental health challenges that foreign staff experience while working in physical stores. This system is accessible via input devices such as the user's smartphone, and a server equipped with natural language processing technology and emotion recognition capabilities plays a crucial role.

[0888] The server first receives input from the user and analyzes its content and sentiment using natural language processing technology. This process may utilize Google Cloud Natural Language API or Azure Cognitive Services. Based on the analysis, the user's intentions and emotional state are understood, and relevant information is retrieved from a knowledge database. This knowledge database contains data necessary to support the user's work performance and mental health.

[0889] Based on the analysis results and information obtained from the knowledge database, the server generates an appropriate response. Using emotion recognition data, the response takes into account the user's emotional state. The generated response is translated into the user's native language and displayed on the user's input device.

[0890] As a concrete example of its use, if a user inputs "Today, no matter what I do during customer service, customers won't smile, and I'm feeling down," the server can categorize this as "Needs suggestions for stress relief" and return advice such as, "Take a few deep breaths and focus on the next customer. If it's difficult to concentrate, consider taking a short break." The prompt itself is "Please tell me how to relieve stress while serving customers."

[0891] Thus, this invention provides real-time support to staff working in physical stores, helping them to perform their duties smoothly.

[0892] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0893] Step 1:

[0894] The user's input device receives input from the user. The input is in text format and includes the user's questions and requests. This information is sent to the server.

[0895] Step 2:

[0896] The server analyzes the received text data using natural language processing techniques. Technologies such as the Google Cloud Natural Language API are used to identify the input language and recognize the user's intent and emotional state. The analysis yields data on key keywords and emotional states.

[0897] Step 3:

[0898] The server searches for relevant information from the knowledge database based on the analysis results. Based on the keywords obtained, it selects data from the database that is useful to the user. This ensures that the information best suited to the user's needs is selected.

[0899] Step 4:

[0900] The server generates an appropriate response based on information obtained from the knowledge database and sentiment analysis results. The response will be considerate of the user's emotional state and may include advice and information to reduce the user's stress.

[0901] Step 5:

[0902] The server translates the generated response into the user's native language. Translation software, such as the Google Translate API, is used to format the response into natural and fluent language.

[0903] Step 6:

[0904] The server sends the translated response to the user's input device. The terminal displays the response in a user-friendly format to guide the user in deciding their next action.

[0905] Step 7:

[0906] The server monitors user feedback and ongoing emotional changes, providing additional support information and counseling resources as needed. This ensures users receive continuous and reassuring support.

[0907] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0908] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0909] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0910] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0911] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0912] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0913] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0914] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0915] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0916] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0917] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0918] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0919] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0920] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0921] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0922] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0923] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0924] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0925] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0926] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0927] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0928] The following is further disclosed regarding the embodiments described above.

[0929] (Claim 1)

[0930] An input terminal that receives user input,

[0931] A server that uses natural language processing to analyze user input and identify the language,

[0932] A means of searching for relevant information from a knowledge base based on user input,

[0933] Means for generating an appropriate response based on analysis results and search results,

[0934] A means of translating the generated response into the user's native language,

[0935] A system including means for sending and displaying a translated response on an input terminal.

[0936] (Claim 2)

[0937] The system according to claim 1, comprising means for analyzing emotions from user input and selecting additional mental health resources based on the analysis results.

[0938] (Claim 3)

[0939] The system according to claim 1, further comprising means for providing emergency support information when an emergency response is required based on user input and sentiment analysis results.

[0940] "Example 1"

[0941] (Claim 1)

[0942] A device that receives information from the user,

[0943] An information processing device that analyzes user information using natural language processing and identifies the language,

[0944] A means of retrieving relevant information from knowledge resources based on the user's information content,

[0945] Means for generating an appropriate response based on analysis results and search results,

[0946] A means of translating the generated response into the user's native language,

[0947] A means for sending and displaying the translated response on an information terminal,

[0948] A method for generating natural and human-like responses tailored to the user's inquiry using a generative AI model,

[0949] A means of evaluating the user's emotions and the urgency of their input, and adjusting the response based on the results,

[0950] A system that includes this.

[0951] (Claim 2)

[0952] The system according to claim 1, comprising means for analyzing emotions from user information and selecting additional support resources based on the analysis results.

[0953] (Claim 3)

[0954] The system according to claim 1, comprising means for providing emergency response information when an emergency response is necessary based on the user's information content and the results of the sentiment evaluation.

[0955] "Application Example 1"

[0956] (Claim 1)

[0957] A terminal that receives user input,

[0958] A computer that analyzes user input using natural language processing technology and identifies the language,

[0959] A means of searching for relevant information from a knowledge base based on user input,

[0960] Means for generating an appropriate response based on analysis results and search results,

[0961] A means of translating the generated response into the user's native language,

[0962] A means of sending and displaying the translated response on a terminal,

[0963] A means of performing sentiment analysis on responses and suggesting additional support according to the user's emotional state,

[0964] It has a multilingual interface and provides a means to enable immediate support for legal challenges faced by foreign workers in the workplace,

[0965] A system that includes this.

[0966] (Claim 2)

[0967] The system according to claim 1, comprising means for referencing consultation cases based on the work environment when generating a response to user input.

[0968] (Claim 3)

[0969] The system according to claim 1, comprising means for generating a response that guides the user to a professional consultation service if it is determined that the user is experiencing high stress levels, based on the user's input and the results of sentiment analysis.

[0970] "Example 2 of combining an emotion engine"

[0971] (Claim 1)

[0972] A device that receives data from users,

[0973] A central processing unit that analyzes user data using natural language processing and identifies the language,

[0974] A device that retrieves relevant information from a database based on the user's data content,

[0975] A device that generates an appropriate response based on analysis results and search results,

[0976] A device that translates the generated response into the user's native language,

[0977] A device that transmits and displays the converted response to a device,

[0978] A device that recognizes a user's emotions in real time and generates a response appropriate to that state,

[0979] A device that accumulates past emotional information and enables personalized support,

[0980] A system that includes this.

[0981] (Claim 2)

[0982] It analyzes the user's emotional changes and provides supplementary support information based on the analysis results.

[0983] The system according to claim 1.

[0984] (Claim 3)

[0985] Providing appropriate support information when emergency assistance is needed based on past emotional history.

[0986] The system according to claim 1.

[0987] "Application example 2 when combining with an emotional engine"

[0988] (Claim 1)

[0989] An input device that receives input from the user,

[0990] A device that analyzes user input using natural language processing technology and identifies the language,

[0991] A means of retrieving relevant data from a knowledge database based on user input,

[0992] Means for generating an appropriate response based on analysis results and search results,

[0993] A means of translating the generated response into the user's native language,

[0994] Means for transmitting and displaying the translated response to an input device,

[0995] A means for acquiring user emotional information using emotion recognition functionality and adjusting responses based on the results,

[0996] A system that includes means for measuring the user's stress level in their work environment and providing mental health information as needed.

[0997] (Claim 2)

[0998] The system according to claim 1, comprising means for suggesting specific advice to support the performance of duties in a physical store based on user input and sentiment analysis results.

[0999] (Claim 3)

[1000] The system according to claim 1, which includes means to improve operational efficiency in physical stores by providing high-priority support information when prompt action is required, based on user input and sentiment analysis results. [Explanation of Symbols]

[1001] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input terminal that receives user input, A server that uses natural language processing to analyze user input and identify the language, A means of searching for relevant information from a knowledge base based on user input, Means for generating an appropriate response based on analysis results and search results, A means of translating the generated response into the user's native language, A system including means for sending and displaying a translated response on an input terminal.

2. The system according to claim 1, comprising means for analyzing emotions from user input and selecting additional mental health resources based on the analysis results.

3. The system according to claim 1, further comprising means for providing emergency support information when an emergency response is necessary based on the user's input and the results of sentiment analysis.

Images