system

Abstract

We provide the system. [Solution] A means of loading multiple knowledge-based models tailored to specific fields, A means for collecting and pre-processing information related to the person receiving assistance, A means by which each knowledge-based model exchanges opinions using pre-processed information, A means for aggregating the results of the aforementioned exchange of opinions and generating proposals, A means of presenting the generated proposals and supporting the final decision-making, Means to securely store data and prepare for future use, A system that includes this.

Description

Technical Field

[0005] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional care certification review meeting, since a plurality of experts such as doctors and nurses need to physically gather, the workload and time constraints of the experts have become major issues. As a result, there are variations in the quality of the review, which affects the fair and prompt certification of care recipients. The present invention aims to solve these problems and realize an efficient and high-quality review process.

Means for Solving the Problems

[0005] This invention proposes a multi-agent system that enables remote discussion by using multiple knowledge-based models tailored to specific fields of expertise. It collects and preprocesses information related to the person receiving care to generate an optimal dataset. The knowledge-based models exchange opinions using the common data, aggregating the results to generate proposals. These proposals are presented to the user to support final decision-making. Furthermore, all data is securely stored and available for future analysis and system improvement. This reduces the burden on professionals and enables rapid and fair care assessment.

[0006] A "knowledge-based model" is an artificial intelligence model that is trained with knowledge in a specific field and has the ability to analyze and process various types of information.

[0007] "Preprocessing" refers to a series of operations or processes that shape and transform raw data for analysis and interpretation.

[0008] "Exchange of opinions" refers to the process by which multiple knowledge-based models share information from the perspective of their respective fields of expertise and collaborate to advance discussions.

[0009] "Aggregation" refers to the act of combining multiple individual data points or opinions into a single, cohesive form.

[0010] A "proposal" refers to a recommended option or judgment that arises as a result of discussion or analysis.

[0011] A "user" is an individual or organization that uses a system to input or output information.

[0012] "Secure storage" means storing data in a way that protects it from unauthorized access and tampering, while also allowing it to be retrieved when needed. [Brief explanation of the drawing]

[0013] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention provides a system that streamlines the care certification review process by utilizing a knowledge-based model with diverse expertise. This system consists of the interaction of a server, terminals, and users.

[0035] The server loads multiple pre-prepared knowledge-based models. Each model specializes in a different field, such as medicine, healthcare, or nutrition, allowing for analysis from different perspectives. This enables the multifaceted analysis necessary for evaluating care recipients.

[0036] The terminal receives data on the care recipient submitted by the user. For example, it can input past medical history, living environment information, and desired certification details. The received data is preprocessed by the server and organized into a format suitable for input into the model.

[0037] The server initiates discussions among the knowledge-based models based on pre-processed data. The exchange of opinions between models takes place in natural language, with each model contributing insights from its area of ​​expertise. The generated opinions are aggregated by the server and compiled into a final proposal.

[0038] The proposed results are presented to the user via their device. Based on these proposals, the user determines the final care level of the person receiving care. The data generated during this process is securely stored on the server and used for future analysis as needed.

[0039] To give a specific example, if the terminal is given input data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup," a medical-focused model will emphasize the importance of diabetes management, and a nutrition model will suggest appropriate meals. Integrating these opinions, the server will propose a conclusion of "Level 2 care needs." This proposal is then reviewed by the user and used for the final determination.

[0040] In this way, by ensuring that each step is executed smoothly, it becomes possible to carry out an efficient and high-quality long-term care certification assessment.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server loads the knowledge-based model. It deploys the domain-specific models into memory along with the necessary computing resources.

[0044] Step 2:

[0045] The terminal receives input data from the user. Information such as the care recipient's health status, past medical records, and living environment is entered. This data becomes basic information necessary for subsequent processes.

[0046] Step 3:

[0047] The server preprocesses the received data. Using natural language processing, it converts unstructured data into a model input format and organizes it as structured information. It also retrieves necessary information from relevant laws and past case databases using a RAG (Random Aggregation) configuration.

[0048] Step 4:

[0049] The server initiates a multi-agent simulation. Pre-processed data is provided to each specialized model, and they engage in natural language exchange. Each model proceeds with the discussion based on its knowledge in its respective field.

[0050] Step 5:

[0051] The server aggregates the discussion content and generates the final proposal. Insights from each model are integrated to form conclusions from multiple perspectives. During this process, a proposal is created that takes into account the necessary weightings and priorities.

[0052] Step 6:

[0053] The terminal presents the generated proposal to the user. The user reviews the proposal and makes a final decision on the care needs assessment level. Additional input and modifications can be made as needed.

[0054] Step 7:

[0055] The server securely stores data. All information, including input data, discussion processes, and final proposals, is properly stored and preserved in a format that can be used for future analysis and improvement.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] The process of assessing the level of care required requires the efficient evaluation of diverse information about the care recipient and the accurate determination of the appropriate level of care needed. However, current methods make it difficult to integrate sufficient expert opinions during the assessment, leading to errors and wasted time. Therefore, technologies are needed to improve the accuracy and efficiency of care assessment.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for loading multiple knowledge base models tailored based on specialized domains, means for acquiring various information related to the person receiving care and processing that information, and means for each knowledge base model to exchange information with each other in natural language based on the pre-processed information. This enables a multifaceted evaluation of the person receiving care and the rapid and accurate proposal of care levels.

[0061] A "specialized field" refers to an area where specific knowledge or skills are studied intensively and deeply understood.

[0062] A "knowledge-based model" is a means of analyzing and evaluating information by utilizing knowledge within a specialized field.

[0063] "Information processing" is the process of converting acquired information into an appropriate format and organizing incomplete data to make it analyzable.

[0064] "Information exchange" is the act of communicating opinions and views between different knowledge-based models to deepen mutual understanding.

[0065] A "proposal" is a conclusion that integrates discussions across knowledge-based models and indicates the optimal course of action for the person receiving care.

[0066] "Care level" refers to a classification that indicates the degree of care needed according to the condition of the person receiving care.

[0067] The system of this invention utilizes a knowledge-based model with diverse expertise to streamline the long-term care certification process. This system is realized through the interaction of servers, terminals, and users.

[0068] The server utilizes machine learning frameworks such as TENSORFLOW® and PyTorch to load multiple knowledge-based models from specialized fields such as medicine, healthcare, and nutrition. These models are pre-configured, allowing for the analysis of care recipient information from multiple perspectives.

[0069] The terminal receives information about the care recipient entered by the user via a secure communication protocol (e.g., HTTPS). This information includes past health history, living situation, and desired care needs assessment. The server then preprocesses the data and formats it into a format suitable for the knowledge base model. This includes information processing such as normalization of numerical data and encoding of categorical data.

[0070] The server inputs pre-processed data into each knowledge-based model and initiates a natural language exchange of ideas. Each model uses its expertise to analyze the data from its own perspective and generates the results in text format. The server then integrates the outputs of each model to create the optimal recommendation.

[0071] This proposal is presented to the user via a terminal, and the user makes the final decision on the care level of the person receiving care based on the presented proposal. During this process, all data is securely stored on a server and used for future analysis as needed.

[0072] For example, if data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup" is entered into the terminal, the medical model will emphasize the importance of diabetes management, and the nutrition model will suggest an appropriate diet. Based on these analysis results, the server generates a conclusion of "Level 2 care needs" and reports it to the user through the terminal.

[0073] An example of a prompt for a generative AI model would be: "Given the data on the care recipient's health status (XX) and living environment (YY), propose the optimal care level." Implementing this system can improve both the accuracy and efficiency of care assessments.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] At system startup, the server loads multiple knowledge-based models, each specializing in areas such as medicine, healthcare, and nutrition, using TensorFlow or PyTorch. Each model encompasses different areas of expertise, enabling a multifaceted approach. In this step, the knowledge-based models are loaded into the server's memory and prepared for processing.

[0077] Step 2:

[0078] The user inputs information about the person receiving care (e.g., health history, living situation) into the terminal using a dedicated interface. The terminal transmits the entered information to the server via HTTPS. The input data is raw and unprocessed, requiring preprocessing on the server side.

[0079] Step 3:

[0080] The server preprocesses the raw data received from the terminal. As part of the preprocessing, it performs tasks such as imputing missing data, normalizing numerical data, and one-hot encoding categorical data. At this stage, the input is the raw data from the terminal, and the output is formatted data that can be used by the model.

[0081] Step 4:

[0082] The server inputs the pre-processed data into each knowledge-based model and begins generating opinions using natural language. Each model analyzes the data based on its own expert perspective. For example, the medical model assesses the need for diabetes management. This process generates model-specific opinions as output, based on the formatted data as input.

[0083] Step 5:

[0084] The server aggregates the opinions generated from each model. This process utilizes natural language processing techniques to ensure consistency between opinions and compile a final proposal. The integrated proposal is then output in a format that can be directly presented to the user as guidance.

[0085] Step 6:

[0086] The terminal displays the final proposal received from the server to the user. Based on the presented information, the user makes the final decision regarding the care level of the person receiving care. At this stage, the proposal is provided in a format that is easy for the user to understand.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] Traditional care needs assessment processes have suffered from the problem of not fully utilizing specialized knowledge in specific fields, making it difficult to comprehensively evaluate the condition of those receiving care. Furthermore, while real-time monitoring of health information and rapid response in the event of an abnormality are required, there is a lack of effective means, leaving challenges in terms of safety and efficiency in the care environment.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server includes means for loading multiple knowledge-based models tailored based on specialized fields, means for collecting and pre-processing information related to the person being cared for, and means for monitoring health information in real time using an information display device. This enables a multifaceted evaluation of the health status of the person being cared for, and allows for rapid intervention, including detection of abnormalities.

[0092] A "knowledge-based model" is a data model that uses information based on specialized fields to analyze specific problems and generate proposals.

[0093] An "information display device" is a device that displays the health information of the person being cared for in real time and presents it to the user immediately.

[0094] "Real-time monitoring" is a process that involves continuously collecting data and analyzing it on the spot to immediately assess the situation.

[0095] "Anomaly detection" is a function that identifies data that exceeds the normal range and notifies the user that there is a problem.

[0096] A "safety system" is a general term for the networks and equipment used to manage and maintain safety within a facility.

[0097] The system of this invention is designed to effectively manage the health of those receiving care and improve safety in nursing care facilities. The server loads multiple knowledge-based models tailored to specific fields of expertise. This enables analysis from different perspectives, such as medical, health, and nutrition. The server uses Python and TensorFlow to build the execution environment for the knowledge-based models.

[0098] The terminal uses smart glasses, which are used by care staff. The smart glasses have the function of monitoring the health information of the person being cared for in real time and displaying the results. The application is implemented using cross-platform development tools such as Flutter (registered trademark). Sensor data such as heart rate and activity level acquired from the glasses is sent to a server, normalized, and then input into each model.

[0099] The care staff, as users of the system, can monitor changes in health conditions in real time and take prompt action based on the information as needed. If suspicious movements or abnormalities are detected, the system immediately issues a warning and prompts a response in conjunction with the facility's safety system. This significantly improves the safety and response efficiency of care facilities.

[0100] For example, if a care recipient's heart rate is detected to be higher than normal, the smart glasses will display the health assessment results and specific countermeasures. In addition, based on information from security cameras, the care recipient's current location can be quickly confirmed, and appropriate safety measures can be taken.

[0101] An example of a prompt would be, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations." Based on such prompts, the generative AI model immediately presents useful information to care staff.

[0102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0103] Step 1:

[0104] The server loads knowledge-based models based on various specialties. These include models with different expertise in medicine, health, and nutrition. The loaded models are ready to analyze data on those receiving care and evaluate them from various perspectives.

[0105] Step 2:

[0106] The smart glasses on the device collect sensor information from the person being cared for in real time. This includes heart rate and activity level. This information is transmitted wirelessly to a server, where the data is normalized. The normalized data is then converted into a format that can be used for subsequent model analysis.

[0107] Step 3:

[0108] The server passes normalized data as input to each knowledge-based model. These models perform evaluations based on the received data and generate expert opinions. Through the exchange of opinions between the models, various elements that constitute the final health assessment are gathered.

[0109] Step 4:

[0110] The care staff, who are the users, receive health assessment results generated through smart glasses. This includes real-time changes in health status and warning messages. For example, if a higher-than-normal heart rate is detected, the glasses will display appropriate countermeasures.

[0111] Step 5:

[0112] The server immediately issues a warning when it detects suspicious activity or anomalies, and works in conjunction with the facility's security system to establish a rapid response system. This allows users to take quick and efficient countermeasures.

[0113] Step 6:

[0114] Users can quickly obtain the necessary information by instructing the generating AI model using prompt sentences. For example, by entering a prompt such as, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations," users can obtain appropriate instructions and suggestions tailored to their specific situation.

[0115] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0116] This invention is an advanced decision support system that combines an emotion engine and improves the care certification assessment process using multiple knowledge-based models tailored to specific fields of expertise. This system is built through server, terminal, and user interaction.

[0117] The server first loads knowledge-based models that are finely tuned for each specialized field. These models include functions specific to various fields such as medicine, healthcare, and nutrition, and are designed to provide comprehensive judgment. Furthermore, an emotion engine is integrated by the server to analyze the user's facial expressions and speech to estimate their psychological state.

[0118] The terminal receives data about the care recipient provided by the user. The information entered includes health status, living environment, and past medical records, and this information is sent to the server. The server receives this data and formats it into an analyzable form through natural language processing and data preprocessing.

[0119] Based on the pre-processed data and the sentiment engine's analysis, the server initiates a discussion among the various knowledge-based models. The models exchange opinions, and evaluation results are generated based on each model's expertise. Based on these results, the server forms the final proposal.

[0120] Users can review suggestions generated through their device. These suggestions are optimized to the user's emotional state and presented in an easily acceptable manner. For example, if the emotional engine identifies anxiety about the level of care needed, suggestions with additional explanations and advice will be provided.

[0121] In this process, the server securely saves all processing steps to a database. This saved data is used for subsequent analysis and system optimization, forming the basis for gaining new insights. These actions improve the efficiency and quality of the care needs assessment process.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The server loads multiple knowledge base models, each finely tuned for a specific field. Models for different fields such as medicine, healthcare, and nutrition are deployed and prepared in memory.

[0125] Step 2:

[0126] The terminal receives information about the person receiving care, necessary for the care needs assessment, from the user. This information includes health status, living environment, and past medical history. The terminal then sends this data to the server.

[0127] Step 3:

[0128] The server preprocesses the input data. Using natural language processing techniques, it structures the raw data and converts it into a format that can be processed by the model. It also retrieves relevant laws and cases from historical databases using a RAG (Random Aggregation) configuration.

[0129] Step 4:

[0130] The server uses an emotion engine to analyze the user's facial expressions and voice data. It estimates the user's emotional state and incorporates that information into subsequent processes.

[0131] Step 5:

[0132] The server feeds pre-processed data into each knowledge-based model and initiates an exchange of ideas. Each model generates an opinion based on its area of ​​expertise and engages in natural language discussions with other models.

[0133] Step 6:

[0134] The server aggregates the results of the exchange of opinions and, taking into account the analysis results of the emotion engine, generates a final proposal. If necessary, the proposal is adjusted to match the user's emotions.

[0135] Step 7:

[0136] The terminal presents the generated suggestions to the user. The suggestions are displayed in a way that promotes reassurance and understanding, allowing the user to make a final decision on the level of care required.

[0137] Step 8:

[0138] The server securely stores all data. Input data, sentiment analysis, discussion processes, and final proposals are fully recorded and used for future system improvements and the development of new care methodologies.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0141] In the process of assessing long-term care needs, there is a need to comprehensively evaluate information on the diverse health status and living environment of those receiving care, and to provide effective proposals that reflect individual conditions and circumstances. Furthermore, it is necessary to provide support that takes into account the user's psychological state. Conventional technologies have had problems such as one-sided evaluations and uniform proposals, which have prevented the quality of decision-making from being sufficiently improved.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes functional means for loading multiple knowledge base models tailored based on specialized fields; preprocessing functional means for collecting information related to the person receiving care using an interactive data acquisition device and converting the input data into an analyzable form; and functional means for each knowledge base model to exchange opinions, including emotion estimation, based on the preprocessed information, and generate evaluation results. This enables a comprehensive evaluation that is multifaceted and considerate of emotions, as well as the provision of individualized suggestions.

[0144] A "specialized field" refers to an area of ​​study or work where specialized knowledge and skills are required.

[0145] A "knowledge-based model" is a software model that collects information and data related to a specific field and uses that information to perform inferences and make decisions.

[0146] An "interactive data acquisition device" is a device that collects data through interaction with the user and effectively processes the user's input.

[0147] "Preprocessing" refers to a series of processes, such as data cleaning and format conversion, performed to transform raw data into a format that is easy to analyze.

[0148] "Emotion estimation" is a process for estimating a user's emotions by judging their psychological state from factors such as their facial expressions and tone of voice.

[0149] "Exchange of opinions" is a process in which multiple knowledge-based models share information from different perspectives to derive an overall judgment.

[0150] "Suggestions" refer to actionable guidelines and advice derived from evaluation results, intended to support users' decision-making.

[0151] "Functional means" refers to mechanical or electronic components or processes implemented to achieve a specific purpose.

[0152] The system of this invention is designed to provide advanced support for the long-term care certification review process, and functions through the mutual cooperation of three parties: a server, a terminal, and a user.

[0153] First, the server loads multiple knowledge-based models finely tuned for specialized fields such as medicine, healthcare, and nutrition. This enables field-specific decision-making. Furthermore, the server integrates an emotion engine to analyze the user's facial expressions and speech to estimate their psychological state. This emotion analysis function allows the server to provide more personalized support.

[0154] The terminal provides an interface for receiving information about the person receiving care from the user. This information includes health status, living environment, and past medical records. The terminal collects this information and sends it to the server.

[0155] The server preprocesses the received data. This preprocessing includes cleaning the data using natural language processing techniques and converting it into an analyzable format. Subsequently, based on the preprocessed data, each knowledge base model exchanges opinions with the others and generates evaluation results from their respective expert perspectives.

[0156] For example, when a care recipient's living environment changes, the server can use an emotion engine to analyze how anxious the user is about that change. Based on this, the server proposes a care plan designed to provide greater reassurance.

[0157] An example of a prompt message to be input into the generating AI model is, "Use the expert knowledge model to provide comprehensive suggestions regarding the care recipient's living environment." This allows the system to accurately provide the support the user needs.

[0158] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0159] Step 1:

[0160] The server loads multiple knowledge-based models, finely tuned based on specific fields. These models are specialized for various areas such as medicine, healthcare, and nutrition, and form the basis for complex data analysis and decision-making. The input requires preliminary data relevant to each field, which is loaded by the server. The output is the setup of each model for use.

[0161] Step 2:

[0162] The terminal collects information about the care recipient from the user. This information includes health status, living environment, and past medical history. The user inputs this information through the terminal's interface. The input data is sent from the terminal to the server and used as the basis for subsequent analysis. The output is formatted data sent to the server.

[0163] Step 3:

[0164] The server performs preprocessing on the data received from the terminal. This preprocessing includes data cleaning and format conversion using natural language processing techniques. Raw data is provided as input and is formatted into a format that is easy to analyze. The output is the formatted data prepared for further analysis.

[0165] Step 4:

[0166] The server uses an emotion engine to estimate the user's psychological state based on pre-processed data. This includes user input, tone of voice, and, in some cases, facial expression analysis. If the emotion engine identifies anxiety or worry in the user, this is used as data within the server. The output is data related to the user's psychological state.

[0167] Step 5:

[0168] The server hands over pre-processed data and sentiment analysis results to the knowledge-based models, initiating opinion exchange and evaluation results generation by the models. Inputs include formatted data and emotional state information, while outputs are evaluation results from each model. The knowledge-based models analyze the data from different expert perspectives using varying approaches, leading to a more comprehensive discussion.

[0169] Step 6:

[0170] The server integrates evaluation results from the models and generates optimized suggestions to present to the user. These suggestions are adjusted to take into account individual circumstances and the user's psychological state. The input requires evaluation results from each model, and the output generates specific suggestions.

[0171] Step 7:

[0172] Users can review the suggestions generated through their device. The suggestions are displayed in an easy-to-understand format, allowing users to easily make decisions based on them. Users can request additional information or provide feedback on the suggestions via their device as needed.

[0173] Step 8:

[0174] The server securely stores all data and evaluation results generated throughout the entire process in a database. This stored data is used for future analysis and system optimization. The input requires all generated data, which is stored in an organized manner. The output is historical data stored in a database, corresponding to anticipated future uses.

[0175] (Application Example 2)

[0176] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0177] Modern technological advancements have created a need for support systems based on specialized knowledge in various fields. However, many existing systems provide uniform responses without considering the user's psychological state or emotions, making them unable to provide flexible support tailored to individual situations. Therefore, there is a need to analyze the user's psychological state and optimize suggestions accordingly. In particular, considering the user's sense of security is crucial in areas such as security, but existing systems fail to address this aspect. Consequently, there is a need to develop new systems that can provide decision-making support tailored to the user's psychological state.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0179] In this invention, the server includes means for loading multiple knowledge data models tailored based on specialized domains, means for collecting and preprocessing information related to the person being assisted, and means for analyzing the psychological state and optimizing suggestions. This enables flexible decision-making support that takes into account the user's psychological state.

[0180] "Multiple knowledge data models tailored to specific areas of expertise" are data models that possess specialized knowledge in individual fields such as medicine, health, and nutrition, and are tailored to each field to support optimal decision-making.

[0181] "Information related to the person receiving support" refers to a collection of specific data about the individual in question that is necessary for the system to provide decision-making support, such as health status, living environment, and past medical records.

[0182] "A means of analyzing psychological state and optimizing suggestions" refers to a technology that uses an emotion engine to analyze the user's facial expressions and statements, optimizes support suggestions based on the results, and enables more personalized responses.

[0183] A "means for exchanging opinions" is a mechanism in which multiple knowledge data models exchange information with each other, discuss from their respective expert perspectives, and form a final judgment.

[0184] "Means for securely recording data and preparing for future use" refers to a recording system that securely stores the processing and proposals performed within the system, making them available for later analysis and system improvement.

[0185] The system for implementing this invention functions through the interaction of a server, a terminal, and a user.

[0186] The server first loads multiple knowledge data models, each tailored to specific professional fields such as medicine and healthcare. These models incorporate different expert perspectives to support comprehensive decision-making. Furthermore, an emotion engine is used to analyze the user's facial expressions and statements, estimating their psychological state. Optimization of suggestions based on emotional state is also ensured at this stage.

[0187] The device collects various pieces of information related to the person receiving assistance from the user. This includes important data such as health status and past medical records, and this information is sent to a server to prepare for further processing and analysis.

[0188] Users can receive suggestions from the server via their device and make decisions based on them. A key feature is that the suggestions are tailored to the user's psychological state and presented in a way that is easily accepted.

[0189] The hardware used here includes the smartphone's camera and microphone, which are used to analyze the user's facial expressions and voice data. The software utilizes generative AI models such as TensorFlow to enable sentiment analysis. Appropriate database technologies such as SQLite are used for data storage.

[0190] For example, if a user expresses security concerns in a particular location, the emotion engine analyzes this and provides security advice for connecting to public Wi-Fi. This allows the user to continue communicating with confidence. A prompt such as, "Think about how a security app can help in a situation where you are feeling anxious," is used.

[0191] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0192] Step 1:

[0193] The terminal receives information about the person receiving assistance from the user as input. Specifically, it collects data such as health status, past medical records, and living environment through the smartphone interface and sends this data to the server. The input here is formatted as text information.

[0194] Step 2:

[0195] The server preprocesses the received information. It analyzes the input data using natural language processing techniques and structures the individual pieces of information. This standardizes the format of each data point, enabling rapid and efficient analysis using knowledge data models.

[0196] Step 3:

[0197] The server simultaneously loads multiple knowledge data models, each tailored to a specific domain. Generative AI models are then used to send pre-processed data to each model, generating opinions from their respective expert perspectives. The output is then sent back, with the models exchanging opinions.

[0198] Step 4:

[0199] The server uses an emotion engine to analyze the user's on-screen statements and psychological nuances collected by the device. The input consists of the user's facial expressions and voice, which are used to estimate their psychological state. The resulting output is a profile of the user's psychological state.

[0200] Step 5:

[0201] The server integrates the output from the knowledge data model and the emotion engine to generate the final recommendation. This recommendation is optimized to take the user's psychological state into account. For example, if the user is showing anxiety, the output will include specific recommendations and precautions to address that anxiety.

[0202] Step 6:

[0203] The user receives the final proposal generated through their device and makes a decision based on it. The final proposal is displayed as text and visual data, presented in a clear and easy-to-understand format for the user.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] This invention provides a system that streamlines the care certification review process by utilizing a knowledge-based model with diverse expertise. This system consists of the interaction of a server, terminals, and users.

[0221] The server loads multiple pre-prepared knowledge-based models. Each model specializes in a different field, such as medicine, healthcare, or nutrition, allowing for analysis from different perspectives. This enables the multifaceted analysis necessary for evaluating care recipients.

[0222] The terminal receives data on the care recipient submitted by the user. For example, it can input past medical history, living environment information, and desired certification details. The received data is preprocessed by the server and organized into a format suitable for input into the model.

[0223] The server initiates discussions among the knowledge-based models based on pre-processed data. The exchange of opinions between models takes place in natural language, with each model contributing insights from its area of ​​expertise. The generated opinions are aggregated by the server and compiled into a final proposal.

[0224] The proposed results are presented to the user via their device. Based on these proposals, the user determines the final care level of the person receiving care. The data generated during this process is securely stored on the server and used for future analysis as needed.

[0225] To give a specific example, if the terminal is given input data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup," a medical-focused model will emphasize the importance of diabetes management, and a nutrition model will suggest appropriate meals. Integrating these opinions, the server will propose a conclusion of "Level 2 care needs." This proposal is then reviewed by the user and used for the final determination.

[0226] In this way, by ensuring that each step is executed smoothly, it becomes possible to carry out an efficient and high-quality long-term care certification assessment.

[0227] The following describes the processing flow.

[0228] Step 1:

[0229] The server loads the knowledge-based model. It deploys the domain-specific models into memory along with the necessary computing resources.

[0230] Step 2:

[0231] The terminal receives input data from the user. Information such as the care recipient's health status, past medical records, and living environment is entered. This data becomes basic information necessary for subsequent processes.

[0232] Step 3:

[0233] The server preprocesses the received data. Using natural language processing, it converts unstructured data into a model input format and organizes it as structured information. It also retrieves necessary information from relevant laws and past case databases using a RAG (Random Aggregation) configuration.

[0234] Step 4:

[0235] The server initiates a multi-agent simulation. Pre-processed data is provided to each specialized model, and they engage in natural language discussions. Each model proceeds with the discussion based on its knowledge in its respective field.

[0236] Step 5:

[0237] The server aggregates the discussion content and generates the final proposal. Insights from each model are integrated to form conclusions from multiple perspectives. During this process, a proposal is created that takes into account the necessary weightings and priorities.

[0238] Step 6:

[0239] The terminal presents the generated suggestions to the user. The user reviews the suggestions and makes a final decision on the care needs assessment level. Additional input and modifications can be made as needed.

[0240] Step 7:

[0241] The server securely stores data. All information, including input data, discussion processes, and final proposals, is properly stored and preserved in a format that can be used for future analysis and improvement.

[0242] (Example 1)

[0243] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0244] The process of assessing the level of care required requires the efficient evaluation of diverse information about the care recipient and the accurate determination of the appropriate level of care needed. However, current methods make it difficult to integrate sufficient expert opinions during the assessment, leading to errors and wasted time. Therefore, technologies are needed to improve the accuracy and efficiency of care assessment.

[0245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0246] In this invention, the server includes means for loading multiple knowledge base models tailored based on specialized domains, means for acquiring various information related to the person receiving care and processing that information, and means for each knowledge base model to exchange information with each other in natural language based on the pre-processed information. This enables a multifaceted evaluation of the person receiving care and the rapid and accurate proposal of care levels.

[0247] A "specialized field" refers to an area where specific knowledge or skills are studied intensively and deeply understood.

[0248] A "knowledge-based model" is a means of analyzing and evaluating information by utilizing knowledge within a specialized field.

[0249] "Information processing" is the process of converting acquired information into an appropriate format and organizing incomplete data to make it analyzable.

[0250] "Information exchange" is the act of communicating opinions and views between different knowledge-based models to deepen mutual understanding.

[0251] A "proposal" is a conclusion that integrates discussions across knowledge-based models and indicates the optimal course of action for the person receiving care.

[0252] "Care level" refers to a classification that indicates the degree of care needed according to the condition of the person receiving care.

[0253] The system of this invention utilizes a knowledge-based model with diverse expertise to streamline the long-term care certification process. This system is realized through the interaction of servers, terminals, and users.

[0254] The server utilizes machine learning frameworks such as TensorFlow and PyTorch to load multiple knowledge-based models from specialized fields such as medicine, healthcare, and nutrition. These models are pre-configured, allowing for the analysis of care recipient information from multiple perspectives.

[0255] The terminal receives information about the care recipient entered by the user via a secure communication protocol (e.g., HTTPS). This information includes past health history, living situation, and desired care needs assessment. The server then preprocesses the data and formats it into a format suitable for the knowledge base model. This includes information processing such as normalization of numerical data and encoding of categorical data.

[0256] The server inputs pre-processed data into each knowledge-based model and initiates a natural language exchange of ideas. Each model uses its expertise to analyze the data from its own perspective and generates the results in text format. The server then integrates the outputs of each model to create optimal recommendations.

[0257] This proposal is presented to the user via a terminal, and the user makes the final decision on the care level of the person receiving care based on the presented proposal. During this process, all data is securely stored on a server and used for future analysis as needed.

[0258] For example, if data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup" is entered into the terminal, the medical model will emphasize the importance of diabetes management, and the nutrition model will suggest an appropriate diet. Based on these analysis results, the server generates a conclusion of "Level 2 care needs" and reports it to the user through the terminal.

[0259] An example of a prompt for a generative AI model would be: "Given the data on the care recipient's health status (XX) and living environment (YY), propose the optimal care level." Implementing this system can improve both the accuracy and efficiency of care assessments.

[0260] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0261] Step 1:

[0262] At system startup, the server loads multiple knowledge-based models, each specializing in areas such as medicine, healthcare, and nutrition, using TensorFlow or PyTorch. Each model encompasses different areas of expertise, enabling a multifaceted approach. In this step, the knowledge-based models are loaded into the server's memory and prepared for processing.

[0263] Step 2:

[0264] The user inputs information about the person receiving care (e.g., health history, living situation) into the terminal using a dedicated interface. The terminal transmits the entered information to the server via HTTPS. The input data is raw and unprocessed, requiring preprocessing on the server side.

[0265] Step 3:

[0266] The server preprocesses the raw data received from the terminal. As part of the preprocessing, it performs tasks such as imputing missing data, normalizing numerical data, and one-hot encoding categorical data. At this stage, the input is the raw data from the terminal, and the output is formatted data that can be used by the model.

[0267] Step 4:

[0268] The server inputs the pre-processed data into each knowledge-based model and begins generating opinions using natural language. Each model analyzes the data based on its own expert perspective. For example, the medical model assesses the need for diabetes management. This process generates model-specific opinions as output, based on the formatted data as input.

[0269] Step 5:

[0270] The server aggregates the opinions generated from each model. This process utilizes natural language processing techniques to ensure consistency between opinions and compile a final proposal. The integrated proposal is then output in a format that can be directly presented to the user as guidance.

[0271] Step 6:

[0272] The terminal displays the final proposal received from the server to the user. Based on the presented information, the user makes the final decision regarding the care level of the person receiving care. At this stage, the proposal is provided in a format that is easy for the user to understand.

[0273] (Application Example 1)

[0274] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0275] Traditional care needs assessment processes have suffered from the problem of not fully utilizing specialized knowledge in specific fields, making it difficult to comprehensively evaluate the condition of those receiving care. Furthermore, while real-time monitoring of health information and rapid response in the event of an abnormality are required, there is a lack of effective means, leaving challenges in terms of safety and efficiency in the care environment.

[0276] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0277] In this invention, the server includes means for loading multiple knowledge-based models tailored based on specialized fields, means for collecting and pre-processing information related to the person being cared for, and means for monitoring health information in real time using an information display device. This enables a multifaceted evaluation of the health status of the person being cared for, and allows for rapid intervention, including detection of abnormalities.

[0278] A "knowledge-based model" is a data model that uses information based on specialized fields to analyze specific problems and generate proposals.

[0279] An "information display device" is a device that displays the health information of the person being cared for in real time and presents it to the user immediately.

[0280] "Real-time monitoring" is a process that involves continuously collecting data and analyzing it on the spot to immediately assess the situation.

[0281] "Anomaly detection" is a function that identifies data that exceeds the normal range and notifies the user that there is a problem.

[0282] A "safety system" is a general term for the networks and equipment used to manage and maintain safety within a facility.

[0283] The system of this invention is designed to effectively manage the health of care recipients in a care facility and improve safety. The server loads a plurality of knowledge-based models adjusted based on specialized fields. This enables analysis from different perspectives such as medicine, healthcare, and nutrition. In the server, an execution environment for the knowledge-based model is constructed using Python and TensorFlow.

[0284] Smart glasses are used as the terminal, and this device is used by care staff. The smart glasses have the function of monitoring the health information of the care recipient in real time and displaying the results. Also, an application is implemented using cross-platform development tools such as Flutter. Sensor data such as the heart rate and activity level obtained from the glasses is transmitted to the server, normalized, and then input into each model.

[0285] The care staff, who are the users, can grasp the changes in the health status in real time and make prompt responses based on the information as needed. When unusual movements or abnormalities are detected, the system immediately issues a warning and coordinates with the in-facility safety system to prompt a response. This can greatly improve the safety and response efficiency of the care facility.

[0286] As a specific example, when it is detected that the heart rate of a certain care recipient is higher than normal, the smart glasses display the evaluation result of the health status and specific countermeasures. In addition, based on the information from the security camera, the current location of the care recipient is quickly confirmed, and appropriate safety measures are taken.

[0287] An example of a prompt sentence is "A 70-year-old care recipient, with a heart rate 20% higher than normal. Please provide an evaluation of the health status and recommendations." Based on such prompts, the generative AI model immediately presents useful information to the care staff.

[0288] The flow of the specific process in Application Example 1 will be described using Figure 12.

[0289] Step 1:

[0290] The server loads knowledge-based models based on various specialties, including models with different expertise in medicine, health, and nutrition. The loaded models are ready to analyze care recipient data and provide evaluations from multiple perspectives.

[0291] Step 2:

[0292] The smart glasses on the device collect sensor information from the person being cared for in real time. This includes heart rate and activity level. This information is transmitted wirelessly to a server, where the data is normalized. The normalized data is then converted into a format that can be used for subsequent model analysis.

[0293] Step 3:

[0294] The server passes normalized data as input to each knowledge-based model. These models perform evaluations based on the received data and generate expert opinions. Through the exchange of opinions between the models, various elements that constitute the final health assessment are gathered.

[0295] Step 4:

[0296] The care staff, who are the users, receive health assessment results generated through smart glasses. This includes real-time changes in health status and warning messages. For example, if a higher-than-normal heart rate is detected, the glasses will display appropriate countermeasures.

[0297] Step 5:

[0298] The server immediately issues a warning when it detects suspicious activity or anomalies, and works in conjunction with the facility's security system to establish a rapid response system. This allows users to take quick and efficient countermeasures.

[0299] Step 6:

[0300] Users can quickly obtain the necessary information by instructing the generating AI model using prompt sentences. For example, by entering a prompt such as, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations," users can obtain appropriate instructions and suggestions tailored to their specific situation.

[0301] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0302] This invention is an advanced decision support system that combines an emotion engine and improves the care certification assessment process using multiple knowledge-based models tailored to specific fields of expertise. This system is built through server, terminal, and user interaction.

[0303] The server first loads knowledge-based models that are finely tuned for each specialized field. These models include functions specific to various fields such as medicine, healthcare, and nutrition, and are designed to provide comprehensive judgment. Furthermore, an emotion engine is integrated by the server to analyze the user's facial expressions and speech to estimate their psychological state.

[0304] The terminal receives data about the care recipient provided by the user. The information entered includes health status, living environment, and past medical records, and this information is sent to the server. The server receives this data and formats it into an analyzable form through natural language processing and data preprocessing.

[0305] Based on the pre-processed data and the sentiment engine's analysis, the server initiates a discussion among the various knowledge-based models. The models exchange opinions, and evaluation results are generated based on each model's expertise. Based on these results, the server forms the final proposal.

[0306] The user can view the proposals generated through the terminal. The proposals are optimized according to the user's emotional state and presented in an easy-to-accept manner. For example, if the emotional engine identifies anxiety about the level of care, proposals with additional explanations and advice are provided.

[0307] In this process, the server securely stores all processing steps in the database. This stored data is used for later analysis and system optimization, serving as a basis for obtaining new insights. These operations improve the efficiency and quality of the care certification review.

[0308] The following describes the processing flow.

[0309] Step 1:

[0310] The server loads multiple knowledge base models fine-tuned by specialty. Models in different specialties such as medicine, health, and nutrition are deployed and prepared in memory.

[0311] Step 2:

[0312] The terminal inputs information about the care recipient required for the care certification review from the user. This information includes health status, living environment, past medical history, etc. The terminal sends this data to the server.

[0313] Step 3:

[0314] The server preprocesses the input data. Using natural language processing technology, the raw data is structured and converted into a form that can be processed by the model. Also, relevant laws and cases are retrieved in RAG composition from the past database.

[0315] Step 4:

[0316] The server uses an emotion engine to analyze the user's facial expressions and voice data. It estimates the user's emotional state and incorporates that information into subsequent processes.

[0317] Step 5:

[0318] The server feeds pre-processed data into each knowledge-based model and initiates an exchange of ideas. Each model generates opinions based on its area of ​​expertise and engages in discussions with other models using natural language.

[0319] Step 6:

[0320] The server aggregates the results of the exchange of opinions and, taking into account the analysis results of the emotion engine, generates a final proposal. If necessary, the proposal is adjusted to match the user's emotions.

[0321] Step 7:

[0322] The terminal presents the generated suggestions to the user. The suggestions are displayed in a way that promotes reassurance and understanding, allowing the user to make a final decision on the level of care required.

[0323] Step 8:

[0324] The server securely stores all data. Input data, sentiment analysis, discussion processes, and final proposals are fully recorded and used for future system improvements and the development of new care methodologies.

[0325] (Example 2)

[0326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0327] In the process of assessing long-term care needs, there is a need to comprehensively evaluate information on the diverse health status and living environment of those receiving care, and to provide effective proposals that reflect individual conditions and circumstances. Furthermore, it is necessary to provide support that takes into account the user's psychological state. Conventional technologies have had problems such as one-sided evaluations and uniform proposals, which have prevented the quality of decision-making from being sufficiently improved.

[0328] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0329] In this invention, the server includes functional means for loading multiple knowledge base models tailored based on specialized fields; preprocessing functional means for collecting information related to the person receiving care using an interactive data acquisition device and converting the input data into an analyzable form; and functional means for each knowledge base model to exchange opinions, including emotion estimation, based on the preprocessed information, and generate evaluation results. This enables a comprehensive evaluation that is multifaceted and considerate of emotions, as well as the provision of individualized suggestions.

[0330] A "specialized field" refers to an area of ​​study or work where specialized knowledge and skills are required.

[0331] A "knowledge-based model" is a software model that collects information and data related to a specific field and uses that information to perform inferences and make decisions.

[0332] An "interactive data acquisition device" is a device that collects data through interaction with the user and effectively processes the user's input.

[0333] "Preprocessing" refers to a series of processes, such as data cleaning and format conversion, performed to transform raw data into a format that is easy to analyze.

[0334] "Emotion estimation" is a process for estimating a user's emotions by judging their psychological state from factors such as their facial expressions and tone of voice.

[0335] "Exchange of opinions" is a process in which multiple knowledge-based models share information from different perspectives to derive an overall judgment.

[0336] "Suggestions" refer to actionable guidelines and advice derived from evaluation results, intended to support users' decision-making.

[0337] "Functional means" refers to mechanical or electronic components or processes implemented to achieve a specific purpose.

[0338] The system of this invention is designed to provide advanced support for the long-term care certification review process, and functions through the mutual cooperation of three parties: a server, a terminal, and a user.

[0339] First, the server loads multiple knowledge-based models finely tuned for specialized fields such as medicine, healthcare, and nutrition. This enables field-specific decision-making. Furthermore, the server integrates an emotion engine to analyze the user's facial expressions and speech to estimate their psychological state. This emotion analysis function allows the server to provide more personalized support.

[0340] The terminal provides an interface for receiving information about the person receiving care from the user. This information includes health status, living environment, and past medical records. The terminal collects this information and sends it to the server.

[0341] The server preprocesses the received data. This preprocessing includes cleaning the data using natural language processing techniques and converting it into an analyzable format. Subsequently, based on the preprocessed data, each knowledge base model exchanges opinions with the others and generates evaluation results from their respective expert perspectives.

[0342] For example, when a care recipient's living environment changes, the server can use an emotion engine to analyze how anxious the user is about that change. Based on this, the server proposes a care plan designed to provide greater reassurance.

[0343] An example of a prompt message to be input into the generating AI model is, "Use the expert knowledge model to provide comprehensive suggestions regarding the care recipient's living environment." This allows the system to accurately provide the support the user needs.

[0344] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0345] Step 1:

[0346] The server loads multiple knowledge-based models, finely tuned based on specific fields. These models are specialized for various areas such as medicine, healthcare, and nutrition, and form the basis for complex data analysis and decision-making. The input requires preliminary data relevant to each field, which is loaded by the server. The output is the setup of each model for use.

[0347] Step 2:

[0348] The terminal collects information about the care recipient from the user. This information includes health status, living environment, and past medical history. The user inputs this information through the terminal's interface. The input data is sent from the terminal to the server and used as the basis for subsequent analysis. The output is formatted data sent to the server.

[0349] Step 3:

[0350] The server performs preprocessing on the data received from the terminal. This preprocessing includes data cleaning and format conversion using natural language processing techniques. Raw data is provided as input and is formatted into a format that is easy to analyze. The output is the formatted data prepared for further analysis.

[0351] Step 4:

[0352] The server uses an emotion engine to estimate the user's psychological state based on pre-processed data. This includes user input, tone of voice, and, in some cases, facial expression analysis. If the emotion engine identifies anxiety or worry in the user, this is used as data within the server. The output is data related to the user's psychological state.

[0353] Step 5:

[0354] The server hands over pre-processed data and sentiment analysis results to the knowledge-based models, initiating opinion exchange and evaluation results generation by the models. Inputs include formatted data and emotional state information, while outputs are evaluation results from each model. The knowledge-based models analyze the data from different expert perspectives using varying approaches, leading to a more comprehensive discussion.

[0355] Step 6:

[0356] The server integrates evaluation results from the models and generates optimized suggestions to present to the user. These suggestions are adjusted to take into account individual circumstances and the user's psychological state. The input requires evaluation results from each model, and the output generates specific suggestions.

[0357] Step 7:

[0358] Users can review the suggestions generated through their device. The suggestions are displayed in an easy-to-understand format, allowing users to easily make decisions based on them. Users can request additional information or provide feedback on the suggestions via their device as needed.

[0359] Step 8:

[0360] The server securely stores all data and evaluation results generated throughout the entire process in a database. This stored data is used for future analysis and system optimization. The input requires all generated data, which is stored in an organized manner. The output is historical data stored in a database, corresponding to anticipated future uses.

[0361] (Application Example 2)

[0362] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0363] Modern technological advancements have created a need for support systems based on specialized knowledge in various fields. However, many existing systems provide uniform responses without considering the user's psychological state or emotions, making them unable to provide flexible support tailored to individual situations. Therefore, there is a need to analyze the user's psychological state and optimize suggestions accordingly. In particular, considering the user's sense of security is crucial in areas such as security, but existing systems fail to address this aspect. Consequently, there is a need to develop new systems that can provide decision-making support tailored to the user's psychological state.

[0364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0365] In this invention, the server includes means for loading multiple knowledge data models tailored based on specialized domains, means for collecting and preprocessing information related to the person being assisted, and means for analyzing the psychological state and optimizing suggestions. This enables flexible decision-making support that takes into account the user's psychological state.

[0366] "Multiple knowledge data models tailored to specific areas of expertise" are data models that possess specialized knowledge in individual fields such as medicine, health, and nutrition, and are tailored to each field to support optimal decision-making.

[0367] "Information related to the person receiving support" refers to a collection of specific data about the individual in question that is necessary for the system to provide decision-making support, such as health status, living environment, and past medical records.

[0368] "A means of analyzing psychological state and optimizing suggestions" refers to a technology that uses an emotion engine to analyze the user's facial expressions and statements, optimizes support suggestions based on the results, and enables more personalized responses.

[0369] A "means for exchanging opinions" is a mechanism in which multiple knowledge data models exchange information with each other, discuss from their respective expert perspectives, and form a final judgment.

[0370] "Means for securely recording data and preparing for future use" refers to a recording system that securely stores the processing and proposals performed within the system, making them available for later analysis and system improvement.

[0371] The system for implementing this invention functions through the interaction of a server, a terminal, and a user.

[0372] The server first loads multiple knowledge data models, each tailored to specific professional fields such as medicine and healthcare. These models incorporate different expert perspectives to support comprehensive decision-making. Furthermore, an emotion engine is used to analyze the user's facial expressions and statements, estimating their psychological state. Optimization of suggestions based on emotional state is also ensured at this stage.

[0373] The device collects various pieces of information related to the person receiving assistance from the user. This includes important data such as health status and past medical records, and this information is sent to a server to prepare for further processing and analysis.

[0374] Users can receive suggestions from the server via their device and make decisions based on them. A key feature is that the suggestions are tailored to the user's psychological state and presented in a way that is easily accepted.

[0375] The hardware used here includes the smartphone's camera and microphone, which are used to analyze the user's facial expressions and voice data. The software utilizes generative AI models such as TensorFlow to enable sentiment analysis. Appropriate database technologies such as SQLite are used for data storage.

[0376] For example, if a user expresses security concerns in a particular location, the emotion engine analyzes this and provides security advice for connecting to public Wi-Fi. This allows the user to continue communicating with confidence. A prompt such as, "Think about how a security app can help in a situation where you are feeling anxious," is used.

[0377] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0378] Step 1:

[0379] The terminal receives information about the person receiving assistance from the user as input. Specifically, it collects data such as health status, past medical records, and living environment through the smartphone interface and sends this data to the server. The input here is formatted as text information.

[0380] Step 2:

[0381] The server preprocesses the received information. It analyzes the input data using natural language processing techniques and structures the individual pieces of information. This standardizes the format of each data point, enabling rapid and efficient analysis using knowledge data models.

[0382] Step 3:

[0383] The server simultaneously loads multiple knowledge data models, each tailored to a specific domain. Generative AI models are then used to send pre-processed data to each model, generating opinions from their respective expert perspectives. The output is then sent back, with the models exchanging opinions.

[0384] Step 4:

[0385] The server uses an emotion engine to analyze the user's on-screen statements and psychological nuances collected by the device. The input consists of the user's facial expressions and voice, which are used to estimate their psychological state. The resulting output is a profile of the user's psychological state.

[0386] Step 5:

[0387] The server integrates the output from the knowledge data model and the emotion engine to generate the final recommendation. This recommendation is optimized to take the user's psychological state into account. For example, if the user is showing anxiety, the output will include specific recommendations and precautions to address that anxiety.

[0388] Step 6:

[0389] The user receives the final proposal generated through their device and makes a decision based on it. The final proposal is displayed as text and visual data, presented in a clear and easy-to-understand format for the user.

[0390] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0391] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0392] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0393] [Third Embodiment]

[0394] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0395] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0396] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0397] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0398] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0400] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0401] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0402] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0403] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0404] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0405] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0406] This invention provides a system that streamlines the care certification review process by utilizing a knowledge-based model with diverse expertise. This system consists of the interaction of a server, terminals, and users.

[0407] The server loads multiple pre-prepared knowledge-based models. Each model specializes in a different field, such as medicine, healthcare, or nutrition, allowing for analysis from different perspectives. This enables the multifaceted analysis necessary for evaluating care recipients.

[0408] The terminal receives data on the care recipient submitted by the user. For example, it can input past medical history, living environment information, and desired certification details. The received data is preprocessed by the server and organized into a format suitable for input into the model.

[0409] The server initiates discussions among the knowledge-based models based on pre-processed data. The exchange of opinions between models takes place in natural language, with each model contributing insights from its area of ​​expertise. The generated opinions are aggregated by the server and compiled into a final proposal.

[0410] The proposed results are presented to the user via their device. Based on these proposals, the user determines the final care level of the person receiving care. The data generated during this process is securely stored on the server and used for future analysis as needed.

[0411] To give a specific example, if the terminal is given input data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup," a medical-focused model will emphasize the importance of diabetes management, and a nutrition model will suggest appropriate meals. Integrating these opinions, the server will propose a conclusion of "Level 2 care needs." This proposal is then reviewed by the user and used for the final determination.

[0412] In this way, by ensuring that each step is executed smoothly, it becomes possible to carry out an efficient and high-quality long-term care certification assessment.

[0413] The following describes the processing flow.

[0414] Step 1:

[0415] The server loads the knowledge-based model. It deploys the domain-specific models into memory along with the necessary computing resources.

[0416] Step 2:

[0417] The terminal receives input data from the user. Information such as the care recipient's health status, past medical records, and living environment is entered. This data becomes basic information necessary for subsequent processes.

[0418] Step 3:

[0419] The server preprocesses the received data. Using natural language processing, it converts unstructured data into a model input format and organizes it as structured information. It also retrieves necessary information from relevant laws and past case databases using a RAG (Random Aggregation) configuration.

[0420] Step 4:

[0421] The server initiates a multi-agent simulation. Pre-processed data is provided to each specialized model, and they engage in natural language discussions. Each model proceeds with the discussion based on its knowledge in its respective field.

[0422] Step 5:

[0423] The server aggregates the discussion content and generates the final proposal. Insights from each model are integrated to form conclusions from multiple perspectives. During this process, a proposal is created that takes into account the necessary weightings and priorities.

[0424] Step 6:

[0425] The terminal presents the generated suggestions to the user. The user reviews the suggestions and makes a final decision on the care needs assessment level. Additional input and modifications can be made as needed.

[0426] Step 7:

[0427] The server securely stores data. All information, including input data, discussion processes, and final proposals, is properly stored and preserved in a format that can be used for future analysis and improvement.

[0428] (Example 1)

[0429] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0430] The process of assessing the level of care required requires the efficient evaluation of diverse information about the care recipient and the accurate determination of the appropriate level of care needed. However, current methods make it difficult to integrate sufficient expert opinions during the assessment, leading to errors and wasted time. Therefore, technologies are needed to improve the accuracy and efficiency of care assessment.

[0431] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0432] In this invention, the server includes means for loading multiple knowledge base models tailored based on specialized domains, means for acquiring various information related to the person receiving care and processing that information, and means for each knowledge base model to exchange information with each other in natural language based on the pre-processed information. This enables a multifaceted evaluation of the person receiving care and the rapid and accurate proposal of care levels.

[0433] A "specialized field" refers to an area where specific knowledge or skills are studied intensively and deeply understood.

[0434] A "knowledge-based model" is a means of analyzing and evaluating information by utilizing knowledge within a specialized field.

[0435] "Information processing" is the process of converting acquired information into an appropriate format and organizing incomplete data to make it analyzable.

[0436] "Information exchange" is the act of communicating opinions and views between different knowledge-based models to deepen mutual understanding.

[0437] A "proposal" is a conclusion that integrates discussions across knowledge-based models and indicates the optimal course of action for the person receiving care.

[0438] "Care level" refers to a classification that indicates the degree of care needed according to the condition of the person receiving care.

[0439] The system of this invention utilizes a knowledge-based model with diverse expertise to streamline the long-term care certification process. This system is realized through the interaction of servers, terminals, and users.

[0440] The server utilizes machine learning frameworks such as TensorFlow and PyTorch to load multiple knowledge-based models from specialized fields such as medicine, healthcare, and nutrition. These models are pre-configured, allowing for the analysis of care recipient information from multiple perspectives.

[0441] The terminal receives information about the care recipient entered by the user via a secure communication protocol (e.g., HTTPS). This information includes past health history, living situation, and desired care needs assessment. The server then preprocesses the data and formats it into a format suitable for the knowledge base model. This includes information processing such as normalization of numerical data and encoding of categorical data.

[0442] The server inputs pre-processed data into each knowledge-based model and initiates a natural language exchange of ideas. Each model uses its expertise to analyze the data from its own perspective and generates the results in text format. The server then integrates the outputs of each model to create optimal recommendations.

[0443] This proposal is presented to the user via a terminal, and the user makes the final decision on the care level of the person receiving care based on the presented proposal. During this process, all data is securely stored on a server and used for future analysis as needed.

[0444] For example, if data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup" is entered into the terminal, the medical model will emphasize the importance of diabetes management, and the nutrition model will suggest an appropriate diet. Based on these analysis results, the server generates a conclusion of "Level 2 care needs" and reports it to the user through the terminal.

[0445] An example of a prompt for a generative AI model would be: "Given the data on the care recipient's health status (XX) and living environment (YY), propose the optimal care level." Implementing this system can improve both the accuracy and efficiency of care assessments.

[0446] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0447] Step 1:

[0448] At system startup, the server loads multiple knowledge-based models, each specializing in areas such as medicine, healthcare, and nutrition, using TensorFlow or PyTorch. Each model encompasses different areas of expertise, enabling a multifaceted approach. In this step, the knowledge-based models are loaded into the server's memory and prepared for processing.

[0449] Step 2:

[0450] The user inputs information about the person receiving care (e.g., health history, living situation) into the terminal using a dedicated interface. The terminal transmits the entered information to the server via HTTPS. The input data is raw and unprocessed, requiring preprocessing on the server side.

[0451] Step 3:

[0452] The server preprocesses the raw data received from the terminal. As part of the preprocessing, it performs tasks such as imputing missing data, normalizing numerical data, and one-hot encoding categorical data. At this stage, the input is the raw data from the terminal, and the output is formatted data that can be used by the model.

[0453] Step 4:

[0454] The server inputs the pre-processed data into each knowledge-based model and begins generating opinions using natural language. Each model analyzes the data based on its own expert perspective. For example, the medical model assesses the need for diabetes management. This process generates model-specific opinions as output, based on the formatted data as input.

[0455] Step 5:

[0456] The server aggregates the opinions generated from each model. This process utilizes natural language processing techniques to ensure consistency between opinions and compile a final proposal. The integrated proposal is then output in a format that can be directly presented to the user as guidance.

[0457] Step 6:

[0458] The terminal displays the final proposal received from the server to the user. Based on the presented information, the user makes the final decision regarding the care level of the person receiving care. At this stage, the proposal is provided in a format that is easy for the user to understand.

[0459] (Application Example 1)

[0460] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0461] Traditional care needs assessment processes have suffered from the problem of not fully utilizing specialized knowledge in specific fields, making it difficult to comprehensively evaluate the condition of those receiving care. Furthermore, while real-time monitoring of health information and rapid response in the event of an abnormality are required, there is a lack of effective means, leaving challenges in terms of safety and efficiency in the care environment.

[0462] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0463] In this invention, the server includes means for loading multiple knowledge-based models tailored based on specialized fields, means for collecting and pre-processing information related to the person being cared for, and means for monitoring health information in real time using an information display device. This enables a multifaceted evaluation of the health status of the person being cared for, and allows for rapid intervention, including detection of abnormalities.

[0464] A "knowledge-based model" is a data model that uses information based on specialized fields to analyze specific problems and generate proposals.

[0465] An "information display device" is a device that displays the health information of the person being cared for in real time and presents it to the user immediately.

[0466] "Real-time monitoring" is a process that involves continuously collecting data and analyzing it on the spot to immediately assess the situation.

[0467] "Anomaly detection" is a function that identifies data that exceeds the normal range and notifies the user that there is a problem.

[0468] A "safety system" is a general term for the networks and equipment used to manage and maintain safety within a facility.

[0469] The system of this invention is designed to effectively manage the health of those receiving care and improve safety in nursing care facilities. The server loads multiple knowledge-based models tailored to specific fields of expertise. This enables analysis from different perspectives, such as medical, health, and nutrition. The server uses Python and TensorFlow to build the execution environment for the knowledge-based models.

[0470] The terminal uses smart glasses, which are used by care staff. The smart glasses have the function of monitoring the health information of the person being cared for in real time and displaying the results. The application is implemented using cross-platform development tools such as Flutter. Sensor data such as heart rate and activity level obtained from the glasses is sent to a server, normalized, and then input into each model.

[0471] The care staff, as users of the system, can monitor changes in health conditions in real time and take prompt action based on the information as needed. If suspicious movements or abnormalities are detected, the system immediately issues a warning and prompts a response in conjunction with the facility's safety system. This significantly improves the safety and response efficiency of care facilities.

[0472] For example, if a care recipient's heart rate is detected to be higher than normal, the smart glasses will display the health assessment results and specific countermeasures. In addition, based on information from security cameras, the care recipient's current location can be quickly confirmed, and appropriate safety measures can be taken.

[0473] An example of a prompt would be, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations." Based on such prompts, the generative AI model immediately presents useful information to care staff.

[0474] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0475] Step 1:

[0476] The server loads knowledge-based models based on various specialties, including models with different expertise in medicine, health, and nutrition. The loaded models are ready to analyze care recipient data and provide evaluations from multiple perspectives.

[0477] Step 2:

[0478] The smart glasses on the device collect sensor information from the person being cared for in real time. This includes heart rate and activity level. This information is transmitted wirelessly to a server, where the data is normalized. The normalized data is then converted into a format that can be used for subsequent model analysis.

[0479] Step 3:

[0480] The server passes normalized data as input to each knowledge-based model. These models perform evaluations based on the received data and generate expert opinions. Through the exchange of opinions between the models, various elements that constitute the final health assessment are gathered.

[0481] Step 4:

[0482] The care staff, who are the users, receive health assessment results generated through smart glasses. This includes real-time changes in health status and warning messages. For example, if a higher-than-normal heart rate is detected, the glasses will display appropriate countermeasures.

[0483] Step 5:

[0484] The server immediately issues a warning when it detects suspicious activity or anomalies, and works in conjunction with the facility's security system to establish a rapid response system. This allows users to take quick and efficient countermeasures.

[0485] Step 6:

[0486] Users can quickly obtain the necessary information by instructing the generating AI model using prompt sentences. For example, by entering a prompt such as, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations," users can obtain appropriate instructions and suggestions tailored to their specific situation.

[0487] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0488] This invention is an advanced decision support system that combines an emotion engine and improves the care certification assessment process using multiple knowledge-based models tailored to specific fields of expertise. This system is built through server, terminal, and user interaction.

[0489] The server first loads knowledge-based models that are finely tuned for each specialized field. These models include functions specific to various fields such as medicine, healthcare, and nutrition, and are designed to provide comprehensive judgment. Furthermore, an emotion engine is integrated by the server to analyze the user's facial expressions and speech to estimate their psychological state.

[0490] The terminal receives data about the care recipient provided by the user. The information entered includes health status, living environment, and past medical records, and this information is sent to the server. The server receives this data and formats it into an analyzable form through natural language processing and data preprocessing.

[0491] Based on the pre-processed data and the sentiment engine's analysis, the server initiates a discussion among the various knowledge-based models. The models exchange opinions, and evaluation results are generated based on each model's expertise. Based on these results, the server forms the final proposal.

[0492] Users can review suggestions generated through their device. These suggestions are optimized to the user's emotional state and presented in an easily acceptable manner. For example, if the emotional engine identifies anxiety about the level of care needed, suggestions with additional explanations and advice will be provided.

[0493] In this process, the server securely saves all processing steps to a database. This saved data is used for subsequent analysis and system optimization, forming the basis for gaining new insights. These actions improve the efficiency and quality of the care needs assessment process.

[0494] The following describes the processing flow.

[0495] Step 1:

[0496] The server loads multiple knowledge base models, each finely tuned for a specific field. Models for different fields such as medicine, healthcare, and nutrition are deployed and prepared in memory.

[0497] Step 2:

[0498] The terminal receives information about the person receiving care, necessary for the care needs assessment, from the user. This information includes health status, living environment, and past medical history. The terminal then sends this data to the server.

[0499] Step 3:

[0500] The server preprocesses the input data. Using natural language processing techniques, it structures the raw data and converts it into a format that can be processed by the model. It also retrieves relevant laws and cases from historical databases using a RAG (Random Aggregation) configuration.

[0501] Step 4:

[0502] The server uses an emotion engine to analyze the user's facial expressions and voice data. It estimates the user's emotional state and incorporates that information into subsequent processes.

[0503] Step 5:

[0504] The server feeds pre-processed data into each knowledge-based model and initiates an exchange of ideas. Each model generates opinions based on its area of ​​expertise and engages in discussions with other models using natural language.

[0505] Step 6:

[0506] The server aggregates the results of the exchange of opinions and, taking into account the analysis results of the emotion engine, generates a final proposal. If necessary, the proposal is adjusted to match the user's emotions.

[0507] Step 7:

[0508] The terminal presents the generated suggestions to the user. The suggestions are displayed in a way that promotes reassurance and understanding, allowing the user to make a final decision on the level of care required.

[0509] Step 8:

[0510] The server securely stores all data. Input data, sentiment analysis, discussion processes, and final proposals are fully recorded and used for future system improvements and the development of new care methodologies.

[0511] (Example 2)

[0512] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0513] In the process of assessing long-term care needs, there is a need to comprehensively evaluate information on the diverse health status and living environment of those receiving care, and to provide effective proposals that reflect individual conditions and circumstances. Furthermore, it is necessary to provide support that takes into account the user's psychological state. Conventional technologies have had problems such as one-sided evaluations and uniform proposals, which have prevented the quality of decision-making from being sufficiently improved.

[0514] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0515] In this invention, the server includes functional means for loading multiple knowledge base models tailored based on specialized fields; preprocessing functional means for collecting information related to the person receiving care using an interactive data acquisition device and converting the input data into an analyzable form; and functional means for each knowledge base model to exchange opinions, including emotion estimation, based on the preprocessed information, and generate evaluation results. This enables a comprehensive evaluation that is multifaceted and considerate of emotions, as well as the provision of individualized suggestions.

[0516] A "specialized field" refers to an area of ​​study or work where specialized knowledge and skills are required.

[0517] A "knowledge-based model" is a software model that collects information and data related to a specific field and uses that information to perform inferences and make decisions.

[0518] An "interactive data acquisition device" is a device that collects data through interaction with the user and effectively processes the user's input.

[0519] "Preprocessing" refers to a series of processes, such as data cleaning and format conversion, performed to transform raw data into a format that is easy to analyze.

[0520] "Emotion estimation" is a process for estimating a user's emotions by judging their psychological state from factors such as their facial expressions and tone of voice.

[0521] "Exchange of opinions" is a process in which multiple knowledge-based models share information from different perspectives to derive an overall judgment.

[0522] "Suggestions" refer to actionable guidelines and advice derived from evaluation results, intended to support users' decision-making.

[0523] "Functional means" refers to mechanical or electronic components or processes implemented to achieve a specific purpose.

[0524] The system of this invention is designed to provide advanced support for the long-term care certification review process, and functions through the mutual cooperation of three parties: a server, a terminal, and a user.

[0525] First, the server loads multiple knowledge-based models finely tuned for specialized fields such as medicine, healthcare, and nutrition. This enables field-specific decision-making. Furthermore, the server integrates an emotion engine to analyze the user's facial expressions and speech to estimate their psychological state. This emotion analysis function allows the server to provide more personalized support.

[0526] The terminal provides an interface for receiving information about the person receiving care from the user. This information includes health status, living environment, and past medical records. The terminal collects this information and sends it to the server.

[0527] The server preprocesses the received data. This preprocessing includes cleaning the data using natural language processing techniques and converting it into an analyzable format. Subsequently, based on the preprocessed data, each knowledge base model exchanges opinions with the others and generates evaluation results from their respective expert perspectives.

[0528] For example, when a care recipient's living environment changes, the server can use an emotion engine to analyze how anxious the user is about that change. Based on this, the server proposes a care plan designed to provide greater reassurance.

[0529] An example of a prompt message to be input into the generating AI model is, "Use the expert knowledge model to provide comprehensive suggestions regarding the care recipient's living environment." This allows the system to accurately provide the support the user needs.

[0530] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0531] Step 1:

[0532] The server loads multiple knowledge-based models, finely tuned based on specific fields. These models are specialized for various areas such as medicine, healthcare, and nutrition, and form the basis for complex data analysis and decision-making. The input requires preliminary data relevant to each field, which is loaded by the server. The output is the setup of each model for use.

[0533] Step 2:

[0534] The terminal collects information about the care recipient from the user. This information includes health status, living environment, and past medical history. The user inputs this information through the terminal's interface. The input data is sent from the terminal to the server and used as the basis for subsequent analysis. The output is formatted data sent to the server.

[0535] Step 3:

[0536] The server performs preprocessing on the data received from the terminal. This preprocessing includes data cleaning and format conversion using natural language processing techniques. Raw data is provided as input and is formatted into a format that is easy to analyze. The output is the formatted data prepared for further analysis.

[0537] Step 4:

[0538] The server uses an emotion engine to estimate the user's psychological state based on pre-processed data. This includes user input, tone of voice, and, in some cases, facial expression analysis. If the emotion engine identifies anxiety or worry in the user, this is used as data within the server. The output is data related to the user's psychological state.

[0539] Step 5:

[0540] The server hands over pre-processed data and sentiment analysis results to the knowledge-based models, initiating opinion exchange and evaluation results generation by the models. Inputs include formatted data and emotional state information, while outputs are evaluation results from each model. The knowledge-based models analyze the data from different expert perspectives using varying approaches, leading to a more comprehensive discussion.

[0541] Step 6:

[0542] The server integrates evaluation results from the models and generates optimized suggestions to present to the user. These suggestions are adjusted to take into account individual circumstances and the user's psychological state. The input requires evaluation results from each model, and the output generates specific suggestions.

[0543] Step 7:

[0544] Users can review the suggestions generated through their device. The suggestions are displayed in an easy-to-understand format, allowing users to easily make decisions based on them. Users can request additional information or provide feedback on the suggestions via their device as needed.

[0545] Step 8:

[0546] The server securely stores all data and evaluation results generated throughout the entire process in a database. This stored data is used for future analysis and system optimization. The input requires all generated data, which is stored in an organized manner. The output is historical data stored in a database, corresponding to anticipated future uses.

[0547] (Application Example 2)

[0548] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0549] Modern technological advancements have created a need for support systems based on specialized knowledge in various fields. However, many existing systems provide uniform responses without considering the user's psychological state or emotions, making them unable to provide flexible support tailored to individual situations. Therefore, there is a need to analyze the user's psychological state and optimize suggestions accordingly. In particular, considering the user's sense of security is crucial in areas such as security, but existing systems fail to address this aspect. Consequently, there is a need to develop new systems that can provide decision-making support tailored to the user's psychological state.

[0550] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0551] In this invention, the server includes means for loading multiple knowledge data models tailored based on specialized domains, means for collecting and preprocessing information related to the person being assisted, and means for analyzing the psychological state and optimizing suggestions. This enables flexible decision-making support that takes into account the user's psychological state.

[0552] "Multiple knowledge data models tailored to specific areas of expertise" are data models that possess specialized knowledge in individual fields such as medicine, health, and nutrition, and are tailored to each field to support optimal decision-making.

[0553] "Information related to the person receiving support" refers to a collection of specific data about the individual in question that is necessary for the system to provide decision-making support, such as health status, living environment, and past medical records.

[0554] "A means of analyzing psychological state and optimizing suggestions" refers to a technology that uses an emotion engine to analyze the user's facial expressions and statements, optimizes support suggestions based on the results, and enables more personalized responses.

[0555] A "means for exchanging opinions" is a mechanism in which multiple knowledge data models exchange information with each other, discuss from their respective expert perspectives, and form a final judgment.

[0556] "Means for securely recording data and preparing for future use" refers to a recording system that securely stores the processing and proposals performed within the system, making them available for later analysis and system improvement.

[0557] The system for implementing this invention functions through the interaction of a server, a terminal, and a user.

[0558] The server first loads multiple knowledge data models, each tailored to specific professional fields such as medicine and healthcare. These models incorporate different expert perspectives to support comprehensive decision-making. Furthermore, an emotion engine is used to analyze the user's facial expressions and statements, estimating their psychological state. Optimization of suggestions based on emotional state is also ensured at this stage.

[0559] The device collects various pieces of information related to the person receiving assistance from the user. This includes important data such as health status and past medical records, and this information is sent to a server to prepare for further processing and analysis.

[0560] Users can receive suggestions from the server via their device and make decisions based on them. A key feature is that the suggestions are tailored to the user's psychological state and presented in a way that is easily accepted.

[0561] The hardware used here includes the smartphone's camera and microphone, which are used to analyze the user's facial expressions and voice data. The software utilizes generative AI models such as TensorFlow to enable sentiment analysis. Appropriate database technologies such as SQLite are used for data storage.

[0562] For example, if a user expresses security concerns in a particular location, the emotion engine analyzes this and provides security advice for connecting to public Wi-Fi. This allows the user to continue communicating with confidence. A prompt such as, "Think about how a security app can help in a situation where you are feeling anxious," is used.

[0563] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0564] Step 1:

[0565] The terminal receives information about the person receiving assistance from the user as input. Specifically, it collects data such as health status, past medical records, and living environment through the smartphone interface and sends this data to the server. The input here is formatted as text information.

[0566] Step 2:

[0567] The server preprocesses the received information. It analyzes the input data using natural language processing techniques and structures the individual pieces of information. This standardizes the format of each data point, enabling rapid and efficient analysis using knowledge data models.

[0568] Step 3:

[0569] The server simultaneously loads multiple knowledge data models, each tailored to a specific domain. Generative AI models are then used to send pre-processed data to each model, generating opinions from their respective expert perspectives. The output is then sent back, with the models exchanging opinions.

[0570] Step 4:

[0571] The server uses an emotion engine to analyze the user's on-screen statements and psychological nuances collected by the device. The input consists of the user's facial expressions and voice, which are used to estimate their psychological state. The resulting output is a profile of the user's psychological state.

[0572] Step 5:

[0573] The server integrates the output from the knowledge data model and the emotion engine to generate the final recommendation. This recommendation is optimized to take the user's psychological state into account. For example, if the user is showing anxiety, the output will include specific recommendations and precautions to address that anxiety.

[0574] Step 6:

[0575] The user receives the final proposal generated through their device and makes a decision based on it. The final proposal is displayed as text and visual data, presented in a clear and easy-to-understand format for the user.

[0576] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0577] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0578] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0579] [Fourth Embodiment]

[0580] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0581] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0582] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0583] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0584] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0585] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0586] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0587] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0588] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0589] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0590] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0591] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0592] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0593] This invention provides a system that streamlines the care certification review process by utilizing a knowledge-based model with diverse expertise. This system consists of the interaction of a server, terminals, and users.

[0594] The server loads multiple pre-prepared knowledge-based models. Each model specializes in a different field, such as medicine, healthcare, or nutrition, allowing for analysis from different perspectives. This enables the multifaceted analysis necessary for evaluating care recipients.

[0595] The terminal receives data on the care recipient submitted by the user. For example, it can input past medical history, living environment information, and desired certification details. The received data is preprocessed by the server and organized into a format suitable for input into the model.

[0596] The server initiates discussions among the knowledge-based models based on pre-processed data. The exchange of opinions between models takes place in natural language, with each model contributing insights from its area of ​​expertise. The generated opinions are aggregated by the server and compiled into a final proposal.

[0597] The proposed results are presented to the user via their device. Based on these proposals, the user determines the final care level of the person receiving care. The data generated during this process is securely stored on the server and used for future analysis as needed.

[0598] To give a specific example, if the terminal is given input data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup," a medical-focused model will emphasize the importance of diabetes management, and a nutrition model will suggest appropriate meals. Integrating these opinions, the server will propose a conclusion of "Level 2 care needs." This proposal is then reviewed by the user and used for the final determination.

[0599] In this way, by ensuring that each step is executed smoothly, it becomes possible to carry out an efficient and high-quality long-term care certification assessment.

[0600] The following describes the processing flow.

[0601] Step 1:

[0602] The server loads the knowledge-based model. It deploys the domain-specific models into memory along with the necessary computing resources.

[0603] Step 2:

[0604] The terminal receives input data from the user. Information such as the care recipient's health status, past medical records, and living environment is entered. This data becomes basic information necessary for subsequent processes.

[0605] Step 3:

[0606] The server preprocesses the received data. Using natural language processing, it converts unstructured data into a model input format and organizes it as structured information. It also retrieves necessary information from relevant laws and past case databases using a RAG (Random Aggregation) configuration.

[0607] Step 4:

[0608] The server initiates a multi-agent simulation. Pre-processed data is provided to each specialized model, and they engage in natural language discussions. Each model proceeds with the discussion based on its knowledge in its respective field.

[0609] Step 5:

[0610] The server aggregates the discussion content and generates the final proposal. Insights from each model are integrated to form conclusions from multiple perspectives. During this process, a proposal is created that takes into account the necessary weightings and priorities.

[0611] Step 6:

[0612] The terminal presents the generated suggestions to the user. The user reviews the suggestions and makes a final decision on the care needs assessment level. Additional input and modifications can be made as needed.

[0613] Step 7:

[0614] The server securely stores data. All information, including input data, discussion processes, and final proposals, is properly stored and preserved in a format that can be used for future analysis and improvement.

[0615] (Example 1)

[0616] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] The process of assessing the level of care required requires the efficient evaluation of diverse information about the care recipient and the accurate determination of the appropriate level of care needed. However, current methods make it difficult to integrate sufficient expert opinions during the assessment, leading to errors and wasted time. Therefore, technologies are needed to improve the accuracy and efficiency of care assessment.

[0618] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0619] In this invention, the server includes means for loading multiple knowledge base models tailored based on specialized domains, means for acquiring various information related to the person receiving care and processing that information, and means for each knowledge base model to exchange information with each other in natural language based on the pre-processed information. This enables a multifaceted evaluation of the person receiving care and the rapid and accurate proposal of care levels.

[0620] A "specialized field" refers to an area where specific knowledge or skills are studied intensively and deeply understood.

[0621] A "knowledge-based model" is a means of analyzing and evaluating information by utilizing knowledge within a specialized field.

[0622] "Information processing" is the process of converting acquired information into an appropriate format and organizing incomplete data to make it analyzable.

[0623] "Information exchange" is the act of communicating opinions and views between different knowledge-based models to deepen mutual understanding.

[0624] A "proposal" is a conclusion that integrates discussions across knowledge-based models and indicates the optimal course of action for the person receiving care.

[0625] "Care level" refers to a classification that indicates the degree of care needed according to the condition of the person receiving care.

[0626] The system of this invention utilizes a knowledge-based model with diverse expertise to streamline the long-term care certification process. This system is realized through the interaction of servers, terminals, and users.

[0627] The server utilizes machine learning frameworks such as TensorFlow and PyTorch to load multiple knowledge-based models from specialized fields such as medicine, healthcare, and nutrition. These models are pre-configured, allowing for the analysis of care recipient information from multiple perspectives.

[0628] The terminal receives information about the care recipient entered by the user via a secure communication protocol (e.g., HTTPS). This information includes past health history, living situation, and desired care needs assessment. The server then preprocesses the data and formats it into a format suitable for the knowledge base model. This includes information processing such as normalization of numerical data and encoding of categorical data.

[0629] The server inputs pre-processed data into each knowledge-based model and initiates a natural language exchange of ideas. Each model uses its expertise to analyze the data from its own perspective and generates the results in text format. The server then integrates the outputs of each model to create optimal recommendations.

[0630] This proposal is presented to the user via a terminal, and the user makes the final decision on the care level of the person receiving care based on the presented proposal. During this process, all data is securely stored on a server and used for future analysis as needed.

[0631] For example, if data such as "70-year-old male, history of diabetes, weight loss in the most recent health checkup" is entered into the terminal, the medical model will emphasize the importance of diabetes management, and the nutrition model will suggest an appropriate diet. Based on these analysis results, the server generates a conclusion of "Level 2 care needs" and reports it to the user through the terminal.

[0632] An example of a prompt for a generative AI model would be: "Given the data on the care recipient's health status (XX) and living environment (YY), propose the optimal care level." Implementing this system can improve both the accuracy and efficiency of care assessments.

[0633] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0634] Step 1:

[0635] At system startup, the server loads multiple knowledge-based models, each specializing in areas such as medicine, healthcare, and nutrition, using TensorFlow or PyTorch. Each model encompasses different areas of expertise, enabling a multifaceted approach. In this step, the knowledge-based models are loaded into the server's memory and prepared for processing.

[0636] Step 2:

[0637] The user inputs information about the person receiving care (e.g., health history, living situation) into the terminal using a dedicated interface. The terminal transmits the entered information to the server via HTTPS. The input data is raw and unprocessed, requiring preprocessing on the server side.

[0638] Step 3:

[0639] The server preprocesses the raw data received from the terminal. As part of the preprocessing, it performs tasks such as imputing missing data, normalizing numerical data, and one-hot encoding categorical data. At this stage, the input is the raw data from the terminal, and the output is formatted data that can be used by the model.

[0640] Step 4:

[0641] The server inputs the pre-processed data into each knowledge-based model and begins generating opinions using natural language. Each model analyzes the data based on its own expert perspective. For example, the medical model assesses the need for diabetes management. This process generates model-specific opinions as output, based on the formatted data as input.

[0642] Step 5:

[0643] The server aggregates the opinions generated from each model. This process utilizes natural language processing techniques to ensure consistency between opinions and compile a final proposal. The integrated proposal is then output in a format that can be directly presented to the user as guidance.

[0644] Step 6:

[0645] The terminal displays the final proposal received from the server to the user. Based on the presented information, the user makes the final decision regarding the care level of the person receiving care. At this stage, the proposal is provided in a format that is easy for the user to understand.

[0646] (Application Example 1)

[0647] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0648] Traditional care needs assessment processes have suffered from the problem of not fully utilizing specialized knowledge in specific fields, making it difficult to comprehensively evaluate the condition of those receiving care. Furthermore, while real-time monitoring of health information and rapid response in the event of an abnormality are required, there is a lack of effective means, leaving challenges in terms of safety and efficiency in the care environment.

[0649] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0650] In this invention, the server includes means for loading multiple knowledge-based models tailored based on specialized fields, means for collecting and pre-processing information related to the person being cared for, and means for monitoring health information in real time using an information display device. This enables a multifaceted evaluation of the health status of the person being cared for, and allows for rapid intervention, including detection of abnormalities.

[0651] A "knowledge-based model" is a data model that uses information based on specialized fields to analyze specific problems and generate proposals.

[0652] An "information display device" is a device that displays the health information of the person being cared for in real time and presents it to the user immediately.

[0653] "Real-time monitoring" is a process that involves continuously collecting data and analyzing it on the spot to immediately assess the situation.

[0654] "Anomaly detection" is a function that identifies data that exceeds the normal range and notifies the user that there is a problem.

[0655] A "safety system" is a general term for the networks and equipment used to manage and maintain safety within a facility.

[0656] The system of this invention is designed to effectively manage the health of those receiving care and improve safety in nursing care facilities. The server loads multiple knowledge-based models tailored to specific fields of expertise. This enables analysis from different perspectives, such as medical, health, and nutrition. The server uses Python and TensorFlow to build the execution environment for the knowledge-based models.

[0657] The terminal uses smart glasses, which are used by care staff. The smart glasses have the function of monitoring the health information of the person being cared for in real time and displaying the results. The application is implemented using cross-platform development tools such as Flutter. Sensor data such as heart rate and activity level obtained from the glasses is sent to a server, normalized, and then input into each model.

[0658] The care staff, as users of the system, can monitor changes in health conditions in real time and take prompt action based on the information as needed. If suspicious movements or abnormalities are detected, the system immediately issues a warning and prompts a response in conjunction with the facility's safety system. This significantly improves the safety and response efficiency of care facilities.

[0659] For example, if a care recipient's heart rate is detected to be higher than normal, the smart glasses will display the health assessment results and specific countermeasures. In addition, based on information from security cameras, the care recipient's current location can be quickly confirmed, and appropriate safety measures can be taken.

[0660] An example of a prompt would be, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations." Based on such prompts, the generative AI model immediately presents useful information to care staff.

[0661] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0662] Step 1:

[0663] The server loads knowledge-based models based on various specialties, including models with different expertise in medicine, health, and nutrition. The loaded models are ready to analyze care recipient data and provide evaluations from multiple perspectives.

[0664] Step 2:

[0665] The smart glasses on the device collect sensor information from the person being cared for in real time. This includes heart rate and activity level. This information is transmitted wirelessly to a server, where the data is normalized. The normalized data is then converted into a format that can be used for subsequent model analysis.

[0666] Step 3:

[0667] The server passes normalized data as input to each knowledge-based model. These models perform evaluations based on the received data and generate expert opinions. Through the exchange of opinions between the models, various elements that constitute the final health assessment are gathered.

[0668] Step 4:

[0669] The care staff, who are the users, receive health assessment results generated through smart glasses. This includes real-time changes in health status and warning messages. For example, if a higher-than-normal heart rate is detected, the glasses will display appropriate countermeasures.

[0670] Step 5:

[0671] The server immediately issues a warning when it detects suspicious activity or anomalies, and works in conjunction with the facility's security system to establish a rapid response system. This allows users to take quick and efficient countermeasures.

[0672] Step 6:

[0673] Users can quickly obtain the necessary information by instructing the generating AI model using prompt sentences. For example, by entering a prompt such as, "A 70-year-old care recipient has a heart rate 20% higher than normal. Please provide a health assessment and recommendations," users can obtain appropriate instructions and suggestions tailored to their specific situation.

[0674] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0675] This invention is an advanced decision support system that combines an emotion engine and improves the care certification assessment process using multiple knowledge-based models tailored to specific fields of expertise. This system is built through server, terminal, and user interaction.

[0676] The server first loads knowledge-based models that are finely tuned for each specialized field. These models include functions specific to various fields such as medicine, healthcare, and nutrition, and are designed to provide comprehensive judgment. Furthermore, an emotion engine is integrated by the server to analyze the user's facial expressions and speech to estimate their psychological state.

[0677] The terminal receives data about the care recipient provided by the user. The information entered includes health status, living environment, and past medical records, and this information is sent to the server. The server receives this data and formats it into an analyzable form through natural language processing and data preprocessing.

[0678] Based on the pre-processed data and the sentiment engine's analysis, the server initiates a discussion among the various knowledge-based models. The models exchange opinions, and evaluation results are generated based on each model's expertise. Based on these results, the server forms the final proposal.

[0679] Users can review suggestions generated through their device. These suggestions are optimized to the user's emotional state and presented in an easily acceptable manner. For example, if the emotional engine identifies anxiety about the level of care needed, suggestions with additional explanations and advice will be provided.

[0680] In this process, the server securely saves all processing steps to a database. This saved data is used for subsequent analysis and system optimization, forming the basis for gaining new insights. These actions improve the efficiency and quality of the care needs assessment process.

[0681] The following describes the processing flow.

[0682] Step 1:

[0683] The server loads multiple knowledge base models, each finely tuned for a specific field. Models for different fields such as medicine, healthcare, and nutrition are deployed and prepared in memory.

[0684] Step 2:

[0685] The terminal receives information about the person receiving care, necessary for the care needs assessment, from the user. This information includes health status, living environment, and past medical history. The terminal then sends this data to the server.

[0686] Step 3:

[0687] The server preprocesses the input data. Using natural language processing techniques, it structures the raw data and converts it into a format that can be processed by the model. It also retrieves relevant laws and cases from historical databases using a RAG (Random Aggregation) configuration.

[0688] Step 4:

[0689] The server uses an emotion engine to analyze the user's facial expressions and voice data. It estimates the user's emotional state and incorporates that information into subsequent processes.

[0690] Step 5:

[0691] The server feeds pre-processed data into each knowledge-based model and initiates an exchange of ideas. Each model generates opinions based on its area of ​​expertise and engages in discussions with other models using natural language.

[0692] Step 6:

[0693] The server aggregates the results of the exchange of opinions and, taking into account the analysis results of the emotion engine, generates a final proposal. If necessary, the proposal is adjusted to match the user's emotions.

[0694] Step 7:

[0695] The terminal presents the generated suggestions to the user. The suggestions are displayed in a way that promotes reassurance and understanding, allowing the user to make a final decision on the level of care required.

[0696] Step 8:

[0697] The server securely stores all data. Input data, sentiment analysis, discussion processes, and final proposals are fully recorded and used for future system improvements and the development of new care methodologies.

[0698] (Example 2)

[0699] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0700] In the process of assessing long-term care needs, there is a need to comprehensively evaluate information on the diverse health status and living environment of those receiving care, and to provide effective proposals that reflect individual conditions and circumstances. Furthermore, it is necessary to provide support that takes into account the user's psychological state. Conventional technologies have had problems such as one-sided evaluations and uniform proposals, which have prevented the quality of decision-making from being sufficiently improved.

[0701] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0702] In this invention, the server includes functional means for loading multiple knowledge base models tailored based on specialized fields; preprocessing functional means for collecting information related to the person receiving care using an interactive data acquisition device and converting the input data into an analyzable form; and functional means for each knowledge base model to exchange opinions, including emotion estimation, based on the preprocessed information, and generate evaluation results. This enables a comprehensive evaluation that is multifaceted and considerate of emotions, as well as the provision of individualized suggestions.

[0703] A "specialized field" refers to an area of ​​study or work where specialized knowledge and skills are required.

[0704] A "knowledge-based model" is a software model that collects information and data related to a specific field and uses that information to perform inferences and make decisions.

[0705] An "interactive data acquisition device" is a device that collects data through interaction with the user and effectively processes the user's input.

[0706] "Preprocessing" refers to a series of processes, such as data cleaning and format conversion, performed to transform raw data into a format that is easy to analyze.

[0707] "Emotion estimation" is a process for estimating a user's emotions by judging their psychological state from factors such as their facial expressions and tone of voice.

[0708] "Exchange of opinions" is a process in which multiple knowledge-based models share information from different perspectives to derive an overall judgment.

[0709] "Suggestions" refer to actionable guidelines and advice derived from evaluation results, intended to support users' decision-making.

[0710] "Functional means" refers to mechanical or electronic components or processes implemented to achieve a specific purpose.

[0711] The system of this invention is designed to provide advanced support for the long-term care certification review process, and functions through the mutual cooperation of three parties: a server, a terminal, and a user.

[0712] First, the server loads multiple knowledge-based models finely tuned for specialized fields such as medicine, healthcare, and nutrition. This enables field-specific decision-making. Furthermore, the server integrates an emotion engine to analyze the user's facial expressions and speech to estimate their psychological state. This emotion analysis function allows the server to provide more personalized support.

[0713] The terminal provides an interface for receiving information about the person receiving care from the user. This information includes health status, living environment, and past medical records. The terminal collects this information and sends it to the server.

[0714] The server preprocesses the received data. This preprocessing includes cleaning the data using natural language processing techniques and converting it into an analyzable format. Subsequently, based on the preprocessed data, each knowledge base model exchanges opinions with the others and generates evaluation results from their respective expert perspectives.

[0715] For example, when a care recipient's living environment changes, the server can use an emotion engine to analyze how anxious the user is about that change. Based on this, the server proposes a care plan designed to provide greater reassurance.

[0716] An example of a prompt message to be input into the generating AI model is, "Use the expert knowledge model to provide comprehensive suggestions regarding the care recipient's living environment." This allows the system to accurately provide the support the user needs.

[0717] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0718] Step 1:

[0719] The server loads multiple knowledge-based models, finely tuned based on specific fields. These models are specialized for various areas such as medicine, healthcare, and nutrition, and form the basis for complex data analysis and decision-making. The input requires preliminary data relevant to each field, which is loaded by the server. The output is the setup of each model for use.

[0720] Step 2:

[0721] The terminal collects information about the care recipient from the user. This information includes health status, living environment, and past medical history. The user inputs this information through the terminal's interface. The input data is sent from the terminal to the server and used as the basis for subsequent analysis. The output is formatted data sent to the server.

[0722] Step 3:

[0723] The server performs preprocessing on the data received from the terminal. This preprocessing includes data cleaning and format conversion using natural language processing techniques. Raw data is provided as input and is formatted into a format that is easy to analyze. The output is the formatted data prepared for further analysis.

[0724] Step 4:

[0725] The server uses an emotion engine to estimate the user's psychological state based on pre-processed data. This includes user input, tone of voice, and, in some cases, facial expression analysis. If the emotion engine identifies anxiety or worry in the user, this is used as data within the server. The output is data related to the user's psychological state.

[0726] Step 5:

[0727] The server hands over pre-processed data and sentiment analysis results to the knowledge-based models, initiating opinion exchange and evaluation results generation by the models. Inputs include formatted data and emotional state information, while outputs are evaluation results from each model. The knowledge-based models analyze the data from different expert perspectives using varying approaches, leading to a more comprehensive discussion.

[0728] Step 6:

[0729] The server integrates evaluation results from the models and generates optimized suggestions to present to the user. These suggestions are adjusted to take into account individual circumstances and the user's psychological state. The input requires evaluation results from each model, and the output generates specific suggestions.

[0730] Step 7:

[0731] Users can review the suggestions generated through their device. The suggestions are displayed in an easy-to-understand format, allowing users to easily make decisions based on them. Users can request additional information or provide feedback on the suggestions via their device as needed.

[0732] Step 8:

[0733] The server securely stores all data and evaluation results generated throughout the entire process in a database. This stored data is used for future analysis and system optimization. The input requires all generated data, which is stored in an organized manner. The output is historical data stored in a database, corresponding to anticipated future uses.

[0734] (Application Example 2)

[0735] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Modern technological advancements have created a need for support systems based on specialized knowledge in various fields. However, many existing systems provide uniform responses without considering the user's psychological state or emotions, making them unable to provide flexible support tailored to individual situations. Therefore, there is a need to analyze the user's psychological state and optimize suggestions accordingly. In particular, considering the user's sense of security is crucial in areas such as security, but existing systems fail to address this aspect. Consequently, there is a need to develop new systems that can provide decision-making support tailored to the user's psychological state.

[0737] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0738] In this invention, the server includes means for loading multiple knowledge data models tailored based on specialized domains, means for collecting and preprocessing information related to the person being assisted, and means for analyzing the psychological state and optimizing suggestions. This enables flexible decision-making support that takes into account the user's psychological state.

[0739] "Multiple knowledge data models tailored to specific areas of expertise" are data models that possess specialized knowledge in individual fields such as medicine, health, and nutrition, and are tailored to each field to support optimal decision-making.

[0740] "Information related to the person receiving support" refers to a collection of specific data about the individual in question that is necessary for the system to provide decision-making support, such as health status, living environment, and past medical records.

[0741] "A means of analyzing psychological state and optimizing suggestions" refers to a technology that uses an emotion engine to analyze the user's facial expressions and statements, optimizes support suggestions based on the results, and enables more personalized responses.

[0742] A "means for exchanging opinions" is a mechanism in which multiple knowledge data models exchange information with each other, discuss from their respective expert perspectives, and form a final judgment.

[0743] "Means for securely recording data and preparing for future use" refers to a recording system that securely stores the processing and proposals performed within the system, making them available for later analysis and system improvement.

[0744] The system for implementing this invention functions through the interaction of a server, a terminal, and a user.

[0745] The server first loads multiple knowledge data models, each tailored to specific professional fields such as medicine and healthcare. These models incorporate different expert perspectives to support comprehensive decision-making. Furthermore, an emotion engine is used to analyze the user's facial expressions and statements, estimating their psychological state. Optimization of suggestions based on emotional state is also ensured at this stage.

[0746] The device collects various pieces of information related to the person receiving assistance from the user. This includes important data such as health status and past medical records, and this information is sent to a server to prepare for further processing and analysis.

[0747] Users can receive suggestions from the server via their device and make decisions based on them. A key feature is that the suggestions are tailored to the user's psychological state and presented in a way that is easily accepted.

[0748] The hardware used here includes the smartphone's camera and microphone, which are used to analyze the user's facial expressions and voice data. The software utilizes generative AI models such as TensorFlow to enable sentiment analysis. Appropriate database technologies such as SQLite are used for data storage.

[0749] For example, if a user expresses security concerns in a particular location, the emotion engine analyzes this and provides security advice for connecting to public Wi-Fi. This allows the user to continue communicating with confidence. A prompt such as, "Think about how a security app can help in a situation where you are feeling anxious," is used.

[0750] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0751] Step 1:

[0752] The terminal receives information about the person receiving assistance from the user as input. Specifically, it collects data such as health status, past medical records, and living environment through the smartphone interface and sends this data to the server. The input here is formatted as text information.

[0753] Step 2:

[0754] The server preprocesses the received information. It analyzes the input data using natural language processing techniques and structures the individual pieces of information. This standardizes the format of each data point, enabling rapid and efficient analysis using knowledge data models.

[0755] Step 3:

[0756] The server simultaneously loads multiple knowledge data models, each tailored to a specific domain. Generative AI models are then used to send pre-processed data to each model, generating opinions from their respective expert perspectives. The output is then sent back, with the models exchanging opinions.

[0757] Step 4:

[0758] The server uses an emotion engine to analyze the user's on-screen statements and psychological nuances collected by the device. The input consists of the user's facial expressions and voice, which are used to estimate their psychological state. The resulting output is a profile of the user's psychological state.

[0759] Step 5:

[0760] The server integrates the output from the knowledge data model and the emotion engine to generate the final recommendation. This recommendation is optimized to take the user's psychological state into account. For example, if the user is showing anxiety, the output will include specific recommendations and precautions to address that anxiety.

[0761] Step 6:

[0762] The user receives the final proposal generated through their device and makes a decision based on it. The final proposal is displayed as text and visual data, presented in a clear and easy-to-understand format for the user.

[0763] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0764] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0765] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0766] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0767] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0768] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0769] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0770] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0771] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0772] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0773] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0774] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0775] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0776] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0777] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0778] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0779] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0780] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0781] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0782] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0783] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0784] The following is further disclosed regarding the embodiments described above.

[0785] (Claim 1)

[0786] A means of loading multiple knowledge-based models tailored to specific fields,

[0787] A means for collecting and pre-processing information related to the person receiving assistance,

[0788] A means by which each knowledge-based model exchanges opinions using pre-processed information,

[0789] A means for aggregating the results of the aforementioned exchange of opinions and generating proposals,

[0790] A means of presenting the generated proposals and supporting the final decision-making,

[0791] Means to securely store data and prepare for future use,

[0792] A system that includes this.

[0793] (Claim 2)

[0794] The system according to claim 1, wherein the information gathering means takes in information including data on the care recipient's past medical and nursing care.

[0795] (Claim 3)

[0796] The system according to claim 1, wherein the opinion exchange means conducts discussions using opinions from different perspectives in specialized fields based on each knowledge base model.

[0797] "Example 1"

[0798] (Claim 1)

[0799] A means of loading multiple knowledge base models tailored based on specialized fields,

[0800] A means for acquiring various information related to the person receiving care and processing that information,

[0801] Based on pre-processed information, each knowledge-based model has a means of exchanging information with each other in natural language,

[0802] A means for integrating the results of the aforementioned information exchange and formulating a proposal,

[0803] A means to display the generated proposals and prompt a final decision,

[0804] Means for securely storing information and preparing for future analysis,

[0805] A system that includes this.

[0806] (Claim 2)

[0807] The system according to claim 1, wherein the information acquisition means includes information relating to the past health history and living conditions of the person receiving care.

[0808] (Claim 3)

[0809] The system according to claim 1, wherein the information exchange means conducts discussions using opinions from different perspectives within specialized fields based on each knowledge base model.

[0810] "Application Example 1"

[0811] (Claim 1)

[0812] A means of loading multiple knowledge-based models tailored to specific fields,

[0813] A means for collecting and pre-processing information related to the person receiving assistance,

[0814] A means by which each knowledge-based model exchanges opinions using pre-processed information,

[0815] A means for aggregating the results of the aforementioned exchange of opinions and generating proposals,

[0816] A means of presenting the generated proposals and supporting the final decision-making,

[0817] Means to securely store data and prepare for future use,

[0818] A means of monitoring health information in real time using an information display device,

[0819] A means of issuing a warning when an anomaly is detected,

[0820] A means of linking information monitoring devices and facility safety systems,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, wherein the information gathering means takes in information including data on the care recipient's past medical and nursing care.

[0824] (Claim 3)

[0825] The system according to claim 1, wherein the opinion exchange means conducts discussions using opinions from different perspectives in specialized fields based on each knowledge base model.

[0826] "Example 2 of combining an emotion engine"

[0827] (Claim 1)

[0828] A functional means for loading multiple knowledge base models tailored based on specialized fields,

[0829] A preprocessing function means that collects information related to the person receiving care using an interactive data collection device and converts the input data into an analyzable format,

[0830] Based on pre-processed information, each knowledge-based model engages in opinion exchange, including sentiment estimation, and generates evaluation results.

[0831] A functional means for integrating the results of the aforementioned exchange of opinions and generating an optimized proposal,

[0832] A functional means that presents generated personalized suggestions and supports the user in making a final decision based on their emotional state,

[0833] Functional means for securely storing data and utilizing it for future analysis and optimization,

[0834] A system that includes this.

[0835] (Claim 2)

[0836] The system according to claim 1, wherein the information gathering means acquires information including the health status, living environment, and past medical history of the person receiving care.

[0837] (Claim 3)

[0838] The system according to claim 1, wherein the opinion exchange means uses opinions from different perspectives in specialized fields based on each knowledge base model to conduct discussions that take into account emotional states.

[0839] "Application example 2 when combining with an emotional engine"

[0840] (Claim 1)

[0841] A means for loading multiple knowledge data models tailored based on specialized fields,

[0842] A means for collecting and preprocessing information related to the person receiving support,

[0843] A means by which each knowledge data model exchanges opinions using preprocessed information,

[0844] A means for aggregating the results of the aforementioned exchange of opinions and generating proposals,

[0845] A means of presenting the generated proposals and supporting the final decision-making,

[0846] A means of analyzing psychological states and optimizing proposals,

[0847] A means to securely record data and prepare it for future use,

[0848] A system that includes this.

[0849] (Claim 2)

[0850] The system according to claim 1, wherein the information gathering means takes in information including information about the past health and support of the person receiving support.

[0851] (Claim 3)

[0852] The system according to claim 1, wherein the opinion exchange means conducts discussions using opinions from different perspectives in specialized fields based on each knowledge data model. [Explanation of Symbols]

[0853] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of loading multiple knowledge-based models tailored to specific fields, A means for collecting and pre-processing information related to the person receiving assistance, A means by which each knowledge-based model exchanges opinions using pre-processed information, A means for aggregating the results of the aforementioned exchange of opinions and generating proposals, A means of presenting the generated proposals and supporting the final decision-making, Means to securely store data and prepare for future use, A system that includes this.

2. The system according to claim 1, wherein the information gathering means takes in information including data on the care recipient's past medical and nursing care.

3. The system according to claim 1, wherein the opinion exchange means conducts discussions using opinions from different perspectives in specialized fields based on each knowledge base model.

Images