System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring behavior data of a person to be cared for; means for transmitting the acquired behavior data to a server; means for analyzing the behavior data received by the server and identifying needs and behavior patterns of the person to be cared for; means for generating predicted support information based on the identified needs and behavior patterns; and means for notifying a nursing staff of the generated support information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As the demand for care increases in an aging society, caring for dementia patients in particular is a major issue. Dementia patients often have difficulty communicating their feelings and needs, making it difficult for newly assigned care staff to respond appropriately. Furthermore, care staff are often busy, making it difficult for them to notice subtle changes in the behavior and speech of care recipients. This creates a risk that appropriate care will not be provided. The present invention aims to provide a system that reduces the burden on care sites and quickly responds to the needs of care recipients. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for acquiring behavioral data of a care recipient, a means for transmitting the acquired behavioral data to a server, a means for the server to analyze the behavioral data received and identify the needs and behavioral patterns of the care recipient, a means for generating predicted support information based on the identified needs and behavioral patterns, and a means for notifying care staff of the generated support information. Furthermore, the system solves this problem by providing a means for the server to learn past behavioral data and predict the future behavior of the care recipient, and a means for the care staff to receive the notification and take specific action for the care recipient based on the content of the notification.

[0006] "Care recipients" refers to elderly, ill, or disabled individuals who require care.

[0007] "Behavioral data" refers to information such as the physical movements and daily actions of the care recipient, as well as changes in facial expressions and voice.

[0008] "Means" refers to a method, device, program, process, etc. used to achieve a specific purpose.

[0009] A "server" refers to a computer system that stores and processes data on a network.

[0010] "Analysis" refers to the process or method of handling collected data and extracting patterns and trends.

[0011] "Needs" refers to the type of support and care the care recipient requires.

[0012] "Behavioral patterns" refer to a series of repeated actions or habits of the person receiving care.

[0013] "Support information" refers to information containing instructions and suggestions necessary for caring for the care recipient.

[0014] "Notification" refers to messages or alerts that inform care staff of specific information.

[0015] "Learning" refers to the process by which an AI system is able to make future predictions and decisions based on past data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects the actions and behavior of care recipients in real time, analyzes this data, and provides appropriate support information to care staff.

[0038] An embodiment of this system is described below.

[0039] Data collection methods

[0040] The device collects behavioral data from sensors attached to the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and changes in facial expressions. The collected data is temporarily stored on the device.

[0041] Data transmission method

[0042] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0043] Data Analysis Methods

[0044] The server analyzes the received data to identify the current situation and behavioral patterns of the care recipient, specifically, walking patterns, meal frequency, and body temperature fluctuations, and if any abnormal behavior is detected, the information is notified to the care staff.

[0045] Needs identification and behavior prediction tools

[0046] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0047] Support information generation method

[0048] The server generates support information that is useful to care staff based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions.

[0049] Notification means

[0050] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other means. This allows the care staff to quickly take appropriate action for the care recipient.

[0051] Specific examples

[0052] For example, consider a case where a care recipient needs to drink fluids around 3:00 PM. The device collects the care recipient's behavioral data and sends it to the server. The server analyzes the data and predicts that the care recipient will need to drink fluids at 3:00 PM based on past behavioral patterns. The server then generates support information such as "Provide fluids at 3:00 PM" and sends a notification to the care staff's device. The care staff receives the notification and provides fluids at the appropriate time, thereby maintaining the care recipient's health.

[0053] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The device collects real-time behavioral data from sensors attached to the care recipient and cameras installed in the device, detecting, for example, walking frequency, distance traveled, and changes in facial expressions.

[0057] Step 2:

[0058] The device performs initial processing of the acquired behavioral data, and performs data compression and noise filtering as needed, ensuring that only the important data is sent to the next step.

[0059] Step 3:

[0060] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using a wireless network (Wi-Fi or mobile data communication).

[0061] Step 4:

[0062] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[0063] Step 5:

[0064] The server retrieves the latest data from the database and applies analytical algorithms to process the data, specifically analyzing behavioral patterns and assessing the care recipient's health.

[0065] Step 6:

[0066] Based on the analysis results, the server identifies the care recipient's current needs and abnormal behavior. For example, if the care recipient wakes up frequently during the night, it may determine that they have a sleep disorder.

[0067] Step 7:

[0068] The server uses past behavioral data to train machine learning algorithms to predict the care recipient's future behavior, including calculating the likelihood of performing certain behaviors at certain times of the day.

[0069] Step 8:

[0070] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 p.m."

[0071] Step 9:

[0072] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[0073] Step 10:

[0074] The device displays the received notification to the care staff, and the notification content includes specific actions to take (e.g., encouraging hydration).

[0075] Step 11:

[0076] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more fluids, taking their temperature, or speaking to them.

[0077] Step 12:

[0078] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous care.

[0079] Example 1

[0080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0081] In an aging society, reducing the burden of caregiving and providing high-quality care to those receiving care are important issues. In particular, there is a demand for an effective system that can grasp the condition of those receiving care in real time and provide appropriate support. Conventional caregiving systems lack the ability to accurately predict and promptly notify the behavior and needs of those receiving care, which places a heavy burden on caregivers and can sometimes prevent them from providing optimal support to those receiving care.

[0082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0083] In this invention, the server includes means for acquiring biometric data of the care recipient, means for transmitting the acquired biometric data to the server, means for analyzing the received biometric data and identifying the condition and behavioral patterns of the care recipient, means for generating predicted support information based on the identified condition and behavioral patterns, and means for notifying the generated support information to the caregiver. This makes it possible to grasp the condition of the care recipient in real time and provide appropriate support promptly.

[0084] "Care recipients" refers to individuals, such as elderly people or people with disabilities, who require care services.

[0085] "Biometric data" refers to physical or physiological information obtained in real time, such as the walking patterns, dietary intake, body temperature, and changes in facial expression of the person receiving care.

[0086] A "server" refers to a computer system that receives data sent from a terminal, analyzes it, and generates necessary information.

[0087] "Terminal" refers to a device that collects data from sensors, cameras, etc. attached to the care recipient and transmits it to a server.

[0088] "Analysis" refers to the process in which the server processes the received biometric data and evaluates the condition and behavioral patterns of the person being cared for.

[0089] "Support information" refers to specific guidelines and suggestions for action that are generated based on analysis results and predictions and provided to caregivers.

[0090] "Notification" refers to a means for transmitting support information generated by the server to the caregiver's terminal and informing the caregiver.

[0091] "Caregiver" refers to an individual whose role is to provide care to a care recipient.

[0092] "Learning algorithm" refers to the computational method used by the server to predict future behavior using past biometric data.

[0093] "Real-time" means that data collection and analysis are almost instantaneous, and that responses are based on real time without delay.

[0094] The present invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system collects behavioral and biological data of care recipients in real time, analyzes this data, and provides appropriate support information to caregivers. Specific embodiments of this system are described below.

[0095] Data collection methods

[0096] The device collects biometric data from sensors worn by the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and facial expressions. The collected data is temporarily stored on the device. For example, walking pattern data is recorded every second, and the care recipient's facial expressions are captured every minute.

[0097] Data transmission method

[0098] The device sends the collected data to the server at a predetermined interval. This communication is performed over a wireless network. For example, the device sends the collected data to the server every five minutes.

[0099] Data Analysis Methods

[0100] The server analyzes the received data to identify the condition and behavioral patterns of the care recipient. Specifically, it analyzes walking patterns, meal intake frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the caregiver. The server analyzes walking patterns and evaluates the risk of tripping. If food intake data is lower than normal, an alert is generated.

[0101] Needs identification and behavior prediction tools

[0102] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's future behavior and needs. For example, if past data shows that the care recipient tends to want food at a certain time of day, the server notifies the caregiver before that time arrives. The server then uses a machine learning model to calculate the probability that the care recipient will start their walk at 10:00 a.m.

[0103] Support information generation method

[0104] The server generates support information that will be useful to caregivers based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions. For example, a specific suggestion such as "recommend taking a walk at 10:00 AM" may be generated.

[0105] Notification means

[0106] The support information generated by the server is sent to the caregiver's device. The device displays the received information to the caregiver in the form of a push notification or other format. This allows the caregiver to quickly take appropriate action for the care recipient. The device displays a notification on the caregiver's smartphone saying, "Please encourage the care recipient to take a walk at 10 a.m."

[0107] Examples and prompts

[0108] For example, consider a case where a care recipient needs hydration around 3:00 PM. The device collects the care recipient's walking data and facial expression data at 2:55 PM and sends the data to the server at 3:00 PM. The server analyzes the data and predicts that hydration will be needed at 3:00 PM based on past patterns, generating support information such as "Provide hydration at 3:00 PM." The device displays this information to the caregiver in the form of a push notification, and the caregiver provides hydration at the appropriate time. An example of a prompt sentence is "Please generate support information for when the care recipient needs hydration at 3:00 PM."

[0109] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on caregivers.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Step 1:

[0112] The device collects biometric data from sensors and cameras attached to the person being cared for.

[0113] Specifically, the device records the walking pattern of the person being cared for every second and captures their facial expressions every minute.

[0114] Input: Biometric data from sensors and cameras.

[0115] Output: Biometric data and its temporary storage.

[0116] Step 2:

[0117] The data collected by the device is temporarily stored in local storage.

[0118] Specific operations include storing walking data and facial expression data in memory or storage.

[0119] Input: Biometric data captured on the terminal.

[0120] Output: Data recorded in the device's local storage.

[0121] Step 3:

[0122] The terminal sends the collected data to the server at a predetermined interval via a wireless network.

[0123] Specifically, the device packages the data every five minutes and sends it to a server using Wi-Fi or the mobile network.

[0124] Input: Biometric data stored on the device.

[0125] Output: Data packages sent over the wireless network.

[0126] Step 4:

[0127] The server receives the data sent from the terminal.

[0128] Specifically, the server takes in data at a receiving port and stores it in a database.

[0129] Input: Data package sent from the terminal.

[0130] Output: Biometric data stored on the server.

[0131] Step 5:

[0132] The server analyzes the data it receives and identifies the condition and behavioral patterns of the person being cared for.

[0133] Specifically, the server analyzes walking patterns to assess the risk of tripping, calculates food intake frequency, and detects abnormalities.

[0134] Input: Biometric data stored on the server.

[0135] Output: Analysis result data (e.g., tripping risk, dietary intake abnormalities).

[0136] Step 6:

[0137] The server compares the collected data with past data and uses learning algorithms to predict the future behavior and needs of the care recipient.

[0138] Specifically, the server runs a machine learning model to predict whether a specific behavioral pattern exists during a specific time period.

[0139] Input: Analysis results and historical data.

[0140] Output: Predictive data about future behavior and needs.

[0141] Step 7:

[0142] The server generates support information that is useful to caregivers based on the analysis results and behavior prediction results.

[0143] As a specific operation, the server generates specific support information such as "recommend taking a walk at 10:00 AM."

[0144] Input: Predictive data about future behaviors and needs.

[0145] Output: Supporting information.

[0146] Step 8:

[0147] The support information generated by the server is sent to the terminal of the caregiver.

[0148] Specifically, the server composes the support information as a text message and sends it to the caregiver's terminal via the notification system.

[0149] Input: Generated assistance information.

[0150] Output: Support information sent to the caregiver's device.

[0151] Step 9:

[0152] The support information received by the device is displayed to the caregiver in the form of a push notification.

[0153] Specifically, the device displays a notification on the caregiver's smartphone saying, "Please encourage the person being cared for to take a walk at 10 a.m."

[0154] Input: Support information sent from the server.

[0155] Output: Assistance notification displayed to caregiver.

[0156] (Application example 1)

[0157] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] In an aging society, it is necessary to provide high-quality care to care recipients while reducing the burden on care staff. However, current care systems have difficulty monitoring the behavior of care recipients in real time, immediately detecting abnormal behavior, and providing appropriate support. In particular, in physical care support activities, it is time-consuming and labor-intensive for care staff to constantly monitor the condition of care recipients. Therefore, an efficient method to protect the health and safety of care recipients is required.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0160] In this invention, the server includes means for acquiring behavioral data of the care recipient, means for transmitting the acquired behavioral data to the server, means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient, means for generating predicted support information based on the identified needs and behavioral patterns, means for notifying the care staff of the generated support information, means for monitoring the behavioral data of the care recipient in real time at the store and detecting abnormal behavior, and means for transmitting a notification to the care staff's terminal when abnormal behavior is detected. This makes it possible to efficiently monitor the condition of the care recipient, quickly detect abnormal behavior, and provide appropriate support.

[0161] "Care recipients" refers to elderly people and individuals with disabilities who require care.

[0162] "Behavioral data" refers to information about the activities of the care recipient, such as their walking patterns, dietary intake, and changes in facial expressions.

[0163] "Server" refers to the computer system that analyzes collected data and identifies behavioral patterns.

[0164] "Cloud" refers to computer resources and storage services provided over the Internet.

[0165] "Care staff" refers to professional people who provide care and support to care recipients.

[0166] "Means of data acquisition" refers to devices such as sensors and cameras used to collect behavioral data of the care recipient.

[0167] "Data transmission means" refers to the communication technology and protocol used to transmit acquired behavioral data to a server.

[0168] "Analysis means" refers to software or algorithms that process the data received by the server and identify the needs and behavioral patterns of the care recipient.

[0169] "Prediction means" refers to a learning algorithm for predicting the future behavior of the care recipient based on past data.

[0170] "Support information generation means" refers to a system that generates specific action plans to be provided to care staff based on identified needs, behavioral patterns, and predictions.

[0171] "Notification means" refers to a method or technology for transmitting the generated support information to the terminal of the care staff.

[0172] "Stores" refer to retail stores and service facilities visited by care recipients.

[0173] "Real-time monitoring means" refers to technology or devices that instantly collect and process behavioral data of care recipients.

[0174] "Abnormal behavior detection means" refers to algorithms or software that analyze collected behavioral data to identify behavior that is out of the ordinary.

[0175] "Terminal" refers to a device used by care staff to receive and display information, such as a smartphone or smart glasses.

[0176] This invention is a system that reduces the burden of caregiving in an aging society and provides high-quality care to care recipients. This system collects and analyzes behavioral data of care recipients in real time and provides appropriate support information to care staff based on that data.

[0177] Data collection methods

[0178] The terminal is connected to sensors worn by the care recipient and cameras installed in the store. These devices are used to collect the care recipient's behavioral data (walking patterns, food intake, changes in facial expressions, etc.) in real time. This behavioral data is temporarily stored on the terminal and later sent to a server.

[0179] Data transmission method

[0180] The device sends the collected behavioral data to a server at predetermined intervals. This communication utilizes a wireless network, using Wi-Fi or Bluetooth as the wireless network technology.

[0181] Data Analysis Methods

[0182] The server analyzes the received behavioral data using software such as Python, TensorFlow, OpenCV (facial recognition), Keras, and Flask (web server). The server uses data analysis to identify the current situation and behavioral patterns of the care recipient. For example, it analyzes walking patterns, meal frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the care staff.

[0183] Needs identification and behavior prediction tools

[0184] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0185] Support information generation method

[0186] Based on the analysis results and behavior prediction results, the server generates support information that will be useful to care staff. This support information is provided in the form of specific action suggestions. For example, if a care recipient needs hydration around 3:00 PM, the server might suggest providing hydration at 3:00 PM.

[0187] Notification means

[0188] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other format, allowing the care staff to quickly take appropriate action for the care recipient.

[0189] Real-time monitoring and abnormal behavior detection in stores

[0190] Using cameras and sensors installed in the store, behavioral data of care recipients is collected and monitored in real time. If an abnormal behavior is detected by the abnormal behavior detection algorithm, the information is immediately sent to the care staff's device.

[0191] Specific examples

[0192] For example, if a care recipient stops standing still for a long time while visiting a store, their behavioral data is collected via a camera and sent to a server. The server analyzes the data and determines that the behavior is abnormal. In that case, a notification is sent to the caregiver's smartphone or smart glasses, requesting that they be given a place to sit.

[0193] Prompt Sentence Examples

[0194] "We would like to develop a system that collects behavioral data of elderly people in stores and analyzes it in real time. We would like to add a function that notifies store staff of appropriate support information based on specific behavioral patterns. Please provide us with Python code that performs facial recognition, analyzes walking patterns, and detects abnormal behavior."

[0195] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0196] Step 1:

[0197] The device collects behavioral data of the care recipient. Specifically, it uses sensors worn by the care recipient and cameras installed in the store to obtain data such as walking patterns, food intake, and changes in facial expressions. The input is the care recipient's real-time behavioral data, and the output is a temporary storage of this data.

[0198] Step 2:

[0199] The device transmits the collected behavioral data to the server at predetermined intervals. This communication is carried out over a wireless network. The input is the behavioral data stored on the device, and the output is the transmission of data to the server. Specific operations include creating and transmitting data packets.

[0200] Step 3:

[0201] The behavioral data received by the server is analyzed. Software such as Python, TensorFlow, OpenCV (face recognition), and Keras is used for the analysis. The input is the behavioral data sent to the server, and the output is the current situation and behavioral patterns of the care recipient as the analysis results. Specific operations include data preprocessing, detection of abnormal behavior, and identification of needs.

[0202] Step 4:

[0203] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's current needs and future behavior. The input is past behavior data and current data, and the output is the prediction result. Specific operations include applying machine learning models, learning data, and making predictions.

[0204] Step 5:

[0205] The server generates support information useful to care staff based on the analysis results and behavior prediction results. The input is the analysis results and behavior prediction results, and the output is support information in the form of specific action suggestions. Specific operations include generating and formatting the support information.

[0206] Step 6:

[0207] The server sends the generated support information to the care staff's terminal. The input is the generated support information, and the output is a notification to the terminal. Specific operations include packetizing and transmitting the notification data.

[0208] Step 7:

[0209] The device displays the support information received by the device to the care staff. The input is the notification data sent from the server, and the output is the provision of information to the care staff. Specific operations include receiving and displaying push notifications.

[0210] Step 8:

[0211] The care staff receives the notification and takes specific actions for the care recipient based on the content of the notification. The input is the notified support information, and the output is specific support actions for the care recipient. Specific actions include the care staff taking actions as instructed.

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] This invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system uses not only behavioral data of care recipients but also an emotion engine to recognize their emotions and provide care based on this.

[0214] System Configuration

[0215] The system includes the following main components and functions:

[0216] 1. Data Collection Methods

[0217] The device collects behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. This data is temporarily stored on the device.

[0218] 2. Data transmission method

[0219] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0220] 3. Data Analysis Methods

[0221] The server analyzes the received data to identify the behavioral patterns and emotional state of the care recipient, for example, by analyzing walking patterns, meal frequency, and facial expressions.

[0222] 4. Needs Identification and Behavior Prediction Tools

[0223] The server predicts the current needs and future behavior of the care recipient based on collected and past data. For example, if past data indicates that the care recipient tends to want to eat at a certain time, it will notify the care staff before that time arrives.

[0224] 5. Emotion Engine

[0225] The server uses an emotion engine to recognize the emotional state (e.g., happiness, stress, anger) of the care recipient from data such as facial expressions, tone of voice, and gestures. This emotional data is also analyzed.

[0226] 6. Support Information Generation Method

[0227] The server generates specific support information for care staff based on the analysis results (behavioral patterns, emotional state) and notifies them of this information. For example, it suggests actions such as "Play relaxing music because the care recipient is feeling stressed."

[0228] 7. Means of notification

[0229] The server sends the generated support information to the care staff's device. Notifications are sent in the form of push notifications or emails. The device displays the received information to the care staff.

[0230] 8. Care staff response

[0231] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as talking to the care recipient to reduce stress or providing meals at specific times.

[0232] Specific examples

[0233] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby simultaneously maintaining the care recipient's good health and emotional state.

[0234] In this way, through a series of processes including data collection, analysis, prediction, emotion recognition, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0235] The processing flow will be explained below.

[0236] Step 1:

[0237] The device collects real-time behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. Specifically, it detects walking frequency, distance traveled, whether or not the care recipient is smiling, and voice intonation.

[0238] Step 2:

[0239] The terminal performs initial processing of the acquired data, noise filtering, and data compression, so that only important data is sent to the next step, improving communication efficiency.

[0240] Step 3:

[0241] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using Wi-Fi or mobile data communication.

[0242] Step 4:

[0243] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[0244] Step 5:

[0245] The server retrieves the latest data from the database and analyzes behavioral patterns and emotional states, applying algorithms that analyze walking patterns, frequency of eating, and changes in facial expressions and tone of voice.

[0246] Step 6:

[0247] The server uses an emotion engine to recognize the care recipient's emotional state from the acquired facial expression, tone of voice, and gesture data, for example, determining whether the care recipient is smiling or has a low voice.

[0248] Step 7:

[0249] The server uses the results of behavioral analysis and emotion recognition to identify the current needs and abnormal behaviors and emotions of the care recipient, for example, frequent nighttime wakings or high stress levels, and responds accordingly.

[0250] Step 8:

[0251] The server uses past behavioral and emotional data to train machine learning algorithms to predict the care recipient's future behavior and emotional state, including calculating the likelihood of engaging in certain behaviors at certain times of the day.

[0252] Step 9:

[0253] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 pm" or "play relaxing music because you are feeling stressed."

[0254] Step 10:

[0255] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[0256] Step 11:

[0257] The device displays the received notification to the caregiver, and the notification content includes specific actions (e.g., encouraging hydration or playing music for relaxation).

[0258] Step 12:

[0259] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more water, talking to them to reduce stress, taking their temperature, or playing music to help them relax.

[0260] Step 13:

[0261] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous and appropriate care.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] In an aging society, the burden of caregiving is increasing, while at the same time there is a demand for providing high-quality care to care recipients. However, conventional systems generally respond based only on the behavioral data of care recipients, making it difficult to provide individualized care that takes into account their emotional state. Therefore, a new system is needed that can detect emotional changes in care recipients and provide optimal care based on that.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0266] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the behavioral patterns and emotional state of the care recipient, means for generating predicted support information based on the identified behavioral patterns and emotional state, and means for notifying the generated support information to the care staff. This enables more accurate care that takes into account not only the behavioral patterns but also the emotional state of the care recipient, and reduces the burden on the care staff.

[0267] "Behavioral data of the care recipient" refers to information that records the specific actions and habits of the care recipient in their daily life, and includes data such as walking patterns, dietary intake status, and excretory behavior.

[0268] "Emotional data" refers to information that indicates the emotional state of the care recipient, and includes data obtained through facial expressions, tone of voice, gestures, and the like.

[0269] "Sensors" are devices that detect changes in the body or environment of the care recipient and collect behavioral and emotional data, and include wearable sensors and fixed cameras.

[0270] "Server" refers to a computer system that analyzes collected data, identifies the behavioral patterns and emotional state of the care recipient, and generates appropriate support information to notify care staff.

[0271] "Behavioral patterns" are data that indicate the tendency for continuity and repetition of specific behaviors in the care recipient's daily life, and include behavioral history by time.

[0272] "Emotional state" refers to the specific emotional state of the care recipient, including emotions such as happiness, stress, and anger.

[0273] "Support information" is information that includes specific instructions and advice for care staff to provide appropriate care based on the behavioral patterns and emotional state of the care recipient.

[0274] "Care staff" refers to professionals who provide direct care and support to care recipients.

[0275] "Notification means" refers to a method or device for transmitting support information from the server to the care staff, and includes push notifications, emails, etc.

[0276] "Analysis means" refers to an analysis algorithm or program for identifying the behavioral patterns and emotional state of the care recipient based on the behavioral data and emotional data received by the server.

[0277] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects and analyzes behavioral and emotional data of care recipients, generates appropriate support information based on that data, and notifies care staff.

[0278] 1. System Configuration

[0279] The system includes the following main components and functions:

[0280] Data collection methods

[0281] Data transmission method

[0282] Data Analysis Methods

[0283] Needs identification and behavior prediction tools

[0284] Emotion Engine

[0285] Support information generation method

[0286] Notification means

[0287] 2. Data Collection Methods

[0288] The device collects behavioral and emotional data via sensors attached to the care recipient or installed cameras. Specific hardware used is wearable sensors or fixed cameras. This data is temporarily stored on the device.

[0289] Example: A wearable sensor attached to the care recipient's arm collects heart rate data, while a camera installed in the living room captures facial expressions.

[0290] 3. Data Transmission Method

[0291] The device sends the collected data to the server at predetermined intervals. This communication is carried out using a wireless network (Wi-Fi or Bluetooth).

[0292] Example: Heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[0293] 4. Data Analysis Methods

[0294] The server analyzes the received data in real time to identify the behavioral patterns and emotional state of the care recipient using machine learning algorithms and emotion recognition engines.

[0295] Example: Using a machine learning model to predict the risk of falls based on the walking patterns of a care recipient, and an emotion recognition engine to determine stress levels based on facial expression data.

[0296] 5. Needs Identification and Behavior Prediction Tools

[0297] The server uses collected and past data to predict the care recipient's current needs and future behavior, including past behavioral history and emotional data trends.

[0298] Example: Using data from the past week, predict that you need to drink water around 3pm and detect trends of increased stress at certain times of the day.

[0299] 6. Emotion Engine

[0300] The server uses an emotion engine to recognize the emotional state of the care recipient from facial expressions, tone of voice, and gestures, and this information is also added to the overall analysis.

[0301] Examples: Analyzing your tone of voice, selecting your favorite music, and analyzing your gestures to determine whether you are feeling anxious.

[0302] 7. Support Information Generation Method

[0303] The server generates specific support information for the care staff based on the analysis results, and this information is formatted for transmission to the care staff's terminal.

[0304] Example: Generate specific care action instructions such as "Provide water and play relaxing music at 3 pm."

[0305] 8. Means of notification

[0306] The support information generated by the server is sent to the care staff's device via push notification or email, and the device displays the received information so that the care staff can check it.

[0307] Example: Using the push notification function, send a message to the care staff's device saying, "Please provide water and play relaxing music at 3 p.m."

[0308] Specific examples

[0309] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby maintaining the care recipient's health and emotional state.

[0310] Prompt Sentence Examples

[0311] For example, you might input the following prompt into a generative AI model:

[0312] "Please explain a system that utilizes AI to reduce the burden of caregiving in an aging society. Please provide specific examples of its operation, including the processes of data collection, analysis, emotion recognition, and notification."

[0313] In this way, this system can improve the quality of care and reduce the burden on care staff through a series of processes: data collection, analysis, prediction, emotion recognition, and notification.

[0314] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0315] Step 1: Data collection

[0316] The device collects behavioral and emotional data using sensors worn by the care recipient and installed cameras. The input is real-time data obtained from the sensors and cameras, and the output is collected data that is temporarily stored on the device. As a specific example of operation, a wearable sensor worn on the arm collects heart rate data, and a camera installed in the living room captures facial expressions.

[0317] Step 2: Send data

[0318] The device sends collected data to a server at predetermined intervals using a wireless network (Wi-Fi or Bluetooth). The input is the collected data stored on the device, and the output is the data to be sent to the server. As a specific example of operation, heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[0319] Step 3: Data analysis

[0320] The server analyzes the data it receives in real time to identify the behavioral patterns and emotional state of the care recipient. The input is the collected data sent to the server, and the output is the analysis results of the behavioral patterns and emotional state. Specific examples of operation include using a machine learning model to predict the risk of falling based on walking patterns, and using an emotion recognition engine to determine stress levels based on facial expression data.

[0321] Step 4: Identifying needs and predicting behavior

[0322] The server predicts the current needs and future behavior of the care recipient based on collected and past data. The input is past data and current analysis data, and the output is predicted needs and behavior. A specific example of how it works is to predict that hydration is necessary around 3:00 pm based on data from the past week, and to detect a tendency for stress to increase at certain times of the day.

[0323] Step 5: Leverage the Emotion Engine

[0324] The server uses an emotion engine to recognize the emotional state of the care recipient from their facial expressions, tone of voice, and gestures. The input is real-time emotional data, and the output is the analysis result of their emotional state. Specific examples of operation include analyzing the tone of voice to select their favorite music, and analyzing their gestures to determine whether they are feeling anxious.

[0325] Step 6: Generate support information

[0326] Based on the analysis results, the server generates specific support information for the care staff. The input is the analysis results of behavioral patterns and emotional states, and the output is the generated support information. As a specific example of operation, it generates instructions for care actions such as "provide water and play relaxing music at 3 p.m."

[0327] Step 7: Notification methods

[0328] The support information generated by the server is sent to the care staff's device via push notification or email. The input is the generated support information, and the output is the information to be notified to the care staff. As a specific example of operation, the push notification function is used to notify the care staff's device, "Please provide fluids and play relaxing music at 3:00 PM."

[0329] Step 8: Care staff response

[0330] The user (care staff) checks the notification and performs specific actions for the care recipient based on the instructions. The input is notification information from the server, and the output is the actual care action for the care recipient. As a specific example of operation, the care staff provides water to the care recipient at 3:00 PM and plays their favorite relaxing music.

[0331] (Application example 2)

[0332] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] In an aging society, it is important to properly manage the stress and fatigue of elderly workers working in production sites, improve their working environment, and maintain their health while continuing to work efficiently. By solving this issue, it is necessary to provide an environment where elderly workers can work with peace of mind and maintain labor productivity.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0335] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the needs, behavioral patterns, and emotional state of the care recipient, means for generating predicted support information based on the identified needs, behavioral patterns, and emotional state, means for notifying the care staff of the generated support information, and means including a factory robot for supporting the care staff in taking specific actions. This makes it possible to appropriately manage stress and fatigue in elderly workers and improve their working environment.

[0336] "Care recipients" are individuals who require assistance, such as the elderly or disabled.

[0337] "Behavioral Data" means information relating to the history or patterns of an individual's activities or behavior.

[0338] "Emotional data" refers to information about an individual's emotional state as expressed through facial expressions, tone of voice, gestures, etc.

[0339] "Server" means a computer system for analyzing collected data and processing results.

[0340] "Needs" refers to the support and services that the care recipient currently requires.

[0341] A "behavioral pattern" is a pattern based on an individual's regular repetition of activities or behaviors.

[0342] "Predicted support information" refers to information about the support that will be needed in the future, generated based on the results of analyzing behavioral and emotional data.

[0343] "Notification" refers to the communication method or means for conveying support information to care staff.

[0344] A "factory robot" is an automated mechanical device that supports workers and assists with work on factory production sites.

[0345] This invention is a system that utilizes factory robots to collect behavioral and emotional data from elderly workers, analyzes this data on a server, and provides support information. The overall operation of the system and the required hardware and software are described below.

[0346] Hardware Configuration

[0347] 1. Factory robots

[0348] They are equipped with high-performance cameras and microphones to collect behavioral and emotional data from workers in the factory.

[0349] 2. Central Server (Server)

[0350] A high-performance computer for data analysis that stores and analyzes data.

[0351] 3. Care staff terminal (user)

[0352] A device, such as a tablet or smartphone, for receiving and displaying support information.

[0353] Software Configuration

[0354] 1. Data Collection Software

[0355] The video data from the camera is converted to grayscale using OpenCV and input into the Keras emotion recognition model.

[0356] 2. Data transmission software

[0357] The behavioral and emotional data obtained using Requests is sent to the server.

[0358] 3. Data Analysis Software

[0359] The received data is analyzed using Python on the server side, and the emotional state is recognized by the emotion engine, which identifies behavioral patterns and emotional states.

[0360] 4. Supporting Information Generation Software

[0361] The server identifies the worker's needs and behavioral patterns based on the analysis results and generates appropriate support information.

[0362] 5. Notification Software

[0363] The generated support information is sent as a push notification to the care staff device.

[0364] System Operation

[0365] 1. Factory robots collect behavioral and emotional data from elderly workers. Specifically, they use cameras to capture facial expressions and microphones to capture emotional data from tone of voice and gestures.

[0366] 2. The collected data is sent to a server via a wireless network.

[0367] 3. The server analyzes the received data and identifies the stress level and fatigue state of the elderly workers.

[0368] 4. Based on the analysis results, including past data, the server predicts future support needs and generates specific support information.

[0369] 5. The generated support information is sent to the care staff terminal.

[0370] 6. The caregiver checks the notification and takes specific action, such as playing relaxing music or encouraging the patient to drink more water.

[0371] Specific examples

[0372] For example, if data indicates that workers tend to feel stressed around 2 p.m., the server will send a command to the factory robot to "play relaxing music" as support information. As a result, the factory robot will play relaxing music, making it possible to reduce stress for the workers.

[0373] Prompt Sentence Examples

[0374] "Collect worker behavioral and emotional data (facial expressions, tone of voice, gestures) and send it to the server. After analyzing the data, please provide assistance in playing relaxing music to help workers reduce stress."

[0375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0376] Step 1:

[0377] Factory robots collect behavioral and emotional data from older workers.

[0378] Specifically, a camera is used to capture the worker's facial expressions, and a microphone is used to obtain emotional data from their tone of voice and gestures.

[0379] Input: Video and audio data of worker

[0380] Output: Worker behavioral and emotional data

[0381] Step 2:

[0382] The behavioral and emotional data collected by the device is transmitted to a server via a wireless network.

[0383] Input: Behavioral and emotional data

[0384] Output: Data sent to the server

[0385] Step 3:

[0386] The server analyzes the received data to identify the stress level and fatigue state of the elderly worker, and uses an emotion recognition model (generative AI model) to identify the worker's emotional state from facial expressions and tone of voice.

[0387] Input: Received data (behavioral data, emotional data)

[0388] Output: Analysis results (stress level, fatigue state, emotional state)

[0389] Step 4:

[0390] The server uses the results of analysis, including past data, to predict future support needs and generate specific support information. For example, if past data predicts that a worker is likely to feel stressed around 2 p.m., it generates instructions to play relaxing music.

[0391] Input: Analysis results, past data

[0392] Output: Generated support information (support needs, specific measures)

[0393] Step 5:

[0394] The server sends the generated support information to the care staff terminal via push notification.

[0395] Input: Generated support information

[0396] Output: Notification to nursing staff terminal

[0397] Step 6:

[0398] The user (caregiver) checks the notification and takes specific action, such as playing relaxing music or encouraging the user to drink more water.

[0399] Input: Notified support information

[0400] Output: Supportive actions for workers (playing relaxing music, encouraging hydration)

[0401] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0402] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0403] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0404] [Second embodiment]

[0405] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0406] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0407] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0408] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0409] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0410] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0411] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0412] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0413] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0414] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0415] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0416] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0417] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects the actions and behavior of care recipients in real time, analyzes this data, and provides appropriate support information to care staff.

[0418] An embodiment of this system is described below.

[0419] Data collection methods

[0420] The device collects behavioral data from sensors attached to the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and changes in facial expressions. The collected data is temporarily stored on the device.

[0421] Data transmission method

[0422] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0423] Data Analysis Methods

[0424] The server analyzes the received data to identify the current situation and behavioral patterns of the care recipient, specifically, walking patterns, meal frequency, and body temperature fluctuations, and if any abnormal behavior is detected, the information is notified to the care staff.

[0425] Needs identification and behavior prediction tools

[0426] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0427] Support information generation method

[0428] The server generates support information that is useful to care staff based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions.

[0429] Notification means

[0430] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other means. This allows the care staff to quickly take appropriate action for the care recipient.

[0431] Specific examples

[0432] For example, consider a case where a care recipient needs to drink fluids around 3:00 PM. The device collects the care recipient's behavioral data and sends it to the server. The server analyzes the data and predicts that the care recipient will need to drink fluids at 3:00 PM based on past behavioral patterns. The server then generates support information such as "Provide fluids at 3:00 PM" and sends a notification to the care staff's device. The care staff receives the notification and provides fluids at the appropriate time, thereby maintaining the care recipient's health.

[0433] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0434] The processing flow will be explained below.

[0435] Step 1:

[0436] The device collects real-time behavioral data from sensors attached to the care recipient and cameras installed in the device, detecting, for example, walking frequency, distance traveled, and changes in facial expressions.

[0437] Step 2:

[0438] The device performs initial processing of the acquired behavioral data, and performs data compression and noise filtering as needed, ensuring that only the important data is sent to the next step.

[0439] Step 3:

[0440] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using a wireless network (Wi-Fi or mobile data communication).

[0441] Step 4:

[0442] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[0443] Step 5:

[0444] The server retrieves the latest data from the database and applies analytical algorithms to process the data, specifically analyzing behavioral patterns and assessing the care recipient's health.

[0445] Step 6:

[0446] Based on the analysis results, the server identifies the care recipient's current needs and abnormal behavior. For example, if the care recipient wakes up frequently during the night, it may determine that they have a sleep disorder.

[0447] Step 7:

[0448] The server uses past behavioral data to train machine learning algorithms to predict the care recipient's future behavior, including calculating the likelihood of performing certain behaviors at certain times of the day.

[0449] Step 8:

[0450] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 p.m."

[0451] Step 9:

[0452] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[0453] Step 10:

[0454] The device displays the received notification to the care staff, and the notification content includes specific actions to take (e.g., encouraging hydration).

[0455] Step 11:

[0456] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more fluids, taking their temperature, or speaking to them.

[0457] Step 12:

[0458] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous care.

[0459] Example 1

[0460] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0461] In an aging society, reducing the burden of caregiving and providing high-quality care to those receiving care are important issues. In particular, there is a demand for an effective system that can grasp the condition of those receiving care in real time and provide appropriate support. Conventional caregiving systems lack the ability to accurately predict and promptly notify the behavior and needs of those receiving care, which places a heavy burden on caregivers and can sometimes prevent them from providing optimal support to those receiving care.

[0462] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0463] In this invention, the server includes means for acquiring biometric data of the care recipient, means for transmitting the acquired biometric data to the server, means for analyzing the received biometric data and identifying the condition and behavioral patterns of the care recipient, means for generating predicted support information based on the identified condition and behavioral patterns, and means for notifying the generated support information to the caregiver. This makes it possible to grasp the condition of the care recipient in real time and provide appropriate support promptly.

[0464] "Care recipients" refers to individuals, such as elderly people or people with disabilities, who require care services.

[0465] "Biometric data" refers to physical or physiological information obtained in real time, such as the walking patterns, dietary intake, body temperature, and changes in facial expression of the person receiving care.

[0466] A "server" refers to a computer system that receives data sent from a terminal, analyzes it, and generates necessary information.

[0467] "Terminal" refers to a device that collects data from sensors, cameras, etc. attached to the care recipient and transmits it to a server.

[0468] "Analysis" refers to the process in which the server processes the received biometric data and evaluates the condition and behavioral patterns of the person being cared for.

[0469] "Support information" refers to specific guidelines and suggestions for action that are generated based on analysis results and predictions and provided to caregivers.

[0470] "Notification" refers to a means for transmitting support information generated by the server to the caregiver's terminal and informing the caregiver.

[0471] "Caregiver" refers to an individual whose role is to provide care to a care recipient.

[0472] "Learning algorithm" refers to the computational method used by the server to predict future behavior using past biometric data.

[0473] "Real-time" means that data collection and analysis are almost instantaneous, and that responses are based on real time without delay.

[0474] The present invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system collects behavioral and biological data of care recipients in real time, analyzes this data, and provides appropriate support information to caregivers. Specific embodiments of this system are described below.

[0475] Data collection methods

[0476] The device collects biometric data from sensors worn by the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and facial expressions. The collected data is temporarily stored on the device. For example, walking pattern data is recorded every second, and the care recipient's facial expressions are captured every minute.

[0477] Data transmission method

[0478] The device sends the collected data to the server at a predetermined interval. This communication is performed over a wireless network. For example, the device sends the collected data to the server every five minutes.

[0479] Data Analysis Methods

[0480] The server analyzes the received data to identify the condition and behavioral patterns of the care recipient. Specifically, it analyzes walking patterns, meal intake frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the caregiver. The server analyzes walking patterns and evaluates the risk of tripping. If food intake data is lower than normal, an alert is generated.

[0481] Needs identification and behavior prediction tools

[0482] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's future behavior and needs. For example, if past data shows that the care recipient tends to want food at a certain time of day, the server notifies the caregiver before that time arrives. The server then uses a machine learning model to calculate the probability that the care recipient will start their walk at 10:00 a.m.

[0483] Support information generation method

[0484] The server generates support information that will be useful to caregivers based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions. For example, a specific suggestion such as "recommend taking a walk at 10:00 AM" may be generated.

[0485] Notification means

[0486] The support information generated by the server is sent to the caregiver's device. The device displays the received information to the caregiver in the form of a push notification or other format. This allows the caregiver to quickly take appropriate action for the care recipient. The device displays a notification on the caregiver's smartphone saying, "Please encourage the care recipient to take a walk at 10 a.m."

[0487] Examples and prompts

[0488] For example, consider a case where a care recipient needs hydration around 3:00 PM. The device collects the care recipient's walking data and facial expression data at 2:55 PM and sends the data to the server at 3:00 PM. The server analyzes the data and predicts that hydration will be needed at 3:00 PM based on past patterns, generating support information such as "Provide hydration at 3:00 PM." The device displays this information to the caregiver in the form of a push notification, and the caregiver provides hydration at the appropriate time. An example of a prompt sentence is "Please generate support information for when the care recipient needs hydration at 3:00 PM."

[0489] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on caregivers.

[0490] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0491] Step 1:

[0492] The device collects biometric data from sensors and cameras attached to the person being cared for.

[0493] Specifically, the device records the walking pattern of the person being cared for every second and captures their facial expressions every minute.

[0494] Input: Biometric data from sensors and cameras.

[0495] Output: Biometric data and its temporary storage.

[0496] Step 2:

[0497] The data collected by the device is temporarily stored in local storage.

[0498] Specific operations include storing walking data and facial expression data in memory or storage.

[0499] Input: Biometric data captured on the terminal.

[0500] Output: Data recorded in the device's local storage.

[0501] Step 3:

[0502] The terminal sends the collected data to the server at a predetermined interval via a wireless network.

[0503] Specifically, the device packages the data every five minutes and sends it to a server using Wi-Fi or the mobile network.

[0504] Input: Biometric data stored on the device.

[0505] Output: Data packages sent over the wireless network.

[0506] Step 4:

[0507] The server receives the data sent from the terminal.

[0508] Specifically, the server takes in data at a receiving port and stores it in a database.

[0509] Input: Data package sent from the terminal.

[0510] Output: Biometric data stored on the server.

[0511] Step 5:

[0512] The server analyzes the data it receives and identifies the condition and behavioral patterns of the person being cared for.

[0513] Specifically, the server analyzes walking patterns to assess the risk of tripping, calculates food intake frequency, and detects abnormalities.

[0514] Input: Biometric data stored on the server.

[0515] Output: Analysis result data (e.g., tripping risk, dietary intake abnormalities).

[0516] Step 6:

[0517] The server compares the collected data with past data and uses learning algorithms to predict the future behavior and needs of the care recipient.

[0518] Specifically, the server runs a machine learning model to predict whether a specific behavioral pattern exists during a specific time period.

[0519] Input: Analysis results and historical data.

[0520] Output: Predictive data about future behavior and needs.

[0521] Step 7:

[0522] The server generates support information that is useful to caregivers based on the analysis results and behavior prediction results.

[0523] As a specific operation, the server generates specific support information such as "recommend taking a walk at 10:00 AM."

[0524] Input: Predictive data about future behaviors and needs.

[0525] Output: Supporting information.

[0526] Step 8:

[0527] The support information generated by the server is sent to the terminal of the caregiver.

[0528] Specifically, the server composes the support information as a text message and sends it to the caregiver's terminal via the notification system.

[0529] Input: Generated assistance information.

[0530] Output: Support information sent to the caregiver's device.

[0531] Step 9:

[0532] The support information received by the device is displayed to the caregiver in the form of a push notification.

[0533] Specifically, the device displays a notification on the caregiver's smartphone saying, "Please encourage the person being cared for to take a walk at 10 a.m."

[0534] Input: Support information sent from the server.

[0535] Output: Assistance notification displayed to caregiver.

[0536] (Application example 1)

[0537] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0538] In an aging society, it is necessary to provide high-quality care to care recipients while reducing the burden on care staff. However, current care systems have difficulty monitoring the behavior of care recipients in real time, immediately detecting abnormal behavior, and providing appropriate support. In particular, in physical care support activities, it is time-consuming and labor-intensive for care staff to constantly monitor the condition of care recipients. Therefore, an efficient method to protect the health and safety of care recipients is required.

[0539] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0540] In this invention, the server includes means for acquiring behavioral data of the care recipient, means for transmitting the acquired behavioral data to the server, means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient, means for generating predicted support information based on the identified needs and behavioral patterns, means for notifying the care staff of the generated support information, means for monitoring the behavioral data of the care recipient in real time at the store and detecting abnormal behavior, and means for transmitting a notification to the care staff's terminal when abnormal behavior is detected. This makes it possible to efficiently monitor the condition of the care recipient, quickly detect abnormal behavior, and provide appropriate support.

[0541] "Care recipients" refers to elderly people and individuals with disabilities who require care.

[0542] "Behavioral data" refers to information about the activities of the care recipient, such as their walking patterns, dietary intake, and changes in facial expressions.

[0543] "Server" refers to the computer system that analyzes collected data and identifies behavioral patterns.

[0544] "Cloud" refers to computer resources and storage services provided over the Internet.

[0545] "Care staff" refers to professional people who provide care and support to care recipients.

[0546] "Means of data acquisition" refers to devices such as sensors and cameras used to collect behavioral data of the care recipient.

[0547] "Data transmission means" refers to the communication technology and protocol used to transmit acquired behavioral data to a server.

[0548] "Analysis means" refers to software or algorithms that process the data received by the server and identify the needs and behavioral patterns of the care recipient.

[0549] "Prediction means" refers to a learning algorithm for predicting the future behavior of the care recipient based on past data.

[0550] "Support information generation means" refers to a system that generates specific action plans to be provided to care staff based on identified needs, behavioral patterns, and predictions.

[0551] "Notification means" refers to a method or technology for transmitting the generated support information to the terminal of the care staff.

[0552] "Stores" refer to retail stores and service facilities visited by care recipients.

[0553] "Real-time monitoring means" refers to technology or devices that instantly collect and process behavioral data of care recipients.

[0554] "Abnormal behavior detection means" refers to algorithms or software that analyze collected behavioral data to identify behavior that is out of the ordinary.

[0555] "Terminal" refers to a device used by care staff to receive and display information, such as a smartphone or smart glasses.

[0556] This invention is a system that reduces the burden of caregiving in an aging society and provides high-quality care to care recipients. This system collects and analyzes behavioral data of care recipients in real time and provides appropriate support information to care staff based on that data.

[0557] Data collection methods

[0558] The terminal is connected to sensors worn by the care recipient and cameras installed in the store. These devices are used to collect the care recipient's behavioral data (walking patterns, food intake, changes in facial expressions, etc.) in real time. This behavioral data is temporarily stored on the terminal and later sent to a server.

[0559] Data transmission method

[0560] The device sends the collected behavioral data to a server at predetermined intervals. This communication utilizes a wireless network, using Wi-Fi or Bluetooth as the wireless network technology.

[0561] Data Analysis Methods

[0562] The server analyzes the received behavioral data using software such as Python, TensorFlow, OpenCV (facial recognition), Keras, and Flask (web server). The server uses data analysis to identify the current situation and behavioral patterns of the care recipient. For example, it analyzes walking patterns, meal frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the care staff.

[0563] Needs identification and behavior prediction tools

[0564] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0565] Support information generation method

[0566] Based on the analysis results and behavior prediction results, the server generates support information that will be useful to care staff. This support information is provided in the form of specific action suggestions. For example, if a care recipient needs hydration around 3:00 PM, the server might suggest providing hydration at 3:00 PM.

[0567] Notification means

[0568] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other format, allowing the care staff to quickly take appropriate action for the care recipient.

[0569] Real-time monitoring and abnormal behavior detection in stores

[0570] Using cameras and sensors installed in the store, behavioral data of care recipients is collected and monitored in real time. If an abnormal behavior is detected by the abnormal behavior detection algorithm, the information is immediately sent to the care staff's device.

[0571] Specific examples

[0572] For example, if a care recipient stops standing still for a long time while visiting a store, their behavioral data is collected via a camera and sent to a server. The server analyzes the data and determines that the behavior is abnormal. In that case, a notification is sent to the caregiver's smartphone or smart glasses, requesting that they be given a place to sit.

[0573] Prompt Sentence Examples

[0574] "We would like to develop a system that collects behavioral data of elderly people in stores and analyzes it in real time. We would like to add a function that notifies store staff of appropriate support information based on specific behavioral patterns. Please provide us with Python code that performs facial recognition, analyzes walking patterns, and detects abnormal behavior."

[0575] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0576] Step 1:

[0577] The device collects behavioral data of the care recipient. Specifically, it uses sensors worn by the care recipient and cameras installed in the store to obtain data such as walking patterns, food intake, and changes in facial expressions. The input is the care recipient's real-time behavioral data, and the output is a temporary storage of this data.

[0578] Step 2:

[0579] The device transmits the collected behavioral data to the server at predetermined intervals. This communication is carried out over a wireless network. The input is the behavioral data stored on the device, and the output is the transmission of data to the server. Specific operations include creating and transmitting data packets.

[0580] Step 3:

[0581] The behavioral data received by the server is analyzed. Software such as Python, TensorFlow, OpenCV (face recognition), and Keras is used for the analysis. The input is the behavioral data sent to the server, and the output is the current situation and behavioral patterns of the care recipient as the analysis results. Specific operations include data preprocessing, detection of abnormal behavior, and identification of needs.

[0582] Step 4:

[0583] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's current needs and future behavior. The input is past behavior data and current data, and the output is the prediction result. Specific operations include applying machine learning models, learning data, and making predictions.

[0584] Step 5:

[0585] The server generates support information useful to care staff based on the analysis results and behavior prediction results. The input is the analysis results and behavior prediction results, and the output is support information in the form of specific action suggestions. Specific operations include generating and formatting the support information.

[0586] Step 6:

[0587] The server sends the generated support information to the care staff's terminal. The input is the generated support information, and the output is a notification to the terminal. Specific operations include packetizing and transmitting the notification data.

[0588] Step 7:

[0589] The device displays the support information received by the device to the care staff. The input is the notification data sent from the server, and the output is the provision of information to the care staff. Specific operations include receiving and displaying push notifications.

[0590] Step 8:

[0591] The care staff receives the notification and takes specific actions for the care recipient based on the content of the notification. The input is the notified support information, and the output is specific support actions for the care recipient. Specific actions include the care staff taking actions as instructed.

[0592] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0593] This invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system uses not only behavioral data of care recipients but also an emotion engine to recognize their emotions and provide care based on this.

[0594] System Configuration

[0595] The system includes the following main components and functions:

[0596] 1. Data Collection Methods

[0597] The device collects behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. This data is temporarily stored on the device.

[0598] 2. Data transmission method

[0599] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0600] 3. Data Analysis Methods

[0601] The server analyzes the received data to identify the behavioral patterns and emotional state of the care recipient, for example, by analyzing walking patterns, meal frequency, and facial expressions.

[0602] 4. Needs Identification and Behavior Prediction Tools

[0603] The server predicts the current needs and future behavior of the care recipient based on collected and past data. For example, if past data indicates that the care recipient tends to want to eat at a certain time, it will notify the care staff before that time arrives.

[0604] 5. Emotion Engine

[0605] The server uses an emotion engine to recognize the emotional state (e.g., happiness, stress, anger) of the care recipient from data such as facial expressions, tone of voice, and gestures. This emotional data is also analyzed.

[0606] 6. Support Information Generation Method

[0607] The server generates specific support information for care staff based on the analysis results (behavioral patterns, emotional state) and notifies them of this information. For example, it suggests actions such as "Play relaxing music because the care recipient is feeling stressed."

[0608] 7. Means of notification

[0609] The server sends the generated support information to the care staff's device. Notifications are sent in the form of push notifications or emails. The device displays the received information to the care staff.

[0610] 8. Care staff response

[0611] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as talking to the care recipient to reduce stress or providing meals at specific times.

[0612] Specific examples

[0613] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby simultaneously maintaining the care recipient's good health and emotional state.

[0614] In this way, through a series of processes including data collection, analysis, prediction, emotion recognition, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0615] The processing flow will be explained below.

[0616] Step 1:

[0617] The device collects real-time behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. Specifically, it detects walking frequency, distance traveled, whether or not the care recipient is smiling, and voice intonation.

[0618] Step 2:

[0619] The terminal performs initial processing of the acquired data, noise filtering, and data compression, so that only important data is sent to the next step, improving communication efficiency.

[0620] Step 3:

[0621] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using Wi-Fi or mobile data communication.

[0622] Step 4:

[0623] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[0624] Step 5:

[0625] The server retrieves the latest data from the database and analyzes behavioral patterns and emotional states, applying algorithms that analyze walking patterns, frequency of eating, and changes in facial expressions and tone of voice.

[0626] Step 6:

[0627] The server uses an emotion engine to recognize the care recipient's emotional state from the acquired facial expression, tone of voice, and gesture data, for example, determining whether the care recipient is smiling or has a low voice.

[0628] Step 7:

[0629] The server uses the results of behavioral analysis and emotion recognition to identify the current needs and abnormal behaviors and emotions of the care recipient, for example, frequent nighttime wakings or high stress levels, and responds accordingly.

[0630] Step 8:

[0631] The server uses past behavioral and emotional data to train machine learning algorithms to predict the care recipient's future behavior and emotional state, including calculating the likelihood of engaging in certain behaviors at certain times of the day.

[0632] Step 9:

[0633] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 pm" or "play relaxing music because you are feeling stressed."

[0634] Step 10:

[0635] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[0636] Step 11:

[0637] The device displays the received notification to the caregiver, and the notification content includes specific actions (e.g., encouraging hydration or playing music for relaxation).

[0638] Step 12:

[0639] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more water, talking to them to reduce stress, taking their temperature, or playing music to help them relax.

[0640] Step 13:

[0641] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous and appropriate care.

[0642] Example 2

[0643] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0644] In an aging society, the burden of caregiving is increasing, while at the same time there is a demand for providing high-quality care to care recipients. However, conventional systems generally respond based only on the behavioral data of care recipients, making it difficult to provide individualized care that takes into account their emotional state. Therefore, a new system is needed that can detect emotional changes in care recipients and provide optimal care based on that.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0646] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the behavioral patterns and emotional state of the care recipient, means for generating predicted support information based on the identified behavioral patterns and emotional state, and means for notifying the generated support information to the care staff. This enables more accurate care that takes into account not only the behavioral patterns but also the emotional state of the care recipient, and reduces the burden on the care staff.

[0647] "Behavioral data of the care recipient" refers to information that records the specific actions and habits of the care recipient in their daily life, and includes data such as walking patterns, dietary intake status, and excretory behavior.

[0648] "Emotional data" refers to information that indicates the emotional state of the care recipient, and includes data obtained through facial expressions, tone of voice, gestures, and the like.

[0649] "Sensors" are devices that detect changes in the body or environment of the care recipient and collect behavioral and emotional data, and include wearable sensors and fixed cameras.

[0650] "Server" refers to a computer system that analyzes collected data, identifies the behavioral patterns and emotional state of the care recipient, and generates appropriate support information to notify care staff.

[0651] "Behavioral patterns" are data that indicate the tendency for continuity and repetition of specific behaviors in the care recipient's daily life, and include behavioral history by time.

[0652] "Emotional state" refers to the specific emotional state of the care recipient, including emotions such as happiness, stress, and anger.

[0653] "Support information" is information that includes specific instructions and advice for care staff to provide appropriate care based on the behavioral patterns and emotional state of the care recipient.

[0654] "Care staff" refers to professionals who provide direct care and support to care recipients.

[0655] "Notification means" refers to a method or device for transmitting support information from the server to the care staff, and includes push notifications, emails, etc.

[0656] "Analysis means" refers to an analysis algorithm or program for identifying the behavioral patterns and emotional state of the care recipient based on the behavioral data and emotional data received by the server.

[0657] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects and analyzes behavioral and emotional data of care recipients, generates appropriate support information based on that data, and notifies care staff.

[0658] 1. System Configuration

[0659] The system includes the following main components and functions:

[0660] Data collection methods

[0661] Data transmission method

[0662] Data Analysis Methods

[0663] Needs identification and behavior prediction tools

[0664] Emotion Engine

[0665] Support information generation method

[0666] Notification means

[0667] 2. Data Collection Methods

[0668] The device collects behavioral and emotional data via sensors attached to the care recipient or installed cameras. Specific hardware used is wearable sensors or fixed cameras. This data is temporarily stored on the device.

[0669] Example: A wearable sensor attached to the care recipient's arm collects heart rate data, while a camera installed in the living room captures facial expressions.

[0670] 3. Data Transmission Method

[0671] The device sends the collected data to the server at predetermined intervals. This communication is carried out using a wireless network (Wi-Fi or Bluetooth).

[0672] Example: Heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[0673] 4. Data Analysis Methods

[0674] The server analyzes the received data in real time to identify the behavioral patterns and emotional state of the care recipient using machine learning algorithms and emotion recognition engines.

[0675] Example: Using a machine learning model to predict the risk of falls based on the walking patterns of a care recipient, and an emotion recognition engine to determine stress levels based on facial expression data.

[0676] 5. Needs Identification and Behavior Prediction Tools

[0677] The server uses collected and past data to predict the care recipient's current needs and future behavior, including past behavioral history and emotional data trends.

[0678] Example: Using data from the past week, predict that you need to drink water around 3pm and detect trends of increased stress at certain times of the day.

[0679] 6. Emotion Engine

[0680] The server uses an emotion engine to recognize the emotional state of the care recipient from facial expressions, tone of voice, and gestures, and this information is also added to the overall analysis.

[0681] Examples: Analyzing your tone of voice, selecting your favorite music, and analyzing your gestures to determine whether you are feeling anxious.

[0682] 7. Support Information Generation Method

[0683] The server generates specific support information for the care staff based on the analysis results, and this information is formatted for transmission to the care staff's terminal.

[0684] Example: Generate specific care action instructions such as "Provide water and play relaxing music at 3 pm."

[0685] 8. Means of notification

[0686] The support information generated by the server is sent to the care staff's device via push notification or email, and the device displays the received information so that the care staff can check it.

[0687] Example: Using the push notification function, send a message to the care staff's device saying, "Please provide water and play relaxing music at 3 p.m."

[0688] Specific examples

[0689] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby maintaining the care recipient's health and emotional state.

[0690] Prompt Sentence Examples

[0691] For example, you might input the following prompt into a generative AI model:

[0692] "Please explain a system that utilizes AI to reduce the burden of caregiving in an aging society. Please provide specific examples of its operation, including the processes of data collection, analysis, emotion recognition, and notification."

[0693] In this way, this system can improve the quality of care and reduce the burden on care staff through a series of processes: data collection, analysis, prediction, emotion recognition, and notification.

[0694] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0695] Step 1: Data collection

[0696] The device collects behavioral and emotional data using sensors worn by the care recipient and installed cameras. The input is real-time data obtained from the sensors and cameras, and the output is collected data that is temporarily stored on the device. As a specific example of operation, a wearable sensor worn on the arm collects heart rate data, and a camera installed in the living room captures facial expressions.

[0697] Step 2: Send data

[0698] The device sends collected data to a server at predetermined intervals using a wireless network (Wi-Fi or Bluetooth). The input is the collected data stored on the device, and the output is the data to be sent to the server. As a specific example of operation, heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[0699] Step 3: Data analysis

[0700] The server analyzes the data it receives in real time to identify the behavioral patterns and emotional state of the care recipient. The input is the collected data sent to the server, and the output is the analysis results of the behavioral patterns and emotional state. Specific examples of operation include using a machine learning model to predict the risk of falling based on walking patterns, and using an emotion recognition engine to determine stress levels based on facial expression data.

[0701] Step 4: Identifying needs and predicting behavior

[0702] The server predicts the current needs and future behavior of the care recipient based on collected and past data. The input is past data and current analysis data, and the output is predicted needs and behavior. A specific example of how it works is to predict that hydration is necessary around 3:00 pm based on data from the past week, and to detect a tendency for stress to increase at certain times of the day.

[0703] Step 5: Leverage the Emotion Engine

[0704] The server uses an emotion engine to recognize the emotional state of the care recipient from their facial expressions, tone of voice, and gestures. The input is real-time emotional data, and the output is the analysis result of their emotional state. Specific examples of operation include analyzing the tone of voice to select their favorite music, and analyzing their gestures to determine whether they are feeling anxious.

[0705] Step 6: Generate support information

[0706] Based on the analysis results, the server generates specific support information for the care staff. The input is the analysis results of behavioral patterns and emotional states, and the output is the generated support information. As a specific example of operation, it generates instructions for care actions such as "provide water and play relaxing music at 3 p.m."

[0707] Step 7: Notification methods

[0708] The support information generated by the server is sent to the care staff's device via push notification or email. The input is the generated support information, and the output is the information to be notified to the care staff. As a specific example of operation, the push notification function is used to notify the care staff's device, "Please provide fluids and play relaxing music at 3:00 PM."

[0709] Step 8: Care staff response

[0710] The user (care staff) checks the notification and performs specific actions for the care recipient based on the instructions. The input is notification information from the server, and the output is the actual care action for the care recipient. As a specific example of operation, the care staff provides water to the care recipient at 3:00 PM and plays their favorite relaxing music.

[0711] (Application example 2)

[0712] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0713] In an aging society, it is important to properly manage the stress and fatigue of elderly workers working in production sites, improve their working environment, and maintain their health while continuing to work efficiently. By solving this issue, it is necessary to provide an environment where elderly workers can work with peace of mind and maintain labor productivity.

[0714] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0715] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the needs, behavioral patterns, and emotional state of the care recipient, means for generating predicted support information based on the identified needs, behavioral patterns, and emotional state, means for notifying the care staff of the generated support information, and means including a factory robot for supporting the care staff in taking specific actions. This makes it possible to appropriately manage stress and fatigue in elderly workers and improve their working environment.

[0716] "Care recipients" are individuals who require assistance, such as the elderly or disabled.

[0717] "Behavioral Data" means information relating to the history or patterns of an individual's activities or behavior.

[0718] "Emotional data" refers to information about an individual's emotional state as expressed through facial expressions, tone of voice, gestures, etc.

[0719] "Server" means a computer system for analyzing collected data and processing results.

[0720] "Needs" refers to the support and services that the care recipient currently requires.

[0721] A "behavioral pattern" is a pattern based on an individual's regular repetition of activities or behaviors.

[0722] "Predicted support information" refers to information about the support that will be needed in the future, generated based on the results of analyzing behavioral and emotional data.

[0723] "Notification" refers to the communication method or means for conveying support information to care staff.

[0724] A "factory robot" is an automated mechanical device that supports workers and assists with work on factory production sites.

[0725] This invention is a system that utilizes factory robots to collect behavioral and emotional data from elderly workers, analyzes this data on a server, and provides support information. The overall operation of the system and the required hardware and software are described below.

[0726] Hardware Configuration

[0727] 1. Factory robots

[0728] They are equipped with high-performance cameras and microphones to collect behavioral and emotional data from workers in the factory.

[0729] 2. Central Server (Server)

[0730] A high-performance computer for data analysis that stores and analyzes data.

[0731] 3. Care staff terminal (user)

[0732] A device, such as a tablet or smartphone, for receiving and displaying support information.

[0733] Software Configuration

[0734] 1. Data Collection Software

[0735] The video data from the camera is converted to grayscale using OpenCV and input into the Keras emotion recognition model.

[0736] 2. Data transmission software

[0737] The behavioral and emotional data obtained using Requests is sent to the server.

[0738] 3. Data Analysis Software

[0739] The received data is analyzed using Python on the server side, and the emotional state is recognized by the emotion engine, which identifies behavioral patterns and emotional states.

[0740] 4. Supporting Information Generation Software

[0741] The server identifies the worker's needs and behavioral patterns based on the analysis results and generates appropriate support information.

[0742] 5. Notification Software

[0743] The generated support information is sent as a push notification to the care staff device.

[0744] System Operation

[0745] 1. Factory robots collect behavioral and emotional data from elderly workers. Specifically, they use cameras to capture facial expressions and microphones to capture emotional data from tone of voice and gestures.

[0746] 2. The collected data is sent to a server via a wireless network.

[0747] 3. The server analyzes the received data and identifies the stress level and fatigue state of the elderly workers.

[0748] 4. Based on the analysis results, including past data, the server predicts future support needs and generates specific support information.

[0749] 5. The generated support information is sent to the care staff terminal.

[0750] 6. The caregiver checks the notification and takes specific action, such as playing relaxing music or encouraging the patient to drink more water.

[0751] Specific examples

[0752] For example, if data indicates that workers tend to feel stressed around 2 p.m., the server will send a command to the factory robot to "play relaxing music" as support information. As a result, the factory robot will play relaxing music, making it possible to reduce stress for the workers.

[0753] Prompt Sentence Examples

[0754] "Collect worker behavioral and emotional data (facial expressions, tone of voice, gestures) and send it to the server. After analyzing the data, please provide assistance in playing relaxing music to help workers reduce stress."

[0755] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0756] Step 1:

[0757] Factory robots collect behavioral and emotional data from older workers.

[0758] Specifically, a camera is used to capture the worker's facial expressions, and a microphone is used to obtain emotional data from their tone of voice and gestures.

[0759] Input: Video and audio data of worker

[0760] Output: Worker behavioral and emotional data

[0761] Step 2:

[0762] The behavioral and emotional data collected by the device is transmitted to a server via a wireless network.

[0763] Input: Behavioral and emotional data

[0764] Output: Data sent to the server

[0765] Step 3:

[0766] The server analyzes the received data to identify the stress level and fatigue state of the elderly worker, and uses an emotion recognition model (generative AI model) to identify the worker's emotional state from facial expressions and tone of voice.

[0767] Input: Received data (behavioral data, emotional data)

[0768] Output: Analysis results (stress level, fatigue state, emotional state)

[0769] Step 4:

[0770] The server uses the results of analysis, including past data, to predict future support needs and generate specific support information. For example, if past data predicts that a worker is likely to feel stressed around 2 p.m., it generates instructions to play relaxing music.

[0771] Input: Analysis results, past data

[0772] Output: Generated support information (support needs, specific measures)

[0773] Step 5:

[0774] The server sends the generated support information to the care staff terminal via push notification.

[0775] Input: Generated support information

[0776] Output: Notification to nursing staff terminal

[0777] Step 6:

[0778] The user (caregiver) checks the notification and takes specific action, such as playing relaxing music or encouraging the user to drink more water.

[0779] Input: Notified support information

[0780] Output: Supportive actions for workers (playing relaxing music, encouraging hydration)

[0781] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0782] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0783] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0784] [Third embodiment]

[0785] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0786] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0787] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0788] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0789] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0790] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0791] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0792] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0793] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0794] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0795] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0796] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0797] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects the actions and behavior of care recipients in real time, analyzes this data, and provides appropriate support information to care staff.

[0798] An embodiment of this system is described below.

[0799] Data collection methods

[0800] The device collects behavioral data from sensors attached to the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and changes in facial expressions. The collected data is temporarily stored on the device.

[0801] Data transmission method

[0802] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0803] Data Analysis Methods

[0804] The server analyzes the received data to identify the current situation and behavioral patterns of the care recipient, specifically, walking patterns, meal frequency, and body temperature fluctuations, and if any abnormal behavior is detected, the information is notified to the care staff.

[0805] Needs identification and behavior prediction tools

[0806] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0807] Support information generation method

[0808] The server generates support information that is useful to care staff based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions.

[0809] Notification means

[0810] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other means. This allows the care staff to quickly take appropriate action for the care recipient.

[0811] Specific examples

[0812] For example, consider a case where a care recipient needs to drink fluids around 3:00 PM. The device collects the care recipient's behavioral data and sends it to the server. The server analyzes the data and predicts that the care recipient will need to drink fluids at 3:00 PM based on past behavioral patterns. The server then generates support information such as "Provide fluids at 3:00 PM" and sends a notification to the care staff's device. The care staff receives the notification and provides fluids at the appropriate time, thereby maintaining the care recipient's health.

[0813] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0814] The processing flow will be explained below.

[0815] Step 1:

[0816] The device collects real-time behavioral data from sensors attached to the care recipient and cameras installed in the device, detecting, for example, walking frequency, distance traveled, and changes in facial expressions.

[0817] Step 2:

[0818] The device performs initial processing of the acquired behavioral data, and performs data compression and noise filtering as needed, ensuring that only the important data is sent to the next step.

[0819] Step 3:

[0820] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using a wireless network (Wi-Fi or mobile data communication).

[0821] Step 4:

[0822] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[0823] Step 5:

[0824] The server retrieves the latest data from the database and applies analytical algorithms to process the data, specifically analyzing behavioral patterns and assessing the care recipient's health.

[0825] Step 6:

[0826] Based on the analysis results, the server identifies the care recipient's current needs and abnormal behavior. For example, if the care recipient wakes up frequently during the night, it may determine that they have a sleep disorder.

[0827] Step 7:

[0828] The server uses past behavioral data to train machine learning algorithms to predict the care recipient's future behavior, including calculating the likelihood of performing certain behaviors at certain times of the day.

[0829] Step 8:

[0830] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 p.m."

[0831] Step 9:

[0832] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[0833] Step 10:

[0834] The device displays the received notification to the care staff, and the notification content includes specific actions to take (e.g., encouraging hydration).

[0835] Step 11:

[0836] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more fluids, taking their temperature, or speaking to them.

[0837] Step 12:

[0838] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous care.

[0839] Example 1

[0840] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0841] In an aging society, reducing the burden of caregiving and providing high-quality care to those receiving care are important issues. In particular, there is a demand for an effective system that can grasp the condition of those receiving care in real time and provide appropriate support. Conventional caregiving systems lack the ability to accurately predict and promptly notify the behavior and needs of those receiving care, which places a heavy burden on caregivers and can sometimes prevent them from providing optimal support to those receiving care.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0843] In this invention, the server includes means for acquiring biometric data of the care recipient, means for transmitting the acquired biometric data to the server, means for analyzing the received biometric data and identifying the condition and behavioral patterns of the care recipient, means for generating predicted support information based on the identified condition and behavioral patterns, and means for notifying the generated support information to the caregiver. This makes it possible to grasp the condition of the care recipient in real time and provide appropriate support promptly.

[0844] "Care recipients" refers to individuals, such as elderly people or people with disabilities, who require care services.

[0845] "Biometric data" refers to physical or physiological information obtained in real time, such as the walking patterns, dietary intake, body temperature, and changes in facial expression of the person receiving care.

[0846] A "server" refers to a computer system that receives data sent from a terminal, analyzes it, and generates necessary information.

[0847] "Terminal" refers to a device that collects data from sensors, cameras, etc. attached to the care recipient and transmits it to a server.

[0848] "Analysis" refers to the process in which the server processes the received biometric data and evaluates the condition and behavioral patterns of the person being cared for.

[0849] "Support information" refers to specific guidelines and suggestions for action that are generated based on analysis results and predictions and provided to caregivers.

[0850] "Notification" refers to a means for transmitting support information generated by the server to the caregiver's terminal and informing the caregiver.

[0851] "Caregiver" refers to an individual whose role is to provide care to a care recipient.

[0852] "Learning algorithm" refers to the computational method used by the server to predict future behavior using past biometric data.

[0853] "Real-time" means that data collection and analysis are almost instantaneous, and that responses are based on real time without delay.

[0854] The present invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system collects behavioral and biological data of care recipients in real time, analyzes this data, and provides appropriate support information to caregivers. Specific embodiments of this system are described below.

[0855] Data collection methods

[0856] The device collects biometric data from sensors worn by the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and facial expressions. The collected data is temporarily stored on the device. For example, walking pattern data is recorded every second, and the care recipient's facial expressions are captured every minute.

[0857] Data transmission method

[0858] The device sends the collected data to the server at a predetermined interval. This communication is performed over a wireless network. For example, the device sends the collected data to the server every five minutes.

[0859] Data Analysis Methods

[0860] The server analyzes the received data to identify the condition and behavioral patterns of the care recipient. Specifically, it analyzes walking patterns, meal intake frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the caregiver. The server analyzes walking patterns and evaluates the risk of tripping. If food intake data is lower than normal, an alert is generated.

[0861] Needs identification and behavior prediction tools

[0862] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's future behavior and needs. For example, if past data shows that the care recipient tends to want food at a certain time of day, the server notifies the caregiver before that time arrives. The server then uses a machine learning model to calculate the probability that the care recipient will start their walk at 10:00 a.m.

[0863] Support information generation method

[0864] The server generates support information that will be useful to caregivers based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions. For example, a specific suggestion such as "recommend taking a walk at 10:00 AM" may be generated.

[0865] Notification means

[0866] The support information generated by the server is sent to the caregiver's device. The device displays the received information to the caregiver in the form of a push notification or other format. This allows the caregiver to quickly take appropriate action for the care recipient. The device displays a notification on the caregiver's smartphone saying, "Please encourage the care recipient to take a walk at 10 a.m."

[0867] Examples and prompts

[0868] For example, consider a case where a care recipient needs hydration around 3:00 PM. The device collects the care recipient's walking data and facial expression data at 2:55 PM and sends the data to the server at 3:00 PM. The server analyzes the data and predicts that hydration will be needed at 3:00 PM based on past patterns, generating support information such as "Provide hydration at 3:00 PM." The device displays this information to the caregiver in the form of a push notification, and the caregiver provides hydration at the appropriate time. An example of a prompt sentence is "Please generate support information for when the care recipient needs hydration at 3:00 PM."

[0869] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on caregivers.

[0870] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0871] Step 1:

[0872] The device collects biometric data from sensors and cameras attached to the person being cared for.

[0873] Specifically, the device records the walking pattern of the person being cared for every second and captures their facial expressions every minute.

[0874] Input: Biometric data from sensors and cameras.

[0875] Output: Biometric data and its temporary storage.

[0876] Step 2:

[0877] The data collected by the device is temporarily stored in local storage.

[0878] Specific operations include storing walking data and facial expression data in memory or storage.

[0879] Input: Biometric data captured on the terminal.

[0880] Output: Data recorded in the device's local storage.

[0881] Step 3:

[0882] The terminal sends the collected data to the server at a predetermined interval via a wireless network.

[0883] Specifically, the device packages the data every five minutes and sends it to a server using Wi-Fi or the mobile network.

[0884] Input: Biometric data stored on the device.

[0885] Output: Data packages sent over the wireless network.

[0886] Step 4:

[0887] The server receives the data sent from the terminal.

[0888] Specifically, the server takes in data at a receiving port and stores it in a database.

[0889] Input: Data package sent from the terminal.

[0890] Output: Biometric data stored on the server.

[0891] Step 5:

[0892] The server analyzes the data it receives and identifies the condition and behavioral patterns of the person being cared for.

[0893] Specifically, the server analyzes walking patterns to assess the risk of tripping, calculates food intake frequency, and detects abnormalities.

[0894] Input: Biometric data stored on the server.

[0895] Output: Analysis result data (e.g., tripping risk, dietary intake abnormalities).

[0896] Step 6:

[0897] The server compares the collected data with past data and uses learning algorithms to predict the future behavior and needs of the care recipient.

[0898] Specifically, the server runs a machine learning model to predict whether a specific behavioral pattern exists during a specific time period.

[0899] Input: Analysis results and historical data.

[0900] Output: Predictive data about future behavior and needs.

[0901] Step 7:

[0902] The server generates support information that is useful to caregivers based on the analysis results and behavior prediction results.

[0903] As a specific operation, the server generates specific support information such as "recommend taking a walk at 10:00 AM."

[0904] Input: Predictive data about future behaviors and needs.

[0905] Output: Supporting information.

[0906] Step 8:

[0907] The support information generated by the server is sent to the terminal of the caregiver.

[0908] Specifically, the server composes the support information as a text message and sends it to the caregiver's terminal via the notification system.

[0909] Input: Generated assistance information.

[0910] Output: Support information sent to the caregiver's device.

[0911] Step 9:

[0912] The support information received by the device is displayed to the caregiver in the form of a push notification.

[0913] Specifically, the device displays a notification on the caregiver's smartphone saying, "Please encourage the person being cared for to take a walk at 10 a.m."

[0914] Input: Support information sent from the server.

[0915] Output: Assistance notification displayed to caregiver.

[0916] (Application example 1)

[0917] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0918] In an aging society, it is necessary to provide high-quality care to care recipients while reducing the burden on care staff. However, current care systems have difficulty monitoring the behavior of care recipients in real time, immediately detecting abnormal behavior, and providing appropriate support. In particular, in physical care support activities, it is time-consuming and labor-intensive for care staff to constantly monitor the condition of care recipients. Therefore, an efficient method to protect the health and safety of care recipients is required.

[0919] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0920] In this invention, the server includes means for acquiring behavioral data of the care recipient, means for transmitting the acquired behavioral data to the server, means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient, means for generating predicted support information based on the identified needs and behavioral patterns, means for notifying the care staff of the generated support information, means for monitoring the behavioral data of the care recipient in real time at the store and detecting abnormal behavior, and means for transmitting a notification to the care staff's terminal when abnormal behavior is detected. This makes it possible to efficiently monitor the condition of the care recipient, quickly detect abnormal behavior, and provide appropriate support.

[0921] "Care recipients" refers to elderly people and individuals with disabilities who require care.

[0922] "Behavioral data" refers to information about the activities of the care recipient, such as their walking patterns, dietary intake, and changes in facial expressions.

[0923] "Server" refers to the computer system that analyzes collected data and identifies behavioral patterns.

[0924] "Cloud" refers to computer resources and storage services provided over the Internet.

[0925] "Care staff" refers to professional people who provide care and support to care recipients.

[0926] "Means of data acquisition" refers to devices such as sensors and cameras used to collect behavioral data of the care recipient.

[0927] "Data transmission means" refers to the communication technology and protocol used to transmit acquired behavioral data to a server.

[0928] "Analysis means" refers to software or algorithms that process the data received by the server and identify the needs and behavioral patterns of the care recipient.

[0929] "Prediction means" refers to a learning algorithm for predicting the future behavior of the care recipient based on past data.

[0930] "Support information generation means" refers to a system that generates specific action plans to be provided to care staff based on identified needs, behavioral patterns, and predictions.

[0931] "Notification means" refers to a method or technology for transmitting the generated support information to the terminal of the care staff.

[0932] "Stores" refer to retail stores and service facilities visited by care recipients.

[0933] "Real-time monitoring means" refers to technology or devices that instantly collect and process behavioral data of care recipients.

[0934] "Abnormal behavior detection means" refers to algorithms or software that analyze collected behavioral data to identify behavior that is out of the ordinary.

[0935] "Terminal" refers to a device used by care staff to receive and display information, such as a smartphone or smart glasses.

[0936] This invention is a system that reduces the burden of caregiving in an aging society and provides high-quality care to care recipients. This system collects and analyzes behavioral data of care recipients in real time and provides appropriate support information to care staff based on that data.

[0937] Data collection methods

[0938] The terminal is connected to sensors worn by the care recipient and cameras installed in the store. These devices are used to collect the care recipient's behavioral data (walking patterns, food intake, changes in facial expressions, etc.) in real time. This behavioral data is temporarily stored on the terminal and later sent to a server.

[0939] Data transmission method

[0940] The device sends the collected behavioral data to a server at predetermined intervals. This communication utilizes a wireless network, using Wi-Fi or Bluetooth as the wireless network technology.

[0941] Data Analysis Methods

[0942] The server analyzes the received behavioral data using software such as Python, TensorFlow, OpenCV (facial recognition), Keras, and Flask (web server). The server uses data analysis to identify the current situation and behavioral patterns of the care recipient. For example, it analyzes walking patterns, meal frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the care staff.

[0943] Needs identification and behavior prediction tools

[0944] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[0945] Support information generation method

[0946] Based on the analysis results and behavior prediction results, the server generates support information that will be useful to care staff. This support information is provided in the form of specific action suggestions. For example, if a care recipient needs hydration around 3:00 PM, the server might suggest providing hydration at 3:00 PM.

[0947] Notification means

[0948] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other format, allowing the care staff to quickly take appropriate action for the care recipient.

[0949] Real-time monitoring and abnormal behavior detection in stores

[0950] Using cameras and sensors installed in the store, behavioral data of care recipients is collected and monitored in real time. If an abnormal behavior is detected by the abnormal behavior detection algorithm, the information is immediately sent to the care staff's device.

[0951] Specific examples

[0952] For example, if a care recipient stops standing still for a long time while visiting a store, their behavioral data is collected via a camera and sent to a server. The server analyzes the data and determines that the behavior is abnormal. In that case, a notification is sent to the caregiver's smartphone or smart glasses, requesting that they be given a place to sit.

[0953] Prompt Sentence Examples

[0954] "We would like to develop a system that collects behavioral data of elderly people in stores and analyzes it in real time. We would like to add a function that notifies store staff of appropriate support information based on specific behavioral patterns. Please provide us with Python code that performs facial recognition, analyzes walking patterns, and detects abnormal behavior."

[0955] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0956] Step 1:

[0957] The device collects behavioral data of the care recipient. Specifically, it uses sensors worn by the care recipient and cameras installed in the store to obtain data such as walking patterns, food intake, and changes in facial expressions. The input is the care recipient's real-time behavioral data, and the output is a temporary storage of this data.

[0958] Step 2:

[0959] The device transmits the collected behavioral data to the server at predetermined intervals. This communication is carried out over a wireless network. The input is the behavioral data stored on the device, and the output is the transmission of data to the server. Specific operations include creating and transmitting data packets.

[0960] Step 3:

[0961] The behavioral data received by the server is analyzed. Software such as Python, TensorFlow, OpenCV (face recognition), and Keras is used for the analysis. The input is the behavioral data sent to the server, and the output is the current situation and behavioral patterns of the care recipient as the analysis results. Specific operations include data preprocessing, detection of abnormal behavior, and identification of needs.

[0962] Step 4:

[0963] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's current needs and future behavior. The input is past behavior data and current data, and the output is the prediction result. Specific operations include applying machine learning models, learning data, and making predictions.

[0964] Step 5:

[0965] The server generates support information useful to care staff based on the analysis results and behavior prediction results. The input is the analysis results and behavior prediction results, and the output is support information in the form of specific action suggestions. Specific operations include generating and formatting the support information.

[0966] Step 6:

[0967] The server sends the generated support information to the care staff's terminal. The input is the generated support information, and the output is a notification to the terminal. Specific operations include packetizing and transmitting the notification data.

[0968] Step 7:

[0969] The device displays the support information received by the device to the care staff. The input is the notification data sent from the server, and the output is the provision of information to the care staff. Specific operations include receiving and displaying push notifications.

[0970] Step 8:

[0971] The care staff receives the notification and takes specific actions for the care recipient based on the content of the notification. The input is the notified support information, and the output is specific support actions for the care recipient. Specific actions include the care staff taking actions as instructed.

[0972] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0973] This invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system uses not only behavioral data of care recipients but also an emotion engine to recognize their emotions and provide care based on this.

[0974] System Configuration

[0975] The system includes the following main components and functions:

[0976] 1. Data Collection Methods

[0977] The device collects behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. This data is temporarily stored on the device.

[0978] 2. Data transmission method

[0979] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[0980] 3. Data Analysis Methods

[0981] The server analyzes the received data to identify the behavioral patterns and emotional state of the care recipient, for example, by analyzing walking patterns, meal frequency, and facial expressions.

[0982] 4. Needs Identification and Behavior Prediction Tools

[0983] The server predicts the current needs and future behavior of the care recipient based on collected and past data. For example, if past data indicates that the care recipient tends to want to eat at a certain time, it will notify the care staff before that time arrives.

[0984] 5. Emotion Engine

[0985] The server uses an emotion engine to recognize the emotional state (e.g., happiness, stress, anger) of the care recipient from data such as facial expressions, tone of voice, and gestures. This emotional data is also analyzed.

[0986] 6. Support Information Generation Method

[0987] The server generates specific support information for care staff based on the analysis results (behavioral patterns, emotional state) and notifies them of this information. For example, it suggests actions such as "Play relaxing music because the care recipient is feeling stressed."

[0988] 7. Means of notification

[0989] The server sends the generated support information to the care staff's device. Notifications are sent in the form of push notifications or emails. The device displays the received information to the care staff.

[0990] 8. Care staff response

[0991] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as talking to the care recipient to reduce stress or providing meals at specific times.

[0992] Specific examples

[0993] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby simultaneously maintaining the care recipient's good health and emotional state.

[0994] In this way, through a series of processes including data collection, analysis, prediction, emotion recognition, and notification, this system can improve the quality of care and reduce the burden on care staff.

[0995] The processing flow will be explained below.

[0996] Step 1:

[0997] The device collects real-time behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. Specifically, it detects walking frequency, distance traveled, whether or not the care recipient is smiling, and voice intonation.

[0998] Step 2:

[0999] The terminal performs initial processing of the acquired data, noise filtering, and data compression, so that only important data is sent to the next step, improving communication efficiency.

[1000] Step 3:

[1001] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using Wi-Fi or mobile data communication.

[1002] Step 4:

[1003] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[1004] Step 5:

[1005] The server retrieves the latest data from the database and analyzes behavioral patterns and emotional states, applying algorithms that analyze walking patterns, frequency of eating, and changes in facial expressions and tone of voice.

[1006] Step 6:

[1007] The server uses an emotion engine to recognize the care recipient's emotional state from the acquired facial expression, tone of voice, and gesture data, for example, determining whether the care recipient is smiling or has a low voice.

[1008] Step 7:

[1009] The server uses the results of behavioral analysis and emotion recognition to identify the current needs and abnormal behaviors and emotions of the care recipient, for example, frequent nighttime wakings or high stress levels, and responds accordingly.

[1010] Step 8:

[1011] The server uses past behavioral and emotional data to train machine learning algorithms to predict the care recipient's future behavior and emotional state, including calculating the likelihood of engaging in certain behaviors at certain times of the day.

[1012] Step 9:

[1013] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 pm" or "play relaxing music because you are feeling stressed."

[1014] Step 10:

[1015] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[1016] Step 11:

[1017] The device displays the received notification to the caregiver, and the notification content includes specific actions (e.g., encouraging hydration or playing music for relaxation).

[1018] Step 12:

[1019] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more water, talking to them to reduce stress, taking their temperature, or playing music to help them relax.

[1020] Step 13:

[1021] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous and appropriate care.

[1022] Example 2

[1023] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1024] In an aging society, the burden of caregiving is increasing, while at the same time there is a demand for providing high-quality care to care recipients. However, conventional systems generally respond based only on the behavioral data of care recipients, making it difficult to provide individualized care that takes into account their emotional state. Therefore, a new system is needed that can detect emotional changes in care recipients and provide optimal care based on that.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1026] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the behavioral patterns and emotional state of the care recipient, means for generating predicted support information based on the identified behavioral patterns and emotional state, and means for notifying the generated support information to the care staff. This enables more accurate care that takes into account not only the behavioral patterns but also the emotional state of the care recipient, and reduces the burden on the care staff.

[1027] "Behavioral data of the care recipient" refers to information that records the specific actions and habits of the care recipient in their daily life, and includes data such as walking patterns, dietary intake status, and excretory behavior.

[1028] "Emotional data" refers to information that indicates the emotional state of the care recipient, and includes data obtained through facial expressions, tone of voice, gestures, and the like.

[1029] "Sensors" are devices that detect changes in the body or environment of the care recipient and collect behavioral and emotional data, and include wearable sensors and fixed cameras.

[1030] "Server" refers to a computer system that analyzes collected data, identifies the behavioral patterns and emotional state of the care recipient, and generates appropriate support information to notify care staff.

[1031] "Behavioral patterns" are data that indicate the tendency for continuity and repetition of specific behaviors in the care recipient's daily life, and include behavioral history by time.

[1032] "Emotional state" refers to the specific emotional state of the care recipient, including emotions such as happiness, stress, and anger.

[1033] "Support information" is information that includes specific instructions and advice for care staff to provide appropriate care based on the behavioral patterns and emotional state of the care recipient.

[1034] "Care staff" refers to professionals who provide direct care and support to care recipients.

[1035] "Notification means" refers to a method or device for transmitting support information from the server to the care staff, and includes push notifications, emails, etc.

[1036] "Analysis means" refers to an analysis algorithm or program for identifying the behavioral patterns and emotional state of the care recipient based on the behavioral data and emotional data received by the server.

[1037] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects and analyzes behavioral and emotional data of care recipients, generates appropriate support information based on that data, and notifies care staff.

[1038] 1. System Configuration

[1039] The system includes the following main components and functions:

[1040] Data collection methods

[1041] Data transmission method

[1042] Data Analysis Methods

[1043] Needs identification and behavior prediction tools

[1044] Emotion Engine

[1045] Support information generation method

[1046] Notification means

[1047] 2. Data Collection Methods

[1048] The device collects behavioral and emotional data via sensors attached to the care recipient or installed cameras. Specific hardware used is wearable sensors or fixed cameras. This data is temporarily stored on the device.

[1049] Example: A wearable sensor attached to the care recipient's arm collects heart rate data, while a camera installed in the living room captures facial expressions.

[1050] 3. Data Transmission Method

[1051] The device sends the collected data to the server at predetermined intervals. This communication is carried out using a wireless network (Wi-Fi or Bluetooth).

[1052] Example: Heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[1053] 4. Data Analysis Methods

[1054] The server analyzes the received data in real time to identify the behavioral patterns and emotional state of the care recipient using machine learning algorithms and emotion recognition engines.

[1055] For example, a machine learning model can be used to predict the risk of falls based on the walking patterns of a care recipient, and an emotion recognition engine can be used to determine stress levels based on facial expression data.

[1056] 5. Needs Identification and Behavior Prediction Tools

[1057] The server uses collected and past data to predict the care recipient's current needs and future behavior, including past behavioral history and emotional data trends.

[1058] Example: Using data from the past week, predict that you need to drink water around 3pm and detect trends of increased stress at certain times of the day.

[1059] 6. Emotion Engine

[1060] The server uses an emotion engine to recognize the emotional state of the care recipient from facial expressions, tone of voice, and gestures, and this information is also added to the overall analysis.

[1061] Examples: Analyzing your tone of voice, selecting your favorite music, and analyzing your gestures to determine whether you are feeling anxious.

[1062] 7. Support Information Generation Method

[1063] The server generates specific support information for the care staff based on the analysis results, and this information is formatted for transmission to the care staff's terminal.

[1064] Example: Generate specific care action instructions such as "Provide water and play relaxing music at 3 pm."

[1065] 8. Means of notification

[1066] The support information generated by the server is sent to the care staff's device via push notification or email, and the device displays the received information so that the care staff can check it.

[1067] Example: Using the push notification function, send a message to the care staff's device saying, "Please provide water and play relaxing music at 3 p.m."

[1068] Specific examples

[1069] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby maintaining the care recipient's health and emotional state.

[1070] Prompt Sentence Examples

[1071] For example, you might input the following prompt into a generative AI model:

[1072] "Please explain a system that utilizes AI to reduce the burden of caregiving in an aging society. Please provide specific examples of its operation, including the processes of data collection, analysis, emotion recognition, and notification."

[1073] In this way, this system can improve the quality of care and reduce the burden on care staff through a series of processes: data collection, analysis, prediction, emotion recognition, and notification.

[1074] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1075] Step 1: Data collection

[1076] The device collects behavioral and emotional data using sensors worn by the care recipient and installed cameras. The input is real-time data obtained from the sensors and cameras, and the output is collected data that is temporarily stored on the device. As a specific example of operation, a wearable sensor worn on the arm collects heart rate data, and a camera installed in the living room captures facial expressions.

[1077] Step 2: Send data

[1078] The device sends collected data to a server at predetermined intervals using a wireless network (Wi-Fi or Bluetooth). The input is the collected data stored on the device, and the output is the data to be sent to the server. As a specific example of operation, heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[1079] Step 3: Data analysis

[1080] The server analyzes the data it receives in real time to identify the behavioral patterns and emotional state of the care recipient. The input is the collected data sent to the server, and the output is the analysis results of the behavioral patterns and emotional state. Specific examples of operation include using a machine learning model to predict the risk of falling based on walking patterns, and using an emotion recognition engine to determine stress levels based on facial expression data.

[1081] Step 4: Identifying needs and predicting behavior

[1082] The server predicts the current needs and future behavior of the care recipient based on collected and past data. The input is past data and current analysis data, and the output is predicted needs and behavior. A specific example of how it works is to predict that hydration is necessary around 3:00 pm based on data from the past week, and to detect a tendency for stress to increase at certain times of the day.

[1083] Step 5: Leverage the Emotion Engine

[1084] The server uses an emotion engine to recognize the emotional state of the care recipient from their facial expressions, tone of voice, and gestures. The input is real-time emotional data, and the output is the analysis result of their emotional state. Specific examples of operation include analyzing the tone of voice to select their favorite music, and analyzing their gestures to determine whether they are feeling anxious.

[1085] Step 6: Generate support information

[1086] Based on the analysis results, the server generates specific support information for the care staff. The input is the analysis results of behavioral patterns and emotional states, and the output is the generated support information. As a specific example of operation, it generates instructions for care actions such as "provide water and play relaxing music at 3 p.m."

[1087] Step 7: Notification methods

[1088] The support information generated by the server is sent to the care staff's device via push notification or email. The input is the generated support information, and the output is the information to be notified to the care staff. As a specific example of operation, the push notification function is used to notify the care staff's device, "Please provide fluids and play relaxing music at 3:00 PM."

[1089] Step 8: Care staff response

[1090] The user (care staff) checks the notification and performs specific actions for the care recipient based on the instructions. The input is notification information from the server, and the output is the actual care action for the care recipient. As a specific example of operation, the care staff provides water to the care recipient at 3:00 PM and plays their favorite relaxing music.

[1091] (Application example 2)

[1092] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1093] In an aging society, it is important to properly manage the stress and fatigue of elderly workers working in production sites, improve their working environment, and maintain their health while continuing to work efficiently. By solving this issue, it is necessary to provide an environment where elderly workers can work with peace of mind and maintain labor productivity.

[1094] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1095] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the needs, behavioral patterns, and emotional state of the care recipient, means for generating predicted support information based on the identified needs, behavioral patterns, and emotional state, means for notifying the care staff of the generated support information, and means including a factory robot for supporting the care staff in taking specific actions. This makes it possible to appropriately manage stress and fatigue in elderly workers and improve their working environment.

[1096] "Care recipients" are individuals who require assistance, such as the elderly or disabled.

[1097] "Behavioral Data" means information relating to the history or patterns of an individual's activities or behavior.

[1098] "Emotional data" refers to information about an individual's emotional state as expressed through facial expressions, tone of voice, gestures, etc.

[1099] "Server" means a computer system for analyzing collected data and processing results.

[1100] "Needs" refers to the support and services that the care recipient currently requires.

[1101] A "behavioral pattern" is a pattern based on an individual's regular repetition of activities or behaviors.

[1102] "Predicted support information" refers to information about the support that will be needed in the future, generated based on the results of analyzing behavioral and emotional data.

[1103] "Notification" refers to the communication method or means for conveying support information to care staff.

[1104] A "factory robot" is an automated mechanical device that supports workers and assists with work on factory production sites.

[1105] This invention is a system that utilizes factory robots to collect behavioral and emotional data from elderly workers, analyzes this data on a server, and provides support information. The overall operation of the system and the required hardware and software are described below.

[1106] Hardware Configuration

[1107] 1. Factory robots

[1108] They are equipped with high-performance cameras and microphones to collect behavioral and emotional data from workers in the factory.

[1109] 2. Central Server (Server)

[1110] A high-performance computer for data analysis that stores and analyzes data.

[1111] 3. Care staff terminal (user)

[1112] A device, such as a tablet or smartphone, for receiving and displaying support information.

[1113] Software Configuration

[1114] 1. Data Collection Software

[1115] The video data from the camera is converted to grayscale using OpenCV and input into the Keras emotion recognition model.

[1116] 2. Data transmission software

[1117] The behavioral and emotional data obtained using Requests is sent to the server.

[1118] 3. Data Analysis Software

[1119] The received data is analyzed using Python on the server side, and the emotional state is recognized by the emotion engine, which identifies behavioral patterns and emotional states.

[1120] 4. Supporting Information Generation Software

[1121] The server identifies the worker's needs and behavioral patterns based on the analysis results and generates appropriate support information.

[1122] 5. Notification Software

[1123] The generated support information is sent as a push notification to the care staff device.

[1124] System Operation

[1125] 1. Factory robots collect behavioral and emotional data from elderly workers. Specifically, they use cameras to capture facial expressions and microphones to capture emotional data from tone of voice and gestures.

[1126] 2. The collected data is sent to a server via a wireless network.

[1127] 3. The server analyzes the received data and identifies the stress level and fatigue state of the elderly workers.

[1128] 4. Based on the analysis results, including past data, the server predicts future support needs and generates specific support information.

[1129] 5. The generated support information is sent to the care staff terminal.

[1130] 6. The caregiver checks the notification and takes specific action, such as playing relaxing music or encouraging the patient to drink more water.

[1131] Specific examples

[1132] For example, if data indicates that workers tend to feel stressed around 2 p.m., the server will send a command to the factory robot to "play relaxing music" as support information. As a result, the factory robot will play relaxing music, making it possible to reduce stress for the workers.

[1133] Prompt Sentence Examples

[1134] "Collect worker behavioral and emotional data (facial expressions, tone of voice, gestures) and send it to the server. After analyzing the data, please provide assistance in playing relaxing music to help workers reduce stress."

[1135] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1136] Step 1:

[1137] Factory robots collect behavioral and emotional data from older workers.

[1138] Specifically, a camera is used to capture the worker's facial expressions, and a microphone is used to obtain emotional data from their tone of voice and gestures.

[1139] Input: Video and audio data of worker

[1140] Output: Worker behavioral and emotional data

[1141] Step 2:

[1142] The behavioral and emotional data collected by the device is transmitted to a server via a wireless network.

[1143] Input: Behavioral and emotional data

[1144] Output: Data sent to the server

[1145] Step 3:

[1146] The server analyzes the received data to identify the stress level and fatigue state of the elderly worker, and uses an emotion recognition model (generative AI model) to identify the worker's emotional state from facial expressions and tone of voice.

[1147] Input: Received data (behavioral data, emotional data)

[1148] Output: Analysis results (stress level, fatigue state, emotional state)

[1149] Step 4:

[1150] The server uses the results of analysis, including past data, to predict future support needs and generate specific support information. For example, if past data predicts that a worker is likely to feel stressed around 2 p.m., it generates instructions to play relaxing music.

[1151] Input: Analysis results, past data

[1152] Output: Generated support information (support needs, specific measures)

[1153] Step 5:

[1154] The server sends the generated support information to the care staff terminal via push notification.

[1155] Input: Generated support information

[1156] Output: Notification to nursing staff terminal

[1157] Step 6:

[1158] The user (caregiver) checks the notification and takes specific action, such as playing relaxing music or encouraging the user to drink more water.

[1159] Input: Notified support information

[1160] Output: Supportive actions for workers (playing relaxing music, encouraging hydration)

[1161] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1162] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1163] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1164] [Fourth embodiment]

[1165] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1166] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1167] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1168] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1169] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1170] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1171] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1172] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1173] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1174] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1175] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1176] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1177] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1178] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects the actions and behavior of care recipients in real time, analyzes this data, and provides appropriate support information to care staff.

[1179] An embodiment of this system is described below.

[1180] Data collection methods

[1181] The device collects behavioral data from sensors attached to the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and changes in facial expressions. The collected data is temporarily stored on the device.

[1182] Data transmission method

[1183] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[1184] Data Analysis Methods

[1185] The server analyzes the received data to identify the current situation and behavioral patterns of the care recipient, specifically, walking patterns, meal frequency, and body temperature fluctuations, and if any abnormal behavior is detected, the information is notified to the care staff.

[1186] Needs identification and behavior prediction tools

[1187] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[1188] Support information generation method

[1189] The server generates support information that is useful to care staff based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions.

[1190] Notification means

[1191] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other means. This allows the care staff to quickly take appropriate action for the care recipient.

[1192] Specific examples

[1193] For example, consider a case where a care recipient needs to drink fluids around 3:00 PM. The device collects the care recipient's behavioral data and sends it to the server. The server analyzes the data and predicts that the care recipient will need to drink fluids at 3:00 PM based on past behavioral patterns. The server then generates support information such as "Provide fluids at 3:00 PM" and sends a notification to the care staff's device. The care staff receives the notification and provides fluids at the appropriate time, thereby maintaining the care recipient's health.

[1194] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on care staff.

[1195] The processing flow will be explained below.

[1196] Step 1:

[1197] The device collects real-time behavioral data from sensors attached to the care recipient and cameras installed in the device, detecting, for example, walking frequency, distance traveled, and changes in facial expressions.

[1198] Step 2:

[1199] The device performs initial processing of the acquired behavioral data, and performs data compression and noise filtering as needed, ensuring that only the important data is sent to the next step.

[1200] Step 3:

[1201] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using a wireless network (Wi-Fi or mobile data communication).

[1202] Step 4:

[1203] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[1204] Step 5:

[1205] The server retrieves the latest data from the database and applies analytical algorithms to process the data, specifically analyzing behavioral patterns and assessing the care recipient's health.

[1206] Step 6:

[1207] Based on the analysis results, the server identifies the care recipient's current needs and abnormal behavior. For example, if the care recipient wakes up frequently during the night, it may determine that they have a sleep disorder.

[1208] Step 7:

[1209] The server uses past behavioral data to train machine learning algorithms to predict the care recipient's future behavior, including calculating the likelihood of performing certain behaviors at certain times of the day.

[1210] Step 8:

[1211] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 p.m."

[1212] Step 9:

[1213] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[1214] Step 10:

[1215] The device displays the received notification to the care staff, and the notification content includes specific actions to take (e.g., encouraging hydration).

[1216] Step 11:

[1217] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more fluids, taking their temperature, or speaking to them.

[1218] Step 12:

[1219] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous care.

[1220] Example 1

[1221] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1222] In an aging society, reducing the burden of caregiving and providing high-quality care to those receiving care are important issues. In particular, there is a demand for an effective system that can grasp the condition of those receiving care in real time and provide appropriate support. Conventional caregiving systems lack the ability to accurately predict and promptly notify the behavior and needs of those receiving care, which places a heavy burden on caregivers and can sometimes prevent them from providing optimal support to those receiving care.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1224] In this invention, the server includes means for acquiring biometric data of the care recipient, means for transmitting the acquired biometric data to the server, means for analyzing the received biometric data and identifying the condition and behavioral patterns of the care recipient, means for generating predicted support information based on the identified condition and behavioral patterns, and means for notifying the generated support information to the caregiver. This makes it possible to grasp the condition of the care recipient in real time and provide appropriate support promptly.

[1225] "Care recipients" refers to individuals, such as elderly people or people with disabilities, who require care services.

[1226] "Biometric data" refers to physical or physiological information obtained in real time, such as the walking patterns, dietary intake, body temperature, and changes in facial expression of the person receiving care.

[1227] A "server" refers to a computer system that receives data sent from a terminal, analyzes it, and generates necessary information.

[1228] "Terminal" refers to a device that collects data from sensors, cameras, etc. attached to the care recipient and transmits it to a server.

[1229] "Analysis" refers to the process in which the server processes the received biometric data and evaluates the condition and behavioral patterns of the person being cared for.

[1230] "Support information" refers to specific guidelines and suggestions for action that are generated based on analysis results and predictions and provided to caregivers.

[1231] "Notification" refers to a means for transmitting support information generated by the server to the caregiver's terminal and informing the caregiver.

[1232] "Caregiver" refers to an individual whose role is to provide care to a care recipient.

[1233] "Learning algorithm" refers to the computational method used by the server to predict future behavior using past biometric data.

[1234] "Real-time" means that data collection and analysis are almost instantaneous, and that responses are based on real time without delay.

[1235] The present invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system collects behavioral and biological data of care recipients in real time, analyzes this data, and provides appropriate support information to caregivers. Specific embodiments of this system are described below.

[1236] Data collection methods

[1237] The device collects biometric data from sensors worn by the care recipient and cameras installed in the device. This data includes the care recipient's walking patterns, food intake, and facial expressions. The collected data is temporarily stored on the device. For example, walking pattern data is recorded every second, and the care recipient's facial expressions are captured every minute.

[1238] Data transmission method

[1239] The device sends the collected data to the server at a predetermined interval. This communication is performed over a wireless network. For example, the device sends the collected data to the server every five minutes.

[1240] Data Analysis Methods

[1241] The server analyzes the received data to identify the condition and behavioral patterns of the care recipient. Specifically, it analyzes walking patterns, meal intake frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the caregiver. The server analyzes walking patterns and evaluates the risk of tripping. If food intake data is lower than normal, an alert is generated.

[1242] Needs identification and behavior prediction tools

[1243] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's future behavior and needs. For example, if past data shows that the care recipient tends to want food at a certain time of day, the server notifies the caregiver before that time arrives. The server then uses a machine learning model to calculate the probability that the care recipient will start their walk at 10:00 a.m.

[1244] Support information generation method

[1245] The server generates support information that will be useful to caregivers based on the analysis results and behavior prediction results. This support information is provided in the form of specific action suggestions. For example, a specific suggestion such as "recommend taking a walk at 10:00 AM" may be generated.

[1246] Notification means

[1247] The support information generated by the server is sent to the caregiver's device. The device displays the received information to the caregiver in the form of a push notification or other format. This allows the caregiver to quickly take appropriate action for the care recipient. The device displays a notification on the caregiver's smartphone saying, "Please encourage the care recipient to take a walk at 10 a.m."

[1248] Examples and prompts

[1249] For example, consider a case where a care recipient needs hydration around 3:00 PM. The device collects the care recipient's walking data and facial expression data at 2:55 PM and sends the data to the server at 3:00 PM. The server analyzes the data and predicts that hydration will be needed at 3:00 PM based on past patterns, generating support information such as "Provide hydration at 3:00 PM." The device displays this information to the caregiver in the form of a push notification, and the caregiver provides hydration at the appropriate time. An example of a prompt sentence is "Please generate support information for when the care recipient needs hydration at 3:00 PM."

[1250] In this way, through a series of processes of data collection, analysis, prediction, and notification, this system can improve the quality of care and reduce the burden on caregivers.

[1251] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1252] Step 1:

[1253] The device collects biometric data from sensors and cameras attached to the person being cared for.

[1254] Specifically, the device records the walking pattern of the person being cared for every second and captures their facial expressions every minute.

[1255] Input: Biometric data from sensors and cameras.

[1256] Output: Biometric data and its temporary storage.

[1257] Step 2:

[1258] The data collected by the device is temporarily stored in local storage.

[1259] Specific operations include storing walking data and facial expression data in memory or storage.

[1260] Input: Biometric data captured on the terminal.

[1261] Output: Data recorded in the device's local storage.

[1262] Step 3:

[1263] The terminal sends the collected data to the server at a predetermined interval via a wireless network.

[1264] Specifically, the device packages the data every five minutes and sends it to a server using Wi-Fi or the mobile network.

[1265] Input: Biometric data stored on the device.

[1266] Output: Data packages sent over the wireless network.

[1267] Step 4:

[1268] The server receives the data sent from the terminal.

[1269] Specifically, the server takes in data at a receiving port and stores it in a database.

[1270] Input: Data package sent from the terminal.

[1271] Output: Biometric data stored on the server.

[1272] Step 5:

[1273] The server analyzes the data it receives and identifies the condition and behavioral patterns of the person being cared for.

[1274] Specifically, the server analyzes walking patterns to assess the risk of tripping, calculates food intake frequency, and detects abnormalities.

[1275] Input: Biometric data stored on the server.

[1276] Output: Analysis result data (e.g., tripping risk, dietary intake abnormalities).

[1277] Step 6:

[1278] The server compares the collected data with past data and uses learning algorithms to predict the future behavior and needs of the care recipient.

[1279] Specifically, the server runs a machine learning model to predict whether a specific behavioral pattern exists during a specific time period.

[1280] Input: Analysis results and historical data.

[1281] Output: Predictive data about future behavior and needs.

[1282] Step 7:

[1283] The server generates support information that is useful to caregivers based on the analysis results and behavior prediction results.

[1284] As a specific operation, the server generates specific support information such as "recommend taking a walk at 10:00 AM."

[1285] Input: Predictive data about future behaviors and needs.

[1286] Output: Supporting information.

[1287] Step 8:

[1288] The support information generated by the server is sent to the terminal of the caregiver.

[1289] Specifically, the server composes the support information as a text message and sends it to the caregiver's terminal via the notification system.

[1290] Input: Generated assistance information.

[1291] Output: Support information sent to the caregiver's device.

[1292] Step 9:

[1293] The support information received by the device is displayed to the caregiver in the form of a push notification.

[1294] Specifically, the device displays a notification on the caregiver's smartphone saying, "Please encourage the person being cared for to take a walk at 10 a.m."

[1295] Input: Support information sent from the server.

[1296] Output: Assistance notification displayed to caregiver.

[1297] (Application example 1)

[1298] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1299] In an aging society, it is necessary to provide high-quality care to care recipients while reducing the burden on care staff. However, current care systems have difficulty monitoring the behavior of care recipients in real time, immediately detecting abnormal behavior, and providing appropriate support. In particular, in physical care support activities, it is time-consuming and labor-intensive for care staff to constantly monitor the condition of care recipients. Therefore, an efficient method to protect the health and safety of care recipients is required.

[1300] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1301] In this invention, the server includes means for acquiring behavioral data of the care recipient, means for transmitting the acquired behavioral data to the server, means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient, means for generating predicted support information based on the identified needs and behavioral patterns, means for notifying the care staff of the generated support information, means for monitoring the behavioral data of the care recipient in real time at the store and detecting abnormal behavior, and means for transmitting a notification to the care staff's terminal when abnormal behavior is detected. This makes it possible to efficiently monitor the condition of the care recipient, quickly detect abnormal behavior, and provide appropriate support.

[1302] "Care recipients" refers to elderly people and individuals with disabilities who require care.

[1303] "Behavioral data" refers to information about the activities of the care recipient, such as their walking patterns, dietary intake, and changes in facial expressions.

[1304] "Server" refers to the computer system that analyzes collected data and identifies behavioral patterns.

[1305] "Cloud" refers to computer resources and storage services provided over the Internet.

[1306] "Care staff" refers to professional people who provide care and support to care recipients.

[1307] "Means of data acquisition" refers to devices such as sensors and cameras used to collect behavioral data of the care recipient.

[1308] "Data transmission means" refers to the communication technology and protocol used to transmit acquired behavioral data to a server.

[1309] "Analysis means" refers to software or algorithms that process the data received by the server and identify the needs and behavioral patterns of the care recipient.

[1310] "Prediction means" refers to a learning algorithm for predicting the future behavior of the care recipient based on past data.

[1311] "Support information generation means" refers to a system that generates specific action plans to be provided to care staff based on identified needs, behavioral patterns, and predictions.

[1312] "Notification means" refers to a method or technology for transmitting the generated support information to the terminal of the care staff.

[1313] "Stores" refer to retail stores and service facilities visited by care recipients.

[1314] "Real-time monitoring means" refers to technology or devices that instantly collect and process behavioral data of care recipients.

[1315] "Abnormal behavior detection means" refers to algorithms or software that analyze collected behavioral data to identify behavior that is out of the ordinary.

[1316] "Terminal" refers to a device used by care staff to receive and display information, such as a smartphone or smart glasses.

[1317] This invention is a system that reduces the burden of caregiving in an aging society and provides high-quality care to care recipients. This system collects and analyzes behavioral data of care recipients in real time and provides appropriate support information to care staff based on that data.

[1318] Data collection methods

[1319] The terminal is connected to sensors worn by the care recipient and cameras installed in the store. These devices are used to collect the care recipient's behavioral data (walking patterns, food intake, changes in facial expressions, etc.) in real time. This behavioral data is temporarily stored on the terminal and later sent to a server.

[1320] Data transmission method

[1321] The device sends the collected behavioral data to a server at predetermined intervals. This communication utilizes a wireless network, using Wi-Fi or Bluetooth as the wireless network technology.

[1322] Data Analysis Methods

[1323] The server analyzes the received behavioral data using software such as Python, TensorFlow, OpenCV (facial recognition), Keras, and Flask (web server). The server uses data analysis to identify the current situation and behavioral patterns of the care recipient. For example, it analyzes walking patterns, meal frequency, and body temperature fluctuations, and if abnormal behavior is detected, it notifies the care staff.

[1324] Needs identification and behavior prediction tools

[1325] The server compares the collected data with past data and uses a learning algorithm to predict the current needs and future behavior of the care recipient. For example, if past data shows that the care recipient tends to want to eat at a certain time of day, it will notify the caregiver before that time arrives.

[1326] Support information generation method

[1327] Based on the analysis results and behavior prediction results, the server generates support information that will be useful to care staff. This support information is provided in the form of specific action suggestions. For example, if a care recipient needs hydration around 3:00 PM, the server might suggest providing hydration at 3:00 PM.

[1328] Notification means

[1329] The server sends the generated support information to the care staff's device, which then displays the received information to the care staff in the form of a push notification or other format, allowing the care staff to quickly take appropriate action for the care recipient.

[1330] Real-time monitoring and abnormal behavior detection in stores

[1331] Using cameras and sensors installed in the store, behavioral data of care recipients is collected and monitored in real time. If an abnormal behavior is detected by the abnormal behavior detection algorithm, the information is immediately sent to the care staff's device.

[1332] Specific examples

[1333] For example, if a care recipient stops standing still for a long time while visiting a store, their behavioral data is collected via a camera and sent to a server. The server analyzes the data and determines that the behavior is abnormal. In that case, a notification is sent to the caregiver's smartphone or smart glasses, requesting that they be given a place to sit.

[1334] Prompt Sentence Examples

[1335] "We would like to develop a system that collects behavioral data of elderly people in stores and analyzes it in real time. We would like to add a function that notifies store staff of appropriate support information based on specific behavioral patterns. Please provide us with Python code that performs facial recognition, analyzes walking patterns, and detects abnormal behavior."

[1336] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1337] Step 1:

[1338] The device collects behavioral data of the care recipient. Specifically, it uses sensors worn by the care recipient and cameras installed in the store to obtain data such as walking patterns, food intake, and changes in facial expressions. The input is the care recipient's real-time behavioral data, and the output is a temporary storage of this data.

[1339] Step 2:

[1340] The device transmits the collected behavioral data to the server at predetermined intervals. This communication is carried out over a wireless network. The input is the behavioral data stored on the device, and the output is the transmission of data to the server. Specific operations include creating and transmitting data packets.

[1341] Step 3:

[1342] The behavioral data received by the server is analyzed. Software such as Python, TensorFlow, OpenCV (face recognition), and Keras is used for the analysis. The input is the behavioral data sent to the server, and the output is the current situation and behavioral patterns of the care recipient as the analysis results. Specific operations include data preprocessing, detection of abnormal behavior, and identification of needs.

[1343] Step 4:

[1344] The server compares the collected data with past data and uses a learning algorithm to predict the care recipient's current needs and future behavior. The input is past behavior data and current data, and the output is the prediction result. Specific operations include applying machine learning models, learning data, and making predictions.

[1345] Step 5:

[1346] The server generates support information useful to care staff based on the analysis results and behavior prediction results. The input is the analysis results and behavior prediction results, and the output is support information in the form of specific action suggestions. Specific operations include generating and formatting the support information.

[1347] Step 6:

[1348] The server sends the generated support information to the care staff's terminal. The input is the generated support information, and the output is a notification to the terminal. Specific operations include packetizing and transmitting the notification data.

[1349] Step 7:

[1350] The device displays the support information received by the device to the care staff. The input is the notification data sent from the server, and the output is the provision of information to the care staff. Specific operations include receiving and displaying push notifications.

[1351] Step 8:

[1352] The care staff receives the notification and takes specific actions for the care recipient based on the content of the notification. The input is the notified support information, and the output is specific support actions for the care recipient. Specific actions include the care staff taking actions as instructed.

[1353] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1354] This invention is a care support system that utilizes AI to reduce the burden of caregiving in an aging society and provide high-quality care to care recipients. This system uses not only behavioral data of care recipients but also an emotion engine to recognize their emotions and provide care based on this.

[1355] System Configuration

[1356] The system includes the following main components and functions:

[1357] 1. Data Collection Methods

[1358] The device collects behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. This data is temporarily stored on the device.

[1359] 2. Data transmission method

[1360] The terminal transmits the collected data to the server at predetermined intervals. This communication is carried out over a wireless network.

[1361] 3. Data Analysis Methods

[1362] The server analyzes the received data to identify the behavioral patterns and emotional state of the care recipient, for example, by analyzing walking patterns, meal frequency, and facial expressions.

[1363] 4. Needs Identification and Behavior Prediction Tools

[1364] The server predicts the current needs and future behavior of the care recipient based on collected and past data. For example, if past data indicates that the care recipient tends to want to eat at a certain time, it will notify the care staff before that time arrives.

[1365] 5. Emotion Engine

[1366] The server uses an emotion engine to recognize the emotional state (e.g., happiness, stress, anger) of the care recipient from data such as facial expressions, tone of voice, and gestures. This emotional data is also analyzed.

[1367] 6. Support Information Generation Method

[1368] The server generates specific support information for care staff based on the analysis results (behavioral patterns, emotional state) and notifies them of this information. For example, it suggests actions such as "Play relaxing music because the care recipient is feeling stressed."

[1369] 7. Means of notification

[1370] The server sends the generated support information to the care staff's device. Notifications are sent in the form of push notifications or emails. The device displays the received information to the care staff.

[1371] 8. Care staff response

[1372] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as talking to the care recipient to reduce stress or providing meals at specific times.

[1373] Specific examples

[1374] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby simultaneously maintaining the care recipient's good health and emotional state.

[1375] In this way, through a series of processes including data collection, analysis, prediction, emotion recognition, and notification, this system can improve the quality of care and reduce the burden on care staff.

[1376] The processing flow will be explained below.

[1377] Step 1:

[1378] The device collects real-time behavioral data and emotion-related data such as facial expressions, tone of voice, and gestures from sensors attached to the care recipient and cameras installed in the device. Specifically, it detects walking frequency, distance traveled, whether or not the care recipient is smiling, and voice intonation.

[1379] Step 2:

[1380] The terminal performs initial processing of the acquired data, noise filtering, and data compression, so that only important data is sent to the next step, improving communication efficiency.

[1381] Step 3:

[1382] The device sends the initially processed data to the server at a predetermined interval. This communication is performed using Wi-Fi or mobile data communication.

[1383] Step 4:

[1384] The server receives the data sent from the device and stores it in a database, where it is saved as time-series data and used for later analysis.

[1385] Step 5:

[1386] The server retrieves the latest data from the database and analyzes behavioral patterns and emotional states, applying algorithms that analyze walking patterns, frequency of eating, and changes in facial expressions and tone of voice.

[1387] Step 6:

[1388] The server uses an emotion engine to recognize the care recipient's emotional state from the acquired facial expression, tone of voice, and gesture data, for example, determining whether the care recipient is smiling or has a low voice.

[1389] Step 7:

[1390] The server uses the results of behavioral analysis and emotion recognition to identify the current needs and abnormal behaviors and emotions of the care recipient, for example, frequent nighttime wakings or high stress levels, and responds accordingly.

[1391] Step 8:

[1392] The server uses past behavioral and emotional data to train machine learning algorithms to predict the care recipient's future behavior and emotional state, including calculating the likelihood of engaging in certain behaviors at certain times of the day.

[1393] Step 9:

[1394] The server generates specific support information based on the prediction results, such as a notification to "remind you to drink water at 3 pm" or "play relaxing music because you are feeling stressed."

[1395] Step 10:

[1396] The support information generated by the server is sent to the care staff's device in the form of push notification or email.

[1397] Step 11:

[1398] The device displays the received notification to the caregiver, and the notification content includes specific actions (e.g., encouraging hydration or playing music for relaxation).

[1399] Step 12:

[1400] The user (caregiver) checks the notification and takes specific actions for the care recipient based on the instructions, such as encouraging them to drink more water, talking to them to reduce stress, taking their temperature, or playing music to help them relax.

[1401] Step 13:

[1402] The device then collects new data on the situation after the caregiver has responded and sends it back to the server. This cycle is repeated to ensure continuous and appropriate care.

[1403] Example 2

[1404] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1405] In an aging society, the burden of caregiving is increasing, while at the same time there is a demand for providing high-quality care to care recipients. However, conventional systems generally respond based only on the behavioral data of care recipients, making it difficult to provide individualized care that takes into account their emotional state. Therefore, a new system is needed that can detect emotional changes in care recipients and provide optimal care based on that.

[1406] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1407] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the behavioral patterns and emotional state of the care recipient, means for generating predicted support information based on the identified behavioral patterns and emotional state, and means for notifying the generated support information to the care staff. This enables more accurate care that takes into account not only the behavioral patterns but also the emotional state of the care recipient, and reduces the burden on the care staff.

[1408] "Behavioral data of the care recipient" refers to information that records the specific actions and habits of the care recipient in their daily life, and includes data such as walking patterns, dietary intake status, and excretory behavior.

[1409] "Emotional data" refers to information that indicates the emotional state of the care recipient, and includes data obtained through facial expressions, tone of voice, gestures, and the like.

[1410] "Sensors" are devices that detect changes in the body or environment of the care recipient and collect behavioral and emotional data, and include wearable sensors and fixed cameras.

[1411] "Server" refers to a computer system that analyzes collected data, identifies the behavioral patterns and emotional state of the care recipient, and generates appropriate support information to notify care staff.

[1412] "Behavioral patterns" are data that indicate the tendency for continuity and repetition of specific behaviors in the care recipient's daily life, and include behavioral history by time.

[1413] "Emotional state" refers to the specific emotional state of the care recipient, including emotions such as happiness, stress, and anger.

[1414] "Support information" is information that includes specific instructions and advice for care staff to provide appropriate care based on the behavioral patterns and emotional state of the care recipient.

[1415] "Care staff" refers to professionals who provide direct care and support to care recipients.

[1416] "Notification means" refers to a method or device for transmitting support information from the server to the care staff, and includes push notifications, emails, etc.

[1417] "Analysis means" refers to an analysis algorithm or program for identifying the behavioral patterns and emotional state of the care recipient based on the behavioral data and emotional data received by the server.

[1418] This invention is a nursing care support system that utilizes AI to reduce the burden of nursing care in an aging society and provide high-quality care to care recipients. This system collects and analyzes behavioral and emotional data of care recipients, generates appropriate support information based on that data, and notifies care staff.

[1419] 1. System Configuration

[1420] The system includes the following main components and functions:

[1421] Data collection methods

[1422] Data transmission method

[1423] Data Analysis Methods

[1424] Needs identification and behavior prediction tools

[1425] Emotion Engine

[1426] Support information generation method

[1427] Notification means

[1428] 2. Data Collection Methods

[1429] The device collects behavioral and emotional data via sensors attached to the care recipient or installed cameras. Specific hardware used is wearable sensors or fixed cameras. This data is temporarily stored on the device.

[1430] Example: A wearable sensor attached to the care recipient's arm collects heart rate data, while a camera installed in the living room captures facial expressions.

[1431] 3. Data Transmission Method

[1432] The device sends the collected data to the server at predetermined intervals. This communication is carried out using a wireless network (Wi-Fi or Bluetooth).

[1433] Example: Heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[1434] 4. Data Analysis Methods

[1435] The server analyzes the received data in real time to identify the behavioral patterns and emotional state of the care recipient using machine learning algorithms and emotion recognition engines.

[1436] Example: Using a machine learning model to predict the risk of falls based on the walking patterns of a care recipient, and an emotion recognition engine to determine stress levels based on facial expression data.

[1437] 5. Needs Identification and Behavior Prediction Tools

[1438] The server uses collected and past data to predict the care recipient's current needs and future behavior, including past behavioral history and emotional data trends.

[1439] Example: Using data from the past week, predict that you need to drink water around 3pm and detect trends of increased stress at certain times of the day.

[1440] 6. Emotion Engine

[1441] The server uses an emotion engine to recognize the emotional state of the care recipient from facial expressions, tone of voice, and gestures, and this information is also added to the overall analysis.

[1442] Examples: Analyzing your tone of voice, selecting your favorite music, and analyzing your gestures to determine whether you are feeling anxious.

[1443] 7. Support Information Generation Method

[1444] The server generates specific support information for the care staff based on the analysis results, and this information is formatted for transmission to the care staff's terminal.

[1445] Example: Generate specific care action instructions such as "Provide water and play relaxing music at 3 pm."

[1446] 8. Means of notification

[1447] The support information generated by the server is sent to the care staff's device via push notification or email, and the device displays the received information so that the care staff can check it.

[1448] Example: Using the push notification function, send a message to the care staff's device saying, "Please provide water and play relaxing music at 3 p.m."

[1449] Specific examples

[1450] For example, consider a case where a care recipient needs to drink water around 3:00 PM. The device collects behavioral and emotional data, which is then sent to the server. The server analyzes the data and predicts that the care recipient will need to drink water at 3:00 PM based on their past behavioral patterns and emotional state. The emotion engine then recognizes that the care recipient's stress level is high. The server then generates support information, such as "Provide water at 3:00 PM and play music to help them relax," and notifies the caregiver's device. The user (caregiver) receives the notification and provides water and plays relaxing music at the appropriate time, thereby maintaining the care recipient's health and emotional state.

[1451] Prompt Sentence Examples

[1452] For example, you might input the following prompt into a generative AI model:

[1453] "Please explain a system that utilizes AI to reduce the burden of caregiving in an aging society. Please provide specific examples of its operation, including the processes of data collection, analysis, emotion recognition, and notification."

[1454] In this way, this system can improve the quality of care and reduce the burden on care staff through a series of processes: data collection, analysis, prediction, emotion recognition, and notification.

[1455] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1456] Step 1: Data collection

[1457] The device collects behavioral and emotional data using sensors worn by the care recipient and installed cameras. The input is real-time data obtained from the sensors and cameras, and the output is collected data that is temporarily stored on the device. As a specific example of operation, a wearable sensor worn on the arm collects heart rate data, and a camera installed in the living room captures facial expressions.

[1458] Step 2: Send data

[1459] The device sends collected data to a server at predetermined intervals using a wireless network (Wi-Fi or Bluetooth). The input is the collected data stored on the device, and the output is the data to be sent to the server. As a specific example of operation, heart rate data and facial expression data collected every hour are sent to a server using a Wi-Fi network.

[1460] Step 3: Data analysis

[1461] The server analyzes the data it receives in real time to identify the behavioral patterns and emotional state of the care recipient. The input is the collected data sent to the server, and the output is the analysis results of the behavioral patterns and emotional state. Specific examples of operation include using a machine learning model to predict the risk of falling based on walking patterns, and using an emotion recognition engine to determine stress levels based on facial expression data.

[1462] Step 4: Identifying needs and predicting behavior

[1463] The server predicts the current needs and future behavior of the care recipient based on collected and past data. The input is past data and current analysis data, and the output is predicted needs and behavior. A specific example of how it works is to predict that hydration is necessary around 3:00 pm based on data from the past week, and to detect a tendency for stress to increase at certain times of the day.

[1464] Step 5: Leverage the Emotion Engine

[1465] The server uses an emotion engine to recognize the emotional state of the care recipient from their facial expressions, tone of voice, and gestures. The input is real-time emotional data, and the output is the analysis result of their emotional state. Specific examples of operation include analyzing the tone of voice to select their favorite music, and analyzing their gestures to determine whether they are feeling anxious.

[1466] Step 6: Generate support information

[1467] Based on the analysis results, the server generates specific support information for the care staff. The input is the analysis results of behavioral patterns and emotional states, and the output is the generated support information. As a specific example of operation, it generates instructions for care actions such as "provide water and play relaxing music at 3 p.m."

[1468] Step 7: Notification methods

[1469] The support information generated by the server is sent to the care staff's device via push notification or email. The input is the generated support information, and the output is the information to be notified to the care staff. As a specific example of operation, the push notification function is used to notify the care staff's device, "Please provide fluids and play relaxing music at 3:00 PM."

[1470] Step 8: Care staff response

[1471] The user (care staff) checks the notification and performs specific actions for the care recipient based on the instructions. The input is notification information from the server, and the output is the actual care action for the care recipient. As a specific example of operation, the care staff provides water to the care recipient at 3:00 PM and plays their favorite relaxing music.

[1472] (Application example 2)

[1473] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] In an aging society, it is important to properly manage the stress and fatigue of elderly workers working in production sites, improve their working environment, and maintain their health while continuing to work efficiently. By solving this issue, it is necessary to provide an environment where elderly workers can work with peace of mind and maintain labor productivity.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1476] In this invention, the server includes means for acquiring behavioral data and emotional data of the care recipient, means for transmitting the acquired behavioral data and emotional data to the server, means for analyzing the behavioral data and emotional data received by the server and identifying the needs, behavioral patterns, and emotional state of the care recipient, means for generating predicted support information based on the identified needs, behavioral patterns, and emotional state, means for notifying the care staff of the generated support information, and means including a factory robot for supporting the care staff in taking specific actions. This makes it possible to appropriately manage stress and fatigue in elderly workers and improve their working environment.

[1477] "Care recipients" are individuals who require assistance, such as the elderly or disabled.

[1478] "Behavioral Data" means information relating to the history or patterns of an individual's activities or behavior.

[1479] "Emotional data" refers to information about an individual's emotional state as expressed through facial expressions, tone of voice, gestures, etc.

[1480] "Server" means a computer system for analyzing collected data and processing results.

[1481] "Needs" refers to the support and services that the care recipient currently requires.

[1482] A "behavioral pattern" is a pattern based on an individual's regular repetition of activities or behaviors.

[1483] "Predicted support information" refers to information about the support that will be needed in the future, generated based on the results of analyzing behavioral and emotional data.

[1484] "Notification" refers to the communication method or means for conveying support information to care staff.

[1485] A "factory robot" is an automated mechanical device that supports workers and assists with work on factory production sites.

[1486] This invention is a system that utilizes factory robots to collect behavioral and emotional data from elderly workers, analyzes this data on a server, and provides support information. The overall operation of the system and the required hardware and software are described below.

[1487] Hardware Configuration

[1488] 1. Factory robots

[1489] They are equipped with high-performance cameras and microphones to collect behavioral and emotional data from workers in the factory.

[1490] 2. Central Server (Server)

[1491] A high-performance computer for data analysis that stores and analyzes data.

[1492] 3. Care staff terminal (user)

[1493] A device, such as a tablet or smartphone, for receiving and displaying support information.

[1494] Software Configuration

[1495] 1. Data Collection Software

[1496] The video data from the camera is converted to grayscale using OpenCV and input into the Keras emotion recognition model.

[1497] 2. Data transmission software

[1498] The behavioral and emotional data obtained using Requests is sent to the server.

[1499] 3. Data Analysis Software

[1500] The received data is analyzed using Python on the server side, and the emotional state is recognized by the emotion engine, which identifies behavioral patterns and emotional states.

[1501] 4. Supporting Information Generation Software

[1502] The server identifies the worker's needs and behavioral patterns based on the analysis results and generates appropriate support information.

[1503] 5. Notification Software

[1504] The generated support information is sent as a push notification to the care staff device.

[1505] System Operation

[1506] 1. Factory robots collect behavioral and emotional data from elderly workers. Specifically, they use cameras to capture facial expressions and microphones to capture emotional data from tone of voice and gestures.

[1507] 2. The collected data is sent to a server via a wireless network.

[1508] 3. The server analyzes the received data and identifies the stress level and fatigue state of the elderly workers.

[1509] 4. Based on the analysis results, including past data, the server predicts future support needs and generates specific support information.

[1510] 5. The generated support information is sent to the care staff terminal.

[1511] 6. The caregiver checks the notification and takes specific action, such as playing relaxing music or encouraging the patient to drink more water.

[1512] Specific examples

[1513] For example, if data indicates that workers tend to feel stressed around 2 p.m., the server will send a command to the factory robot to "play relaxing music" as support information. As a result, the factory robot will play relaxing music, making it possible to reduce stress for the workers.

[1514] Prompt Sentence Examples

[1515] "Collect worker behavioral and emotional data (facial expressions, tone of voice, gestures) and send it to the server. After analyzing the data, please provide assistance in playing relaxing music to help workers reduce stress."

[1516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1517] Step 1:

[1518] Factory robots collect behavioral and emotional data from older workers.

[1519] Specifically, a camera is used to capture the worker's facial expressions, and a microphone is used to obtain emotional data from their tone of voice and gestures.

[1520] Input: Video and audio data of worker

[1521] Output: Worker behavioral and emotional data

[1522] Step 2:

[1523] The behavioral and emotional data collected by the device is transmitted to a server via a wireless network.

[1524] Input: Behavioral and emotional data

[1525] Output: Data sent to the server

[1526] Step 3:

[1527] The server analyzes the received data to identify the stress level and fatigue state of the elderly worker, and uses an emotion recognition model (generative AI model) to identify the worker's emotional state from facial expressions and tone of voice.

[1528] Input: Received data (behavioral data, emotional data)

[1529] Output: Analysis results (stress level, fatigue state, emotional state)

[1530] Step 4:

[1531] The server uses the results of analysis, including past data, to predict future support needs and generate specific support information. For example, if past data predicts that a worker is likely to feel stressed around 2 p.m., it generates instructions to play relaxing music.

[1532] Input: Analysis results, past data

[1533] Output: Generated support information (support needs, specific measures)

[1534] Step 5:

[1535] The server sends the generated support information to the care staff terminal via push notification.

[1536] Input: Generated support information

[1537] Output: Notification to nursing staff terminal

[1538] Step 6:

[1539] The user (caregiver) checks the notification and takes specific action, such as playing relaxing music or encouraging the user to drink more water.

[1540] Input: Notified support information

[1541] Output: Supportive actions for workers (playing relaxing music, encouraging hydration)

[1542] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1543] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1544] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1545] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1546] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1547] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1548] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1549] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1550] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1551] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1552] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1553] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1554] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1555] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1556] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1557] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1558] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1559] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1560] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1561] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1562] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1563] The following is further disclosed regarding the above embodiment.

[1564] (Claim 1)

[1565] A means for acquiring behavioral data of a care recipient;

[1566] means for transmitting the acquired behavioral data to a server;

[1567] A means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient;

[1568] a means for generating predicted support information based on identified needs and behavioral patterns;

[1569] A means for notifying the generated support information to the care staff;

[1570] A system including:

[1571] (Claim 2)

[1572] 2. The system according to claim 1, further comprising means for the server to learn past behavioral data and predict future behavior of the care recipient.

[1573] (Claim 3)

[1574] The system according to claim 1, further comprising a means for a care staff member to receive the notification and take specific action for the care recipient based on the content of the notification.

[1575] "Example 1"

[1576] (Claim 1)

[1577] A means for acquiring biometric data of the care recipient;

[1578] means for transmitting the acquired biometric data to a server;

[1579] A means for analyzing the received biometric data by the server and identifying the condition and behavioral patterns of the care recipient;

[1580] a means for generating predicted assistance information based on the identified state and behavior pattern;

[1581] a means for notifying a caregiver of the generated support information;

[1582] A system including:

[1583] (Claim 2)

[1584] 2. The system according to claim 1, further comprising means for the server to learn past biological data and predict future behavior of the care recipient.

[1585] (Claim 3)

[1586] 10. The system according to claim 1, further comprising means for a caregiver to receive the notification and take specific measures for the care recipient based on the content of the notification.

[1587] "Application Example 1"

[1588] (Claim 1)

[1589] A means for acquiring behavioral data of a care recipient;

[1590] means for transmitting the acquired behavioral data to a server;

[1591] A means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient;

[1592] a means for generating predicted support information based on identified needs and behavioral patterns;

[1593] A means for notifying the generated support information to the care staff;

[1594] a means for monitoring behavioral data of the care recipient in real time in the store and detecting abnormal behavior;

[1595] means for sending a notification to a terminal of a care staff member when abnormal behavior is detected;

[1596] A system including:

[1597] (Claim 2)

[1598] 2. The system according to claim 1, further comprising means for the server to learn past behavioral data and predict future behavior of the care recipient.

[1599] (Claim 3)

[1600] The system according to claim 1, further comprising a means for a care staff member to receive the notification and take specific action for the care recipient based on the content of the notification.

[1601] "Example 2: Combining Emotion Engines"

[1602] (Claim 1)

[1603] A means for acquiring behavioral data and emotional data of a care recipient;

[1604] means for transmitting the acquired behavioral data and emotion data to a server;

[1605] a means for analyzing the behavioral data and emotional data received by the server and identifying the behavioral pattern and emotional state of the care recipient;

[1606] means for generating predicted support information based on the identified behavioral patterns and emotional states;

[1607] A means for notifying the generated support information to the care staff;

[1608] A system including:

[1609] (Claim 2)

[1610] 10. The system of claim 1, wherein the server further comprises means for learning past behavioral data and emotional data and predicting future behavioral and emotional states of the care recipient.

[1611] (Claim 3)

[1612] The system according to claim 1, further comprising a means for a care staff member to receive the notification and take specific action for the care recipient based on the content of the notification.

[1613] "Application example 2 when combining emotion engines"

[1614] (Claim 1)

[1615] A means for acquiring behavioral data and emotional data of a care recipient;

[1616] means for transmitting the acquired behavioral data and emotion data to a server;

[1617] means for analyzing the behavioral data and emotional data received by the server to identify the needs, behavioral patterns, and emotional state of the care recipient;

[1618] means for generating predicted support information based on identified needs, behavioral patterns, and emotional states;

[1619] A means for notifying the generated support information to the care staff;

[1620] a means including a factory robot that assists care staff in carrying out specific actions;

[1621] A system including:

[1622] (Claim 2)

[1623] 2. The system according to claim 1, further comprising means for the server to learn past behavioral data and emotional data and predict future behaviors and emotions of the care recipient.

[1624] (Claim 3)

[1625] The system according to claim 1, further comprising a means for a care staff member to receive the notification and take specific action for the care recipient based on the content of the notification. [Explanation of symbols]

[1626] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring behavioral data of a care recipient; means for transmitting the acquired behavioral data to a server; A means for analyzing the behavioral data received by the server and identifying the needs and behavioral patterns of the care recipient; a means for generating predicted support information based on identified needs and behavioral patterns; A means for notifying the generated support information to the care staff; A system including:

2. The system according to claim 1, further comprising means for the server to learn past behavioral data and predict future behavior of the care recipient.

3. The system according to claim 1 , further comprising a means for a care staff member to receive the notification and take specific action for the care recipient based on the content of the notification.

Images