System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving video data of a care facility or home; artificial intelligence means for analyzing the video data; means for generating a suggestion to a caregiver based on the analysis; means for notifying the caregiver of the suggestion; and means for calculating and paying an information provision fee from the caregiver.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's rapidly aging society, the demand for nursing care is increasing rapidly, and the resulting burden on caregivers is becoming a serious social problem. In particular, in cases of young carers and elderly caregivers, caregivers face challenges such as social isolation and increased mental burdens. Another problem is the difficulty of quickly grasping changes in the psychological and physical state of those receiving care and providing appropriate responses. Therefore, there is a need to provide a system that reduces the burden on caregivers and enables appropriate care for those receiving care. [Means for solving the problem]

[0005] This invention provides a system that includes a means for receiving video data from nursing care facilities or homes, a means for analyzing the video data using artificial intelligence, and a means for generating and notifying caregivers of optimal recommendations based on the analysis results. Specifically, the system receives video data in real time and analyzes the facial expressions, movements, and environmental changes of the care recipient to detect changes in their psychological and physical state. Based on the analysis results, the system also notifies caregivers of emergency alerts and daily care advice, encouraging appropriate responses. The system also includes a system for calculating and paying fees for information provided by local residents for caregiving information. In this way, the burden on caregivers is reduced and the support system for the entire community is strengthened.

[0006] "Nursing care facilities and homes" refers to places where people who require nursing care, such as the elderly and disabled, live and their surrounding environments.

[0007] "Video Data" means video information collected using cameras or sensors, including real-time or recorded video.

[0008] "Artificial intelligence" refers to a system that uses technologies such as machine learning and deep learning to analyze data and make decisions.

[0009] "Analysis" refers to the process of extracting specific information and specific points from video data and evaluating and judging them.

[0010] "Recommendation" means a course of action or advice provided to a caregiver based on the analysis results.

[0011] "Notification" refers to the means by which information such as suggestions or emergency alerts is communicated to caregivers.

[0012] "Alert" means a warning or notification that immediately notifies a caregiver of an emergency involving a care recipient.

[0013] "Information provision fee" refers to the compensation paid to local residents for providing nursing care information.

[0014] "Server" refers to a central computer system that processes data for the entire system, and receives, analyzes, and generates proposals for video data.

[0015] "Terminal" refers to a device installed in a care facility or home, which collects data through cameras and sensors and transmits it to a server.

[0016] "Caregivers" refer to people who provide daily care for elderly people or people with disabilities, including family members in the home and staff at nursing homes.

[0017] "Local residents" refers to people who live outside of nursing care facilities or homes and who cooperate in providing nursing care information, and are eligible to receive fees for providing information. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[0040] System Configuration

[0041] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[0042] Server Roles

[0043] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0044] Data reception

[0045] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0046] Data analysis

[0047] The server inputs the received video data into the AI ​​model and begins analysis, which detects the facial expressions and movements of the care recipient as well as changes in the surrounding environment to determine their psychological state and risk of falling.

[0048] Proposal generation

[0049] Based on the analysis results, the server generates appropriate suggestions and advice for caregivers, such as suggestions for safe layouts, how to interact with patients depending on their psychological state, and recommendations for appropriate care products.

[0050] Emergency Alerts

[0051] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[0052] Calculation and payment of information fees

[0053] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0054] Device Role

[0055] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0056] Data collection

[0057] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0058] Data Preprocessing

[0059] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0060] Data transmission

[0061] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[0062] Real-time notifications

[0063] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[0064] User Roles

[0065] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0066] Receive advice

[0067] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[0068] Real-time support

[0069] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[0070] Information provision

[0071] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0072] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and realizes appropriate care for those receiving care.

[0073] The processing flow will be explained below.

[0074] Step 1: Data collection

[0075] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[0076] Step 2: Data Preprocessing

[0077] The device compresses the collected video data and performs any necessary filtering, such as masking personal information using facial recognition technology.

[0078] Step 3: Send data

[0079] The terminal sends the pre-processed data to the server, which involves sending the packets over the Internet.

[0080] Step 4: Receiving Data

[0081] The server receives the video data sent from the device in real time. This is completed by opening a receiving socket and receiving the data packet.

[0082] Step 5: Data analysis

[0083] The server inputs the received video data into the AI ​​model and begins analysis, which monitors the facial expressions and movements of the care recipient as well as changes in the surrounding environment to assess their psychological state and risk of falling.

[0084] Step 6: Proposal Generation

[0085] Based on the analysis results, the server generates optimal suggestions and advice for caregivers, such as changing the room layout, safety measures, and how to interact with the patient depending on their psychological state.

[0086] Step 7: Proposal Notification

[0087] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[0088] Step 8: Emergency Notification

[0089] If the server detects an emergency situation in the care recipient (such as a fall or a sudden change in physical condition), it immediately generates an alert and notifies the caregiver.

[0090] Step 9: Provide information

[0091] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout or the way they interact with the person they are caring for.

[0092] Step 10: Calculation and payment of information fee

[0093] The server calculates the fee for providing information based on the care information provided by local residents, determines the reward based on the frequency and quality of the information provided, and processes the payment.

[0094] Example 1

[0095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0096] Caring for care recipients in nursing homes and at home requires understanding their situation in real time and responding quickly, but existing systems have not been able to adequately achieve this. A particular challenge is accurately assessing the care recipient's psychological state and risk of falling, and quickly providing appropriate suggestions and emergency alerts. It is also important to properly calculate information provision fees and promote information collection with the cooperation of local residents. A new system is needed to solve these issues.

[0097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0098] In this invention, the server includes: means for receiving video data from a nursing facility or home; artificial intelligence means for analyzing the video data; means for compressing and filtering the video data; means for generating suggestions to caregivers based on the analysis results; means for notifying the caregivers of the suggestions; means for calculating and paying a fee for providing information from the caregiver; means for analyzing the facial expressions and movements of the care recipient to determine their psychological state and risk of falling; and means for generating and notifying alerts in real time when an emergency occurs. This allows for real-time understanding of the care recipient's condition and rapid response. Furthermore, it can encourage information provision from local residents and improve the overall quality of care.

[0099] "Video data from nursing care facilities and homes" refers to video and image data that records the living environment and behavior of care recipients, obtained by cameras and sensors installed in nursing care facilities and homes.

[0100] "Artificial intelligence means" refers to technologies that use algorithms such as machine learning and deep learning to analyze data and detect specific patterns or anomalies.

[0101] "Compression and filtering" refers to the process of compressing large amounts of data to reduce its size so that it can be transmitted and stored efficiently, and the process of removing and processing unnecessary data or data related to privacy.

[0102] The "means for generating suggestions to caregivers based on the analysis results" is a system that automatically generates specific guidelines for action and advice to be provided to caregivers based on information obtained from the results of data analysis.

[0103] The "means for notifying the caregiver of the suggestion" is a function for sending the generated suggestion or advice as a message to the device used by the caregiver.

[0104] The "means for calculating and paying information fees" is a system for calculating and paying compensation based on the quantity and quality of information provided by local residents.

[0105] "Means for analyzing the facial expressions and movements of care recipients to determine their psychological state and risk of falling" refers to technology that analyzes facial expressions and movement data of care recipients obtained from video data to assess their psychological state and risk of falling.

[0106] "Means for generating and notifying alerts in real time when an emergency occurs" refers to a system that generates a warning message the moment something abnormal occurs in the person being cared for and immediately notifies the caregiver.

[0107] "Data collection" refers to the acquisition of video and environmental data from within nursing facilities and homes using cameras and sensors.

[0108] "Means for monitoring the progress of data transmission and retransmitting if an error occurs" is a function that monitors the progress of collected data being transmitted to the server in real time and automatically attempts to retransmit if an error occurs.

[0109] The "means for a caregiver who receives an emergency alert to respond immediately" refers to a notification and instruction function that enables a caregiver to respond quickly when an emergency alert is received.

[0110] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[0111] System Configuration

[0112] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[0113] Server Roles

[0114] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0115] Data reception

[0116] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0117] Data analysis

[0118] The server inputs the received video data into an AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. Specifically, AI frameworks such as TENSORFLOW (registered trademark) and PyTorch are used. For example, if the AI ​​determines that the care recipient is feeling anxious, it outputs the result of the analysis.

[0119] Proposal generation

[0120] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. These suggestions include suggestions for safe layouts, how to interact with the person depending on their psychological state, and recommendations for appropriate care products. A generative AI model is used to generate the suggestions. For example, a suggestion might be, "The person receiving care seems to be feeling anxious. Create a calming environment and talk to them."

[0121] Emergency Alerts

[0122] In the event of an emergency, the server will immediately generate an alert and notify the caregiver, for example, an alert saying "The care recipient has fallen" will be sent immediately.

[0123] Calculation and payment of information fees

[0124] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0125] Device Role

[0126] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0127] Data collection

[0128] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0129] Data Preprocessing

[0130] The device compresses the collected data and performs necessary filtering to protect privacy, including masking personal information using facial recognition technology, such as blurring individuals' faces from recorded data.

[0131] Data transmission

[0132] Once preprocessing is complete, the data is sent from the device to the server, enabling efficient data utilization. The device transmits data via Wi-Fi or a wired network and has the ability to automatically attempt retransmission if an error occurs.

[0133] Real-time notifications

[0134] If an emergency occurs, the device will issue an alert on the spot, allowing caregivers to respond immediately. For example, if a fall is detected, the device will issue an alert on the spot.

[0135] User Roles

[0136] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0137] Receive advice

[0138] Caregivers can receive and check suggestions and advice from the server on their smartphones or dedicated devices. For example, they might receive a notification on their smartphone app saying, "The person being cared for may be feeling anxious. Please check immediately."

[0139] Real-time support

[0140] When an emergency alert is received, the caregiver will immediately rush to the scene and take appropriate action. For example, if an alert is received that a person is likely to fall, the caregiver will quickly rush to the care recipient's side.

[0141] Information provision

[0142] Local residents provide daily care data and proposal information to the server. This strengthens the support system throughout the community. In doing so, they can receive a fee for providing the information. For example, they can provide daily monitoring information to the server and receive a fee in return.

[0143] Prompt Sentence Examples

[0144] Below is an example of a prompt sentence to input to the generative AI model.

[0145] Sample prompt: "Detect the risk of a care recipient falling while walking in the living room and provide appropriate advice to the caregiver."

[0146] The above is a specific embodiment of the system of the present invention. This system allows caregivers to receive appropriate suggestions and respond quickly in emergencies. It also encourages information provision from local residents and improves the overall quality of care.

[0147] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0148] Step 1: Data collection

[0149] The device collects video data and environmental data using cameras and sensors installed in nursing care facilities or homes. The device continuously records the room of the care recipient, acquiring video data of the living room, for example. This data is temporarily stored in local storage. The input is real-time video data, and the output is collected video data.

[0150] Step 2: Data Preprocessing

[0151] The device compresses the collected video data and filters it for privacy purposes, for example, by using facial recognition technology to blur the faces of people other than the care recipient. The input is the collected video data, and the output is the pre-processed video data.

[0152] Step 3: Send data

[0153] The preprocessed data is sent from the device to the server. The device transmits the data over Wi-Fi or a wired network and monitors the progress of the transmission. If an error occurs, the device automatically attempts to retransmit. The input is the preprocessed video data, and the output is the data sent to the server.

[0154] Step 4: Receiving Data

[0155] The server receives video data sent from the device via the Internet. The server stores the received data in a storage device and prepares it for analysis. The input is the data sent from the device, and the output is the stored video data.

[0156] Step 5: Data analysis

[0157] The server inputs the received video data into an AI model for analysis. This analysis detects the facial expressions, movements, and environmental changes of the care recipient to determine their psychological state and risk of falling. For example, using TensorFlow or PyTorch, the analysis result may be "the care recipient is feeling anxious." The input is the received video data, and the output is the analysis result.

[0158] Step 6: Proposal Generation

[0159] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. Using a generative AI model, it creates suggestions such as, "The care recipient seems to be feeling anxious. Create a calming environment and talk to them." The input is the analysis results, and the output is the generated suggestions.

[0160] Step 7: Proposal Notification

[0161] The server notifies the caregiver of the generated suggestion by sending a notification message to the caregiver's smartphone or dedicated device. The input is the generated suggestion, and the output is the sent notification.

[0162] Step 8: Emergency Alerts

[0163] When an emergency occurs, the server immediately generates an alert and notifies the caregiver. For example, it generates and sends an alert saying, "The care recipient has fallen." The input is the emergency analysis result, and the output is an emergency alert.

[0164] Step 9: Calculation and payment of information fee

[0165] The server calculates the information provision fee according to the amount of care information provided by local residents and processes the payment. The server calculates the reward using a point system and pays via digital wallet or bank transfer. The input is the provided information data, and the output is the calculated information provision fee and payment information.

[0166] Step 10: Receive advice

[0167] The user receives and checks the suggestions and advice sent from the server on a smartphone or dedicated device. For example, they may receive a notification saying, "The care recipient may be feeling anxious. Please check immediately." The input is the notification from the server, and the output is the suggestions displayed on the user's device.

[0168] Step 11: Real-time support

[0169] When a user receives an emergency alert, they immediately rush to the scene to respond. For example, when an alert is received indicating a high possibility of a fall, a caregiver will quickly rush to the care recipient's side. The input is the emergency alert, and the output is the user's response action.

[0170] Step 12: Provide information

[0171] Users provide daily care data and suggestion information to the server. Local residents input observation data and insights into the app and send them to the server. The input is the observation data and suggestion information, and the output is the information sent to the server.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In addition to the challenge of ensuring the safety of care recipients while reducing the burden on caregivers in nursing homes and homes, it is also necessary to monitor the safety of workers in factories and improve work efficiency. However, with current systems, it is difficult to solve both challenges simultaneously.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions, means for calculating and paying an information provision fee from the caregiver, means for receiving video data from a factory and analyzing the movements of workers and machines, means for immediately issuing emergency alerts based on the analysis, and means for generating suggestions to improve work efficiency and notifying a manager. This enables safety to be ensured and efficient operation to be achieved in both nursing care and factory environments.

[0177] "Means for receiving video data from within a care facility or home" refers to a device or system that has the function of collecting video data transmitted from cameras or sensors installed in a care facility or home.

[0178] "Artificial intelligence means for analyzing the video data" refers to a system that uses AI technology to analyze the received video data and executes a process to determine the condition and situation of the care recipient or worker.

[0179] The "means for generating suggestions to caregivers based on the analysis results" refers to an algorithm or program that generates useful suggestions or advice for caregivers based on the analysis results.

[0180] The "means for notifying the caregiver of the suggestions" refers to a communication means or application that notifies the caregiver of the generated suggestions or advice to their smartphone or dedicated terminal.

[0181] The "means for calculating and paying the information provision fee from the caregiver" is a system for calculating and paying the information provision fee in return based on the information provided by the caregiver or local residents.

[0182] "Means for receiving video data from within the factory and analyzing the movements of workers and machines" refers to a system that has the function of collecting video data sent from cameras and sensors installed within the factory and analyzing the movements of workers and machines based on that data.

[0183] The "means for immediately issuing an emergency alert based on the analysis" refers to a communication means or application that generates an alert in real time and notifies an administrator if an abnormality is detected during the analysis.

[0184] The "means for generating proposals for improving work efficiency and notifying the administrator" refers to an algorithm or program that generates proposals and improvement measures for improving work efficiency from the analysis results and notifies the administrator.

[0185] "Means for collecting video data using cameras and sensors installed in nursing care facilities or homes" refers to devices or systems that have the function of collecting video data from cameras and sensors installed in nursing care facilities or homes.

[0186] The "means for compressing and filtering the video data" refers to a system that compresses collected video data and performs filtering processing to protect privacy and reduce data volume.

[0187] The "means for transmitting the collected data to the server" refers to a communication means or protocol for transmitting the pre-processed video data to the server.

[0188] It is a "Japanese company."

[0189] "Means for collecting and preprocessing data from cameras and sensors installed within the factory" refers to a system that collects video data from cameras and sensors installed within the factory, preprocesses it, and puts it in a format suitable for analysis.

[0190] The "means for transmitting collected data to a cloud service" refers to a communication means or protocol for transmitting preprocessed data to a cloud service server.

[0191] "The facial expression of the person receiving care" is a change in facial expression.

[0192] High quality data and the ability to send alerts in case of emergency are required.

[0193] It is useful for management.

[0194] Overall system configuration

[0195] This invention is a system that collects and analyzes video data from nursing homes, homes, and factories. The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in the nursing home, home, or factory and collects video data and sends it to the server. The server analyzes the video data and generates optimal suggestions and emergency alerts for caregivers and factory managers. The user receives this information and takes appropriate action.

[0196] Server Roles

[0197] The roles of the server are as follows:

[0198] 1. Data reception: The server receives video data sent via the Internet from nursing care facilities, homes, and factories, allowing real-time information on the living and working conditions of care recipients and workers.

[0199] 2. Data analysis: The server inputs the received video data into the generative AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient and caregiver, as well as changes in the surrounding environment, and determines their psychological state, risk of falls, safety risks, etc.

[0200] 3. Proposal Generation: Based on the analysis results, the server generates appropriate suggestions for caregivers and suggestions and advice for factory managers to improve work efficiency. These suggestions include recommendations for safe layouts, how to interact with employees depending on their psychological state, and efficient work methods.

[0201] 4. Emergency Alert: In case of an emergency, the server will immediately generate an alert to notify caregivers or factory managers, allowing for prompt action.

[0202] 5. Calculation and payment of information fees: The server calculates the information fees according to the amount of information provided by local residents and workers, and processes the payments. This promotes information gathering across the entire region or factory.

[0203] Device Role

[0204] The roles of the terminal are as follows:

[0205] 1. Data Collection: Cameras and sensors connected to devices collect video and environmental data 24 / 7. For example, cameras in nursing homes continuously record activity in living rooms, or cameras in factories monitor activity in work areas.

[0206] 2. Data preprocessing: The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0207] 3. Data transmission: After preprocessing, the data is transmitted from the terminal to the server, enabling efficient data utilization.

[0208] 4. Real-time notification: The device will generate an immediate alert if an emergency occurs, allowing caregivers or factory managers to respond immediately.

[0209] User Roles

[0210] The user roles are as follows:

[0211] 1. Receiving advice: Caregivers and factory managers receive and review suggestions and advice from the server on their smartphones or dedicated devices. For example, they may receive suggestions on how to deal with care recipients who are in an unstable psychological state or specific work methods to improve work efficiency.

[0212] 2. Real-time response: When an emergency alert is received, caregivers and factory managers rush to the scene and take appropriate action, such as tidying up the surrounding environment if the risk of falling is deemed high, or temporarily suspending work if the work risk is deemed high.

[0213] 3. Information provision: Local residents and factory workers provide daily care and work data to the server. This strengthens the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0214] Examples of specific examples and prompts

[0215] For example, if heavy machinery in a factory makes a dangerous move, AI will detect the abnormality and immediately send an alert to the manager via email. It will also analyze the movements of workers and make suggestions to improve work efficiency.

[0216] Example prompts to input to a generative AI model:

[0217] "Write a Python program to monitor the movement of workers and heavy equipment in a factory, assess safety in real time, and generate alerts if anything goes wrong."

[0218] In this way, the system of the present invention is realized through cooperation between servers, terminals, and users, reducing the burden on caregivers and factory managers, ensuring appropriate care and safety for care recipients and workers, and improving work efficiency.

[0219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0220] Step 1: Data collection

[0221] The device collects video data using cameras and sensors installed in nursing homes, homes, and factories. This data monitors the dynamics of care recipients, workers, and the environment 24 / 7. Specifically, cameras continuously record images of living rooms and factory work areas, while sensors detect changes in the environment. The input is real-time data from the cameras and sensors, and the output is the collected video data and sensor data.

[0222] Step 2: Data Preprocessing

[0223] The device compresses the collected data and performs necessary filtering to protect privacy. Specifically, it uses facial recognition technology to mask personal information from the collected data. This process reduces the amount of data and improves transmission efficiency. The input is the collected video data and sensor data, and the output is the compressed and filtered data.

[0224] Step 3: Send data

[0225] The terminal sends the preprocessed data to the server. Specifically, the compressed and filtered data is uploaded to the cloud server via the Internet. The input is the preprocessed data, and the output is the data sent to the server.

[0226] Step 4: Data analysis

[0227] The server analyzes the received data. Specifically, it uses a generative AI model to analyze the data and evaluate the condition of the care recipient and worker, as well as changes in the surrounding environment. This analysis determines factors such as fall risks, safety risks, and efficient work methods. The input is the data sent to the server, and the output is the analysis results.

[0228] Step 5: Proposal Generation

[0229] The server generates suggestions and advice for caregivers and factory managers based on the analysis results. Specifically, it creates suggestions including recommendations for safe layouts, how to interact with patients based on their psychological state, and efficient work methods. The input is the analysis results, and the output is the generated suggestions.

[0230] Step 6: Notification

[0231] The server notifies the caregiver or factory manager of the generated suggestions and advice. Specifically, it sends notifications to smartphones or dedicated devices and provides emergency alerts in real time. The input is the generated suggestions, and the output is the notified information.

[0232] Step 7: Emergency response

[0233] Users (caregivers and factory managers) receive emergency alerts from the server and immediately rush to the scene to take appropriate action. Specifically, if a high risk of falling is determined, they will organize the environment around the care recipient, and if a dangerous situation occurs in the factory, they will temporarily suspend work. The input is the alert notification from the server, and the output is the response action.

[0234] Step 8: Calculation and payment of information fee

[0235] The server calculates the information fee based on the information provided by local residents and workers, and processes the payment. Specifically, it evaluates the quantity and quality of the information provided, determines the reward, and makes the payment. The input is the provided information, and the output is the calculated information fee and payment processing.

[0236] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0237] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides suggestions and emergency alerts to caregivers. It also includes the calculation and payment of information provision fees from local residents.

[0238] System Configuration

[0239] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies the caregiver of optimal suggestions and emergency alerts. Furthermore, by combining it with an emotion engine, the system recognizes the emotions of the care recipient and makes suggestions based on those emotions. The user receives this information and takes appropriate action.

[0240] Server Roles

[0241] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0242] Data reception

[0243] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0244] Data analysis

[0245] The server inputs the received video data into the AI ​​and emotion engine and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. The emotion engine also simultaneously analyzes the care recipient's emotions.

[0246] Proposal generation

[0247] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver, such as suggestions for safe layouts, how to interact with the patient based on their psychological state, recommendations for appropriate care products, and how to respond based on their emotions.

[0248] sentiment analysis

[0249] The emotion engine recognizes and determines the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling anxious or stressed, the analysis results are fed back to the server.

[0250] Emergency Alerts

[0251] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[0252] Calculation and payment of information fees

[0253] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0254] Device Role

[0255] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0256] Data collection

[0257] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0258] Data Preprocessing

[0259] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0260] Data transmission

[0261] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[0262] Real-time notifications

[0263] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[0264] User Roles

[0265] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0266] Receive advice

[0267] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[0268] Emotion-based responses

[0269] Based on the analysis results of the emotion engine, the caregiver will respond according to the care recipient's emotions. For example, if the care recipient is feeling stressed, the system will suggest actions such as playing relaxing music.

[0270] Real-time support

[0271] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[0272] Information provision

[0273] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0274] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and provides appropriate care to those receiving care. Furthermore, by combining it with an emotion engine, it becomes possible to provide advanced responses based on the emotions of the care recipient.

[0275] The processing flow will be explained below.

[0276] Step 1: Data collection

[0277] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[0278] Step 2: Data Preprocessing

[0279] The device compresses the collected video data and performs necessary filtering (such as masking personal information using facial recognition technology) to ensure that the data complies with privacy standards.

[0280] Step 3: Send data

[0281] The terminal transmits the pre-processed data to the server. This transmission is carried out over the Internet and arrives at the server as packets.

[0282] Step 4: Receiving Data

[0283] The server receives the video data transmitted from the terminal via the Internet in real time and receives the data packets using a receiving socket.

[0284] Step 5: Data analysis

[0285] The server inputs the received video data into the AI ​​model and emotion engine and begins analysis. This analysis monitors the care recipient's facial expressions, movements, and changes in the surrounding environment to evaluate their psychological state, risk of falling, and emotions (e.g., joy, anger, sadness, and happiness).

[0286] Step 6: Sentiment analysis

[0287] The server uses an emotion engine to recognize the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling stressed, the emotion engine immediately sends the analysis results back to the server.

[0288] Step 7: Proposal Generation

[0289] Based on the results of data analysis and sentiment analysis, the server generates optimal suggestions and advice for caregivers, including suggestions for safe layouts, how to interact with patients based on their psychological state, recommendations for appropriate caregiving products, and emotional responses.

[0290] Step 8: Proposal Notification

[0291] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[0292] Step 9: Emergency Notification

[0293] If the server detects an emergency situation in the care recipient (e.g., a fall or a sudden change in physical condition), it immediately generates an emergency alert and notifies the caregiver, enabling a prompt response.

[0294] Step 10: Provide information

[0295] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout and how they interact with the care recipient based on their emotions.

[0296] Step 11: Calculation and payment of information fee

[0297] The server calculates the fee for providing information based on the care information provided by local residents. It determines the fee based on the frequency and quality of the information provided and processes the payment. This promotes information collection throughout the community.

[0298] Example 2

[0299] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0300] To improve the quality of care in nursing facilities and homes, it is necessary to grasp the condition of care recipients in real time and respond promptly and appropriately. However, currently, there are limited means of understanding the emotions and condition of care recipients, which may result in delayed responses to emergencies. In addition, there is a lack of a system for efficiently utilizing information provided by local residents and appropriately rewarding them for providing that information. A system is needed to solve these problems and provide a safe and secure environment for both caregivers and care recipients.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0302] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for analyzing the facial expressions, movements, and environmental changes of the care recipient in the video data and collected data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions and emergency alerts, and means for calculating and paying fees for information provided by the caregivers. This enables real-time understanding of the care recipient's condition and recognition of their emotions, enabling prompt and appropriate responses. Furthermore, by appropriately managing and paying fees for information provided by local residents, the quality and quantity of information can be improved, strengthening the nursing care support system throughout the community.

[0303] "Video data from nursing homes and homes" refers to video and environmental information obtained from cameras and sensors installed in nursing homes and homes.

[0304] An "artificial intelligence means" is a computer system that performs data analysis using machine learning algorithms and deep learning.

[0305] The "means for generating suggestions" is a system that generates appropriate suggestions and advice for the caregiver based on the analyzed data.

[0306] "Means of notifying caregivers" refers to a system that notifies caregivers of suggestions and emergency alerts via their smartphones or dedicated devices.

[0307] The "means for calculating and paying information fees" is a system that calculates fees based on the quantity and quality of information provided by local residents and pays them electronically.

[0308] "Means for collecting video data using cameras and sensors" refers to a system that collects data using cameras and sensors installed in nursing facilities and homes.

[0309] "Means for compressing data and filtering for privacy protection" refers to technologies that compress collected video data for efficient storage and transmission, and perform masking and filtering to protect personal information.

[0310] "Means for transmitting data to a server" refers to a mechanism for transmitting collected data to a server via the Internet.

[0311] The "emotion engine" is an algorithm that analyzes and determines the emotions of the person receiving care from their facial expressions and voice.

[0312] "Means for notifying caregivers of emergency alerts in real time" refers to a system that immediately sends an alert to caregivers when an emergency situation is detected.

[0313] The system of the present invention receives video data from nursing homes and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides optimal suggestions and emergency alerts to caregivers. It also calculates and pays rewards for information provided by local residents. The detailed configuration and operation of the system are described below.

[0314] Server Roles

[0315] The server acts as a central device and performs the following processes:

[0316] 1. Data Reception

[0317] The server receives video data sent from care facilities and homes via the Internet.

[0318] Specific hardware: A high-performance server is required, such as a data center server equipped with an Intel Xeon processor.

[0319] Specific software: RTSP (Real-Time Streaming Protocol) is used for data streaming.

[0320] 2. Data Analysis

[0321] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[0322] Specific hardware: A server equipped with a GPU (NVIDIA Tesla) is used.

[0323] Specific software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[0324] 3. Proposal generation

[0325] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver.

[0326] Specific software: A rule-based expert system is used to generate recommendations based on the analysis results.

[0327] 4. Sentiment analysis

[0328] The emotion engine identifies the emotions of the care recipient and feeds the results back to the server.

[0329] Specific software: Libraries such as OpenCV and Amazon Rekognition are used for emotion recognition.

[0330] 5. Emergency Alerts

[0331] In the event of an emergency, the server will instantly generate an alert and notify the caregiver.

[0332] Specific operation: Use an anomaly detection algorithm (e.g., a fall detection algorithm).

[0333] 6. Calculation and payment of information fees

[0334] The server calculates and pays rewards based on the amount of information provided by local residents.

[0335] Specific software: SQL databases and electronic payment systems (PayPal API and Stripe API).

[0336] Device Role

[0337] The devices are installed in nursing homes and homes and perform the following functions:

[0338] 1. Data Collection

[0339] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[0340] Specific hardware: IP cameras (e.g., Axis), various sensors (e.g., temperature and humidity sensors).

[0341] Example: An in-room camera continuously records what is happening in the living room.

[0342] 2. Data Preprocessing

[0343] The collected data is compressed and filtered for privacy purposes.

[0344] Specific software: Personal information masking using facial recognition technology and H.264 / HEVC compression technology.

[0345] 3. Data Transmission

[0346] The pre-processed data is sent to the server.

[0347] Specific software: Data is transmitted using HTTPS (Hypertext Transfer Protocol Secure).

[0348] 4. Real-time notifications

[0349] If an emergency occurs, an alert will be generated immediately.

[0350] Specific operation: When the device detects data that exceeds a certain threshold, it will sound an alert using the built-in speaker and light.

[0351] User Roles

[0352] Users interact with the system as caregivers or local residents and perform the following functions:

[0353] 1. Receiving advice

[0354] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[0355] Example: Caregivers can view the suggestions on a smartphone app.

[0356] 2. Emotional responses

[0357] Caregivers respond appropriately based on the analysis results of the emotion engine.

[0358] Example: If the care recipient is feeling stressed, the app will suggest playing relaxing music, and the user can follow the suggestion.

[0359] 3. Real-time support

[0360] If an emergency alert is received, we will rush to the scene immediately.

[0361] Example: A caregiver is notified of a fall risk and rushes to the scene to respond.

[0362] 4. Information provision

[0363] Local residents provide daily care data and suggestion information to the server and receive rewards.

[0364] Example: Residents enter their daily observations and suggestions through a dedicated smartphone app, which are then sent to a server. Rewards are then paid at the end of the month.

[0365] Prompt Sentence Examples

[0366] "Please suggest appropriate ways to respond when the person you are caring for is feeling anxious."

[0367] By combining the above elements, a more advanced and secure care system can be realized for both caregivers and those receiving care.

[0368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0369] Server Processing

[0370] Step 1: Receiving data

[0371] The server receives video data transmitted from care facilities and homes via the Internet.

[0372] Input: Real-time streaming data transmitted via RTSP.

[0373] Output: Received video data.

[0374] Specific operation: The server monitors the specified IP address and port and captures streaming data in real time.

[0375] Step 2: Data analysis

[0376] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[0377] Input: Received video data.

[0378] Output: Analysis results (e.g., facial expressions, movements, and environmental information of the care recipient).

[0379] Specific operation: The AI ​​module processes the video data frame by frame and performs facial recognition and movement analysis using deep learning (TensorFlow or PyTorch). The emotion engine (Amazon Rekognition, etc.) analyzes the facial expressions and voice data of the care recipient to identify their emotions.

[0380] Step 3: Proposal Generation

[0381] The server generates appropriate suggestions and advice for the caregiver based on the analysis results.

[0382] Input: Results of data analysis.

[0383] Output: Suggestions and advice.

[0384] How it works: A rule-based expert system generates suggestions based on the analysis results and in accordance with predefined patterns and rules.

[0385] Step 4: Emergency alert generation

[0386] If the server detects an emergency, it will immediately generate an alert and notify the caregiver.

[0387] Input: Results of data analysis (detection of emergency situations).

[0388] Output: Emergency alert notification.

[0389] Specific behavior: Anomaly detection algorithms identify high-risk behaviors (e.g., falls) in real time and send emergency notifications via email or SMS.

[0390] Step 5: Calculation and payment of information fee

[0391] The server calculates and pays rewards based on the amount of information provided by local residents.

[0392] Input: Informational log.

[0393] Output: Calculated reward amount and notification of payment completion.

[0394] Specific operation: The server analyzes the information provision log in the database, calculates the reward amount, and then pays the reward using an electronic payment system (PayPal or Stripe).

[0395] Terminal handling

[0396] Step 1: Data collection

[0397] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[0398] Input: The care recipient's real-time living environment.

[0399] Output: Collected video and environmental data.

[0400] Specific operation: IP cameras and temperature and humidity sensors continuously collect data and temporarily store it locally.

[0401] Step 2: Data Preprocessing

[0402] The device compresses the collected data and filters it for privacy purposes.

[0403] Input: Collected video and environmental data.

[0404] Output: Preprocessed data.

[0405] Specific operation: Uses facial recognition algorithms to mask personal information and compresses data using H.264 / HEVC compression technology.

[0406] Step 3: Send data

[0407] The pre-processed data is sent to a server.

[0408] Input: Preprocessed data.

[0409] Output: The data sent to the server.

[0410] Specific operation: Encrypts data using the HTTPS protocol and sends a POST request to the specified server URL.

[0411] Step 4: Real-time notifications

[0412] In the event of an emergency, the device will issue an alert on the spot.

[0413] Input: Emergency detection.

[0414] Output: Alert notification.

[0415] Specific operation: When the sensor inside the device detects data that exceeds the threshold, an alarm will sound from the built-in speaker and an emergency message will be displayed on the screen.

[0416] User Action

[0417] Step 1: Receive advice

[0418] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[0419] Input: Notification of suggestions and advice.

[0420] Output: Caregiver reactions and how they responded.

[0421] Specific operation: The smartphone app receives a push notification from the server and displays the content.

[0422] Step 2: Respond based on emotions

[0423] The caregiver responds based on the results of the emotion analysis of the care recipient.

[0424] Input: Sentiment analysis results.

[0425] Output: Implementing the response method.

[0426] Specific operations: Commands such as playing relaxing music and changing the color of smart lights can be given through a smartphone app.

[0427] Step 3: Real-time response

[0428] When an emergency alert is received, caregivers will rush to the scene immediately.

[0429] Input: Emergency alert notification.

[0430] Output: On-site response actions.

[0431] Specific actions: Upon receiving an emergency notification, the caregiver rushes to the scene and takes action such as assisting the person being cared for who has fallen.

[0432] Step 4: Provide information

[0433] Local residents provide daily care data and suggestion information to the server and receive rewards.

[0434] Input: Daily care data and proposal information.

[0435] Output: Receipt of reward.

[0436] Specific operation: Data is entered and sent through a dedicated smartphone app, and the server calculates the reward amount at the end of the month and makes the payment electronically.

[0437] (Application example 2)

[0438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] In nursing care facilities and at home, it is necessary to monitor the safety and health of elderly people and those receiving care in real time and to take appropriate measures promptly. However, current systems face many challenges, such as limited human resources and difficulties in early detection of emergencies. In addition, there is a lack of mechanisms for efficiently managing information provided by local residents and promoting their cooperation. As a result, there is a risk that the burden on caregivers will increase and the quality of life of those receiving care will decline.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0441] In this invention, the server includes a means for receiving video data from a nursing facility or home, an artificial intelligence means for analyzing the video data, a means for generating an alert in the event of an emergency and notifying the alert via a communication device, and a means for calculating and paying an information provision fee based on the amount of information provided by local residents. This enables real-time monitoring of the safety and health of care recipients and rapid response in emergencies. Furthermore, by efficiently managing information provided by local residents and promoting their cooperation, it is expected that the burden on caregivers will be reduced and the quality of life of care recipients will be improved.

[0442] A "nursing home" is a facility that provides necessary care and support to elderly people and people with physical disabilities who require nursing care.

[0443] "Within the home" refers to the interior of an individual home, that is, the dwelling, where the people who live there go about their daily lives.

[0444] "Video data" refers to information about moving images or videos captured by a video capture device such as a camera.

[0445] "Artificial intelligence" is a general term for technologies that mimic human intelligence and includes algorithms for analyzing data and solving problems.

[0446] An "alert" is a warning or notification that notifies of an emergency or abnormality.

[0447] "Communications equipment" means equipment for sending and receiving data and information.

[0448] "Local residents" refers to people who live within a certain area.

[0449] "Information fee" means the remuneration paid for providing information.

[0450] "Data compression" is a technical process used to reduce the volume of data.

[0451] "Filtering" is a technical technique for removing or extracting specific elements from data.

[0452] "Analysis" means investigating and analyzing data in detail to derive its meaning and trends.

[0453] An "emotion engine" is a technology for recognizing and analyzing human emotions from video and audio data.

[0454] The system of the present invention receives and analyzes video data from within a care facility or home, and provides suggestions and emergency alerts to caregivers. The system is composed of three parties: a server, a terminal, and a user, and functions as follows.

[0455] Server Roles

[0456] Data reception:

[0457] The server receives video data in real time transmitted from nursing homes and homes via the Internet. The hardware used is a high-performance data server, and the software used is FFmpeg for data streaming.

[0458] Data Analysis:

[0459] The received video data is analyzed using an AI model and emotion engine, using TensorFlow and Keras software. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling.

[0460] Suggestion generation:

[0461] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[0462] Emergency Alert:

[0463] If an emergency is detected, the server immediately generates an alert and notifies the caregiver via a communication device. Notifications are sent via SMS using the Twilio API and push notifications are provided using AWS (registered trademark) SNS.

[0464] Calculation and payment of information fees:

[0465] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[0466] Device Role

[0467] Data collection:

[0468] The device operates cameras and sensors 24 / 7 to collect video and environmental data. The hardware used is a general-purpose surveillance camera and multi-purpose sensor.

[0469] Data preprocessing:

[0470] The device compresses the collected data, filters it for privacy, and uses facial recognition technology to mask personal information to protect privacy.

[0471] Data transmission:

[0472] The pre-processed data is sent from the device to the server, where it is efficiently transmitted using high-speed communication technology.

[0473] User Roles

[0474] Advice received:

[0475] Caregivers can receive suggestions and advice from the server on their smartphones or dedicated devices and check them according to the situation, for example, they can receive suggestions on how to deal with a care recipient who is in an unstable psychological state.

[0476] Emergency Response:

[0477] When an emergency alert is received, caregivers can immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of falling, they can take measures such as tidying up the surrounding environment.

[0478] Information provided:

[0479] Local residents will strengthen the support system by providing care data and proposal information to the server on a daily basis. By receiving a fee for providing information, cooperation throughout the community will be promoted.

[0480] Specific examples

[0481] For example, if a security camera detects abnormal activity at night, the server receives and analyzes the video data. If the AI ​​detects the presence of a suspicious individual, it uses the Twilio API to send an emergency alert via SMS to nearby security personnel. It also uses the Stripe API to reward local residents for providing information.

[0482] Prompt Sentence Examples

[0483] You are the developer of a system that uses AI models to analyze video data from nursing homes and homes to detect abnormal behavior and emotions. Collect video data and analyze it in real time using AI. In the event of an emergency, use the Twilio API to send SMS alerts and the Stripe API to process rewards.

[0484] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0485] Step 1:

[0486] Data collection

[0487] The device will:

[0488] The device collects video and environmental data 24 hours a day from cameras and sensors installed in care facilities and homes. The hardware used is a surveillance camera and a multi-purpose sensor, which continuously monitors the movements of the care recipient and their surrounding environment.

[0489] Input and Output:

[0490] Input: Real-time video data and environmental data from cameras and sensors

[0491] Output: Collected video data and sensor data

[0492] Step 2:

[0493] Data Preprocessing

[0494] The device will:

[0495] The device compresses the collected video and environmental data and filters it for privacy, specifically masking personal information using facial recognition technology.

[0496] Input and Output:

[0497] Input: Collected video data and environmental data

[0498] Output: Compressed and filtered data

[0499] Step 3:

[0500] Data transmission

[0501] The device will:

[0502] The device then transmits the pre-processed data to the server using high-speed communication technology, enabling efficient use of the data.

[0503] Input and Output:

[0504] Input: Compressed and filtered data

[0505] Output: Data sent to the server

[0506] Step 4:

[0507] Data reception

[0508] Server actions:

[0509] The server receives video and environmental data in real time, transmitted via the Internet, using FFmpeg software for data streaming.

[0510] Input and Output:

[0511] Input: Data sent from the terminal

[0512] Output: Received data

[0513] Step 5:

[0514] Data analysis

[0515] Server actions:

[0516] The server inputs the received video and environmental data into an AI model and emotion engine for analysis using TensorFlow and Keras software, which detects the facial expressions and movements of the care recipient and changes in the surrounding environment to determine their psychological state and risk of falling.

[0517] Input and Output:

[0518] Input: Received data

[0519] Output: Analysis results

[0520] Step 6:

[0521] Proposal generation

[0522] Server actions:

[0523] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[0524] Input and Output:

[0525] Input: Analysis results

[0526] Output: Generated suggestions and advice

[0527] Step 7:

[0528] Emergency alert sending

[0529] Server actions:

[0530] If an emergency is detected, the server immediately generates an alert and notifies the user via communication devices. The Twilio API is used for SMS notifications, and AWS SNS is used for push notifications.

[0531] Input and Output:

[0532] Input: Analysis results

[0533] Output: Emergency alert notification

[0534] Step 8:

[0535] Information Fee Calculation and Payment

[0536] Server actions:

[0537] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[0538] Input and Output:

[0539] Input: Information provided by local residents

[0540] Output: Calculation and payment of information fee

[0541] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0542] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0543] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0544] [Second embodiment]

[0545] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0546] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0547] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0548] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0549] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0551] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0552] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0553] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0554] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0555] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0556] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0557] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[0558] System Configuration

[0559] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[0560] Server Roles

[0561] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0562] Data reception

[0563] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0564] Data analysis

[0565] The server inputs the received video data into the AI ​​model and begins analysis, which detects the facial expressions and movements of the care recipient as well as changes in the surrounding environment to determine their psychological state and risk of falling.

[0566] Proposal generation

[0567] Based on the analysis results, the server generates appropriate suggestions and advice for caregivers, such as suggestions for safe layouts, how to interact with patients depending on their psychological state, and recommendations for appropriate care products.

[0568] Emergency Alerts

[0569] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[0570] Calculation and payment of information fees

[0571] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0572] Device Role

[0573] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0574] Data collection

[0575] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0576] Data Preprocessing

[0577] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0578] Data transmission

[0579] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[0580] Real-time notifications

[0581] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[0582] User Roles

[0583] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0584] Receive advice

[0585] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[0586] Real-time support

[0587] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[0588] Information provision

[0589] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0590] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and realizes appropriate care for those receiving care.

[0591] The processing flow will be explained below.

[0592] Step 1: Data collection

[0593] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[0594] Step 2: Data Preprocessing

[0595] The device compresses the collected video data and performs any necessary filtering, such as masking personal information using facial recognition technology.

[0596] Step 3: Send data

[0597] The terminal sends the pre-processed data to the server, which involves sending the packets over the Internet.

[0598] Step 4: Receiving Data

[0599] The server receives the video data sent from the device in real time. This is completed by opening a receiving socket and receiving the data packet.

[0600] Step 5: Data analysis

[0601] The server inputs the received video data into the AI ​​model and begins analysis, which monitors the facial expressions and movements of the care recipient as well as changes in the surrounding environment to assess their psychological state and risk of falling.

[0602] Step 6: Proposal Generation

[0603] Based on the analysis results, the server generates optimal suggestions and advice for caregivers, such as changing the room layout, safety measures, and how to interact with the patient depending on their psychological state.

[0604] Step 7: Proposal Notification

[0605] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[0606] Step 8: Emergency Notification

[0607] If the server detects an emergency situation in the care recipient (such as a fall or a sudden change in physical condition), it immediately generates an alert and notifies the caregiver.

[0608] Step 9: Provide information

[0609] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout or the way they interact with the person they are caring for.

[0610] Step 10: Calculation and payment of information fee

[0611] The server calculates the fee for providing information based on the care information provided by local residents, determines the reward based on the frequency and quality of the information provided, and processes the payment.

[0612] Example 1

[0613] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] Caring for care recipients in nursing homes and at home requires understanding their situation in real time and responding quickly, but existing systems have not been able to adequately achieve this. A particular challenge is accurately assessing the care recipient's psychological state and risk of falling, and quickly providing appropriate suggestions and emergency alerts. It is also important to properly calculate information provision fees and promote information collection with the cooperation of local residents. A new system is needed to solve these issues.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0616] In this invention, the server includes: means for receiving video data from a nursing facility or home; artificial intelligence means for analyzing the video data; means for compressing and filtering the video data; means for generating suggestions to caregivers based on the analysis results; means for notifying the caregivers of the suggestions; means for calculating and paying a fee for providing information from the caregiver; means for analyzing the facial expressions and movements of the care recipient to determine their psychological state and risk of falling; and means for generating and notifying alerts in real time when an emergency occurs. This allows for real-time understanding of the care recipient's condition and rapid response. Furthermore, it can encourage information provision from local residents and improve the overall quality of care.

[0617] "Video data from nursing care facilities and homes" refers to video and image data that records the living environment and behavior of care recipients, obtained by cameras and sensors installed in nursing care facilities and homes.

[0618] "Artificial intelligence means" refers to technologies that use algorithms such as machine learning and deep learning to analyze data and detect specific patterns or anomalies.

[0619] "Compression and filtering" refers to the process of compressing large amounts of data to reduce its size so that it can be transmitted and stored efficiently, and the process of removing and processing unnecessary data or data related to privacy.

[0620] The "means for generating suggestions to caregivers based on the analysis results" is a system that automatically generates specific guidelines for action and advice to be provided to caregivers based on information obtained from the results of data analysis.

[0621] The "means for notifying the caregiver of the suggestion" is a function for sending the generated suggestion or advice as a message to the device used by the caregiver.

[0622] The "means for calculating and paying information fees" is a system for calculating and paying compensation based on the quantity and quality of information provided by local residents.

[0623] "Means for analyzing the facial expressions and movements of care recipients to determine their psychological state and risk of falling" refers to technology that analyzes facial expressions and movement data of care recipients obtained from video data to assess their psychological state and risk of falling.

[0624] "Means for generating and notifying alerts in real time when an emergency occurs" refers to a system that generates a warning message the moment something abnormal occurs in the person being cared for and immediately notifies the caregiver.

[0625] "Data collection" refers to the acquisition of video and environmental data from within nursing facilities and homes using cameras and sensors.

[0626] "Means for monitoring the progress of data transmission and retransmitting if an error occurs" is a function that monitors the progress of collected data being transmitted to the server in real time and automatically attempts to retransmit if an error occurs.

[0627] The "means for a caregiver who receives an emergency alert to respond immediately" refers to a notification and instruction function that enables a caregiver to respond quickly when an emergency alert is received.

[0628] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[0629] System Configuration

[0630] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[0631] Server Roles

[0632] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0633] Data reception

[0634] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0635] Data analysis

[0636] The server inputs the received video data into an AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. Specifically, AI frameworks such as TensorFlow and PyTorch are used. For example, if the AI ​​determines that the care recipient is feeling anxious, it outputs the result of the analysis.

[0637] Proposal generation

[0638] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. These suggestions include suggestions for safe layouts, how to interact with the person depending on their psychological state, and recommendations for appropriate care products. A generative AI model is used to generate the suggestions. For example, a suggestion might be, "The person receiving care seems to be feeling anxious. Create a calming environment and talk to them."

[0639] Emergency Alerts

[0640] In the event of an emergency, the server will immediately generate an alert and notify the caregiver, for example, an alert saying "The care recipient has fallen" will be sent immediately.

[0641] Calculation and payment of information fees

[0642] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0643] Device Role

[0644] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0645] Data collection

[0646] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0647] Data Preprocessing

[0648] The device compresses the collected data and performs necessary filtering to protect privacy, including masking personal information using facial recognition technology, such as blurring individuals' faces from recorded data.

[0649] Data transmission

[0650] Once preprocessing is complete, the data is sent from the device to the server, enabling efficient data utilization. The device transmits data via Wi-Fi or a wired network and has the ability to automatically attempt retransmission if an error occurs.

[0651] Real-time notifications

[0652] If an emergency occurs, the device will issue an alert on the spot, allowing caregivers to respond immediately. For example, if a fall is detected, the device will issue an alert on the spot.

[0653] User Roles

[0654] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0655] Receive advice

[0656] Caregivers can receive and check suggestions and advice from the server on their smartphones or dedicated devices. For example, they might receive a notification on their smartphone app saying, "The person being cared for may be feeling anxious. Please check immediately."

[0657] Real-time support

[0658] When an emergency alert is received, the caregiver will immediately rush to the scene and take appropriate action. For example, if an alert is received that a person is likely to fall, the caregiver will quickly rush to the care recipient's side.

[0659] Information provision

[0660] Local residents provide daily care data and proposal information to the server. This strengthens the support system throughout the community. In doing so, they can receive a fee for providing the information. For example, they can provide daily monitoring information to the server and receive a fee in return.

[0661] Prompt Sentence Examples

[0662] Below is an example of a prompt sentence to input to the generative AI model.

[0663] Sample prompt: "Detect the risk of a care recipient falling while walking in the living room and provide appropriate advice to the caregiver."

[0664] The above is a specific embodiment of the system of the present invention. This system allows caregivers to receive appropriate suggestions and respond quickly in emergencies. It also encourages information provision from local residents and improves the overall quality of care.

[0665] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0666] Step 1: Data collection

[0667] The device collects video data and environmental data using cameras and sensors installed in nursing care facilities or homes. The device continuously records the room of the care recipient, acquiring video data of the living room, for example. This data is temporarily stored in local storage. The input is real-time video data, and the output is collected video data.

[0668] Step 2: Data Preprocessing

[0669] The device compresses the collected video data and filters it for privacy purposes, for example, by using facial recognition technology to blur the faces of people other than the care recipient. The input is the collected video data, and the output is the pre-processed video data.

[0670] Step 3: Send data

[0671] The preprocessed data is sent from the device to the server. The device transmits the data over Wi-Fi or a wired network and monitors the progress of the transmission. If an error occurs, the device automatically attempts to retransmit. The input is the preprocessed video data, and the output is the data sent to the server.

[0672] Step 4: Receiving Data

[0673] The server receives video data sent from the device via the Internet. The server stores the received data in a storage device and prepares it for analysis. The input is the data sent from the device, and the output is the stored video data.

[0674] Step 5: Data analysis

[0675] The server inputs the received video data into an AI model for analysis. This analysis detects the facial expressions, movements, and environmental changes of the care recipient to determine their psychological state and risk of falling. For example, using TensorFlow or PyTorch, the analysis result may be "the care recipient is feeling anxious." The input is the received video data, and the output is the analysis result.

[0676] Step 6: Proposal Generation

[0677] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. Using a generative AI model, it creates suggestions such as, "The care recipient seems to be feeling anxious. Create a calming environment and talk to them." The input is the analysis results, and the output is the generated suggestions.

[0678] Step 7: Proposal Notification

[0679] The server notifies the caregiver of the generated suggestion by sending a notification message to the caregiver's smartphone or dedicated device. The input is the generated suggestion, and the output is the sent notification.

[0680] Step 8: Emergency Alerts

[0681] When an emergency occurs, the server immediately generates an alert and notifies the caregiver. For example, it generates and sends an alert saying, "The care recipient has fallen." The input is the emergency analysis result, and the output is an emergency alert.

[0682] Step 9: Calculation and payment of information fee

[0683] The server calculates the information provision fee according to the amount of care information provided by local residents and processes the payment. The server calculates the reward using a point system and pays via digital wallet or bank transfer. The input is the provided information data, and the output is the calculated information provision fee and payment information.

[0684] Step 10: Receive advice

[0685] The user receives and checks the suggestions and advice sent from the server on a smartphone or dedicated device. For example, they may receive a notification saying, "The care recipient may be feeling anxious. Please check immediately." The input is the notification from the server, and the output is the suggestions displayed on the user's device.

[0686] Step 11: Real-time support

[0687] When a user receives an emergency alert, they immediately rush to the scene to respond. For example, when an alert is received indicating a high possibility of a fall, a caregiver will quickly rush to the care recipient's side. The input is the emergency alert, and the output is the user's response action.

[0688] Step 12: Provide information

[0689] Users provide daily care data and suggestion information to the server. Local residents input observation data and insights into the app and send them to the server. The input is the observation data and suggestion information, and the output is the information sent to the server.

[0690] (Application example 1)

[0691] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] In addition to the challenge of ensuring the safety of care recipients while reducing the burden on caregivers in nursing homes and homes, it is also necessary to monitor the safety of workers in factories and improve work efficiency. However, with current systems, it is difficult to solve both challenges simultaneously.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0694] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions, means for calculating and paying an information provision fee from the caregiver, means for receiving video data from a factory and analyzing the movements of workers and machines, means for immediately issuing emergency alerts based on the analysis, and means for generating suggestions to improve work efficiency and notifying a manager. This enables safety to be ensured and efficient operation to be achieved in both nursing care and factory environments.

[0695] "Means for receiving video data from within a care facility or home" refers to a device or system that has the function of collecting video data transmitted from cameras or sensors installed in a care facility or home.

[0696] "Artificial intelligence means for analyzing the video data" refers to a system that uses AI technology to analyze the received video data and executes a process to determine the condition and situation of the care recipient or worker.

[0697] The "means for generating suggestions to caregivers based on the analysis results" refers to an algorithm or program that generates useful suggestions or advice for caregivers based on the analysis results.

[0698] The "means for notifying the caregiver of the suggestions" refers to a communication means or application that notifies the caregiver of the generated suggestions or advice to their smartphone or dedicated terminal.

[0699] The "means for calculating and paying the information provision fee from the caregiver" is a system for calculating and paying the information provision fee in return based on the information provided by the caregiver or local residents.

[0700] "Means for receiving video data from within the factory and analyzing the movements of workers and machines" refers to a system that has the function of collecting video data sent from cameras and sensors installed within the factory and analyzing the movements of workers and machines based on that data.

[0701] The "means for immediately issuing an emergency alert based on the analysis" refers to a communication means or application that generates an alert in real time and notifies an administrator if an abnormality is detected during the analysis.

[0702] The "means for generating proposals for improving work efficiency and notifying the administrator" refers to an algorithm or program that generates proposals and improvement measures for improving work efficiency from the analysis results and notifies the administrator.

[0703] "Means for collecting video data using cameras and sensors installed in nursing care facilities or homes" refers to devices or systems that have the function of collecting video data from cameras and sensors installed in nursing care facilities or homes.

[0704] The "means for compressing and filtering the video data" refers to a system that compresses collected video data and performs filtering processing to protect privacy and reduce data volume.

[0705] The "means for transmitting the collected data to the server" refers to a communication means or protocol for transmitting the pre-processed video data to the server.

[0706] It is a "Japanese company."

[0707] "Means for collecting and preprocessing data from cameras and sensors installed within the factory" refers to a system that collects video data from cameras and sensors installed within the factory, preprocesses it, and puts it in a format suitable for analysis.

[0708] The "means for transmitting collected data to a cloud service" refers to a communication means or protocol for transmitting preprocessed data to a cloud service server.

[0709] "The facial expression of the person receiving care" is a change in facial expression.

[0710] High quality data and the ability to send alerts in case of emergency are required.

[0711] It is useful for management.

[0712] Overall system configuration

[0713] This invention is a system that collects and analyzes video data from nursing homes, homes, and factories. The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in the nursing home, home, or factory and collects video data and sends it to the server. The server analyzes the video data and generates optimal suggestions and emergency alerts for caregivers and factory managers. The user receives this information and takes appropriate action.

[0714] Server Roles

[0715] The roles of the server are as follows:

[0716] 1. Data reception: The server receives video data sent via the Internet from nursing care facilities, homes, and factories, allowing real-time information on the living and working conditions of care recipients and workers.

[0717] 2. Data analysis: The server inputs the received video data into the generative AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient and caregiver, as well as changes in the surrounding environment, and determines their psychological state, risk of falls, safety risks, etc.

[0718] 3. Proposal Generation: Based on the analysis results, the server generates appropriate suggestions for caregivers and suggestions and advice for factory managers to improve work efficiency. These suggestions include recommendations for safe layouts, how to interact with employees depending on their psychological state, and efficient work methods.

[0719] 4. Emergency Alert: In case of an emergency, the server will immediately generate an alert to notify caregivers or factory managers, allowing for prompt action.

[0720] 5. Calculation and payment of information fees: The server calculates the information fees according to the amount of information provided by local residents and workers, and processes the payments. This promotes information gathering across the entire region or factory.

[0721] Device Role

[0722] The roles of the terminal are as follows:

[0723] 1. Data Collection: Cameras and sensors connected to devices collect video and environmental data 24 / 7. For example, cameras in nursing homes continuously record activity in living rooms, or cameras in factories monitor activity in work areas.

[0724] 2. Data preprocessing: The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0725] 3. Data transmission: After preprocessing, the data is transmitted from the terminal to the server, enabling efficient data utilization.

[0726] 4. Real-time notification: The device will generate an immediate alert if an emergency occurs, allowing caregivers or factory managers to respond immediately.

[0727] User Roles

[0728] The user roles are as follows:

[0729] 1. Receiving advice: Caregivers and factory managers receive and review suggestions and advice from the server on their smartphones or dedicated devices. For example, they may receive suggestions on how to deal with care recipients who are in an unstable psychological state or specific work methods to improve work efficiency.

[0730] 2. Real-time response: When an emergency alert is received, caregivers and factory managers rush to the scene and take appropriate action, such as tidying up the surrounding environment if the risk of falling is deemed high, or temporarily suspending work if the work risk is deemed high.

[0731] 3. Information provision: Local residents and factory workers provide daily care and work data to the server. This strengthens the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0732] Examples of specific examples and prompts

[0733] For example, if heavy machinery in a factory makes a dangerous move, AI will detect the abnormality and immediately send an alert to the manager via email. It will also analyze the movements of workers and make suggestions to improve work efficiency.

[0734] Example prompts to input to a generative AI model:

[0735] "Write a Python program to monitor the movement of workers and heavy equipment in a factory, assess safety in real time, and generate alerts if anything goes wrong."

[0736] In this way, the system of the present invention is realized through cooperation between servers, terminals, and users, reducing the burden on caregivers and factory managers, ensuring appropriate care and safety for care recipients and workers, and improving work efficiency.

[0737] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0738] Step 1: Data collection

[0739] The device collects video data using cameras and sensors installed in nursing homes, homes, and factories. This data monitors the dynamics of care recipients, workers, and the environment 24 / 7. Specifically, cameras continuously record images of living rooms and factory work areas, while sensors detect changes in the environment. The input is real-time data from the cameras and sensors, and the output is the collected video data and sensor data.

[0740] Step 2: Data Preprocessing

[0741] The device compresses the collected data and performs necessary filtering to protect privacy. Specifically, it uses facial recognition technology to mask personal information from the collected data. This process reduces the amount of data and improves transmission efficiency. The input is the collected video data and sensor data, and the output is the compressed and filtered data.

[0742] Step 3: Send data

[0743] The terminal sends the preprocessed data to the server. Specifically, the compressed and filtered data is uploaded to the cloud server via the Internet. The input is the preprocessed data, and the output is the data sent to the server.

[0744] Step 4: Data analysis

[0745] The server analyzes the received data. Specifically, it uses a generative AI model to analyze the data and evaluate the condition of the care recipient and worker, as well as changes in the surrounding environment. This analysis determines factors such as fall risks, safety risks, and efficient work methods. The input is the data sent to the server, and the output is the analysis results.

[0746] Step 5: Proposal Generation

[0747] The server generates suggestions and advice for caregivers and factory managers based on the analysis results. Specifically, it creates suggestions including recommendations for safe layouts, how to interact with patients based on their psychological state, and efficient work methods. The input is the analysis results, and the output is the generated suggestions.

[0748] Step 6: Notification

[0749] The server notifies the caregiver or factory manager of the generated suggestions and advice. Specifically, it sends notifications to smartphones or dedicated devices and provides emergency alerts in real time. The input is the generated suggestions, and the output is the notified information.

[0750] Step 7: Emergency response

[0751] Users (caregivers and factory managers) receive emergency alerts from the server and immediately rush to the scene to take appropriate action. Specifically, if a high risk of falling is determined, they will organize the environment around the care recipient, and if a dangerous situation occurs in the factory, they will temporarily suspend work. The input is the alert notification from the server, and the output is the response action.

[0752] Step 8: Calculation and payment of information fee

[0753] The server calculates the information fee based on the information provided by local residents and workers, and processes the payment. Specifically, it evaluates the quantity and quality of the information provided, determines the reward, and makes the payment. The input is the provided information, and the output is the calculated information fee and payment processing.

[0754] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0755] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides suggestions and emergency alerts to caregivers. It also includes the calculation and payment of information provision fees from local residents.

[0756] System Configuration

[0757] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies the caregiver of optimal suggestions and emergency alerts. Furthermore, by combining it with an emotion engine, the system recognizes the emotions of the care recipient and makes suggestions based on those emotions. The user receives this information and takes appropriate action.

[0758] Server Roles

[0759] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[0760] Data reception

[0761] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[0762] Data analysis

[0763] The server inputs the received video data into the AI ​​and emotion engine and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. The emotion engine also simultaneously analyzes the care recipient's emotions.

[0764] Proposal generation

[0765] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver, such as suggestions for safe layouts, how to interact with the patient based on their psychological state, recommendations for appropriate care products, and how to respond based on their emotions.

[0766] sentiment analysis

[0767] The emotion engine recognizes and determines the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling anxious or stressed, the analysis results are fed back to the server.

[0768] Emergency Alerts

[0769] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[0770] Calculation and payment of information fees

[0771] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[0772] Device Role

[0773] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[0774] Data collection

[0775] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[0776] Data Preprocessing

[0777] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[0778] Data transmission

[0779] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[0780] Real-time notifications

[0781] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[0782] User Roles

[0783] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[0784] Receive advice

[0785] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[0786] Emotion-based responses

[0787] Based on the analysis results of the emotion engine, the caregiver will respond according to the care recipient's emotions. For example, if the care recipient is feeling stressed, the system will suggest actions such as playing relaxing music.

[0788] Real-time support

[0789] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[0790] Information provision

[0791] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[0792] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and provides appropriate care to those receiving care. Furthermore, by combining it with an emotion engine, it becomes possible to provide advanced responses based on the emotions of the care recipient.

[0793] The processing flow will be explained below.

[0794] Step 1: Data collection

[0795] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[0796] Step 2: Data Preprocessing

[0797] The device compresses the collected video data and performs necessary filtering (such as masking personal information using facial recognition technology) to ensure that the data complies with privacy standards.

[0798] Step 3: Send data

[0799] The terminal transmits the pre-processed data to the server. This transmission is carried out over the Internet and arrives at the server as packets.

[0800] Step 4: Receiving Data

[0801] The server receives the video data transmitted from the terminal via the Internet in real time and receives the data packets using a receiving socket.

[0802] Step 5: Data analysis

[0803] The server inputs the received video data into the AI ​​model and emotion engine and begins analysis. This analysis monitors the care recipient's facial expressions, movements, and changes in the surrounding environment to evaluate their psychological state, risk of falling, and emotions (e.g., joy, anger, sadness, and happiness).

[0804] Step 6: Sentiment analysis

[0805] The server uses an emotion engine to recognize the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling stressed, the emotion engine immediately sends the analysis results back to the server.

[0806] Step 7: Proposal Generation

[0807] Based on the results of data analysis and sentiment analysis, the server generates optimal suggestions and advice for caregivers, including suggestions for safe layouts, how to interact with patients based on their psychological state, recommendations for appropriate caregiving products, and emotional responses.

[0808] Step 8: Proposal Notification

[0809] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[0810] Step 9: Emergency Notification

[0811] If the server detects an emergency situation in the care recipient (e.g., a fall or a sudden change in physical condition), it immediately generates an emergency alert and notifies the caregiver, enabling a prompt response.

[0812] Step 10: Provide information

[0813] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout and how they interact with the care recipient based on their emotions.

[0814] Step 11: Calculation and payment of information fee

[0815] The server calculates the fee for providing information based on the care information provided by local residents. It determines the fee based on the frequency and quality of the information provided and processes the payment. This promotes information collection throughout the community.

[0816] Example 2

[0817] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0818] To improve the quality of care in nursing facilities and homes, it is necessary to grasp the condition of care recipients in real time and respond promptly and appropriately. However, currently, there are limited means of understanding the emotions and condition of care recipients, which may result in delayed responses to emergencies. In addition, there is a lack of a system for efficiently utilizing information provided by local residents and appropriately rewarding them for providing that information. A system is needed to solve these problems and provide a safe and secure environment for both caregivers and care recipients.

[0819] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0820] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for analyzing the facial expressions, movements, and environmental changes of the care recipient in the video data and collected data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions and emergency alerts, and means for calculating and paying fees for information provided by the caregivers. This enables real-time understanding of the care recipient's condition and recognition of their emotions, enabling prompt and appropriate responses. Furthermore, by appropriately managing and paying fees for information provided by local residents, the quality and quantity of information can be improved, strengthening the nursing care support system throughout the community.

[0821] "Video data from nursing homes and homes" refers to video and environmental information obtained from cameras and sensors installed in nursing homes and homes.

[0822] An "artificial intelligence means" is a computer system that performs data analysis using machine learning algorithms and deep learning.

[0823] The "means for generating suggestions" is a system that generates appropriate suggestions and advice for the caregiver based on the analyzed data.

[0824] "Means of notifying caregivers" refers to a system that notifies caregivers of suggestions and emergency alerts via their smartphones or dedicated devices.

[0825] The "means for calculating and paying information fees" is a system that calculates fees based on the quantity and quality of information provided by local residents and pays them electronically.

[0826] "Means for collecting video data using cameras and sensors" refers to a system that collects data using cameras and sensors installed in nursing facilities and homes.

[0827] "Means for compressing data and filtering for privacy protection" refers to technologies that compress collected video data for efficient storage and transmission, and perform masking and filtering to protect personal information.

[0828] "Means for transmitting data to a server" refers to a mechanism for transmitting collected data to a server via the Internet.

[0829] The "emotion engine" is an algorithm that analyzes and determines the emotions of the person receiving care from their facial expressions and voice.

[0830] "Means for notifying caregivers of emergency alerts in real time" refers to a system that immediately sends an alert to caregivers when an emergency situation is detected.

[0831] The system of the present invention receives video data from nursing homes and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides optimal suggestions and emergency alerts to caregivers. It also calculates and pays rewards for information provided by local residents. The detailed configuration and operation of the system are described below.

[0832] Server Roles

[0833] The server acts as a central device and performs the following processes:

[0834] 1. Data Reception

[0835] The server receives video data sent from care facilities and homes via the Internet.

[0836] Specific hardware: A high-performance server is required, such as a data center server equipped with an Intel Xeon processor.

[0837] Specific software: RTSP (Real-Time Streaming Protocol) is used for data streaming.

[0838] 2. Data Analysis

[0839] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[0840] Specific hardware: A server equipped with a GPU (NVIDIA Tesla) is used.

[0841] Specific software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[0842] 3. Proposal generation

[0843] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver.

[0844] Specific software: A rule-based expert system is used to generate recommendations based on the analysis results.

[0845] 4. Sentiment analysis

[0846] The emotion engine identifies the emotions of the care recipient and feeds the results back to the server.

[0847] Specific software: Libraries such as OpenCV and Amazon Rekognition are used for emotion recognition.

[0848] 5. Emergency Alerts

[0849] In the event of an emergency, the server will instantly generate an alert and notify the caregiver.

[0850] Specific operation: Use an anomaly detection algorithm (e.g., a fall detection algorithm).

[0851] 6. Calculation and payment of information fees

[0852] The server calculates and pays rewards based on the amount of information provided by local residents.

[0853] Specific software: SQL databases and electronic payment systems (PayPal API and Stripe API).

[0854] Device Role

[0855] The devices are installed in nursing homes and homes and perform the following functions:

[0856] 1. Data Collection

[0857] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[0858] Specific hardware: IP cameras (e.g., Axis), various sensors (e.g., temperature and humidity sensors).

[0859] Example: An in-room camera continuously records what is happening in the living room.

[0860] 2. Data Preprocessing

[0861] The collected data is compressed and filtered for privacy purposes.

[0862] Specific software: Personal information masking using facial recognition technology and H.264 / HEVC compression technology.

[0863] 3. Data Transmission

[0864] The pre-processed data is sent to the server.

[0865] Specific software: Data is transmitted using HTTPS (Hypertext Transfer Protocol Secure).

[0866] 4. Real-time notifications

[0867] If an emergency occurs, an alert will be generated immediately.

[0868] Specific operation: When the device detects data that exceeds a certain threshold, it will sound an alert using the built-in speaker and light.

[0869] User Roles

[0870] Users interact with the system as caregivers or local residents and perform the following functions:

[0871] 1. Receiving advice

[0872] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[0873] Example: Caregivers can view the suggestions on a smartphone app.

[0874] 2. Emotional responses

[0875] Caregivers respond appropriately based on the analysis results of the emotion engine.

[0876] Example: If the care recipient is feeling stressed, the app will suggest playing relaxing music, and the user can follow the suggestion.

[0877] 3. Real-time support

[0878] If an emergency alert is received, we will rush to the scene immediately.

[0879] Example: A caregiver is notified of a fall risk and rushes to the scene to respond.

[0880] 4. Information provision

[0881] Local residents provide daily care data and suggestion information to the server and receive rewards.

[0882] Example: Residents enter their daily observations and suggestions through a dedicated smartphone app, which are then sent to a server. Rewards are then paid at the end of the month.

[0883] Prompt Sentence Examples

[0884] "Please suggest appropriate ways to respond when the person you are caring for is feeling anxious."

[0885] By combining the above elements, a more advanced and secure care system can be realized for both caregivers and those receiving care.

[0886] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0887] Server Processing

[0888] Step 1: Receiving data

[0889] The server receives video data transmitted from care facilities and homes via the Internet.

[0890] Input: Real-time streaming data transmitted via RTSP.

[0891] Output: Received video data.

[0892] Specific operation: The server monitors the specified IP address and port and captures streaming data in real time.

[0893] Step 2: Data analysis

[0894] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[0895] Input: Received video data.

[0896] Output: Analysis results (e.g., facial expressions, movements, and environmental information of the care recipient).

[0897] Specific operation: The AI ​​module processes the video data frame by frame and performs facial recognition and movement analysis using deep learning (TensorFlow or PyTorch). The emotion engine (Amazon Rekognition, etc.) analyzes the facial expressions and voice data of the care recipient to identify their emotions.

[0898] Step 3: Proposal Generation

[0899] The server generates appropriate suggestions and advice for the caregiver based on the analysis results.

[0900] Input: Results of data analysis.

[0901] Output: Suggestions and advice.

[0902] How it works: A rule-based expert system generates suggestions based on the analysis results and in accordance with predefined patterns and rules.

[0903] Step 4: Emergency alert generation

[0904] If the server detects an emergency, it will immediately generate an alert and notify the caregiver.

[0905] Input: Results of data analysis (detection of emergency situations).

[0906] Output: Emergency alert notification.

[0907] Specific behavior: Anomaly detection algorithms identify high-risk behaviors (e.g., falls) in real time and send emergency notifications via email or SMS.

[0908] Step 5: Calculation and payment of information fee

[0909] The server calculates and pays rewards based on the amount of information provided by local residents.

[0910] Input: Informational log.

[0911] Output: Calculated reward amount and notification of payment completion.

[0912] Specific operation: The server analyzes the information provision log in the database, calculates the reward amount, and then pays the reward using an electronic payment system (PayPal or Stripe).

[0913] Terminal handling

[0914] Step 1: Data collection

[0915] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[0916] Input: The care recipient's real-time living environment.

[0917] Output: Collected video and environmental data.

[0918] Specific operation: IP cameras and temperature and humidity sensors continuously collect data and temporarily store it locally.

[0919] Step 2: Data Preprocessing

[0920] The device compresses the collected data and filters it for privacy purposes.

[0921] Input: Collected video and environmental data.

[0922] Output: Preprocessed data.

[0923] Specific operation: Uses facial recognition algorithms to mask personal information and compresses data using H.264 / HEVC compression technology.

[0924] Step 3: Send data

[0925] The pre-processed data is sent to a server.

[0926] Input: Preprocessed data.

[0927] Output: The data sent to the server.

[0928] Specific operation: Encrypts data using the HTTPS protocol and sends a POST request to the specified server URL.

[0929] Step 4: Real-time notifications

[0930] In the event of an emergency, the device will issue an alert on the spot.

[0931] Input: Emergency detection.

[0932] Output: Alert notification.

[0933] Specific operation: When the sensor inside the device detects data that exceeds the threshold, an alarm will sound from the built-in speaker and an emergency message will be displayed on the screen.

[0934] User Action

[0935] Step 1: Receive advice

[0936] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[0937] Input: Notification of suggestions and advice.

[0938] Output: Caregiver reactions and how they responded.

[0939] Specific operation: The smartphone app receives a push notification from the server and displays the content.

[0940] Step 2: Respond based on emotions

[0941] The caregiver responds based on the results of the emotion analysis of the care recipient.

[0942] Input: Sentiment analysis results.

[0943] Output: Implementing the response method.

[0944] Specific operations: Commands such as playing relaxing music and changing the color of smart lights can be given through a smartphone app.

[0945] Step 3: Real-time response

[0946] When an emergency alert is received, caregivers will rush to the scene immediately.

[0947] Input: Emergency alert notification.

[0948] Output: On-site response actions.

[0949] Specific actions: Upon receiving an emergency notification, the caregiver rushes to the scene and takes action such as assisting the person being cared for who has fallen.

[0950] Step 4: Provide information

[0951] Local residents provide daily care data and suggestion information to the server and receive rewards.

[0952] Input: Daily care data and proposal information.

[0953] Output: Receipt of reward.

[0954] Specific operation: Data is entered and sent through a dedicated smartphone app, and the server calculates the reward amount at the end of the month and makes the payment electronically.

[0955] (Application example 2)

[0956] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0957] In nursing care facilities and at home, it is necessary to monitor the safety and health of elderly people and those receiving care in real time and to take appropriate measures promptly. However, current systems face many challenges, such as limited human resources and difficulties in early detection of emergencies. In addition, there is a lack of mechanisms for efficiently managing information provided by local residents and promoting their cooperation. As a result, there is a risk that the burden on caregivers will increase and the quality of life of those receiving care will decline.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0959] In this invention, the server includes a means for receiving video data from a nursing facility or home, an artificial intelligence means for analyzing the video data, a means for generating an alert in the event of an emergency and notifying the alert via a communication device, and a means for calculating and paying an information provision fee based on the amount of information provided by local residents. This enables real-time monitoring of the safety and health of care recipients and rapid response in emergencies. Furthermore, by efficiently managing information provided by local residents and promoting their cooperation, it is expected that the burden on caregivers will be reduced and the quality of life of care recipients will be improved.

[0960] A "nursing home" is a facility that provides necessary care and support to elderly people and people with physical disabilities who require nursing care.

[0961] "Within the home" refers to the interior of an individual home, that is, the dwelling, where the people who live there go about their daily lives.

[0962] "Video data" refers to information about moving images or videos captured by a video capture device such as a camera.

[0963] "Artificial intelligence" is a general term for technologies that mimic human intelligence and includes algorithms for analyzing data and solving problems.

[0964] An "alert" is a warning or notification that notifies of an emergency or abnormality.

[0965] "Communications equipment" means equipment for sending and receiving data and information.

[0966] "Local residents" refers to people who live within a certain area.

[0967] "Information fee" means the remuneration paid for providing information.

[0968] "Data compression" is a technical process used to reduce the volume of data.

[0969] "Filtering" is a technical technique for removing or extracting specific elements from data.

[0970] "Analysis" means investigating and analyzing data in detail to derive its meaning and trends.

[0971] An "emotion engine" is a technology for recognizing and analyzing human emotions from video and audio data.

[0972] The system of the present invention receives and analyzes video data from within a care facility or home, and provides suggestions and emergency alerts to caregivers. The system is composed of three parties: a server, a terminal, and a user, and functions as follows.

[0973] Server Roles

[0974] Data reception:

[0975] The server receives video data in real time transmitted from nursing homes and homes via the Internet. The hardware used is a high-performance data server, and the software used is FFmpeg for data streaming.

[0976] Data Analysis:

[0977] The received video data is analyzed using an AI model and emotion engine, using TensorFlow and Keras software. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling.

[0978] Suggestion generation:

[0979] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[0980] Emergency Alert:

[0981] If an emergency is detected, the server immediately generates an alert and notifies the caregiver via communication devices, using the Twilio API for SMS notifications and AWS SNS for push notifications.

[0982] Calculation and payment of information fees:

[0983] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[0984] Device Role

[0985] Data collection:

[0986] The device operates cameras and sensors 24 / 7 to collect video and environmental data. The hardware used is a general-purpose surveillance camera and multi-purpose sensor.

[0987] Data preprocessing:

[0988] The device compresses the collected data, filters it for privacy, and uses facial recognition technology to mask personal information to protect privacy.

[0989] Data transmission:

[0990] The pre-processed data is sent from the device to the server, where it is efficiently transmitted using high-speed communication technology.

[0991] User Roles

[0992] Advice received:

[0993] Caregivers can receive suggestions and advice from the server on their smartphones or dedicated devices and check them according to the situation, for example, they can receive suggestions on how to deal with a care recipient who is in an unstable psychological state.

[0994] Emergency Response:

[0995] When an emergency alert is received, caregivers can immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of falling, they can take measures such as tidying up the surrounding environment.

[0996] Information provided:

[0997] Local residents will strengthen the support system by providing care data and proposal information to the server on a daily basis. By receiving a fee for providing information, cooperation throughout the community will be promoted.

[0998] Specific examples

[0999] For example, if a security camera detects abnormal activity at night, the server receives and analyzes the video data. If the AI ​​detects the presence of a suspicious individual, it uses the Twilio API to send an emergency alert via SMS to nearby security personnel. It also uses the Stripe API to reward local residents for providing information.

[1000] Prompt Sentence Examples

[1001] You are the developer of a system that uses AI models to analyze video data from nursing homes and homes to detect abnormal behavior and emotions. Collect video data and analyze it in real time using AI. In the event of an emergency, use the Twilio API to send SMS alerts and the Stripe API to process rewards.

[1002] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1003] Step 1:

[1004] Data collection

[1005] The device will:

[1006] The device collects video and environmental data 24 hours a day from cameras and sensors installed in care facilities and homes. The hardware used is a surveillance camera and a multi-purpose sensor, which continuously monitors the movements of the care recipient and their surrounding environment.

[1007] Input and Output:

[1008] Input: Real-time video data and environmental data from cameras and sensors

[1009] Output: Collected video data and sensor data

[1010] Step 2:

[1011] Data Preprocessing

[1012] The device will:

[1013] The device compresses the collected video and environmental data and filters it for privacy, specifically masking personal information using facial recognition technology.

[1014] Input and Output:

[1015] Input: Collected video data and environmental data

[1016] Output: Compressed and filtered data

[1017] Step 3:

[1018] Data transmission

[1019] The device will:

[1020] The device then transmits the pre-processed data to the server using high-speed communication technology, enabling efficient use of the data.

[1021] Input and Output:

[1022] Input: Compressed and filtered data

[1023] Output: Data sent to the server

[1024] Step 4:

[1025] Data reception

[1026] Server actions:

[1027] The server receives video and environmental data in real time, transmitted via the Internet, using FFmpeg software for data streaming.

[1028] Input and Output:

[1029] Input: Data sent from the terminal

[1030] Output: Received data

[1031] Step 5:

[1032] Data analysis

[1033] Server actions:

[1034] The server inputs the received video and environmental data into an AI model and emotion engine for analysis using TensorFlow and Keras software, which detects the facial expressions and movements of the care recipient and changes in the surrounding environment to determine their psychological state and risk of falling.

[1035] Input and Output:

[1036] Input: Received data

[1037] Output: Analysis results

[1038] Step 6:

[1039] Proposal generation

[1040] Server actions:

[1041] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[1042] Input and Output:

[1043] Input: Analysis results

[1044] Output: Generated suggestions and advice

[1045] Step 7:

[1046] Emergency alert sending

[1047] Server actions:

[1048] If an emergency is detected, the server immediately generates an alert and notifies the user via communication devices. The Twilio API is used for SMS notifications, and AWS SNS is used for push notifications.

[1049] Input and Output:

[1050] Input: Analysis results

[1051] Output: Emergency alert notification

[1052] Step 8:

[1053] Information Fee Calculation and Payment

[1054] Server actions:

[1055] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[1056] Input and Output:

[1057] Input: Information provided by local residents

[1058] Output: Calculation and payment of information fee

[1059] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1060] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1061] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1062] [Third embodiment]

[1063] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1064] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1065] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1066] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1067] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1068] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1069] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1070] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1071] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1072] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1073] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1074] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1075] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[1076] System Configuration

[1077] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[1078] Server Roles

[1079] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1080] Data reception

[1081] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1082] Data analysis

[1083] The server inputs the received video data into the AI ​​model and begins analysis, which detects the facial expressions and movements of the care recipient as well as changes in the surrounding environment to determine their psychological state and risk of falling.

[1084] Proposal generation

[1085] Based on the analysis results, the server generates appropriate suggestions and advice for caregivers, such as suggestions for safe layouts, how to interact with patients depending on their psychological state, and recommendations for appropriate care products.

[1086] Emergency Alerts

[1087] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[1088] Calculation and payment of information fees

[1089] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1090] Device Role

[1091] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1092] Data collection

[1093] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1094] Data Preprocessing

[1095] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1096] Data transmission

[1097] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[1098] Real-time notifications

[1099] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[1100] User Roles

[1101] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1102] Receive advice

[1103] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[1104] Real-time support

[1105] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[1106] Information provision

[1107] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1108] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and realizes appropriate care for those receiving care.

[1109] The processing flow will be explained below.

[1110] Step 1: Data collection

[1111] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[1112] Step 2: Data Preprocessing

[1113] The device compresses the collected video data and performs any necessary filtering, such as masking personal information using facial recognition technology.

[1114] Step 3: Send data

[1115] The terminal sends the pre-processed data to the server, which involves sending the packets over the Internet.

[1116] Step 4: Receiving Data

[1117] The server receives the video data sent from the device in real time. This is completed by opening a receiving socket and receiving the data packet.

[1118] Step 5: Data analysis

[1119] The server inputs the received video data into the AI ​​model and begins analysis, which monitors the facial expressions and movements of the care recipient as well as changes in the surrounding environment to assess their psychological state and risk of falling.

[1120] Step 6: Proposal Generation

[1121] Based on the analysis results, the server generates optimal suggestions and advice for caregivers, such as changing the room layout, safety measures, and how to interact with the patient depending on their psychological state.

[1122] Step 7: Proposal Notification

[1123] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[1124] Step 8: Emergency Notification

[1125] If the server detects an emergency situation in the care recipient (such as a fall or a sudden change in physical condition), it immediately generates an alert and notifies the caregiver.

[1126] Step 9: Provide information

[1127] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout or the way they interact with the person they are caring for.

[1128] Step 10: Calculation and payment of information fee

[1129] The server calculates the fee for providing information based on the care information provided by local residents, determines the reward based on the frequency and quality of the information provided, and processes the payment.

[1130] Example 1

[1131] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1132] Caring for care recipients in nursing homes and at home requires understanding their situation in real time and responding quickly, but existing systems have not been able to adequately achieve this. A particular challenge is accurately assessing the care recipient's psychological state and risk of falling, and quickly providing appropriate suggestions and emergency alerts. It is also important to properly calculate information provision fees and promote information collection with the cooperation of local residents. A new system is needed to solve these issues.

[1133] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1134] In this invention, the server includes: means for receiving video data from a nursing facility or home; artificial intelligence means for analyzing the video data; means for compressing and filtering the video data; means for generating suggestions to caregivers based on the analysis results; means for notifying the caregivers of the suggestions; means for calculating and paying a fee for providing information from the caregiver; means for analyzing the facial expressions and movements of the care recipient to determine their psychological state and risk of falling; and means for generating and notifying alerts in real time when an emergency occurs. This allows for real-time understanding of the care recipient's condition and rapid response. Furthermore, it can encourage information provision from local residents and improve the overall quality of care.

[1135] "Video data from nursing care facilities and homes" refers to video and image data that records the living environment and behavior of care recipients, obtained by cameras and sensors installed in nursing care facilities and homes.

[1136] "Artificial intelligence means" refers to technologies that use algorithms such as machine learning and deep learning to analyze data and detect specific patterns or anomalies.

[1137] "Compression and filtering" refers to the process of compressing large amounts of data to reduce its size so that it can be transmitted and stored efficiently, and the process of removing and processing unnecessary data or data related to privacy.

[1138] The "means for generating suggestions to caregivers based on the analysis results" is a system that automatically generates specific guidelines for action and advice to be provided to caregivers based on information obtained from the results of data analysis.

[1139] The "means for notifying the caregiver of the suggestion" is a function for sending the generated suggestion or advice as a message to the device used by the caregiver.

[1140] The "means for calculating and paying information fees" is a system for calculating and paying compensation based on the quantity and quality of information provided by local residents.

[1141] "Means for analyzing the facial expressions and movements of care recipients to determine their psychological state and risk of falling" refers to technology that analyzes facial expressions and movement data of care recipients obtained from video data to assess their psychological state and risk of falling.

[1142] "Means for generating and notifying alerts in real time when an emergency occurs" refers to a system that generates a warning message the moment something abnormal occurs in the person being cared for and immediately notifies the caregiver.

[1143] "Data collection" refers to the acquisition of video and environmental data from within nursing facilities and homes using cameras and sensors.

[1144] "Means for monitoring the progress of data transmission and retransmitting if an error occurs" is a function that monitors the progress of collected data being transmitted to the server in real time and automatically attempts to retransmit if an error occurs.

[1145] The "means for a caregiver who receives an emergency alert to respond immediately" refers to a notification and instruction function that enables a caregiver to respond quickly when an emergency alert is received.

[1146] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[1147] System Configuration

[1148] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[1149] Server Roles

[1150] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1151] Data reception

[1152] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1153] Data analysis

[1154] The server inputs the received video data into an AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. Specifically, AI frameworks such as TensorFlow and PyTorch are used. For example, if the AI ​​determines that the care recipient is feeling anxious, it outputs the result of the analysis.

[1155] Proposal generation

[1156] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. These suggestions include suggestions for safe layouts, how to interact with the person depending on their psychological state, and recommendations for appropriate care products. A generative AI model is used to generate the suggestions. For example, a suggestion might be, "The person receiving care seems to be feeling anxious. Create a calming environment and talk to them."

[1157] Emergency Alerts

[1158] In the event of an emergency, the server will immediately generate an alert and notify the caregiver, for example, an alert saying "The care recipient has fallen" will be sent immediately.

[1159] Calculation and payment of information fees

[1160] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1161] Device Role

[1162] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1163] Data collection

[1164] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1165] Data Preprocessing

[1166] The device compresses the collected data and performs necessary filtering to protect privacy, including masking personal information using facial recognition technology, such as blurring individuals' faces from recorded data.

[1167] Data transmission

[1168] Once preprocessing is complete, the data is sent from the device to the server, enabling efficient data utilization. The device transmits data via Wi-Fi or a wired network and has the ability to automatically attempt retransmission if an error occurs.

[1169] Real-time notifications

[1170] If an emergency occurs, the device will issue an alert on the spot, allowing caregivers to respond immediately. For example, if a fall is detected, the device will issue an alert on the spot.

[1171] User Roles

[1172] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1173] Receive advice

[1174] Caregivers can receive and check suggestions and advice from the server on their smartphones or dedicated devices. For example, they might receive a notification on their smartphone app saying, "The person being cared for may be feeling anxious. Please check immediately."

[1175] Real-time support

[1176] When an emergency alert is received, the caregiver will immediately rush to the scene and take appropriate action. For example, if an alert is received that a person is likely to fall, the caregiver will quickly rush to the care recipient's side.

[1177] Information provision

[1178] Local residents provide daily care data and proposal information to the server. This strengthens the support system throughout the community. In doing so, they can receive a fee for providing the information. For example, they can provide daily monitoring information to the server and receive a fee in return.

[1179] Prompt Sentence Examples

[1180] Below is an example of a prompt sentence to input to the generative AI model.

[1181] Sample prompt: "Detect the risk of a care recipient falling while walking in the living room and provide appropriate advice to the caregiver."

[1182] The above is a specific embodiment of the system of the present invention. This system allows caregivers to receive appropriate suggestions and respond quickly in emergencies. It also encourages information provision from local residents and improves the overall quality of care.

[1183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1184] Step 1: Data collection

[1185] The device collects video data and environmental data using cameras and sensors installed in nursing care facilities or homes. The device continuously records the room of the care recipient, acquiring video data of the living room, for example. This data is temporarily stored in local storage. The input is real-time video data, and the output is collected video data.

[1186] Step 2: Data Preprocessing

[1187] The device compresses the collected video data and filters it for privacy purposes, for example, by using facial recognition technology to blur the faces of people other than the care recipient. The input is the collected video data, and the output is the pre-processed video data.

[1188] Step 3: Send data

[1189] The preprocessed data is sent from the device to the server. The device transmits the data over Wi-Fi or a wired network and monitors the progress of the transmission. If an error occurs, the device automatically attempts to retransmit. The input is the preprocessed video data, and the output is the data sent to the server.

[1190] Step 4: Receiving Data

[1191] The server receives video data sent from the device via the Internet. The server stores the received data in a storage device and prepares it for analysis. The input is the data sent from the device, and the output is the stored video data.

[1192] Step 5: Data analysis

[1193] The server inputs the received video data into an AI model for analysis. This analysis detects the facial expressions, movements, and environmental changes of the care recipient to determine their psychological state and risk of falling. For example, using TensorFlow or PyTorch, the analysis result may be "the care recipient is feeling anxious." The input is the received video data, and the output is the analysis result.

[1194] Step 6: Proposal Generation

[1195] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. Using a generative AI model, it creates suggestions such as, "The care recipient seems to be feeling anxious. Create a calming environment and talk to them." The input is the analysis results, and the output is the generated suggestions.

[1196] Step 7: Proposal Notification

[1197] The server notifies the caregiver of the generated suggestion by sending a notification message to the caregiver's smartphone or dedicated device. The input is the generated suggestion, and the output is the sent notification.

[1198] Step 8: Emergency Alerts

[1199] When an emergency occurs, the server immediately generates an alert and notifies the caregiver. For example, it generates and sends an alert saying, "The care recipient has fallen." The input is the emergency analysis result, and the output is an emergency alert.

[1200] Step 9: Calculation and payment of information fee

[1201] The server calculates the information provision fee according to the amount of care information provided by local residents and processes the payment. The server calculates the reward using a point system and pays via digital wallet or bank transfer. The input is the provided information data, and the output is the calculated information provision fee and payment information.

[1202] Step 10: Receive advice

[1203] The user receives and checks the suggestions and advice sent from the server on a smartphone or dedicated device. For example, they may receive a notification saying, "The care recipient may be feeling anxious. Please check immediately." The input is the notification from the server, and the output is the suggestions displayed on the user's device.

[1204] Step 11: Real-time support

[1205] When a user receives an emergency alert, they immediately rush to the scene to respond. For example, when an alert is received indicating a high possibility of a fall, a caregiver will quickly rush to the care recipient's side. The input is the emergency alert, and the output is the user's response action.

[1206] Step 12: Provide information

[1207] Users provide daily care data and suggestion information to the server. Local residents input observation data and insights into the app and send them to the server. The input is the observation data and suggestion information, and the output is the information sent to the server.

[1208] (Application example 1)

[1209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1210] In addition to the challenge of ensuring the safety of care recipients while reducing the burden on caregivers in nursing homes and homes, it is also necessary to monitor the safety of workers in factories and improve work efficiency. However, with current systems, it is difficult to solve both challenges simultaneously.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1212] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions, means for calculating and paying an information provision fee from the caregiver, means for receiving video data from a factory and analyzing the movements of workers and machines, means for immediately issuing emergency alerts based on the analysis, and means for generating suggestions to improve work efficiency and notifying a manager. This enables safety to be ensured and efficient operation to be achieved in both nursing care and factory environments.

[1213] "Means for receiving video data from within a care facility or home" refers to a device or system that has the function of collecting video data transmitted from cameras or sensors installed in a care facility or home.

[1214] "Artificial intelligence means for analyzing the video data" refers to a system that uses AI technology to analyze the received video data and executes a process to determine the condition and situation of the care recipient or worker.

[1215] The "means for generating suggestions to caregivers based on the analysis results" refers to an algorithm or program that generates useful suggestions or advice for caregivers based on the analysis results.

[1216] The "means for notifying the caregiver of the suggestions" refers to a communication means or application that notifies the caregiver of the generated suggestions or advice to their smartphone or dedicated terminal.

[1217] The "means for calculating and paying the information provision fee from the caregiver" is a system for calculating and paying the information provision fee in return based on the information provided by the caregiver or local residents.

[1218] "Means for receiving video data from within the factory and analyzing the movements of workers and machines" refers to a system that has the function of collecting video data sent from cameras and sensors installed within the factory and analyzing the movements of workers and machines based on that data.

[1219] The "means for immediately issuing an emergency alert based on the analysis" refers to a communication means or application that generates an alert in real time and notifies an administrator if an abnormality is detected during the analysis.

[1220] The "means for generating proposals for improving work efficiency and notifying the administrator" refers to an algorithm or program that generates proposals and improvement measures for improving work efficiency from the analysis results and notifies the administrator.

[1221] "Means for collecting video data using cameras and sensors installed in nursing care facilities or homes" refers to devices or systems that have the function of collecting video data from cameras and sensors installed in nursing care facilities or homes.

[1222] The "means for compressing and filtering the video data" refers to a system that compresses collected video data and performs filtering processing to protect privacy and reduce data volume.

[1223] The "means for transmitting the collected data to the server" refers to a communication means or protocol for transmitting the pre-processed video data to the server.

[1224] It is a "Japanese company."

[1225] "Means for collecting and preprocessing data from cameras and sensors installed within the factory" refers to a system that collects video data from cameras and sensors installed within the factory, preprocesses it, and puts it in a format suitable for analysis.

[1226] The "means for transmitting collected data to a cloud service" refers to a communication means or protocol for transmitting preprocessed data to a cloud service server.

[1227] "The facial expression of the person receiving care" is a change in facial expression.

[1228] High quality data and the ability to send alerts in case of emergency are required.

[1229] It is useful for management.

[1230] Overall system configuration

[1231] This invention is a system that collects and analyzes video data from nursing homes, homes, and factories. The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in the nursing home, home, or factory and collects video data and sends it to the server. The server analyzes the video data and generates optimal suggestions and emergency alerts for caregivers and factory managers. The user receives this information and takes appropriate action.

[1232] Server Roles

[1233] The roles of the server are as follows:

[1234] 1. Data reception: The server receives video data sent via the Internet from nursing care facilities, homes, and factories, allowing real-time information on the living and working conditions of care recipients and workers.

[1235] 2. Data analysis: The server inputs the received video data into the generative AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient and caregiver, as well as changes in the surrounding environment, and determines their psychological state, risk of falls, safety risks, etc.

[1236] 3. Proposal Generation: Based on the analysis results, the server generates appropriate suggestions for caregivers and suggestions and advice for factory managers to improve work efficiency. These suggestions include recommendations for safe layouts, how to interact with employees depending on their psychological state, and efficient work methods.

[1237] 4. Emergency Alert: In case of an emergency, the server will immediately generate an alert to notify caregivers or factory managers, allowing for prompt action.

[1238] 5. Calculation and payment of information fees: The server calculates the information fees according to the amount of information provided by local residents and workers, and processes the payments. This promotes information gathering across the entire region or factory.

[1239] Device Role

[1240] The roles of the terminal are as follows:

[1241] 1. Data Collection: Cameras and sensors connected to devices collect video and environmental data 24 / 7. For example, cameras in nursing homes continuously record activity in living rooms, or cameras in factories monitor activity in work areas.

[1242] 2. Data preprocessing: The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1243] 3. Data transmission: After preprocessing, the data is transmitted from the terminal to the server, enabling efficient data utilization.

[1244] 4. Real-time notification: The device will generate an immediate alert if an emergency occurs, allowing caregivers or factory managers to respond immediately.

[1245] User Roles

[1246] The user roles are as follows:

[1247] 1. Receiving advice: Caregivers and factory managers receive and review suggestions and advice from the server on their smartphones or dedicated devices. For example, they may receive suggestions on how to deal with care recipients who are in an unstable psychological state or specific work methods to improve work efficiency.

[1248] 2. Real-time response: When an emergency alert is received, caregivers and factory managers rush to the scene and take appropriate action, such as tidying up the surrounding environment if the risk of falling is deemed high, or temporarily suspending work if the work risk is deemed high.

[1249] 3. Information provision: Local residents and factory workers provide daily care and work data to the server. This strengthens the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1250] Examples of specific examples and prompts

[1251] For example, if heavy machinery in a factory makes a dangerous move, AI will detect the abnormality and immediately send an alert to the manager via email. It will also analyze the movements of workers and make suggestions to improve work efficiency.

[1252] Example prompts to input to a generative AI model:

[1253] "Write a Python program to monitor the movement of workers and heavy equipment in a factory, assess safety in real time, and generate alerts if anything goes wrong."

[1254] In this way, the system of the present invention is realized through cooperation between servers, terminals, and users, reducing the burden on caregivers and factory managers, ensuring appropriate care and safety for care recipients and workers, and improving work efficiency.

[1255] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1256] Step 1: Data collection

[1257] The device collects video data using cameras and sensors installed in nursing homes, homes, and factories. This data monitors the dynamics of care recipients, workers, and the environment 24 / 7. Specifically, cameras continuously record images of living rooms and factory work areas, while sensors detect changes in the environment. The input is real-time data from the cameras and sensors, and the output is the collected video data and sensor data.

[1258] Step 2: Data Preprocessing

[1259] The device compresses the collected data and performs necessary filtering to protect privacy. Specifically, it uses facial recognition technology to mask personal information from the collected data. This process reduces the amount of data and improves transmission efficiency. The input is the collected video data and sensor data, and the output is the compressed and filtered data.

[1260] Step 3: Send data

[1261] The terminal sends the preprocessed data to the server. Specifically, the compressed and filtered data is uploaded to the cloud server via the Internet. The input is the preprocessed data, and the output is the data sent to the server.

[1262] Step 4: Data analysis

[1263] The server analyzes the received data. Specifically, it uses a generative AI model to analyze the data and evaluate the condition of the care recipient and worker, as well as changes in the surrounding environment. This analysis determines factors such as fall risks, safety risks, and efficient work methods. The input is the data sent to the server, and the output is the analysis results.

[1264] Step 5: Proposal Generation

[1265] The server generates suggestions and advice for caregivers and factory managers based on the analysis results. Specifically, it creates suggestions including recommendations for safe layouts, how to interact with patients based on their psychological state, and efficient work methods. The input is the analysis results, and the output is the generated suggestions.

[1266] Step 6: Notification

[1267] The server notifies the caregiver or factory manager of the generated suggestions and advice. Specifically, it sends notifications to smartphones or dedicated devices and provides emergency alerts in real time. The input is the generated suggestions, and the output is the notified information.

[1268] Step 7: Emergency response

[1269] Users (caregivers and factory managers) receive emergency alerts from the server and immediately rush to the scene to take appropriate action. Specifically, if a high risk of falling is determined, they will organize the environment around the care recipient, and if a dangerous situation occurs in the factory, they will temporarily suspend work. The input is the alert notification from the server, and the output is the response action.

[1270] Step 8: Calculation and payment of information fee

[1271] The server calculates the information fee based on the information provided by local residents and workers, and processes the payment. Specifically, it evaluates the quantity and quality of the information provided, determines the reward, and makes the payment. The input is the provided information, and the output is the calculated information fee and payment processing.

[1272] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1273] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides suggestions and emergency alerts to caregivers. It also includes the calculation and payment of information provision fees from local residents.

[1274] System Configuration

[1275] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies the caregiver of optimal suggestions and emergency alerts. Furthermore, by combining it with an emotion engine, the system recognizes the emotions of the care recipient and makes suggestions based on those emotions. The user receives this information and takes appropriate action.

[1276] Server Roles

[1277] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1278] Data reception

[1279] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1280] Data analysis

[1281] The server inputs the received video data into the AI ​​and emotion engine and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. The emotion engine also simultaneously analyzes the care recipient's emotions.

[1282] Proposal generation

[1283] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver, such as suggestions for safe layouts, how to interact with the patient based on their psychological state, recommendations for appropriate care products, and how to respond based on their emotions.

[1284] sentiment analysis

[1285] The emotion engine recognizes and determines the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling anxious or stressed, the analysis results are fed back to the server.

[1286] Emergency Alerts

[1287] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[1288] Calculation and payment of information fees

[1289] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1290] Device Role

[1291] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1292] Data collection

[1293] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1294] Data Preprocessing

[1295] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1296] Data transmission

[1297] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[1298] Real-time notifications

[1299] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[1300] User Roles

[1301] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1302] Receive advice

[1303] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[1304] Emotion-based responses

[1305] Based on the analysis results of the emotion engine, the caregiver will respond according to the care recipient's emotions. For example, if the care recipient is feeling stressed, the system will suggest actions such as playing relaxing music.

[1306] Real-time support

[1307] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[1308] Information provision

[1309] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1310] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and provides appropriate care to those receiving care. Furthermore, by combining it with an emotion engine, it becomes possible to provide advanced responses based on the emotions of the care recipient.

[1311] The processing flow will be explained below.

[1312] Step 1: Data collection

[1313] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[1314] Step 2: Data Preprocessing

[1315] The device compresses the collected video data and performs necessary filtering (such as masking personal information using facial recognition technology) to ensure that the data complies with privacy standards.

[1316] Step 3: Send data

[1317] The terminal transmits the pre-processed data to the server. This transmission is carried out over the Internet and arrives at the server as packets.

[1318] Step 4: Receiving Data

[1319] The server receives the video data transmitted from the terminal via the Internet in real time and receives the data packets using a receiving socket.

[1320] Step 5: Data analysis

[1321] The server inputs the received video data into the AI ​​model and emotion engine and begins analysis. This analysis monitors the care recipient's facial expressions, movements, and changes in the surrounding environment to evaluate their psychological state, risk of falling, and emotions (e.g., joy, anger, sadness, and happiness).

[1322] Step 6: Sentiment analysis

[1323] The server uses an emotion engine to recognize the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling stressed, the emotion engine immediately sends the analysis results back to the server.

[1324] Step 7: Proposal Generation

[1325] Based on the results of data analysis and sentiment analysis, the server generates optimal suggestions and advice for caregivers, including suggestions for safe layouts, how to interact with patients based on their psychological state, recommendations for appropriate caregiving products, and emotional responses.

[1326] Step 8: Proposal Notification

[1327] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[1328] Step 9: Emergency Notification

[1329] If the server detects an emergency situation in the care recipient (e.g., a fall or a sudden change in physical condition), it immediately generates an emergency alert and notifies the caregiver, enabling a prompt response.

[1330] Step 10: Provide information

[1331] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout and how they interact with the care recipient based on their emotions.

[1332] Step 11: Calculation and payment of information fee

[1333] The server calculates the fee for providing information based on the care information provided by local residents. It determines the fee based on the frequency and quality of the information provided and processes the payment. This promotes information collection throughout the community.

[1334] Example 2

[1335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1336] To improve the quality of care in nursing facilities and homes, it is necessary to grasp the condition of care recipients in real time and respond promptly and appropriately. However, currently, there are limited means of understanding the emotions and condition of care recipients, which may result in delayed responses to emergencies. In addition, there is a lack of a system for efficiently utilizing information provided by local residents and appropriately rewarding them for providing that information. A system is needed to solve these problems and provide a safe and secure environment for both caregivers and care recipients.

[1337] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1338] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for analyzing the facial expressions, movements, and environmental changes of the care recipient in the video data and collected data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions and emergency alerts, and means for calculating and paying fees for information provided by the caregivers. This enables real-time understanding of the care recipient's condition and recognition of their emotions, enabling prompt and appropriate responses. Furthermore, by appropriately managing and paying fees for information provided by local residents, the quality and quantity of information can be improved, strengthening the nursing care support system throughout the community.

[1339] "Video data from nursing homes and homes" refers to video and environmental information obtained from cameras and sensors installed in nursing homes and homes.

[1340] An "artificial intelligence means" is a computer system that performs data analysis using machine learning algorithms and deep learning.

[1341] The "means for generating suggestions" is a system that generates appropriate suggestions and advice for the caregiver based on the analyzed data.

[1342] "Means of notifying caregivers" refers to a system that notifies caregivers of suggestions and emergency alerts via their smartphones or dedicated devices.

[1343] The "means for calculating and paying information fees" is a system that calculates fees based on the quantity and quality of information provided by local residents and pays them electronically.

[1344] "Means for collecting video data using cameras and sensors" refers to a system that collects data using cameras and sensors installed in nursing facilities and homes.

[1345] "Means for compressing data and filtering for privacy protection" refers to technologies that compress collected video data for efficient storage and transmission, and perform masking and filtering to protect personal information.

[1346] "Means for transmitting data to a server" refers to a mechanism for transmitting collected data to a server via the Internet.

[1347] The "emotion engine" is an algorithm that analyzes and determines the emotions of the person receiving care from their facial expressions and voice.

[1348] "Means for notifying caregivers of emergency alerts in real time" refers to a system that immediately sends an alert to caregivers when an emergency situation is detected.

[1349] The system of the present invention receives video data from nursing homes and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides optimal suggestions and emergency alerts to caregivers. It also calculates and pays rewards for information provided by local residents. The detailed configuration and operation of the system are described below.

[1350] Server Roles

[1351] The server acts as a central device and performs the following processes:

[1352] 1. Data Reception

[1353] The server receives video data sent from care facilities and homes via the Internet.

[1354] Specific hardware: A high-performance server is required, such as a data center server equipped with an Intel Xeon processor.

[1355] Specific software: RTSP (Real-Time Streaming Protocol) is used for data streaming.

[1356] 2. Data Analysis

[1357] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[1358] Specific hardware: A server equipped with a GPU (NVIDIA Tesla) is used.

[1359] Specific software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[1360] 3. Proposal generation

[1361] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver.

[1362] Specific software: A rule-based expert system is used to generate recommendations based on the analysis results.

[1363] 4. Sentiment analysis

[1364] The emotion engine identifies the emotions of the care recipient and feeds the results back to the server.

[1365] Specific software: Libraries such as OpenCV and Amazon Rekognition are used for emotion recognition.

[1366] 5. Emergency Alerts

[1367] In the event of an emergency, the server will instantly generate an alert and notify the caregiver.

[1368] Specific operation: Use an anomaly detection algorithm (e.g., a fall detection algorithm).

[1369] 6. Calculation and payment of information fees

[1370] The server calculates and pays rewards based on the amount of information provided by local residents.

[1371] Specific software: SQL databases and electronic payment systems (PayPal API and Stripe API).

[1372] Device Role

[1373] The devices are installed in nursing homes and homes and perform the following functions:

[1374] 1. Data Collection

[1375] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[1376] Specific hardware: IP cameras (e.g., Axis), various sensors (e.g., temperature and humidity sensors).

[1377] Example: An in-room camera continuously records what is happening in the living room.

[1378] 2. Data Preprocessing

[1379] The collected data is compressed and filtered for privacy purposes.

[1380] Specific software: Personal information masking using facial recognition technology and H.264 / HEVC compression technology.

[1381] 3. Data Transmission

[1382] The pre-processed data is sent to the server.

[1383] Specific software: Data is transmitted using HTTPS (Hypertext Transfer Protocol Secure).

[1384] 4. Real-time notifications

[1385] If an emergency occurs, an alert will be generated immediately.

[1386] Specific operation: When the device detects data that exceeds a certain threshold, it will sound an alert using the built-in speaker and light.

[1387] User Roles

[1388] Users interact with the system as caregivers or local residents and perform the following functions:

[1389] 1. Receiving advice

[1390] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[1391] Example: Caregivers can view the suggestions on a smartphone app.

[1392] 2. Emotional responses

[1393] Caregivers respond appropriately based on the analysis results of the emotion engine.

[1394] Example: If the care recipient is feeling stressed, the app will suggest playing relaxing music, and the user can follow the suggestion.

[1395] 3. Real-time support

[1396] If an emergency alert is received, we will rush to the scene immediately.

[1397] Example: A caregiver is notified of a fall risk and rushes to the scene to respond.

[1398] 4. Information provision

[1399] Local residents provide daily care data and suggestion information to the server and receive rewards.

[1400] Example: Residents enter their daily observations and suggestions through a dedicated smartphone app, which are then sent to a server. Rewards are then paid at the end of the month.

[1401] Prompt Sentence Examples

[1402] "Please suggest appropriate ways to respond when the person you are caring for is feeling anxious."

[1403] By combining the above elements, a more advanced and secure care system can be realized for both caregivers and those receiving care.

[1404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1405] Server Processing

[1406] Step 1: Receiving data

[1407] The server receives video data transmitted from care facilities and homes via the Internet.

[1408] Input: Real-time streaming data transmitted via RTSP.

[1409] Output: Received video data.

[1410] Specific operation: The server monitors the specified IP address and port and captures streaming data in real time.

[1411] Step 2: Data analysis

[1412] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[1413] Input: Received video data.

[1414] Output: Analysis results (e.g., facial expressions, movements, and environmental information of the care recipient).

[1415] Specific operation: The AI ​​module processes the video data frame by frame and performs facial recognition and movement analysis using deep learning (TensorFlow or PyTorch). The emotion engine (Amazon Rekognition, etc.) analyzes the facial expressions and voice data of the care recipient to identify their emotions.

[1416] Step 3: Proposal Generation

[1417] The server generates appropriate suggestions and advice for the caregiver based on the analysis results.

[1418] Input: Results of data analysis.

[1419] Output: Suggestions and advice.

[1420] How it works: A rule-based expert system generates suggestions based on the analysis results and in accordance with predefined patterns and rules.

[1421] Step 4: Emergency alert generation

[1422] If the server detects an emergency, it will immediately generate an alert and notify the caregiver.

[1423] Input: Results of data analysis (detection of emergency situations).

[1424] Output: Emergency alert notification.

[1425] Specific behavior: Anomaly detection algorithms identify high-risk behaviors (e.g., falls) in real time and send emergency notifications via email or SMS.

[1426] Step 5: Calculation and payment of information fee

[1427] The server calculates and pays rewards based on the amount of information provided by local residents.

[1428] Input: Informational log.

[1429] Output: Calculated reward amount and notification of payment completion.

[1430] Specific operation: The server analyzes the information provision log in the database, calculates the reward amount, and then pays the reward using an electronic payment system (PayPal or Stripe).

[1431] Terminal handling

[1432] Step 1: Data collection

[1433] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[1434] Input: The care recipient's real-time living environment.

[1435] Output: Collected video and environmental data.

[1436] Specific operation: IP cameras and temperature and humidity sensors continuously collect data and temporarily store it locally.

[1437] Step 2: Data Preprocessing

[1438] The device compresses the collected data and filters it for privacy purposes.

[1439] Input: Collected video and environmental data.

[1440] Output: Preprocessed data.

[1441] Specific operation: Uses facial recognition algorithms to mask personal information and compresses data using H.264 / HEVC compression technology.

[1442] Step 3: Send data

[1443] The pre-processed data is sent to a server.

[1444] Input: Preprocessed data.

[1445] Output: The data sent to the server.

[1446] Specific operation: Encrypts data using the HTTPS protocol and sends a POST request to the specified server URL.

[1447] Step 4: Real-time notifications

[1448] In the event of an emergency, the device will issue an alert on the spot.

[1449] Input: Emergency detection.

[1450] Output: Alert notification.

[1451] Specific operation: When the sensor inside the device detects data that exceeds the threshold, an alarm will sound from the built-in speaker and an emergency message will be displayed on the screen.

[1452] User Action

[1453] Step 1: Receive advice

[1454] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[1455] Input: Notification of suggestions and advice.

[1456] Output: Caregiver reactions and how they responded.

[1457] Specific operation: The smartphone app receives a push notification from the server and displays the content.

[1458] Step 2: Respond based on emotions

[1459] The caregiver responds based on the results of the emotion analysis of the care recipient.

[1460] Input: Sentiment analysis results.

[1461] Output: Implementing the response method.

[1462] Specific operations: Commands such as playing relaxing music and changing the color of smart lights can be given through a smartphone app.

[1463] Step 3: Real-time response

[1464] When an emergency alert is received, caregivers will rush to the scene immediately.

[1465] Input: Emergency alert notification.

[1466] Output: On-site response actions.

[1467] Specific actions: Upon receiving an emergency notification, the caregiver rushes to the scene and takes action such as assisting the person being cared for who has fallen.

[1468] Step 4: Provide information

[1469] Local residents provide daily care data and suggestion information to the server and receive rewards.

[1470] Input: Daily care data and proposal information.

[1471] Output: Receipt of reward.

[1472] Specific operation: Data is entered and sent through a dedicated smartphone app, and the server calculates the reward amount at the end of the month and makes the payment electronically.

[1473] (Application example 2)

[1474] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1475] In nursing care facilities and at home, it is necessary to monitor the safety and health of elderly people and those receiving care in real time and to take appropriate measures promptly. However, current systems face many challenges, such as limited human resources and difficulties in early detection of emergencies. In addition, there is a lack of mechanisms for efficiently managing information provided by local residents and promoting their cooperation. As a result, there is a risk that the burden on caregivers will increase and the quality of life of those receiving care will decline.

[1476] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1477] In this invention, the server includes a means for receiving video data from a nursing facility or home, an artificial intelligence means for analyzing the video data, a means for generating an alert in the event of an emergency and notifying the alert via a communication device, and a means for calculating and paying an information provision fee based on the amount of information provided by local residents. This enables real-time monitoring of the safety and health of care recipients and rapid response in emergencies. Furthermore, by efficiently managing information provided by local residents and promoting their cooperation, it is expected that the burden on caregivers will be reduced and the quality of life of care recipients will be improved.

[1478] A "nursing home" is a facility that provides necessary care and support to elderly people and people with physical disabilities who require nursing care.

[1479] "Within the home" refers to the interior of an individual home, that is, the dwelling, where the people who live there go about their daily lives.

[1480] "Video data" refers to information about moving images or videos captured by a video capture device such as a camera.

[1481] "Artificial intelligence" is a general term for technologies that mimic human intelligence and includes algorithms for analyzing data and solving problems.

[1482] An "alert" is a warning or notification that notifies of an emergency or abnormality.

[1483] "Communications equipment" means equipment for sending and receiving data and information.

[1484] "Local residents" refers to people who live within a certain area.

[1485] "Information fee" means the remuneration paid for providing information.

[1486] "Data compression" is a technical process used to reduce the volume of data.

[1487] "Filtering" is a technical technique for removing or extracting specific elements from data.

[1488] "Analysis" means investigating and analyzing data in detail to derive its meaning and trends.

[1489] An "emotion engine" is a technology for recognizing and analyzing human emotions from video and audio data.

[1490] The system of the present invention receives and analyzes video data from within a care facility or home, and provides suggestions and emergency alerts to caregivers. The system is composed of three parties: a server, a terminal, and a user, and functions as follows.

[1491] Server Roles

[1492] Data reception:

[1493] The server receives video data in real time transmitted from nursing homes and homes via the Internet. The hardware used is a high-performance data server, and the software used is FFmpeg for data streaming.

[1494] Data Analysis:

[1495] The received video data is analyzed using an AI model and emotion engine, using TensorFlow and Keras software. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling.

[1496] Suggestion generation:

[1497] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[1498] Emergency Alert:

[1499] If an emergency is detected, the server immediately generates an alert and notifies the caregiver via communication devices, using the Twilio API for SMS notifications and AWS SNS for push notifications.

[1500] Calculation and payment of information fees:

[1501] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[1502] Device Role

[1503] Data collection:

[1504] The device operates cameras and sensors 24 / 7 to collect video and environmental data. The hardware used is a general-purpose surveillance camera and multi-purpose sensor.

[1505] Data preprocessing:

[1506] The device compresses the collected data, filters it for privacy, and uses facial recognition technology to mask personal information to protect privacy.

[1507] Data transmission:

[1508] The pre-processed data is sent from the device to the server, where it is efficiently transmitted using high-speed communication technology.

[1509] User Roles

[1510] Advice received:

[1511] Caregivers can receive suggestions and advice from the server on their smartphones or dedicated devices and check them according to the situation, for example, they can receive suggestions on how to deal with a care recipient who is in an unstable psychological state.

[1512] Emergency Response:

[1513] When an emergency alert is received, caregivers can immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of falling, they can take measures such as tidying up the surrounding environment.

[1514] Information provided:

[1515] Local residents will strengthen the support system by providing care data and proposal information to the server on a daily basis. By receiving a fee for providing information, cooperation throughout the community will be promoted.

[1516] Specific examples

[1517] For example, if a security camera detects abnormal activity at night, the server receives and analyzes the video data. If the AI ​​detects the presence of a suspicious individual, it uses the Twilio API to send an emergency alert via SMS to nearby security personnel. It also uses the Stripe API to reward local residents for providing information.

[1518] Prompt Sentence Examples

[1519] You are the developer of a system that uses AI models to analyze video data from nursing homes and homes to detect abnormal behavior and emotions. Collect video data and analyze it in real time using AI. In the event of an emergency, use the Twilio API to send SMS alerts and the Stripe API to process rewards.

[1520] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1521] Step 1:

[1522] Data collection

[1523] The device will:

[1524] The device collects video and environmental data 24 hours a day from cameras and sensors installed in care facilities and homes. The hardware used is a surveillance camera and a multi-purpose sensor, which continuously monitors the movements of the care recipient and their surrounding environment.

[1525] Input and Output:

[1526] Input: Real-time video data and environmental data from cameras and sensors

[1527] Output: Collected video data and sensor data

[1528] Step 2:

[1529] Data Preprocessing

[1530] The device will:

[1531] The device compresses the collected video and environmental data and filters it for privacy, specifically masking personal information using facial recognition technology.

[1532] Input and Output:

[1533] Input: Collected video data and environmental data

[1534] Output: Compressed and filtered data

[1535] Step 3:

[1536] Data transmission

[1537] The device will:

[1538] The device then transmits the pre-processed data to the server using high-speed communication technology, enabling efficient use of the data.

[1539] Input and Output:

[1540] Input: Compressed and filtered data

[1541] Output: Data sent to the server

[1542] Step 4:

[1543] Data reception

[1544] Server actions:

[1545] The server receives video and environmental data in real time, transmitted via the Internet, using FFmpeg software for data streaming.

[1546] Input and Output:

[1547] Input: Data sent from the terminal

[1548] Output: Received data

[1549] Step 5:

[1550] Data analysis

[1551] Server actions:

[1552] The server inputs the received video and environmental data into an AI model and emotion engine for analysis using TensorFlow and Keras software, which detects the facial expressions and movements of the care recipient and changes in the surrounding environment to determine their psychological state and risk of falling.

[1553] Input and Output:

[1554] Input: Received data

[1555] Output: Analysis results

[1556] Step 6:

[1557] Proposal generation

[1558] Server actions:

[1559] Based on the analysis results, the server generates optimal measures and advice for caregivers, such as how to interact with the care recipient based on their psychological state, recommendations for appropriate care products, and suggestions for safe layouts.

[1560] Input and Output:

[1561] Input: Analysis results

[1562] Output: Generated suggestions and advice

[1563] Step 7:

[1564] Emergency alert sending

[1565] Server actions:

[1566] If an emergency is detected, the server immediately generates an alert and notifies the user via communication devices. The Twilio API is used for SMS notifications, and AWS SNS is used for push notifications.

[1567] Input and Output:

[1568] Input: Analysis results

[1569] Output: Emergency alert notification

[1570] Step 8:

[1571] Information Fee Calculation and Payment

[1572] Server actions:

[1573] Based on the amount of care information provided by local residents, the system calculates the fee for providing information and processes payments. This process uses the web framework Django and the payment system Stripe API.

[1574] Input and Output:

[1575] Input: Information provided by local residents

[1576] Output: Calculation and payment of information fee

[1577] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1578] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1579] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1580] [Fourth embodiment]

[1581] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1582] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1583] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1584] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1585] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1586] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1587] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1588] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1589] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1592] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1593] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1594] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[1595] System Configuration

[1596] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[1597] Server Roles

[1598] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1599] Data reception

[1600] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1601] Data analysis

[1602] The server inputs the received video data into the AI ​​model and begins analysis, which detects the facial expressions and movements of the care recipient as well as changes in the surrounding environment to determine their psychological state and risk of falling.

[1603] Proposal generation

[1604] Based on the analysis results, the server generates appropriate suggestions and advice for caregivers, such as suggestions for safe layouts, how to interact with patients depending on their psychological state, and recommendations for appropriate care products.

[1605] Emergency Alerts

[1606] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[1607] Calculation and payment of information fees

[1608] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1609] Device Role

[1610] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1611] Data collection

[1612] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1613] Data Preprocessing

[1614] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1615] Data transmission

[1616] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[1617] Real-time notifications

[1618] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[1619] User Roles

[1620] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1621] Receive advice

[1622] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[1623] Real-time support

[1624] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[1625] Information provision

[1626] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1627] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and realizes appropriate care for those receiving care.

[1628] The processing flow will be explained below.

[1629] Step 1: Data collection

[1630] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[1631] Step 2: Data Preprocessing

[1632] The device compresses the collected video data and performs any necessary filtering, such as masking personal information using facial recognition technology.

[1633] Step 3: Send data

[1634] The terminal sends the pre-processed data to the server, which involves sending the packets over the Internet.

[1635] Step 4: Receiving Data

[1636] The server receives the video data sent from the device in real time. This is completed by opening a receiving socket and receiving the data packet.

[1637] Step 5: Data analysis

[1638] The server inputs the received video data into the AI ​​model and begins analysis, which monitors the facial expressions and movements of the care recipient as well as changes in the surrounding environment to assess their psychological state and risk of falling.

[1639] Step 6: Proposal Generation

[1640] Based on the analysis results, the server generates optimal suggestions and advice for caregivers, such as changing the room layout, safety measures, and how to interact with the patient depending on their psychological state.

[1641] Step 7: Proposal Notification

[1642] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[1643] Step 8: Emergency Notification

[1644] If the server detects an emergency situation in the care recipient (such as a fall or a sudden change in physical condition), it immediately generates an alert and notifies the caregiver.

[1645] Step 9: Provide information

[1646] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout or the way they interact with the person they are caring for.

[1647] Step 10: Calculation and payment of information fee

[1648] The server calculates the fee for providing information based on the care information provided by local residents, determines the reward based on the frequency and quality of the information provided, and processes the payment.

[1649] Example 1

[1650] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1651] Caring for care recipients in nursing homes and at home requires understanding their situation in real time and responding quickly, but existing systems have not been able to adequately achieve this. A particular challenge is accurately assessing the care recipient's psychological state and risk of falling, and quickly providing appropriate suggestions and emergency alerts. It is also important to properly calculate information provision fees and promote information collection with the cooperation of local residents. A new system is needed to solve these issues.

[1652] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1653] In this invention, the server includes: means for receiving video data from a nursing facility or home; artificial intelligence means for analyzing the video data; means for compressing and filtering the video data; means for generating suggestions to caregivers based on the analysis results; means for notifying the caregivers of the suggestions; means for calculating and paying a fee for providing information from the caregiver; means for analyzing the facial expressions and movements of the care recipient to determine their psychological state and risk of falling; and means for generating and notifying alerts in real time when an emergency occurs. This allows for real-time understanding of the care recipient's condition and rapid response. Furthermore, it can encourage information provision from local residents and improve the overall quality of care.

[1654] "Video data from nursing care facilities and homes" refers to video and image data that records the living environment and behavior of care recipients, obtained by cameras and sensors installed in nursing care facilities and homes.

[1655] "Artificial intelligence means" refers to technologies that use algorithms such as machine learning and deep learning to analyze data and detect specific patterns or anomalies.

[1656] "Compression and filtering" refers to the process of compressing large amounts of data to reduce its size so that it can be transmitted and stored efficiently, and the process of removing and processing unnecessary data or data related to privacy.

[1657] The "means for generating suggestions to caregivers based on the analysis results" is a system that automatically generates specific guidelines for action and advice to be provided to caregivers based on information obtained from the results of data analysis.

[1658] The "means for notifying the caregiver of the suggestion" is a function for sending the generated suggestion or advice as a message to the device used by the caregiver.

[1659] The "means for calculating and paying information fees" is a system for calculating and paying compensation based on the quantity and quality of information provided by local residents.

[1660] "Means for analyzing the facial expressions and movements of care recipients to determine their psychological state and risk of falling" refers to technology that analyzes facial expressions and movement data of care recipients obtained from video data to assess their psychological state and risk of falling.

[1661] "Means for generating and notifying alerts in real time when an emergency occurs" refers to a system that generates a warning message the moment something abnormal occurs in the person being cared for and immediately notifies the caregiver.

[1662] "Data collection" refers to the acquisition of video and environmental data from within nursing facilities and homes using cameras and sensors.

[1663] "Means for monitoring the progress of data transmission and retransmitting if an error occurs" is a function that monitors the progress of collected data being transmitted to the server in real time and automatically attempts to retransmit if an error occurs.

[1664] The "means for a caregiver who receives an emergency alert to respond immediately" refers to a notification and instruction function that enables a caregiver to respond quickly when an emergency alert is received.

[1665] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI), and provides suggestions to caregivers. It also includes an emergency alert function and calculation and payment of information provision fees.

[1666] System Configuration

[1667] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies optimal suggestions and emergency alerts to caregivers. The user receives this information and takes appropriate action.

[1668] Server Roles

[1669] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1670] Data reception

[1671] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1672] Data analysis

[1673] The server inputs the received video data into an AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. Specifically, AI frameworks such as TensorFlow and PyTorch are used. For example, if the AI ​​determines that the care recipient is feeling anxious, it outputs the result of the analysis.

[1674] Proposal generation

[1675] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. These suggestions include suggestions for safe layouts, how to interact with the person depending on their psychological state, and recommendations for appropriate care products. A generative AI model is used to generate the suggestions. For example, a suggestion might be, "The person receiving care seems to be feeling anxious. Create a calming environment and talk to them."

[1676] Emergency Alerts

[1677] In the event of an emergency, the server will immediately generate an alert and notify the caregiver, for example, an alert saying "The care recipient has fallen" will be sent immediately.

[1678] Calculation and payment of information fees

[1679] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1680] Device Role

[1681] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1682] Data collection

[1683] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1684] Data Preprocessing

[1685] The device compresses the collected data and performs necessary filtering to protect privacy, including masking personal information using facial recognition technology, such as blurring individuals' faces from recorded data.

[1686] Data transmission

[1687] Once preprocessing is complete, the data is sent from the device to the server, enabling efficient data utilization. The device transmits data via Wi-Fi or a wired network and has the ability to automatically attempt retransmission if an error occurs.

[1688] Real-time notifications

[1689] If an emergency occurs, the device will issue an alert on the spot, allowing caregivers to respond immediately. For example, if a fall is detected, the device will issue an alert on the spot.

[1690] User Roles

[1691] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1692] Receive advice

[1693] Caregivers can receive and check suggestions and advice from the server on their smartphones or dedicated devices. For example, they might receive a notification on their smartphone app saying, "The person being cared for may be feeling anxious. Please check immediately."

[1694] Real-time support

[1695] When an emergency alert is received, the caregiver will immediately rush to the scene and take appropriate action. For example, if an alert is received that a person is likely to fall, the caregiver will quickly rush to the care recipient's side.

[1696] Information provision

[1697] Local residents provide daily care data and proposal information to the server. This strengthens the support system throughout the community. In doing so, they can receive a fee for providing the information. For example, they can provide daily monitoring information to the server and receive a fee in return.

[1698] Prompt Sentence Examples

[1699] Below is an example of a prompt sentence to input to the generative AI model.

[1700] Sample prompt: "Detect the risk of a care recipient falling while walking in the living room and provide appropriate advice to the caregiver."

[1701] The above is a specific embodiment of the system of the present invention. This system allows caregivers to receive appropriate suggestions and respond quickly in emergencies. It also encourages information provision from local residents and improves the overall quality of care.

[1702] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1703] Step 1: Data collection

[1704] The device collects video data and environmental data using cameras and sensors installed in nursing care facilities or homes. The device continuously records the room of the care recipient, acquiring video data of the living room, for example. This data is temporarily stored in local storage. The input is real-time video data, and the output is collected video data.

[1705] Step 2: Data Preprocessing

[1706] The device compresses the collected video data and filters it for privacy purposes, for example, by using facial recognition technology to blur the faces of people other than the care recipient. The input is the collected video data, and the output is the pre-processed video data.

[1707] Step 3: Send data

[1708] The preprocessed data is sent from the device to the server. The device transmits the data over Wi-Fi or a wired network and monitors the progress of the transmission. If an error occurs, the device automatically attempts to retransmit. The input is the preprocessed video data, and the output is the data sent to the server.

[1709] Step 4: Receiving Data

[1710] The server receives video data sent from the device via the Internet. The server stores the received data in a storage device and prepares it for analysis. The input is the data sent from the device, and the output is the stored video data.

[1711] Step 5: Data analysis

[1712] The server inputs the received video data into an AI model for analysis. This analysis detects the facial expressions, movements, and environmental changes of the care recipient to determine their psychological state and risk of falling. For example, using TensorFlow or PyTorch, the analysis result may be "the care recipient is feeling anxious." The input is the received video data, and the output is the analysis result.

[1713] Step 6: Proposal Generation

[1714] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver. Using a generative AI model, it creates suggestions such as, "The care recipient seems to be feeling anxious. Create a calming environment and talk to them." The input is the analysis results, and the output is the generated suggestions.

[1715] Step 7: Proposal Notification

[1716] The server notifies the caregiver of the generated suggestion by sending a notification message to the caregiver's smartphone or dedicated device. The input is the generated suggestion, and the output is the sent notification.

[1717] Step 8: Emergency Alerts

[1718] When an emergency occurs, the server immediately generates an alert and notifies the caregiver. For example, it generates and sends an alert saying, "The care recipient has fallen." The input is the emergency analysis result, and the output is an emergency alert.

[1719] Step 9: Calculation and payment of information fee

[1720] The server calculates the information provision fee according to the amount of care information provided by local residents and processes the payment. The server calculates the reward using a point system and pays via digital wallet or bank transfer. The input is the provided information data, and the output is the calculated information provision fee and payment information.

[1721] Step 10: Receive advice

[1722] The user receives and checks the suggestions and advice sent from the server on a smartphone or dedicated device. For example, they may receive a notification saying, "The care recipient may be feeling anxious. Please check immediately." The input is the notification from the server, and the output is the suggestions displayed on the user's device.

[1723] Step 11: Real-time support

[1724] When a user receives an emergency alert, they immediately rush to the scene to respond. For example, when an alert is received indicating a high possibility of a fall, a caregiver will quickly rush to the care recipient's side. The input is the emergency alert, and the output is the user's response action.

[1725] Step 12: Provide information

[1726] Users provide daily care data and suggestion information to the server. Local residents input observation data and insights into the app and send them to the server. The input is the observation data and suggestion information, and the output is the information sent to the server.

[1727] (Application example 1)

[1728] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1729] In addition to the challenge of ensuring the safety of care recipients while reducing the burden on caregivers in nursing homes and homes, it is also necessary to monitor the safety of workers in factories and improve work efficiency. However, with current systems, it is difficult to solve both challenges simultaneously.

[1730] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1731] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions, means for calculating and paying an information provision fee from the caregiver, means for receiving video data from a factory and analyzing the movements of workers and machines, means for immediately issuing emergency alerts based on the analysis, and means for generating suggestions to improve work efficiency and notifying a manager. This enables safety to be ensured and efficient operation to be achieved in both nursing care and factory environments.

[1732] "Means for receiving video data from within a care facility or home" refers to a device or system that has the function of collecting video data transmitted from cameras or sensors installed in a care facility or home.

[1733] "Artificial intelligence means for analyzing the video data" refers to a system that uses AI technology to analyze the received video data and executes a process to determine the condition and situation of the care recipient or worker.

[1734] The "means for generating suggestions to caregivers based on the analysis results" refers to an algorithm or program that generates useful suggestions or advice for caregivers based on the analysis results.

[1735] The "means for notifying the caregiver of the suggestions" refers to a communication means or application that notifies the caregiver of the generated suggestions or advice to their smartphone or dedicated terminal.

[1736] The "means for calculating and paying the information provision fee from the caregiver" is a system for calculating and paying the information provision fee in return based on the information provided by the caregiver or local residents.

[1737] "Means for receiving video data from within the factory and analyzing the movements of workers and machines" refers to a system that has the function of collecting video data sent from cameras and sensors installed within the factory and analyzing the movements of workers and machines based on that data.

[1738] The "means for immediately issuing an emergency alert based on the analysis" refers to a communication means or application that generates an alert in real time and notifies an administrator if an abnormality is detected during the analysis.

[1739] The "means for generating proposals for improving work efficiency and notifying the administrator" refers to an algorithm or program that generates proposals and improvement measures for improving work efficiency from the analysis results and notifies the administrator.

[1740] "Means for collecting video data using cameras and sensors installed in nursing care facilities or homes" refers to devices or systems that have the function of collecting video data from cameras and sensors installed in nursing care facilities or homes.

[1741] The "means for compressing and filtering the video data" refers to a system that compresses collected video data and performs filtering processing to protect privacy and reduce data volume.

[1742] The "means for transmitting the collected data to the server" refers to a communication means or protocol for transmitting the pre-processed video data to the server.

[1743] It is a "Japanese company."

[1744] "Means for collecting and preprocessing data from cameras and sensors installed within the factory" refers to a system that collects video data from cameras and sensors installed within the factory, preprocesses it, and puts it in a format suitable for analysis.

[1745] The "means for transmitting collected data to a cloud service" refers to a communication means or protocol for transmitting preprocessed data to a cloud service server.

[1746] "The facial expression of the person receiving care" is a change in facial expression.

[1747] High quality data and the ability to send alerts in case of emergency are required.

[1748] It is useful for management.

[1749] Overall system configuration

[1750] This invention is a system that collects and analyzes video data from nursing homes, homes, and factories. The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in the nursing home, home, or factory and collects video data and sends it to the server. The server analyzes the video data and generates optimal suggestions and emergency alerts for caregivers and factory managers. The user receives this information and takes appropriate action.

[1751] Server Roles

[1752] The roles of the server are as follows:

[1753] 1. Data reception: The server receives video data sent via the Internet from nursing care facilities, homes, and factories, allowing real-time information on the living and working conditions of care recipients and workers.

[1754] 2. Data analysis: The server inputs the received video data into the generative AI model and begins analysis. This analysis detects the facial expressions and movements of the care recipient and caregiver, as well as changes in the surrounding environment, and determines their psychological state, risk of falls, safety risks, etc.

[1755] 3. Proposal Generation: Based on the analysis results, the server generates appropriate suggestions for caregivers and suggestions and advice for factory managers to improve work efficiency. These suggestions include recommendations for safe layouts, how to interact with employees depending on their psychological state, and efficient work methods.

[1756] 4. Emergency Alert: In case of an emergency, the server will immediately generate an alert to notify caregivers or factory managers, allowing for prompt action.

[1757] 5. Calculation and payment of information fees: The server calculates the information fees according to the amount of information provided by local residents and workers, and processes the payments. This promotes information gathering across the entire region or factory.

[1758] Device Role

[1759] The roles of the terminal are as follows:

[1760] 1. Data Collection: Cameras and sensors connected to devices collect video and environmental data 24 / 7. For example, cameras in nursing homes continuously record activity in living rooms, or cameras in factories monitor activity in work areas.

[1761] 2. Data preprocessing: The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1762] 3. Data transmission: After preprocessing, the data is transmitted from the terminal to the server, enabling efficient data utilization.

[1763] 4. Real-time notification: The device will generate an immediate alert if an emergency occurs, allowing caregivers or factory managers to respond immediately.

[1764] User Roles

[1765] The user roles are as follows:

[1766] 1. Receiving advice: Caregivers and factory managers receive and review suggestions and advice from the server on their smartphones or dedicated devices. For example, they may receive suggestions on how to deal with care recipients who are in an unstable psychological state or specific work methods to improve work efficiency.

[1767] 2. Real-time response: When an emergency alert is received, caregivers and factory managers rush to the scene and take appropriate action, such as tidying up the surrounding environment if the risk of falling is deemed high, or temporarily suspending work if the work risk is deemed high.

[1768] 3. Information provision: Local residents and factory workers provide daily care and work data to the server. This strengthens the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1769] Examples of specific examples and prompts

[1770] For example, if heavy machinery in a factory makes a dangerous move, AI will detect the abnormality and immediately send an alert to the manager via email. It will also analyze the movements of workers and make suggestions to improve work efficiency.

[1771] Example prompts to input to a generative AI model:

[1772] "Write a Python program to monitor the movement of workers and heavy equipment in a factory, assess safety in real time, and generate alerts if anything goes wrong."

[1773] In this way, the system of the present invention is realized through cooperation between servers, terminals, and users, reducing the burden on caregivers and factory managers, ensuring appropriate care and safety for care recipients and workers, and improving work efficiency.

[1774] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1775] Step 1: Data collection

[1776] The device collects video data using cameras and sensors installed in nursing homes, homes, and factories. This data monitors the dynamics of care recipients, workers, and the environment 24 / 7. Specifically, cameras continuously record images of living rooms and factory work areas, while sensors detect changes in the environment. The input is real-time data from the cameras and sensors, and the output is the collected video data and sensor data.

[1777] Step 2: Data Preprocessing

[1778] The device compresses the collected data and performs necessary filtering to protect privacy. Specifically, it uses facial recognition technology to mask personal information from the collected data. This process reduces the amount of data and improves transmission efficiency. The input is the collected video data and sensor data, and the output is the compressed and filtered data.

[1779] Step 3: Send data

[1780] The terminal sends the preprocessed data to the server. Specifically, the compressed and filtered data is uploaded to the cloud server via the Internet. The input is the preprocessed data, and the output is the data sent to the server.

[1781] Step 4: Data analysis

[1782] The server analyzes the received data. Specifically, it uses a generative AI model to analyze the data and evaluate the condition of the care recipient and worker, as well as changes in the surrounding environment. This analysis determines factors such as fall risks, safety risks, and efficient work methods. The input is the data sent to the server, and the output is the analysis results.

[1783] Step 5: Proposal Generation

[1784] The server generates suggestions and advice for caregivers and factory managers based on the analysis results. Specifically, it creates suggestions including recommendations for safe layouts, how to interact with patients based on their psychological state, and efficient work methods. The input is the analysis results, and the output is the generated suggestions.

[1785] Step 6: Notification

[1786] The server notifies the caregiver or factory manager of the generated suggestions and advice. Specifically, it sends notifications to smartphones or dedicated devices and provides emergency alerts in real time. The input is the generated suggestions, and the output is the notified information.

[1787] Step 7: Emergency response

[1788] Users (caregivers and factory managers) receive emergency alerts from the server and immediately rush to the scene to take appropriate action. Specifically, if a high risk of falling is determined, they will organize the environment around the care recipient, and if a dangerous situation occurs in the factory, they will temporarily suspend work. The input is the alert notification from the server, and the output is the response action.

[1789] Step 8: Calculation and payment of information fee

[1790] The server calculates the information fee based on the information provided by local residents and workers, and processes the payment. Specifically, it evaluates the quantity and quality of the information provided, determines the reward, and makes the payment. The input is the provided information, and the output is the calculated information fee and payment processing.

[1791] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1792] The system of the present invention receives video data from care facilities and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides suggestions and emergency alerts to caregivers. It also includes the calculation and payment of information provision fees from local residents.

[1793] System Configuration

[1794] The system mainly consists of three components: a server, a terminal, and a user. The terminal is installed in a care facility or at home and collects video data and sends it to the server. The server analyzes the video data and generates and notifies the caregiver of optimal suggestions and emergency alerts. Furthermore, by combining it with an emotion engine, the system recognizes the emotions of the care recipient and makes suggestions based on those emotions. The user receives this information and takes appropriate action.

[1795] Server Roles

[1796] The server is a central device that receives and analyzes video data sent from nursing homes and homes in real time. Specifically, the following processes are performed:

[1797] Data reception

[1798] The server receives video data sent from care facilities and homes via the Internet, allowing the real-time living conditions of care recipients to be monitored.

[1799] Data analysis

[1800] The server inputs the received video data into the AI ​​and emotion engine and begins analysis. This analysis detects the facial expressions and movements of the care recipient, as well as changes in the surrounding environment, and determines their psychological state and risk of falling. The emotion engine also simultaneously analyzes the care recipient's emotions.

[1801] Proposal generation

[1802] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver, such as suggestions for safe layouts, how to interact with the patient based on their psychological state, recommendations for appropriate care products, and how to respond based on their emotions.

[1803] sentiment analysis

[1804] The emotion engine recognizes and determines the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling anxious or stressed, the analysis results are fed back to the server.

[1805] Emergency Alerts

[1806] In the event of an emergency, the server immediately generates an alert and notifies caregivers, allowing for prompt response.

[1807] Calculation and payment of information fees

[1808] The server calculates the fee for providing information according to the amount of care information provided by local residents and processes the payment, which promotes information collection throughout the community.

[1809] Device Role

[1810] The devices are installed in nursing homes or homes and use cameras and sensors to collect data and send it to a server.

[1811] Data collection

[1812] The cameras and sensors connected to the device collect video and environmental data 24 / 7. For example, an in-room camera continuously records what is happening in the living room.

[1813] Data Preprocessing

[1814] The device compresses the collected data and performs necessary filtering for privacy purposes, including masking personal information using facial recognition technology.

[1815] Data transmission

[1816] After preprocessing is complete, the data is sent from the device to the server, allowing for efficient use of the data.

[1817] Real-time notifications

[1818] The device will issue an alert on the spot if an emergency occurs, allowing caregivers to respond immediately.

[1819] User Roles

[1820] Users interact with the system as caregivers or local residents, and are required to receive information from the server and take appropriate action.

[1821] Receive advice

[1822] Caregivers can receive and review suggestions and advice from the server on their smartphones or dedicated devices, such as suggestions on how to respond to a care recipient who is in an unstable psychological state.

[1823] Emotion-based responses

[1824] Based on the analysis results of the emotion engine, the caregiver will respond according to the care recipient's emotions. For example, if the care recipient is feeling stressed, the system will suggest actions such as playing relaxing music.

[1825] Real-time support

[1826] When an emergency alert is received, caregivers will immediately rush to the scene and take appropriate action. For example, if it is determined that there is a high risk of a fall, they will tidy up the surrounding environment.

[1827] Information provision

[1828] Local residents provide daily care data and proposal information to the server, strengthening the support system for the entire community. In doing so, they can receive a fee for providing the information.

[1829] As described above, the system of the present invention is realized by the cooperation of the server, terminals, and users, and reduces the burden on caregivers and provides appropriate care to those receiving care. Furthermore, by combining it with an emotion engine, it becomes possible to provide advanced responses based on the emotions of the care recipient.

[1830] The processing flow will be explained below.

[1831] Step 1: Data collection

[1832] The device uses cameras and sensors installed in care facilities and homes to collect video and environmental data 24 hours a day. For example, a camera in the living room continuously records the behavior and facial expressions of the care recipient.

[1833] Step 2: Data Preprocessing

[1834] The device compresses the collected video data and performs necessary filtering (such as masking personal information using facial recognition technology) to ensure that the data complies with privacy standards.

[1835] Step 3: Send data

[1836] The terminal transmits the pre-processed data to the server. This transmission is carried out over the Internet and arrives at the server as packets.

[1837] Step 4: Receiving Data

[1838] The server receives the video data transmitted from the terminal via the Internet in real time and receives the data packets using a receiving socket.

[1839] Step 5: Data analysis

[1840] The server inputs the received video data into the AI ​​model and emotion engine and begins analysis. This analysis monitors the care recipient's facial expressions, movements, and changes in the surrounding environment to evaluate their psychological state, risk of falling, and emotions (e.g., joy, anger, sadness, and happiness).

[1841] Step 6: Sentiment analysis

[1842] The server uses an emotion engine to recognize the emotions of the care recipient from their facial expressions and voice. For example, if the care recipient is feeling stressed, the emotion engine immediately sends the analysis results back to the server.

[1843] Step 7: Proposal Generation

[1844] Based on the results of data analysis and sentiment analysis, the server generates optimal suggestions and advice for caregivers, including suggestions for safe layouts, how to interact with patients based on their psychological state, recommendations for appropriate caregiving products, and emotional responses.

[1845] Step 8: Proposal Notification

[1846] The server notifies the caregiver of the generated suggestions and advice by sending a push notification to their smartphone or dedicated device.

[1847] Step 9: Emergency Notification

[1848] If the server detects an emergency situation in the care recipient (e.g., a fall or a sudden change in physical condition), it immediately generates an emergency alert and notifies the caregiver, enabling a prompt response.

[1849] Step 10: Provide information

[1850] The user (caregiver) receives suggestions and advice from the server and takes appropriate action based on them, such as changing the room layout and how they interact with the care recipient based on their emotions.

[1851] Step 11: Calculation and payment of information fee

[1852] The server calculates the fee for providing information based on the care information provided by local residents. It determines the fee based on the frequency and quality of the information provided and processes the payment. This promotes information collection throughout the community.

[1853] Example 2

[1854] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1855] To improve the quality of care in nursing facilities and homes, it is necessary to grasp the condition of care recipients in real time and respond promptly and appropriately. However, currently, there are limited means of understanding the emotions and condition of care recipients, which may result in delayed responses to emergencies. In addition, there is a lack of a system for efficiently utilizing information provided by local residents and appropriately rewarding them for providing that information. A system is needed to solve these problems and provide a safe and secure environment for both caregivers and care recipients.

[1856] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1857] In this invention, the server includes means for receiving video data from a nursing facility or home, artificial intelligence means for analyzing the video data, means for analyzing the facial expressions, movements, and environmental changes of the care recipient in the video data and collected data, means for generating suggestions to caregivers based on the analysis results, means for notifying the caregivers of the suggestions and emergency alerts, and means for calculating and paying fees for information provided by the caregivers. This enables real-time understanding of the care recipient's condition and recognition of their emotions, enabling prompt and appropriate responses. Furthermore, by appropriately managing and paying fees for information provided by local residents, the quality and quantity of information can be improved, strengthening the nursing care support system throughout the community.

[1858] "Video data from nursing homes and homes" refers to video and environmental information obtained from cameras and sensors installed in nursing homes and homes.

[1859] An "artificial intelligence means" is a computer system that performs data analysis using machine learning algorithms and deep learning.

[1860] The "means for generating suggestions" is a system that generates appropriate suggestions and advice for the caregiver based on the analyzed data.

[1861] "Means of notifying caregivers" refers to a system that notifies caregivers of suggestions and emergency alerts via their smartphones or dedicated devices.

[1862] The "means for calculating and paying information fees" is a system that calculates fees based on the quantity and quality of information provided by local residents and pays them electronically.

[1863] "Means for collecting video data using cameras and sensors" refers to a system that collects data using cameras and sensors installed in nursing facilities and homes.

[1864] "Means for compressing data and filtering for privacy protection" refers to technologies that compress collected video data for efficient storage and transmission, and perform masking and filtering to protect personal information.

[1865] "Means for transmitting data to a server" refers to a mechanism for transmitting collected data to a server via the Internet.

[1866] The "emotion engine" is an algorithm that analyzes and determines the emotions of the person receiving care from their facial expressions and voice.

[1867] "Means for notifying caregivers of emergency alerts in real time" refers to a system that immediately sends an alert to caregivers when an emergency situation is detected.

[1868] The system of the present invention receives video data from nursing homes and homes, analyzes the data using artificial intelligence (AI) and an emotion engine, and provides optimal suggestions and emergency alerts to caregivers. It also calculates and pays rewards for information provided by local residents. The detailed configuration and operation of the system are described below.

[1869] Server Roles

[1870] The server acts as a central device and performs the following processes:

[1871] 1. Data Reception

[1872] The server receives video data sent from care facilities and homes via the Internet.

[1873] Specific hardware: A high-performance server is required, such as a data center server equipped with an Intel Xeon processor.

[1874] Specific software: RTSP (Real-Time Streaming Protocol) is used for data streaming.

[1875] 2. Data Analysis

[1876] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[1877] Specific hardware: A server equipped with a GPU (NVIDIA Tesla) is used.

[1878] Specific software: Deep learning frameworks such as TensorFlow and PyTorch are used.

[1879] 3. Proposal generation

[1880] Based on the analysis results, the server generates appropriate suggestions and advice for the caregiver.

[1881] Specific software: A rule-based expert system is used to generate recommendations based on the analysis results.

[1882] 4. Sentiment analysis

[1883] The emotion engine identifies the emotions of the care recipient and feeds the results back to the server.

[1884] Specific software: Libraries such as OpenCV and Amazon Rekognition are used for emotion recognition.

[1885] 5. Emergency Alerts

[1886] In the event of an emergency, the server will instantly generate an alert and notify the caregiver.

[1887] Specific operation: Use an anomaly detection algorithm (e.g., a fall detection algorithm).

[1888] 6. Calculation and payment of information fees

[1889] The server calculates and pays rewards based on the amount of information provided by local residents.

[1890] Specific software: SQL databases and electronic payment systems (PayPal API and Stripe API).

[1891] Device Role

[1892] The devices are installed in nursing homes and homes and perform the following functions:

[1893] 1. Data Collection

[1894] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[1895] Specific hardware: IP cameras (e.g., Axis), various sensors (e.g., temperature and humidity sensors).

[1896] Example: An in-room camera continuously records what is happening in the living room.

[1897] 2. Data Preprocessing

[1898] The collected data is compressed and filtered for privacy purposes.

[1899] Specific software: Personal information masking using facial recognition technology and H.264 / HEVC compression technology.

[1900] 3. Data Transmission

[1901] The pre-processed data is sent to the server.

[1902] Specific software: Data is transmitted using HTTPS (Hypertext Transfer Protocol Secure).

[1903] 4. Real-time notifications

[1904] If an emergency occurs, an alert will be generated immediately.

[1905] Specific operation: When the device detects data that exceeds a certain threshold, it will sound an alert using the built-in speaker and light.

[1906] User Roles

[1907] Users interact with the system as caregivers or local residents and perform the following functions:

[1908] 1. Receiving advice

[1909] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[1910] Example: Caregivers can view the suggestions on a smartphone app.

[1911] 2. Emotional responses

[1912] Caregivers respond appropriately based on the analysis results of the emotion engine.

[1913] Example: If the care recipient is feeling stressed, the app will suggest playing relaxing music, and the user can follow the suggestion.

[1914] 3. Real-time support

[1915] If an emergency alert is received, we will rush to the scene immediately.

[1916] Example: A caregiver is notified of a fall risk and rushes to the scene to respond.

[1917] 4. Information provision

[1918] Local residents provide daily care data and suggestion information to the server and receive rewards.

[1919] Example: Residents enter their daily observations and suggestions through a dedicated smartphone app, which are then sent to a server. Rewards are then paid at the end of the month.

[1920] Prompt Sentence Examples

[1921] "Please suggest appropriate ways to respond when the person you are caring for is feeling anxious."

[1922] By combining the above elements, a more advanced and secure care system can be realized for both caregivers and those receiving care.

[1923] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1924] Server Processing

[1925] Step 1: Receiving data

[1926] The server receives video data transmitted from care facilities and homes via the Internet.

[1927] Input: Real-time streaming data transmitted via RTSP.

[1928] Output: Received video data.

[1929] Specific operation: The server monitors the specified IP address and port and captures streaming data in real time.

[1930] Step 2: Data analysis

[1931] The server inputs the received video data into the AI ​​and emotion engine for data analysis.

[1932] Input: Received video data.

[1933] Output: Analysis results (e.g., facial expressions, movements, and environmental information of the care recipient).

[1934] Specific operation: The AI ​​module processes the video data frame by frame and performs facial recognition and movement analysis using deep learning (TensorFlow or PyTorch). The emotion engine (Amazon Rekognition, etc.) analyzes the facial expressions and voice data of the care recipient to identify their emotions.

[1935] Step 3: Proposal Generation

[1936] The server generates appropriate suggestions and advice for the caregiver based on the analysis results.

[1937] Input: Results of data analysis.

[1938] Output: Suggestions and advice.

[1939] How it works: A rule-based expert system generates suggestions based on the analysis results and in accordance with predefined patterns and rules.

[1940] Step 4: Emergency alert generation

[1941] If the server detects an emergency, it will immediately generate an alert and notify the caregiver.

[1942] Input: Results of data analysis (detection of emergency situations).

[1943] Output: Emergency alert notification.

[1944] Specific behavior: Anomaly detection algorithms identify high-risk behaviors (e.g., falls) in real time and send emergency notifications via email or SMS.

[1945] Step 5: Calculation and payment of information fee

[1946] The server calculates and pays rewards based on the amount of information provided by local residents.

[1947] Input: Informational log.

[1948] Output: Calculated reward amount and notification of payment completion.

[1949] Specific operation: The server analyzes the information provision log in the database, calculates the reward amount, and then pays the reward using an electronic payment system (PayPal or Stripe).

[1950] Terminal handling

[1951] Step 1: Data collection

[1952] The device uses cameras and sensors to collect video and environmental data 24 hours a day.

[1953] Input: The care recipient's real-time living environment.

[1954] Output: Collected video and environmental data.

[1955] Specific operation: IP cameras and temperature and humidity sensors continuously collect data and temporarily store it locally.

[1956] Step 2: Data Preprocessing

[1957] The device compresses the collected data and filters it for privacy purposes.

[1958] Input: Collected video and environmental data.

[1959] Output: Preprocessed data.

[1960] Specific operation: Uses facial recognition algorithms to mask personal information and compresses data using H.264 / HEVC compression technology.

[1961] Step 3: Send data

[1962] The pre-processed data is sent to a server.

[1963] Input: Preprocessed data.

[1964] Output: The data sent to the server.

[1965] Specific operation: Encrypts data using the HTTPS protocol and sends a POST request to the specified server URL.

[1966] Step 4: Real-time notifications

[1967] In the event of an emergency, the device will issue an alert on the spot.

[1968] Input: Emergency detection.

[1969] Output: Alert notification.

[1970] Specific operation: When the sensor inside the device detects data that exceeds the threshold, an alarm will sound from the built-in speaker and an emergency message will be displayed on the screen.

[1971] User Action

[1972] Step 1: Receive advice

[1973] Caregivers receive and check suggestions and advice from the server on their smartphones or dedicated devices.

[1974] Input: Notification of suggestions and advice.

[1975] Output: Caregiver reactions and how they responded.

[1976] Specific operation: The smartphone app receives a push notification from the server and displays the content.

[1977] Step 2: Respond based on emotions

[1978] The caregiver responds based on the results of the emotion analysis of the care recipient.

[1979] Input: Sentiment analysis results.

[1980] Output: Implementing the response method.

[1981] Specific operations: Commands such as playing relaxing music and changing the color of smart lights can be given through a smartphone app.

[1982] Step 3: Real-time response

[1983] When an emergency alert is received, caregivers will rush to the scene immediately.

[1984] Input: Emergency alert notification.

[1985] Output: On-site response actions.

[1986] Specific actions: Upon receiving an emergency notification, the caregiver rushes to the scene and takes action such as assisting the person being cared for who has fallen.

[1987] Step 4: Provide information

[1988] Local residents provide daily care data and suggestion information to the server and receive rewards.

[1989] Input: Daily care data and proposal information.

[1990] Output: Receipt of reward.

[1991] Specific operation: Data is entered and sent through a dedicated smartphone app, and the server calculates the reward amount at the end of the month and makes the payment electronically.

[1992] (Application example 2)

[1993] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1994] In nursing care facilities and at home, it is necessary to monitor the safety and health of elderly people and those receiving care in real time and to take appropriate measures promptly. However, current systems face many challenges, such as limited human resources and d...

Claims

1. A means for receiving video data from a care facility or home; an artificial intelligence means for analyzing the video data; means for generating recommendations to a caregiver based on the analysis results; means for notifying a caregiver of said suggestion; means for calculating and paying a fee for providing information from the caregiver; A system including:

2. A means of collecting video data using cameras and sensors installed in care facilities and homes, and means for compressing and filtering said video data; means for transmitting the collected data to a server; The system of claim 1 further comprising:

3. A means for analyzing the facial expressions, movements, and environmental changes of the care recipient in the video data and collected data; a means for notifying a caregiver of an emergency alert based on the analysis result in real time; The system of claim 1 further comprising:

Images