Information processing device and information processing method

Abstract

To provide an information processing device and information processing method for appropriately supporting a caregiver in providing care to a care recipient.SOLUTION: An information processing device disclosed herein comprises: a cause determination unit for determining whether behavior of a care recipient is abnormal behavior caused by dementia or not on the basis of (1) dementia level information of the care recipient and (2) at least one of environmental information, excretion information, and sleep information of the care recipient; and a support information output unit for outputting support information for supporting a caregiver in providing care to the care recipient based on a determination result of the cause determination unit and sensor information corresponding to a sensing result related to the caregiver providing care to the care recipient, or the care recipient.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and the like.

Background Art

[0002] Conventionally, systems used in medical sites, nursing facilities, and the like are known. For example, Patent Document 1 discloses a method of instructing an assistance method for moving a care recipient.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided are an information processing apparatus, an information processing method, and the like that appropriately support the assistance of a care recipient by an assistant.

Means for Solving the Problems

[0005] The information processing apparatus according to the present embodiment includes: a factor determination unit that determines whether the abnormal behavior is a dementia factor and whether it is an excretion disorder factor based on (1) the dementia level information of the care recipient and (2) at least one of the environmental information, excretion information, and sleep information of the care recipient when it is determined that the behavior of the care recipient is abnormal; and a support information output unit that outputs support information for supporting the assistance of the care recipient by the assistant based on the determination result of the factor determination unit and sensor information that is the sensing result regarding the assistant who assists the care recipient or the care recipient.

Brief Description of the Drawings

[0006] [Figure 1] Configuration example of an information processing system including an information processing apparatus. [Figure 2A] An example of a care bed, which is a type of caregiving device. [Figure 2B] An example of a lift, a type of assistive device used in elderly care. [Figure 2C] An example of a sensing device. [Figure 2D] An example of a sensing device. [Figure 3] Example of a server system configuration. [Figure 4] Example configuration of a mobile terminal device. [Figure 5] A diagram illustrating a neural network. [Figure 6] An example of input and output for a neural network used for factor determination. [Figure 7] A flowchart explaining the learning process for determining the cause. [Figure 8] A flowchart explaining the factor determination process. [Figure 9] An example of input and output for a neural network used to output support information. [Figure 10] Example configuration of a neural network for outputting support information. [Figure 11] Example configuration of a neural network for outputting support information. [Figure 12] Example configuration of a neural network for outputting support information. [Figure 13] An example of the relationship between a neural network for cause determination and a neural network for outputting supporting information. [Figure 14] An example of the first correspondence information. [Figure 15] An example of second-order information. [Figure 16] An example of third-party correspondence information. [Figure 17] An example of a settings screen. [Figure 18] A flowchart explaining the setup process. [Figure 19] A flowchart explaining the process for outputting each piece of support information. [Figure 20] A flowchart explaining the determination of when to initiate an assistance sequence. [Figure 21] Flowchart for explaining a meal assistance sequence. [Figure 22] Flowchart for explaining an excretion assistance sequence. [Figure 23] Flowchart for explaining a transfer and movement assistance sequence. [Figure 24A] Diagram for explaining transitions between multiple assistance sequences. [Figure 24B] Diagram for explaining transitions between multiple assistance sequences. [Figure 25A] Example of a setting screen. [Figure 25B] Example of a display screen for adding data. [Figure 25C] Example of a display screen for determining input data. [Figure 25D] Example of a display screen for presenting learning results. [Figure 26] Basic configuration example of a neural network in this embodiment. [Figure 27] Explanation diagram of the process for determining the structure of a neural network by classification. [Figure 28] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 29] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 30] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 31] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 32] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 33] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 34] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 35] Specific examples of input data when supporting the assistance of a care recipient by a caregiver. [Figure 36]Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 37] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 38] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 39] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 40] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 41] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 42] Specific examples of input data used when supporting caregivers assisting those receiving care. [Figure 43] A concrete example of output data when supporting meal assistance. [Figure 44] A concrete example of output data when supporting toileting assistance. [Figure 45] Specific examples of output data when supporting transfer and mobility assistance. [Modes for carrying out the invention]

[0007] This embodiment will be described below with reference to the drawings. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted. This embodiment described below is not intended to unduly limit the content described in the claims. Furthermore, not all of the configurations described in this embodiment are necessarily essential components.

[0008] 1. Example System Configuration Figure 1 shows an example configuration of an information processing system 10 including an information processing device according to this embodiment. The information processing system 10 according to this embodiment provides instructions to care workers in a nursing care facility, for example, by digitizing the "intuition" and "tacit knowledge" of care workers that are currently used for tasks, so that appropriate assistance can be provided regardless of the care worker's skill level. The information processing system 10 shown in Figure 1 includes a server system 100, a care worker device 200, a care device 300, and a sensor group 400. However, the configuration of the information processing system 10 is not limited to Figure 1, and various modifications such as omitting some parts or adding other components are possible. Furthermore, the same applies to Figures 3 and 4, which will be described later, in which modifications such as omitting or adding components are possible.

[0009] The information processing device in this embodiment corresponds to, for example, a server system 100. However, the method of this embodiment is not limited thereto, and the processing of the information processing device may be performed by distributed processing using the server system 100 and other devices. For example, the information processing device in this embodiment may include a server system 100 and a caregiver device 200. An example in which the information processing device is a server system 100 will be described below.

[0010] The server system 100 is connected to the caregiver device 200, the care device 300, and the sensor group 400, for example, via a network NW. The network NW here is a public communication network such as the Internet, but it may also be a LAN (Local Area Network). For example, the caregiver device 200, the care device 300, and the sensor group 400 are installed in a care facility. The server system 100 processes information from the sensor group 400 and, based on the processing results, outputs information to the caregiver device 200 and remotely controls the care device 300, etc.

[0011] Figure 1 shows an example where the caregiver device 200, the care device 300, and the sensor group 400 can each communicate with the server system 100 via a network NW, but this is not the only example. For example, a relay device (not shown) may be provided in a care facility. The relay device is a device that can communicate with the server system 100 via a network NW. Information output by the sensor group 400 may be aggregated by the relay device using the LAN within the care facility, and the relay device may transmit this information to the server system 100. Alternatively, information from the server system 100 may be transmitted to the relay device, and the relay device may transmit the necessary information to the caregiver device 200 or the care device 300. For example, it is assumed that multiple caregiver devices 200 and multiple care devices 300 are used simultaneously in a care facility. The relay device may perform a process to select the caregiver device 200 or care device 300 to which information from the server system 100 is to be transmitted. Alternatively, the relay device may be an administrator terminal used by the administrator of the care facility and may operate based on the administrator's input. For example, information from the server system 100 may be displayed on the display unit of the relay device, and an administrator who views the display results may select the caregiver device 200 or nursing care device 300 as the destination. Furthermore, as described above, the information processing device of this embodiment can be implemented in various modified forms, and for example, the relay device described above may be included in the information processing device.

[0012] The server system 100 may consist of one server or may include multiple servers. For example, the server system 100 may include a database server and an application server. The database server stores various data, as described later using Figure 3. The application server performs processing, as described later using Figures 7, 8, 18 to 23, etc. The multiple servers mentioned here may be physical servers or virtual servers. Furthermore, if a virtual server is used, the virtual server may be located on a single physical server or distributed across multiple physical servers. As described above, the specific configuration of the server system 100 in this embodiment can be modified in various ways.

[0013] The caregiver device 200 is a device used by caregivers who provide care to those receiving care (patients, residents) in care facilities, etc., and is used to present information to the caregiver or to allow the caregiver to input information. For example, the caregiver device 200 may be a device carried or worn by the caregiver. For example, the caregiver device 200 includes a portable terminal device 210 and a wearable device 220. The portable terminal device 210 is, for example, a smartphone, but may be another portable device. The wearable device 220 is a device that can be worn by the caregiver, and may be, for example, a headset including earphones or headphones and a microphone. The wearable device 220 may also be a glasses-type device, a wristwatch-type device, or a device of another shape. The caregiver device 200 may also be another device such as a PC (Personal Computer).

[0014] The care device 300 is a device used in care facilities and the like to provide care (including assistance) to those receiving care. While the caregiver device 200 is primarily a device for presenting information to caregivers, the care device 300 is a device for directly providing assistance to those receiving care. For example, the care device 300 may include a care bed 310 whose bottom (the bottom can be a plate or a mesh, and its shape is not specified) has an adjustable angle and height, and a lift 320 for transferring a person being cared for from the care bed 310 to a wheelchair. The care device 300 may also include other devices such as wheelchairs, walkers, rehabilitation equipment, and meal delivery carts.

[0015] Figure 2A shows an example of a care bed 310. The care bed 310 has multiple adjustable bottom heights and angles. This allows for flexible adjustment of the posture of the person being cared for while lying on the care bed 310. Figure 2B shows an example of a lift 320. The lift 320 is a device used, for example, to transfer people who have a low ADL (Activities of Daily Living) score and who are difficult to transfer manually.

[0016] The sensor group 400 includes multiple sensors placed in a nursing care facility or similar location. The sensor group 400 may also include a motion sensor 410, an imaging sensor 420, and an odor sensor 430. The motion sensor 410 may be an acceleration sensor, a gyroscope, or any other sensor capable of detecting movement. The motion sensor 410 may be a sensor that detects the movement of the person being cared for, or a sensor that detects the movement of the caregiver. The imaging sensor 420 is a sensor that converts an image of an object formed through a lens into an electrical signal. The odor sensor 430 is a sensor that detects and quantifies odors. Furthermore, the sensor group 400 can include various sensors such as a temperature sensor, a humidity sensor, an illuminance sensor, a magnetic sensor, a position sensor, and a pressure sensor.

[0017] In Figure 1, the caregiver device 200, the care device 300, and the sensor group 400 are shown separately. For example, the sensors included in the sensor group 400 may be placed in rooms, dining rooms, corridors, stairwells, etc., within the care facility. For example, cameras including imaging sensors 420 may be placed in various locations within the care facility. Sensing devices for sensing information necessary for care may also be used. By installing sensors in various locations within the care facility, it is possible not only to sense the necessary information but also to identify the location of the sensors.

[0018] For example, Figure 2C shows an example of a sensing device 440 placed on the mattress of a care bed 310. The sensing device 440 shown in Figure 2C includes, for example, an odor sensor 430 to detect whether or not the person being cared for has urinated or defecated. The sensing device 430 may also be capable of determining whether or not the person is ill based on body odor or breath. Figure 2D shows an example of a sensing device 450 placed under the mattress on the care bed 310 (placed between the care bed 310 and the mattress). The sensing device 450 shown in Figure 2D includes, for example, a pressure sensor to detect the heart rate, respiratory rate, and activity level of the person being cared for. The sensing device 450 may also be capable of determining whether or not the person is asleep or out of bed.

[0019] However, the method of this embodiment is not limited to the above examples, and the sensors included in the sensor group 400 may be provided in the caregiver device 200 or the care device 300. For example, the sensors included in the sensor group 400 may be a camera, acceleration sensor, gyro sensor, GPS (Global Positioning System) sensor, etc., included in the mobile terminal device 210. In addition, the care device 300 may be provided with a motion sensor for detecting the posture of the care device 300, and a camera for imaging the person being cared for and the caregiver using the care device 300.

[0020] Figure 3 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit 130.

[0021] The processing unit 110 of this embodiment is composed of the following hardware. The hardware may include at least one of a circuit that processes digital signals and a circuit that processes analog signals. For example, the hardware may consist of one or more circuit devices or one or more circuit elements mounted on a circuit board. One or more circuit devices may be, for example, an IC (Integrated Circuit) or an FPGA (Field-Programmable Gate Array). One or more circuit elements may be, for example, a resistor or a capacitor.

[0022] The processing unit 110 may also be implemented by the following processor. The server system 100 of this embodiment includes a memory for storing information and a processor that operates based on the information stored in the memory. The information includes, for example, programs and various types of data. The processor includes hardware. Various types of processors can be used, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, or it may be a register, or it may be a magnetic storage device such as a hard disk drive (HDD), or it may be an optical storage device such as an optical disk drive. For example, the memory stores instructions that can be read by the computer, and the functions of the processing unit 110 are realized as processing when the processor executes these instructions. The instructions here may be instructions from an instruction set that constitutes a program, or they may be instructions that instruct the hardware circuit of the processor to operate.

[0023] The processing unit 110 includes a factor determination unit 111, a support information output unit 112, a setting unit 113, and a learning unit 114.

[0024] The factor determination unit 111 determines whether the care recipient's behavior is abnormal behavior due to dementia factors, based on input that includes at least the care recipient's dementia level information. For example, the factor determination unit 111 determines whether the care recipient's behavior is abnormal behavior due to dementia factors, based on (1) the care recipient's dementia level information and (2) at least one of the care recipient's environmental information, excretion information, and sleep information. Details of each piece of information will be described later.

[0025] The support information output unit 112 outputs support information to assist the caregiver in assisting the person being assisted, based on the determination result output by the factor determination unit 111 and sensor information, which is the sensing result regarding the caregiver or the person being assisted. Details of the support information will be described later.

[0026] The setting unit 113 performs setting processing when using the information processing system 10 according to this embodiment. For example, a caregiver who is a user of the information processing system 10 may be able to set which of the many support information items to output. In this case, as will be described later with reference to Figures 17 and 18, the setting unit 113 performs processing to receive setting operations by the caregiver and processing to update the setting information. The setting unit 113 may also perform setting processing to add user-specific custom support information to the output. Specific examples will be described later with reference to Figures 25A to 25D, etc.

[0027] The learning unit 114 outputs a trained model by performing machine learning based on training data. Here, machine learning is, for example, supervised learning. In supervised learning, training data is a dataset in which input data corresponding to the model's input is associated with ground truth data representing the appropriate output data when that input data is given. The learning unit 114 may also generate a trained model by performing machine learning using, for example, a neural network. Hereinafter, a neural network will be denoted as NN. For example, the learning unit 114 performs the process of generating a factor determination NN 121 and a support information output NN 122. Details of the processing in the learning unit 114 will be described later. However, in this embodiment, machine learning is not mandatory, and the learning unit 114 can be omitted. Furthermore, even when machine learning is performed, the learning process can be executed on a learning device different from the server system 100, and in this case as well, the learning unit 114 can be omitted.

[0028] The memory unit 120 is the work area of ​​the processing unit 110 and stores various information. The memory unit 120 can be implemented using various types of memory, and the memory may be a semiconductor memory such as SRAM, DRAM, ROM, or flash memory, or it may be a register, a magnetic storage device, or an optical storage device.

[0029] The memory unit 120 stores information used for processing in the factor determination unit 111 and information used for processing in the support information output unit 112. For example, the memory unit 120 may store a factor determination NN 121 and a support information output NN 122 obtained by machine learning using a neural network (NN). The factor determination NN 121 and support information output NN 122 here include information that defines the structure of the NN, as well as parameters used in calculations using that structure. Specifically, the parameters are weights whose values ​​are determined by machine learning.

[0030] The memory unit 120 may also store first correspondence information 123, second correspondence information 124, and third correspondence information 125. The first correspondence information 123 is information that associates a caregiver with information indicating whether or not to output each support information to the caregiver. The second correspondence information 124 is information that associates support information with sensor information necessary for outputting the support information. The third correspondence information 125 is information that associates a given care facility with sensor information that can be obtained at the care facility. Specific examples of each correspondence information will be described later using Figures 14 to 16. The memory unit 120 may also store other information.

[0031] The communication unit 130 is an interface for communication via a network NW and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. The communication unit 130 may operate according to the control of the processing unit 110, or it may include a separate communication control processor from the processing unit 110. The communication unit 130 is an interface for communication according to, for example, TCP / IP (Transmission Control Protocol / Internet Protocol). However, various variations of the specific communication method are possible.

[0032] Figure 4 is a block diagram showing an example of a caregiver device 200, specifically a detailed configuration example of a portable terminal device 210. The portable terminal device 210 includes, for example, a processing unit 211, a storage unit 212, a communication unit 213, a display unit 214, and an operation unit 215.

[0033] The processing unit 211 is comprised of hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 211 may also be implemented by a processor. Various types of processors can be used, such as a CPU, GPU, or DSP. The functions of the processing unit 211 are realized as processing when the processor executes instructions stored in the memory of the mobile terminal device 210.

[0034] The memory unit 212 is the work area of ​​the processing unit 211 and is implemented by various types of memory such as SRAM, DRAM, and ROM.

[0035] The communication unit 213 is an interface for communication via a network NW and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 213 communicates with the server system 100, for example, via the network NW.

[0036] The display unit 214 is an interface for displaying various information, and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 215 is an interface for receiving user input. The operation unit 215 may be a button or the like provided on the mobile terminal device 210. Alternatively, the display unit 214 and the operation unit 215 may be a touch panel configured as an integrated unit.

[0037] The mobile terminal device 210 may also include components not shown in Figure 4, such as a light-emitting unit, a vibration unit, and a sound output unit. The light-emitting unit is, for example, an LED (light-emitting diode) and provides notification by emitting light. The vibration unit is, for example, a motor and provides notification by vibration. The sound output unit is, for example, a speaker and provides notification by sound. As mentioned above, the mobile terminal device 210 may also include sensors included in the sensor group 400.

[0038] 2. Factor identification and output of support information The information processing device of this embodiment performs processes to determine the factors in the behavior of the person being cared for, and processes to output support information that assists the caregiver in caring for the person being cared for. In this way, it becomes possible to have the caregiver provide appropriate care tailored to the person being cared for, taking into account factors such as dementia. Below, machine learning will be described as a specific example of a method for determining factors and outputting support information. However, the method of this embodiment is not limited to using machine learning, and various modifications can be implemented. Furthermore, although an example using NN as machine learning will be described below, other machine learning methods such as SVM (support vector machine) may be used, or methods that are advanced versions of NN or SVM may be used.

[0039] 2.1 A brief explanation of NN Figure 5 shows a basic example of a neural network (NN). Each circle in Figure 5 is called a node or neuron. In the example in Figure 5, the NN has an input layer, two or more hidden layers, and an output layer. The input layer is I, the hidden layers are H1 and Hn, and the output layer is O. In the example in Figure 5, the input layer has 2 nodes, the hidden layers each have 5 nodes, and the output layer has 1 node. However, the number of hidden layers and the number of nodes in each layer can be varied in various ways. Also, Figure 5 shows an example where each node in a given layer is connected to all nodes in the next layer, but this configuration can also be varied in various ways.

[0040] The input layer receives input values ​​and outputs them to the hidden layer H1. In the example in Figure 5, input layer I receives two types of input values. Each node in the input layer may perform some processing on the input values ​​and output the values ​​after that processing.

[0041] In a neural network (NN), weights are assigned between two connected nodes. In Figure 5, W1 represents the weights between the input layer I and the first hidden layer H1. W1 represents the set of weights between a given node in the input layer and a given node in the first hidden layer. For example, W1 in Figure 5 contains information with 10 weights.

[0042] At each node of the first hidden layer H1, the output of the node in the input layer I connected to that node is weighted and added using a weight W1, and then a bias is added. Furthermore, at each node, the output of that node is obtained by applying a nonlinear activation function to the summation result. The activation function may be a ReLU function, a sigmoid function, or any other function.

[0043] The same applies to subsequent layers. That is, in a given layer, the output of the previous layer is weighted and added using the weight W, a bias is added, and then the activation function is applied to determine the output for the next layer. The NN takes the output of the output layer as its own output.

[0044] As can be seen from the above explanation, in order to obtain desired output data from input data using a neural network (NN), it is necessary to set appropriate weights and biases. For training, training data is prepared in which a given input data is associated with ground truth data that represents the correct output data for that input data. The NN training process is the process of finding the most probable weights based on the training data. Various learning methods such as backpropagation are known for NN training. In this embodiment, these learning methods can be broadly applied, so a detailed explanation is omitted.

[0045] Furthermore, the NN is not limited to the configuration shown in Figure 5. For example, a Convolutional Neural Network (CNN) may be used as the NN. A CNN has convolutional layers and pooling layers. The convolutional layers perform convolution operations. Specifically, convolution operations refer to filtering. The pooling layers perform processing to reduce the vertical and horizontal size of the data. In a CNN, the characteristics of the filters used in the convolution operations are learned by performing learning processing using methods such as backpropagation. That is, the weights in the NN include the filter characteristics in the CNN. Other network configurations, such as Recurrent Neural Networks (RNNs), may also be used as the NN.

[0046] 2.2 Factor Determination Figure 6 illustrates the input and output data of the factor determination NN121 used for factor determination. The input data for factor determination includes, for example, dementia level information. The input data also includes at least one of environmental information, sleep information, and excretion information. Figure 6 shows an example where the input data includes all of environmental information, sleep information, and excretion information. The input data may also include other information. For example, as shown in Figure 6, the input data may include medication information and dietary and fluid information. Furthermore, the configuration of the factor determination NN121 is not limited to Figure 6, and various modifications are possible.

[0047] Dementia level information represents the degree of progression of dementia in the person receiving care. For example, dementia level information may be an MMSE (Mini-Mental State Examination) score, a Revised Hasegawa Dementia Scale-Revised (HDS-R) score, or other information representing the results of dementia tests. Dementia level information may also be based on brain images obtained using CT (Computed Tomography) or MRI (Magnetic Resonance Imaging). For example, dementia level information may represent the results of a diagnosis made by a physician based on brain images, the brain images themselves, or the results of some kind of image processing applied to the brain images.

[0048] Environmental information represents the living environment of the person receiving care. This information includes temperature information, humidity information, illuminance information, and atmospheric pressure information. For example, temperature sensors, humidity sensors, illuminance sensors, and atmospheric pressure sensors are placed in regularly used areas such as the person receiving care's room or dining room, and temperature information, humidity information, illuminance information, and atmospheric pressure information are obtained based on the output of each sensor.

[0049] Environmental information may also include information related to sound. For example, microphones may be placed in living environments such as rooms, and the information collected by these microphones may be used as environmental information. Environmental information may include information related to sound pressure, or information representing the results of frequency analysis. Environmental information may also include information related to the time when a particular sound occurs.

[0050] Furthermore, environmental information may include information about the care bed 310 used by the person receiving care. Information regarding the care bed 310 may include information that identifies the model of the care bed 310, or information such as the type and firmness of the mattress used with the care bed 310. Furthermore, information regarding the care bed 310 may include information representing the results of the care bed 310's operation. For example, information such as the angle and height of the care bed 310's base, or the time the care bed 310 was operated, may be used as environmental information.

[0051] Sleep information is information that represents the sleep state of the person being cared for. For example, sleep information may be detected using a sensing device 450 as shown in Figure 2D. Alternatively, sleep information may be detected using a wristwatch-type device that includes a photoelectric sensor for detecting pulse rate. Sleep information includes, for example, the time of sleep onset, wake-up time, daily sleep duration, sleep depth, number and time of awakenings during sleep, and information such as heart rate, respiratory rate, and activity level during sleep.

[0052] Excretion information includes information representing the excretion status of the person being cared for. For example, excretion information may be detected using a sensing device 440 as shown in Figure 2C. The sensing device 440 outputs whether or not the person being cared for has excreted, the type of excretion, and the timing at which it was determined that excretion had occurred, based on, for example, an odor sensor 430. Excretion information includes information such as the number of times excretion occurred in a given span, the interval between excretion, and the type of excretion. Furthermore, excretion information may also include captured images of the diaper after excretion, and information such as comments added by the caregiver.

[0053] Medication information is information that identifies the medications being administered to the person receiving care. For example, medication information may include the name of the medication taken by the person receiving care, the dosage, the time of administration, etc. Medication information may also include information from prescriptions issued to the person receiving care.

[0054] Food and fluid information refers to information that represents the food and fluids consumed by the person receiving care. For example, food and fluid information may include the time of the meal, the menu, and the actual amount eaten. It may also include information that identifies how easy the food was to eat, such as the hardness and size of the food. Furthermore, it may include the time of fluid intake, the type of fluid (water, tea, etc.), and the amount consumed.

[0055] During the learning phase, training data for creating the factor determination NN121 is obtained by associating the above input data over a predetermined period with the correct answer data. The predetermined period here may be a fixed period such as one day. Alternatively, the predetermined period may be set based on the time when the person being cared for exhibits any abnormal behavior.

[0056] Furthermore, the correct answer data may be provided by experts with specialized knowledge, such as doctors. When a person being cared for exhibits abnormal behavior, the expert diagnoses the person being cared for and identifies the factors causing the abnormal behavior. Here, the correct answer data is information that represents the identified factors. For example, the correct answer data indicates whether the behavior is due to dementia, environmental factors, sleep disorders, or excretory disorders. For example, if the input data corresponding to one period of time for one person being cared for is associated with the correct answer data and this is used as one dataset, then by increasing the number of people being cared for or the period of time, training data containing a large number of datasets can be obtained.

[0057] The learning unit 114 of the server system 100 acquires training data for factor determination and creates a factor determination NN 121 by performing machine learning based on the training data.

[0058] Figure 7 is a flowchart illustrating the learning process for generating the NN121 for factor determination. When this process begins, in step S101, the learning unit 114 first acquires input data for learning. The input data here is as described above and includes, for example, dementia level information, environmental information, sleep information, and excretion information. The input data may also include other information such as medication information and dietary and fluid information.

[0059] In step S102, the learning unit 114 obtains the correct answer data associated with the input data. For example, the learning unit 114 executes the processes in steps S101 and S102 by reading one of the training data sets obtained during the learning phase.

[0060] In step S103, the learning unit 114 updates the weights of the neural network (NN). Specifically, the learning unit 114 inputs the input data obtained in step S101 into the factor determination NN 121 and obtains output data by performing forward operations using the weights at that stage. The learning unit 114 then calculates an objective function based on this output data and the correct answer data. The objective function here is, for example, an error function based on the difference between the output data and the correct answer data, or a cross-entropy function based on the distribution of the output data and the distribution of the correct answer data.

[0061] For example, if the output layer of the factor determination NN121 is a known softmax layer, the output of the output layer is probability data whose sum is 1. For example, the output layer contains four nodes, the first to the fourth node. The output value of the first node represents the "likelihood that the care recipient's behavior is a dementia factor." The output value of the second node represents the "likelihood that the care recipient's behavior is an environmental factor." The output value of the third node represents the "likelihood that the care recipient's behavior is a sleep disorder factor." The output value of the fourth node represents the "likelihood that the care recipient's behavior is an excretory disorder factor." The ground truth data is data where the value of the correct factor is 1 and the other values ​​are 0. For example, if an expert identifies a factor as related to dementia, the data where the probability of the dementia factor is 1 and the probability of the other three factors is 0 will be used as the correct answer data.

[0062] The learning unit 114 updates the weights, for example, so that the error function decreases. Methods for updating weights include the backpropagation method described above, and these methods can be broadly applied in this embodiment as well.

[0063] In step S104, the learning unit 114 determines whether or not to terminate the learning process. For example, multiple datasets included in the training data may be divided into training data and validation data. The learning unit 114 may terminate the learning process if the process of updating the weights has been performed using all the training data, or it may terminate the learning process if the accuracy rate using the validation data exceeds a given threshold.

[0064] If the learning process is not terminated, the learning unit 114 returns to step S101 and continues processing. That is, the learning unit 114 reads a new dataset from the training data and updates the weights based on that dataset.

[0065] When the learning process is terminated, the learning unit 114 stores the factor determination NN 121 at that stage in the storage unit 120 as a trained model. Note that Figure 7 is an example of the learning process, and the method of this embodiment is not limited to this. For example, batch learning and other methods are widely known in machine learning, and these methods can be broadly applied in this embodiment.

[0066] Figure 8 is a flowchart illustrating the processing of the factor determination unit 111 during the inference stage. When this process begins, in step S201, the factor determination unit 111 first determines whether the person being cared for has performed any abnormal behavior that suggests dementia. The factor determination unit 111 may also automatically determine whether the person being cared for is performing abnormal behavior based on sensor information related to the person being cared for. For example, the sensor group 400 includes a motion sensor 410, an imaging sensor 420, a microphone, etc., and the factor determination unit 111 determines the presence or absence of abnormal behavior by detecting the person being cared for's movements and vocalizations. Alternatively, the caregiver may observe the person being cared for and input the observation results using the caregiver device 200, etc. In this case, the factor determination unit 111 executes the process in step S201 based on the caregiver's input. If it is determined that the person being cared for is not performing abnormal behavior, the factor determination unit 111 terminates the process without performing steps S202 and beyond.

[0067] If it is determined that the person being cared for has exhibited abnormal behavior, in step S202, the factor determination unit 111 acquires input data related to the person being cared for. For example, the memory unit 120 acquires and stores information such as dementia level information related to the person being cared for, and sensor information collected by the sensor group 400, via the communication unit 130. Dementia level information may be acquired, for example, at a care facility and transmitted from the equipment at the care facility to the server system 100. The factor determination unit 111 processes the collected data to read out data related to the person being cared for in question, such as dementia level information, environmental information, sleep information, and excretion information corresponding to a predetermined period, as input data.

[0068] In step S203, the factor determination unit 111 reads the factor determination NN121 from the memory unit 120. It then inputs the input data obtained in step S202 into the factor determination NN121 and performs forward calculations to obtain output data. The output data of the factor determination NN121 is, for example, four probability values ​​representing the likelihood of each factor, as described above. The factor determination unit 111 determines, for example, that the factor with the highest probability value is the factor causing the abnormal behavior of the person being cared for. For example, if the value representing the likelihood of the dementia factor is greater than the likelihoods of the other three factors, the factor determination unit 111 determines that the abnormal behavior is due to the dementia factor. Furthermore, the output of the factor determination unit 111 is not limited to this; it may also be the four probability values ​​themselves, or a value calculated based on them.

[0069] The factor determination unit 111 periodically performs the process shown in Figure 8. The frequency of the process is arbitrary, but it may be, for example, once a day. In this way, the presence or absence of abnormal behavior by the person being cared for, and the factors if such abnormal behavior occurs, can be determined periodically. For example, the factor determination unit 111 may perform the process shown in Figure 8 every morning, and the care plan for the day may be determined based on the processing results. Furthermore, if abnormal behavior is observed in the person being cared for, the process shown in Figure 8 may be performed without waiting for the next processing timing, and various variations of the process performed by the factor determination unit 111 are possible.

[0070] Furthermore, the above describes an example of determining whether the behavior of the person being cared for is abnormal, as a process separate from the process using NN121 for factor determination (see step S201 in Figure 8). However, a NN that performs determination including whether or not the behavior is abnormal may also be created.

[0071] For example, the factor determination NN121 may receive sensor information representing the care recipient's behavior in addition to the input data shown in Figure 6. The factor determination NN121 may also include a node that outputs the "likelihood that there is no abnormality in the care recipient's behavior" in addition to the nodes that output the likelihood of the four factors shown in Figure 6. During the learning phase, training data is created using data from cases where abnormal behavior occurred as well as data from cases where there was no abnormal behavior. Specifically, the ground truth data associated with the input data includes data representing "no abnormal behavior." In this case, the factor determination unit 111 can estimate the presence or absence of abnormal behavior and the factors if abnormal behavior occurred by inputting the input data into the factor determination NN121.

[0072] 2.3 Assistance Support 2.3.1 Inputs and Outputs Figure 9 is a diagram illustrating the schematic input data for the NN122 used for outputting support information. As shown in Figure 9, the input data may include sensor information. The sensor information includes information sensed from the person being assisted or information sensed from the caregiver. The sensor information is output from, for example, the sensors included in the sensor group 400.

[0073] Furthermore, the sensor information may include information sensing the living environment of the person being cared for. In this case, the sensor information corresponds, for example, to the environmental information mentioned above. For example, the sensor information may include outputs from temperature sensors, humidity sensors, illuminance sensors, pressure sensors, microphones, etc.

[0074] The input data may also include attribute data of the person being cared for and physical assessment data representing their physical condition. Attribute data of the person being cared for includes information such as age, gender, height, weight, medical history, and medication history. Physical assessment data includes information such as ADL (Activities of Daily Living) scores, rehabilitation history, fall risk, and pressure ulcer risk.

[0075] The input data may also include attribute data of caregivers and data related to care facilities. Caregiver attribute data includes the caregiver's age, gender, height, weight, caregiving experience, and qualifications. Information regarding care facilities includes the care schedule at the facility, the number and usage status of care devices, and statistical data on the number of care recipients and their care needs.

[0076] Figures 28 to 42 illustrate the details of the data used as input when supporting the caregiver's assistance to the person being cared for in this embodiment, and more specifically, they illustrate examples of input data for the NN122 for outputting support information. As shown in Figures 28 to 42, various types of information can be used as input data in this embodiment. Note that it is not necessary to acquire all of the input data shown in Figures 28 to 42, and some information may be omitted. In addition, other information not shown in Figures 28 to 42 may be added.

[0077] Furthermore, the output data of the NN122 for support information output is information used to support the execution of each assistance action when the assistance provided by the caregiver to the person being cared for is subdivided into multiple assistance actions. For example, the output data of the NN122 for support information output is support information for determining the timing of the start of assistance, the movements and vocalizations during assistance, and the type and amount of items to be provided to the person being cared for.

[0078] Figures 43 to 45 illustrate the details of the data used in this embodiment to support the caregiver's assistance to the person being cared for, and more specifically, they show examples of support information, which is output data from the NN122 for support information output.

[0079] Figure 43 shows an example of support information output during meal assistance, where caregivers assist a person receiving care with eating. For example, during meal assistance, caregivers understand the characteristics of the person receiving care and explain them clearly to the person receiving care to facilitate the meal. For instance, if a person receiving care has difficulty chewing, knowing this allows caregivers to take measures to prevent aspiration, and it is also useful to advise the person receiving care by saying, "The rice has been softened, so please chew it well." The output data for Number 1 in Figure 43 is support information for "communicating the user's characteristics" to the caregiver. This may be data representing the characteristics of the person receiving care, or it may be information converted to be easily understood by the caregiver. As mentioned above, caregivers may also communicate the characteristics of the person receiving care to the person receiving care, and the output data for Number 1 in Figure 43 may include data for that purpose. The same applies to Numbers 2 and beyond; the output data shown in Figure 43 includes information to support various actions of the caregiver during meal assistance.

[0080] Figure 44 shows an example of support information output during toileting assistance, where assistance is provided to the person being cared for in urinary incontinence. Note that toileting assistance may be performed in a toilet or using diapers. Numbers 66-72 represent the output data when toileting assistance is performed in a toilet, and Numbers 73-75 represent the output data when toileting assistance is performed using diapers.

[0081] Figure 45 shows examples of support information output during transfer assistance and mobility assistance, which involve assisting a person being assisted with transferring or moving. Note that the presence or type of equipment used in transfer and mobility assistance varies depending on the condition of the person being assisted and the availability of equipment such as lifts. In the example in Figure 45, Numbers 92-103 represent output data when assistance is provided using a wheelchair, Numbers 104-107 represent output data when assistance is provided using a cane, and Numbers 108-112 represent output data when assistance is provided using a lift.

[0082] As shown in Figures 43 to 45, support information may include information supporting at least one of the following: assistance with eating, assistance with excretion, and assistance with transfers and mobility. This makes it possible to appropriately support the assistance that is most needed in care facilities. For example, supporting assistance with eating can help reduce incidents such as aspiration and improve the nutritional status of those receiving assistance. Supporting assistance with excretion can help reduce incontinence, reduce the workload and risks associated with dealing with incontinence, alleviate excretory disorders, and reduce the risk of falls. Furthermore, supporting assistance with transfers and mobility can reduce the risk of falls and allow for prior preparation of those who need assistance.

[0083] 2.3.2 Example Configuration of a Network Interface for Outputting Support Information Figures 10 to 12 show more specific configuration examples of the NN122 for outputting support information shown in Figure 9. As shown in Figure 10, the NN122 for outputting support information may be a collection of multiple NNs, each of which outputs one piece of support information. Support information 1 in Figure 10 corresponds to one of the support information shown in Figures 43 to 45. Input data group 1 represents one or more input data from the multiple input data shown in Figures 28 to 42 that are necessary for outputting support information 1. The same applies to support information 2 and beyond.

[0084] As shown in Figure 11, the support information output NN122 may also be a collection of multiple NNs capable of outputting multiple related output data together. In the example in Figure 11, the support information output NN122 includes a NN for outputting meal assistance support information, a NN for outputting excretion assistance support information, and a NN for outputting transfer assistance support information.

[0085] For example, a neural network (NN) for outputting meal assistance support information outputs multiple pieces of meal assistance support information. The output data of the neural network (NN) for outputting meal assistance support information corresponds to the multiple support information shown in Figure 43. The input data of the NN for outputting meal assistance support information represents the multiple input data necessary for outputting meal assistance support information from among the multiple input data shown in Figures 28 to 42. The output of the NN for outputting support information for excretion assistance corresponds to the multiple support information shown in Figure 44. The output of the NN for outputting support information for transfer assistance corresponds to the multiple support information shown in Figure 45.

[0086] As shown in Figure 12, the NN122 for outputting support information may also be a single NN. The input data for the NN in Figure 12 is the set of all the data shown in Figures 28 to 42, and the output data is the set of all the support information shown in Figures 43 to 45.

[0087] Furthermore, the configuration of the NN122 for outputting support information is not limited to Figures 10 to 12. For example, by dividing the NN for outputting meal assistance support information into several parts, an intermediate configuration between Figures 10 and 11 may be used. In addition, various modifications can be made to the specific configuration of the NN122 for outputting support information.

[0088] Figure 13 shows an example of the relationship between NN121 for factor determination and NN122 for support information output. The input data for factor determination in Figure 13 is the input in Figure 6, and includes dementia level information, etc. The input data for support information output NN122 is the input in Figure 9, and specifically the data shown in Figures 28 to 42. Note that the input data for factor determination and the input data for support information output may partially overlap.

[0089] In the example shown in Figure 13, the output data of the factor determination NN121 is used as part of the input data for the support information output NN122. The output data of the factor determination NN121 may be information that identifies a single factor, as described above, or it may be information based on multiple probability values. If the support information output NN122 includes multiple NNs, as shown in Figure 10 or Figure 11, the output data of the factor determination NN121 may be input to all NNs or to some of them. In this way, it becomes possible to output support information based on the result of the factor determination in the factor determination unit 111. As a result, it becomes possible to make caregivers understand the degree of dementia progression of the person being cared for, and to provide various types of care according to that degree of progression.

[0090] Furthermore, if no abnormal behavior is observed in the person being cared for, the output of the factor determination NN121 may be treated as 0. Also, as mentioned above, the factor determination NN121 may be capable of outputting information indicating "no abnormal behavior".

[0091] However, in the method of this embodiment, it is sufficient that the determination result by the factor determination unit 111 is used for outputting support information, and the specific method is not limited to the example in Figure 13.

[0092] 2.3.3 Training and Inference Processes The learning process for the NN122 for outputting support information in the learning unit 114 is the same as when creating the NN121 for factor determination. The training data used to create the NN122 for outputting support information includes a dataset in which the input data is associated with ground truth data representing the assistance results performed by skilled caregivers using tacit knowledge.

[0093] For example, when assisting a person receiving care with eating, at least the sensors in the sensor group 400 necessary for eating assistance are turned on. As a result, data related to eating assistance from the input data shown in Figures 28 to 42 is acquired by the sensor group 400 and stored in the storage unit 120 of the server system 100. In addition, data representing the results of the caregiver's assistance, such as the posture the skilled caregiver had the person receiving care assume (corresponding to Numbers 9-12 in Figure 43), the timing of providing food with a spoon (corresponding to Number 26 in Figure 43), and the amount provided per bite (corresponding to Number 25 in Figure 43), are stored in the storage unit 120 as correct data.

[0094] The learning unit 114 inputs the input data from the training data into the support information output NN 122 and performs forward operations using the weights to obtain the output data. The learning unit 114 also calculates an objective function (for example, an error function such as the mean squared error function) based on the output data and the ground truth data, and updates the weights to reduce the error using methods such as backpropagation. The support information output NN 122 at the end of the learning process is stored in the storage unit 120 as a trained model.

[0095] As shown in Figure 13, the output of the factor determination NN121 may be included as input to the support information output NN122. In this case, the input data in the training data includes information representing the factors of the abnormal behavior of the person being cared for. For example, as explained in the learning process of the factor determination NN121, the correct answer data provided by an expert such as a doctor may be used as one of the input data in the training data. Alternatively, if the learning of the factor determination NN121 has been completed first, as shown in Figure 8, inference processing using the factor determination NN121 may be performed, and the result may be used as one of the input data in the training data.

[0096] The ground truth data, as in the example above, represents information showing the results of assistance provided by skilled caregivers using tacit knowledge. Skilled caregivers can naturally provide assistance that is appropriate for the person being cared for, taking into account factors such as the progression of dementia in that person. In other words, by using the results of assistance provided by skilled caregivers as ground truth data, it becomes possible to machine-learn appropriate assistance in response to the factors causing abnormal behavior. The processing after the training data is acquired is the same in this case as well. That is, the learning unit 114 performs forward operations using the input data from the training data, calculates the error function from the output data and the correct answer data, and updates the weights to minimize the error.

[0097] The support information output unit 112 of the server system 100 acquires the input data shown in Figures 28 to 42 during the inference phase. Note that the input data here only needs to include data that can output the desired support information, and it is not necessary to acquire all the input data shown in Figures 28 to 42. The support information output unit 112 also acquires the judgment result of the factor determination unit 111 as one of the input data. The support information output unit 112 reads the trained support information output NN 122 from the storage unit 120 and inputs the input data into the support information output NN 122. Furthermore, when using a neural network capable of outputting multiple pieces of support information, as shown in Figures 11 and 12, and only some of the support information needs to be output, there is a possibility that some of the input data may not be acquired. In this case, the support information output unit 112 may, for example, set the values ​​of the unacquired input data to 0. The support information output unit 112 obtains the support information as output data by performing forward calculations.

[0098] 3. Processing Flow Next, we will explain the specific procedures involved in supporting caregivers in assisting those receiving care at care facilities and similar settings.

[0099] 3.1 Estimation of the factors behind behavior First, the server system 100, separate from the process of starting and executing a specific assistance sequence, determines whether or not abnormal behavior is observed in the person being assisted, and if so, what the cause of that abnormal behavior is.

[0100] For example, the factor determination unit 111 periodically performs the above-described process using Figure 8. In this way, the presence or absence of abnormal behavior and the cause of the abnormal behavior can be determined for each of the multiple people being assisted. In the following explanation, we will assume that the results of the factor determination by the factor determination unit 111 have been obtained.

[0101] 3.2 Assistance Support 3.2.1 User Settings Examples of support information in this embodiment are shown in Figures 43 to 45. The support information output unit 112 may output all of this support information. However, if the amount of information provided is too large, an inexperienced caregiver may not be able to fully grasp the content or recognize the differences in importance between each process. Also, a caregiver with some experience may find the notification of support information bothersome, as they can perform the given care appropriately even without support. Therefore, in this embodiment, the user, who is the caregiver, may be able to set the support information that is output.

[0102] The storage unit 120 of the server system 100 may store the first correspondence information 123. Figure 14 shows a specific example of the first correspondence information. As shown in Figure 14, the first correspondence information 123 is information that associates a caregiver ID that identifies the caregiver, support information, and information representing the output settings of said support information.

[0103] The output settings include active and inactive. If the given support information is set to active, the support information output unit 112 outputs the support information to the caregiver in question. If the given support information is set to inactive, the support information output unit 112 does not output the support information to the caregiver in question. In this way, it becomes possible to flexibly set the output support information for each caregiver.

[0104] However, as shown in Figures 28 to 42, the types of input data assumed in this embodiment are very numerous, so it is not always possible for a nursing care facility to acquire all of the input data. For example, due to constraints such as budget or the structure of the nursing care facility, it may not be possible to install the sensors necessary to acquire the given input data. In this case, the lack of input data may prevent the acquisition of the given supporting information with sufficient accuracy.

[0105] Therefore, the output settings for support information may include not only active / inactive but also "output disabled." Output disabled indicates that support information is not output because the necessary input data cannot be obtained. Inactive, on the other hand, indicates that support information is not output even though the necessary input data can be obtained, and is therefore different from output disabled.

[0106] For example, the storage unit 120 of the server system 100 may store the second correspondence information 124 and the third correspondence information 125. Figure 15 shows a specific example of the second correspondence information 124. As shown in Figure 15, the second correspondence information 124 includes support information and a set of required input data essential for outputting the support information. The support information is one of the multiple data shown in Figures 43 to 45. The set of required input data is one or more of the data shown in Figures 28 to 42. The set of required input data may be, for example, data specified by the user. Alternatively, a support information output NN122 may be created for each of the multiple candidate input data sets, and the candidate input data set with the highest accuracy rate using validation data may be selected as the required input data set. Furthermore, the set of required input data is not limited to one set; multiple candidate input data sets with an accuracy rate above a predetermined threshold may be used as the required input data set.

[0107] Figure 16 shows a specific example of the third correspondence information 125. The third correspondence information 125 is information that associates a nursing care facility with input data that can be obtained at that nursing care facility. For example, a person in charge of a nursing care facility may select input data that can be obtained at that nursing care facility and send the selection result to the server system 100. Alternatively, a group of sensors 400 installed at a nursing care facility associates information that identifies the nursing care facility with sensor information and sends it to the server system 100. The processing unit 110 of the server system 100 may create the third correspondence information 125 based on the acquisition history of sensor information.

[0108] The configuration unit 113 of the server system 100 determines whether or not each piece of support information can be output for each nursing care facility based on the second correspondence information 124 and the third correspondence information 125. Specifically, the configuration unit 113 determines whether or not the support information can be output based on whether or not the set of essential input data required for outputting the support information is included in the set of input data that can be obtained at the target nursing care facility.

[0109] Furthermore, there is a correspondence between the input data and the sensor used to acquire that input data. Therefore, the storage unit 120 may store a fourth correspondence information that associates the input data with one or more sensors used to acquire the input data. By using the fourth correspondence information in addition to the second correspondence information 124 and the third correspondence information 125, it becomes possible to determine on a sensor-by-sensor basis whether or not support information can be output. Alternatively, instead of providing a separate fourth correspondence information, the input data of the second correspondence information 124 and the third correspondence information 125 may be replaced with sensor information.

[0110] Furthermore, the device containing a given sensor is not limited to one. For example, if a motion sensor 410 and an imaging sensor 420 are required, a device such as a smartphone containing both a camera and an accelerometer may be used, or two separate devices may be used. Also, among cameras, multiple models with different resolutions and magnifications may be available. Therefore, the storage unit 120 may store a fifth correspondence information that associates a sensor with a device containing that sensor. In this case, it becomes possible to manage data on a device-by-device basis. For example, if the nursing care facility specifies a device that has already been installed, the server system 100 will identify the sensors included in that device and the input data that can be acquired using those sensors. This improves user convenience because nursing care facility staff and caregivers do not need to know the sensors included in the device or the input data that can be acquired with that device.

[0111] The above describes an example of determining whether support information can be output on a care facility basis (see, for example, Figure 16). However, the method of this embodiment is not limited to this. For example, if a nursing care facility has a first space for residents with high care needs and a second space for residents with low care needs, it is conceivable that the first space would have many sensors and the second space would have fewer sensors. In this case, the server system 100 may manage the support information that can be output in the first space and the support information that can be output in the second space separately. In addition, various variations can be implemented in specific methods, such as managing whether or not support information can be output for each person receiving care.

[0112] Figure 17 shows an example of a settings screen for configuring the support information to be output. The processes described below are realized, for example, by the storage unit 212 of the mobile terminal device 210 storing a web application program that communicates with the server system 100, and the processing unit 211 operating according to the web application program. For example, the display of the display screen and the reception of user operations are performed using the display unit 214 and the operation unit 215 according to the web application program. Furthermore, the generation and updating of the display screen and database control according to user operations are performed by the settings unit 113 of the server system 100. However, the method of this embodiment is not limited to using a web application program, and various modifications such as using a so-called native application are possible. Also, although Figure 17 shows an example in which the settings screen is displayed on the display unit 214 of the mobile terminal device 210, the settings screen may be displayed on another caregiver device 200.

[0113] For example, the settings screen allows you to select active, inactive, or not output for each of the multiple support information items. Figure 17 shows an example of a settings screen that includes objects OB1 to OB3 corresponding to three pieces of support information: "Diaper change timing," which supports toileting assistance, and "Amount to be served with a spoon" and "Timing of serving food with a spoon," which support meal assistance.

[0114] For example, if the corresponding support information is active, the object is displayed in the first mode. If the corresponding support information is inactive, the object is displayed in the second mode. If the corresponding support information cannot be output, the object is displayed in the third mode. The display mode here may be controlled using the size, shape, and color of the object, or using the size, font, and color of the text contained in the object. In addition, various modifications can be made to the specific display mode.

[0115] Figure 17 shows an example where objects OB1 to OB3 are buttons, and their colors differ depending on whether they are active, inactive, or unavailable for output. For example, "Diaper change timing" is active, "Amount to be served with a spoon" is unavailable for output, and "Timing of serving food with a spoon" is inactive. In this case, the support information output unit 112 outputs support information representing "Diaper change timing" but does not output support information representing "Timing of serving food with a spoon." Also, in the target nursing facility, it may be difficult to accurately determine "Amount to be served with a spoon" due to a lack of sensors, so outputting "Amount to be served with a spoon" is not permitted. By displaying objects OB1 to OB3 in different ways, it is possible to clearly present the current settings to the caregiver.

[0116] The caregiver can switch between active and inactive states by operating the control panel 215 of the mobile terminal device 210. For example, if the caregiver performs an operation to select "diaper change timing," information indicating this is sent to the server system 100. The setting unit 113 updates the output setting corresponding to "diaper change timing" for the target caregiver ID in the first correspondence information 123 to inactive. The setting unit 113 also generates a display screen that shows the corresponding object OB1 in a second mode corresponding to inactivity and sends it to the mobile terminal device 210 via the communication unit 130. The display unit 214 displays the display screen.

[0117] Similarly, if an object corresponding to inactive support information is selected, the setting unit 113 updates the output settings corresponding to the caregiver and support information to be active. The display unit 214 also changes the display mode of the selected object to the first mode.

[0118] On the other hand, even if an object corresponding to support information that cannot be output is selected, the display unit 214 maintains the display in the third mode indicating that output is not possible. In this case, the setting unit 113 does not perform the update process for the first correspondence information 123.

[0119] Furthermore, if an operation is performed to select an object corresponding to support information that cannot be output, the input data necessary for outputting that support information may be suggested. For example, the server system 100 may identify the necessary input data based on the second correspondence information 124 and perform the process of displaying that input data on the display unit 214 of the mobile terminal device 210. As mentioned above, the input data here may be replaced with sensors or devices. For example, the setting unit 113 may identify the sensor or device necessary for outputting the support information selected by the user and perform the process of displaying that sensor or device on the display unit 214 of the mobile terminal device 210.

[0120] Figure 18 is a flowchart illustrating the above configuration process. First, the caregiver performs a configuration change operation using their own caregiver device 200. In step S301, the configuration unit 113 of the server system 100 receives the configuration change operation via the network NW.

[0121] In step S302, the setting unit 113 performs a process to display the setting screen on the caregiver device 200 based on the first correspondence information 123 at that time and the caregiver ID representing the caregiver who performed the setting change operation. The process in step S302 may be a process of creating an image corresponding to the setting screen and sending the image to the caregiver device 200, or a process of sending information for generating the setting screen to the caregiver device 200. The information for generating the setting screen may be an extraction result obtained by extracting some data corresponding to the caregiver ID from the first correspondence information 123. Alternatively, the information for generating the setting screen may be a processed result obtained by performing some processing on the extraction result. As a result, for example, the screen corresponding to Figure 17 is displayed on the display unit 214 of the mobile terminal device 210.

[0122] In step S303, the setting unit 113 determines the user operation to be performed on the caregiver device 200. If no operation to change settings is detected, the setting unit 113 terminates the process.

[0123] Furthermore, if an operation is performed to deactivate active support information, or to activate inactive support information, in step S304, the setting unit 113 reflects the setting change. Specifically, the setting unit 113 performs a process to update the first correspondence information 123 based on information from the caregiver device 200.

[0124] Furthermore, if the operation to select support information that cannot be output is performed, in step S305, the setting unit 113 identifies the input data, sensor, or device that is missing for the output of the support information. In step S306, the setting unit 113 performs a process to present the identified input data, sensor, or device to the caregiver. The process in step S306 may be a process to transmit the display image itself, similar to the process in step S302, or a process to transmit the information used to generate the display image. In addition, the presentation here is not limited to display, and presentation processing using sound or other means may also be performed.

[0125] 3.2.2 Output processing of support information Figure 19 is a flowchart illustrating the output process of support information by the support information output unit 112. In step S401, the support information output unit 112 acquires input data corresponding to the support information to be output. Specifically, the storage unit 120 of the server system 100 stores one or more input data for outputting the target support information from among the multiple input data shown in Figures 28 to 42. The correspondence between the support information and the input data is performed, for example, using the second correspondence information 124 described above.

[0126] In step S402, the support information output unit 112 obtains support information by inputting the necessary input data to the support information output NN122. In step S403, the support information output unit 112 determines whether or not notification based on the support information is necessary. If notification is necessary, in step S404, the support information output unit 112 performs notification processing. Notification may be by voice using the earphones of a headset, by display using the display unit 214 of the mobile terminal device 210, or by other means. If notification is not necessary, or after notification processing has been performed, the support information output unit 112 terminates processing.

[0127] As mentioned above, the number of support information items to be output can be changed depending on the type of sensors installed in the care facility and the settings of the caregivers. However, in all cases, the processing flow for each support information item to be output is the same, as shown in Figure 19: identification of input data, calculation by neural network, and notification as needed.

[0128] If the server system 100 has sufficient processing capacity, the support information output unit 112 may continuously perform the processing shown in Figure 19 for all support information set as output targets, and may appropriately perform notification processing for those that are determined to require notification.

[0129] Furthermore, to reduce the processing load, the processing shown in Figure 19 may be limited to the support information needed at that time. For example, as shown in Figures 43 to 45, the support information can be classified into necessary situations, such as support information needed for meal assistance and support information needed for excretion assistance. Therefore, the support information output unit 112 may identify the support information needed in the current situation and execute the processing shown in Figure 19 on the identified support information. For example, the support information output unit 112 may determine whether or not to start assistance for meal assistance, excretion assistance, and transfer / mobility assistance. The support information output unit 112 performs the processing shown in Figure 19 on the support information related to the assistance that has been determined to be started. The start determination will be described later using Figure 20.

[0130] Furthermore, in the case of meal assistance, it is possible to classify it chronologically, such as assistance performed before the meal, assistance performed during the meal, and assistance performed after the meal. Therefore, the support information output unit 112 can define the order in which the processes shown in Figure 19 are executed among multiple support information. In addition, depending on the assistance, there may be constraints on the execution order or necessity of each assistance, such as the second assistance being necessary only if the first assistance has been performed.

[0131] Therefore, the assistance support provided by the information processing system 10 of this embodiment may be performed according to an assistance sequence that combines multiple assistance actions. Specifically, the support information output unit 112 sequentially outputs multiple support information actions according to the assistance sequence, thereby supporting the assistance provided by the caregiver to the person being assisted.

[0132] Below, we will explain examples of assistance sequences for meal assistance, toileting assistance, and transfer / mobility assistance. Specifically, we will first determine when to start each assistance sequence, and then explain the specific flow of each assistance sequence.

[0133] As will be explained later using Figures 21 to 23, in the following assistance sequences, for the sake of explanation, only some of the support information from Figures 43 to 45 is output. However, it will be easily understood by those skilled in the art that it is possible to modify each assistance sequence described later by omitting the output of some of the support information, or by adding the output of other support information shown in Figures 43 to 45.

[0134] 3.2.3 Start determination In this embodiment, assistance may include assistance with eating, assistance with excretion, and assistance with transfers and mobility. However, these assistance does not need to be provided at all times; a specific assistance sequence is executed only when the person receiving assistance needs such assistance and a caregiver capable of performing the assistance exists. That is, in this embodiment, a determination is made first to determine when to start the assistance sequence, and depending on the result of that determination, the start or waiting of the assistance sequence may be decided.

[0135] Figure 20 is a flowchart illustrating the start determination. This process is performed periodically, for example, for each person receiving assistance. First, in step S501, the support information output unit 112 acquires at least a portion of the input data shown in Figures 28 to 42. In step S502, the support information output unit 112 obtains support information by inputting the acquired input data into the support information output NN122. The support information here is information that identifies at least one of the following: the start timing for meal assistance, the start timing for excretion assistance, and the start timing for transfer / mobility assistance. For example, the support information output unit 112 may determine whether or not to start each assistance at the time the process in Figure 20 is performed. Alternatively, the support information output unit 112 may output information that identifies a specific time, such as how many minutes later each assistance will start.

[0136] In step S503, the support information output unit 112 determines whether the current timing is the start timing of the assistance sequence. If it is determined that it is not the start timing, the support information output unit 112 terminates processing and waits again until the processing shown in Figure 20 is performed.

[0137] For example, the support information output unit 112 determines the start timing of the meal assistance sequence by taking as input data information not only about the meal schedule at the nursing care facility, but also the individual person being cared for, such as their complexion, body temperature, weight, medication, past meal history, excretion history, and rehabilitation history.

[0138] Furthermore, regarding assistance with excretion, there may be cases where a general schedule is set, such as five times a day. Therefore, the support information output unit 112 determines the timing of the start of the excretion assistance sequence by taking as input data information not only the excretion assistance schedule at the care facility, but also the amount and timing of meals for the person being cared for, the amount and timing of fluid intake, whether or not laxatives were administered, past excretion history, rehabilitation records, and the condition of pressure ulcers.

[0139] Furthermore, the support information output unit 112 determines the start timing of the assistance sequence related to transfer and mobility assistance by taking as input data whether or not an event requiring the assistance recipient to move has occurred, such as meals or recreation, as well as the assistance recipient's ADL, medical history, etc.

[0140] If it is determined that the current timing is the start timing of the assistance sequence, in step S504, the support information output unit 112 performs a process to determine the caregiver who will assist the person receiving assistance. For example, the support information output unit 112 may hold information such as the work shifts of caregivers at the care facility and the assignment of care recipients, and may determine the caregiver based on this information.

[0141] In step S505, the support information output unit 112 performs notification processing to instruct the caregiver's device 200 of the determined caregiver to start the assistance sequence. For example, the support information output unit 112 may perform processing to play an audio message such as "Please start assisting Mr. / Ms. A with his / her meal" on a wearable device 220 such as a headset. Alternatively, the support information output unit 112 may perform processing to display the same text on the display unit 214 of the mobile terminal device 210.

[0142] In step S506, the support information output unit 112 determines the caregiver's response to the notification process. For example, three types of responses may be set for the caregiver: "OK", "Later", and "transfer". The caregiver's response may be given by voice. For example, the caregiver's response may be obtained based on the detection results from the headset microphone. Alternatively, the caregiver's response may be realized by other means, such as text input.

[0143] "OK" is a response indicating that the instructed assistance sequence can be started. In this case, in step S507, the support information output unit 112 proceeds to the specific assistance sequence. For example, the support information output unit 112 starts the processing shown in Figures 21, 22, and 23.

[0144] "Later" is a response indicating that the assistance sequence cannot be started immediately, but it is expected to be possible to start it after a predetermined time has elapsed. For example, this applies when the caregiver is currently performing another task, but can start the instructed assistance sequence once that task is completed. In this case, in step S508, the support information output unit 112 waits for a predetermined time, and after waiting, returns to step S505 and performs the notification process again to the same caregiver.

[0145] "Transfer" is a response indicating that it is difficult to perform the assistance sequence and that a request for assistance from another caregiver is being made. In this case, the support information output unit 112 returns to step S504 and selects another caregiver. The process from step S505 onwards is similar.

[0146] However, the processing when "transfer" is selected is not limited to this. For example, if a given caregiver selects "transfer," the support information output unit 112 may send a notification to multiple caregivers simultaneously. Then, it may select a caregiver who responds with "OK" from among the multiple caregivers and start a specific care sequence targeting that caregiver.

[0147] 3.2.4 Assistance with meals Figure 21 is a flowchart illustrating a specific assistance sequence when providing meal assistance. First, when the meal assistance sequence begins, in step S601, the support information output unit 112 controls the sensors necessary for supporting meal assistance from among the sensor group 400 installed in the care facility. In step S601, the support information output unit 112 may remotely control the on / off status of the sensors included in the sensor group 400. Alternatively, the support information output unit 112 may instruct devices within the care facility, such as the caregiver device 200, to turn on the sensors or devices, and the caregiver may turn on the sensors according to the instructions. From this point onward, although not explicitly shown in the flowchart, the sensor group 400 periodically transmits sensor information to the server system 100, and the support information output unit 112 is capable of acquiring the input data necessary for outputting support information.

[0148] In step S602, the support information output unit 112 outputs support information for providing meals tailored to the person being cared for, based on the support information output NN122. For example, in step S602, the support information output unit 112 outputs support information instructing meals tailored to the person being cared for's allergies and medications tailored to their medical condition.

[0149] Next, in step S603, the support information output unit 112 determines whether the person being cared for and the caregiver have moved to a location where they will eat. The meal may be taken in the person being cared for's room or in a dining room, etc. The process in step S603 is performed by inputting information that can identify the location of the person being cared for and the caregiver, such as a camera or RFID (radio frequency identifier). In addition, the process in step S603 may be performed by determining that the person being cared for and the caregiver have moved to a location where they will eat when, for example, the person being cared for and the meal are captured on the screen of a camera carried by the caregiver.

[0150] If at least one of the person being assisted and / or the caregiver is not in position, in step S604, the support information output unit 112 waits for a certain period of time and then performs the process of step S603 again.

[0151] Once the person being cared for and the caregiver are in position, in step S605, the support information output unit 112 determines the minimum amount of food to be served. Here, the minimum amount may be less than the amount served. In other words, the caregiver is not required to make the person eat all of the served food, and once the minimum amount is reached, they are not required to force the person to eat any more. The processing in step S605 is performed by inputting data such as the person being cared for's complexion, care records, weight changes, and meal schedule.

[0152] In step S606, the support information output unit 112 notifies the caregiver of the determined minimum supply amount. The notification may be made by voice using earphones such as a headset, or by display using the display unit 214 of the portable terminal device 210.

[0153] In step S607, the support information output unit 112 determines the timing of serving food with a spoon and the amount of food served with the spoon. The timing of serving food with a spoon refers to the timing at which a bite-sized portion of food placed on the spoon is placed in the mouth of the person being assisted. The amount of food served with the spoon refers to the amount of food served in one bite. The processing in step S607 is performed, for example, by using input data related to the chewing state of the person being assisted. Input data related to the chewing state includes, for example, information about the state of the person being assisted's mouth, throat, facial expression, complexion, posture, changes in eating in response to verbal cues, swallowing timing, time the food is in the mouth, eating rhythm, etc., and may also be, for example, an image of the person being assisted. Input data related to the chewing state also includes information about jaw movement, cheek movement, overall facial movement, and body movement, and may also be, for example, sensor information from the motion sensor 410. Furthermore, input data related to the chewing state may include audio data representing the voice quality and volume in response to verbal cues during meals, as well as information representing differences in timing and quantity of past meals, seasonal differences, and differences due to physical condition. The information representing the eating rhythm may be captured images or sensor information from the motion sensor 410. In addition, the sensors used to acquire the above information can be modified in various ways.

[0154] In step S608, the support information output unit 112 informs the caregiver of the timing for serving the meal with the spoon and the amount to be served with the spoon. For example, in step S607, the support information output unit 112 determines whether the current timing is the timing for serving the meal with the spoon. If the support information output unit 112 determines that it is the timing for serving, it notifies the caregiver of this fact in step S608, and does not notify the caregiver if it determines that it is not the timing for serving. In addition, if the support information output unit 112 determines that it is not the timing for serving, it may notify the caregiver that it is not the timing for serving when the caregiver attempts to serve the meal to the person being cared for.

[0155] Furthermore, if the support information output unit 112 determines, for example, that it is time to serve, in step S607 it may determine the amount to serve with a spoon, and in step S608 it may inform the caregiver of the determined amount in grams or in stages such as more / normal / less. Alternatively, in step S607 the support information output unit 112 may use input data representing the amount of food the caregiver actually put on the spoon to obtain support information indicating whether the amount served with the spoon is appropriate. In this case, the input data may include, for example, the output of a camera that images the caregiver's hands. If the amount of food on the spoon is too much or too little, in step S608 the support information output unit 112 may provide notification prompting the caregiver to change the amount of food on the spoon.

[0156] Furthermore, the facial expression of the person being cared for may be used to determine whether the caregiver's assistance is correct or not. For example, the support information output unit 112 may not output any specific instructions when the person being cared for is smiling, but may output instructions when the person being cared for is making an unpleasant face. For example, the support information output unit 112 may use an image of the person being cared for's face or the result of facial expression determination processing based on that image as input data in step S607. In this way, it is possible to determine whether the pace of eating is appropriate or not based on the person being cared for's facial expression. Alternatively, the support information output unit 112 may determine the degree of relaxation from heart rate (pulse) analysis. The support information output unit 112 may not output any specific instructions when the person being cared for is highly relaxed, but may output instructions when the person being cared for is not relaxed. For example, the support information output unit 112 may use information representing heart rate, pulse rate, or the results of their analysis as input data in step S607. Furthermore, the use of facial expressions and levels of relaxation to determine whether assistance is appropriate also applies to steps other than S607 in Figure 21, as well as to Figures 22 and 23, which will be discussed later.

[0157] In step S609, the support information output unit 112 determines whether or not the person being assisted has finished eating. If not, the process returns to step S607. By repeating the processes in steps S607 and S608 in this way, it becomes possible to continuously inform the caregiver of the timing and amount of food to be served for each bite. As a result, it becomes possible to allow the person being assisted to eat at an appropriate pace.

[0158] If it is determined that the meal has been completed, in step S610, the support information output unit 112 instructs the recording of the meal results. For example, as the meal results, an image of any leftover food is acquired. The instruction to record may involve instructing the caregiver to take an image using a portable terminal device 210 or the like, or it may involve automatically taking an image by remotely controlling a camera placed in an appropriate location.

[0159] In step S611, the support information output unit 112 obtains support information indicating whether or not the person being cared for needs to be hydrated. If hydration is necessary, in step S612, the support information output unit 112 notifies the caregiver to provide hydration. Alternatively, in step S611, the support information output unit 112 may obtain support information indicating the specific amount to be hydrated and notify the caregiver of that amount in step S612.

[0160] If hydration is not required, or after processing in step S612, the meal assistance sequence is terminated.

[0161] Figure 21 shows an example of a meal assistance sequence, and various variations of the sequence are possible. For example, when meal assistance is performed on the care bed 310, control may be added to switch the care bed 310 to a meal mode suitable for eating (for example, a mode that raises the backrest to a set angle in the range of 30 to 90 degrees, a mode that raises the knees to a set angle in the range of 0 to 30 degrees, a mode that lowers the feet to a set angle in the range of 0 to 90 degrees, or a mode that tilts the bed so that the head side is higher to a set angle in the range of 0 to 20 degrees). For example, the support information output unit 112 may obtain support information indicating whether the person being assisted and the served meal are in a state suitable for starting a meal by taking the output of a camera or the like as input data. If it is determined that the person being assisted and the meal are properly set up, the support information output unit 112 performs a notification process to ask the caregiver if it is OK to move the care bed 310. If the caregiver answers "OK", the care bed 310 may be changed to meal mode.

[0162] Furthermore, the support information output unit 112 can output various support information related to meal assistance to caregivers and cooks even before the meal is ready at the dining area.

[0163] 3.2.5 Assistance with excretion Figure 22 is a flowchart illustrating a specific assistance sequence when providing assistance with excretion. When the excretion assistance sequence begins, in step S701, the support information output unit 112 controls the sensors among the sensor group 400 installed in the care facility, etc., to turn on the sensors necessary to support excretion assistance.

[0164] Next, in step S702, the support information output unit 112 determines whether the caregiver has moved to the position where the assistance will be performed. For example, if the person being assisted has defecated into a diaper on the care bed 310, it is assumed that the assistance will be performed in the person being assisted's room. In this case, the processing in step S702 is performed by using, for example, the output of a camera placed in the room, or the output of a camera on a portable terminal device 210 carried by the caregiver, or the output of an RFID as input data.

[0165] If the caregiver is not in position, in step S703, the support information output unit 112 waits for a certain period of time and then performs the process of step S702 again. If the caregiver is in position, in step S704, the support information output unit 112 requests support information regarding diaper removal. In step S705, the support information output unit 112 performs notification processing of the requested support information.

[0166] For example, if the posture of the person being cared for, the caregiver, or the direction in which the diaper is removed is not appropriate, feces may adhere to the person being cared for's clothes or sheets, which is undesirable. Therefore, the support information output unit 112 may, for example, in step S704, determine whether the caregiver's movements when removing the diaper are appropriate. For example, the support information output unit 112 may acquire the correct movements and compare them with the actual movements of the caregiver. If it is determined that the movements are inappropriate, in step S705, the support information output unit 112 may notify that the movements are inappropriate, or it may give specific instructions on how to perform the appropriate movements.

[0167] In step S706, the support information output unit 112 requests support information regarding diaper application. In step S707, the support information output unit 112 performs notification processing for the requested support information.

[0168] The support information output unit 112 may, for example, determine in step S706 whether the caregiver's movements when putting on a new diaper are appropriate. For example, the support information output unit 112 may acquire the correct movements and compare them with the actual movements of the caregiver. If it is determined that the movements are inappropriate, in step S707, the support information output unit 112 may notify that the movements are inappropriate or may give specific instructions on appropriate movements.

[0169] While soiling of sheets, etc., when removing diapers is a problem, the presence of a caregiver makes it easier for the caregiver to recognize the soiling and to deal with it relatively easily. On the other hand, if stool leakage occurs due to insufficient fitting, the caregiver may not be present when the leakage occurs. Furthermore, considering the burden on caregivers, it is not easy to increase the frequency of excretion assistance excessively, and there is a risk that stool leakage may be left unattended for a long time. In light of the above, the support information output unit 112 may set conditions so that notification in step S707 is more likely to occur than in step S705. For example, if notification in steps S705 and S707 is performed when the degree of deviation between the correct answer and the actual movement exceeds a threshold, the threshold in step S707 is set to be smaller than the threshold in step S705.

[0170] Alternatively, in step S706, the sensor information from a sensor installed on the diaper may be used as input data to determine in detail whether the diaper is properly fitted. In this case as well, it becomes possible to provide assistance that places more emphasis on proper diaper fitting.

[0171] In step S708, the support information output unit 112 instructs the recording of the excretion status. Specifically, the support information output unit 112 instructs the imaging of the urine and stool, and the measurement of the weight of the urine and stool. The weight measurement may be the weight of the diaper, or the weight of the garbage if a garbage can is being transported. The recording instruction here may involve having a caregiver perform the imaging and weight measurement, or it may involve remotely controlling the camera or sensor.

[0172] Figure 22 shows an example of an excretion assistance sequence, and various variations of the specific sequence are possible. For example, when excretion assistance is performed on a care bed 310, control may be added to change the height of the care bed 310 to a height suitable for excretion assistance (for example, a height of 50mm to 100mm from the floor to the top of the bottom so that the caregiver does not need to bend over). For example, if the caregiver responds "OK" in step S506 of Figure 20, changing the height of the care bed 310 will create a state where it is easier to provide toileting assistance when the caregiver arrives. If the care bed 310 has a speaker, the support information output unit 112 may output an audio message explaining the purpose of the height change to the person being cared for before the height change is performed. The support information output unit 112 may also notify the caregiver when the height change of the care bed 310 is complete.

[0173] 3.2.6 Transfer assistance or mobility assistance Figure 23 is a flowchart illustrating a specific assistance sequence when providing transfer assistance or mobility assistance. When the transfer / mobility assistance sequence begins, in step S801, the support information output unit 112 controls the sensors among the sensor group 400 installed in the care facility, etc., that are necessary to support the transfer / mobility assistance.

[0174] Next, in step S802, the support information output unit 112 determines whether a lift is necessary for the transfer and mobility assistance of the person being assisted. The processing in step S802 is carried out by inputting data such as the difference in physique between the caregiver and the person being assisted, the person being assisted's ADL, the time required for the transfer, and the inventory of lifts at the care facility.

[0175] If a lift is not needed, the caregiver manually transfers the person being cared for to the wheelchair. In step S803, the support information output unit 112 requests support information regarding the manual transfer. In step S804, the support information output unit 112 processes a notification for the requested support information. The support information output unit 112 may also notify the caregiver whether or not the wheelchair needs to be locked before the transfer. If the caregiver replies that it does, the support information output unit 112 controls the wheelchair to lock. Alternatively, the support information output unit 112 may automatically determine whether or not locking is necessary, and if it is determined that it is necessary, it may instruct the caregiver to lock the wheelchair before the processing in step S803.

[0176] The support information output unit 112 may, for example, determine in step S803 whether the caregiver's use of their body during manual transfer is appropriate. For example, the support information output unit 112 may acquire correct movements such as the caregiver's posture and the position where the person being assisted is placed on the caregiver's feet, and compare these movements with the actual movements of the caregiver. The caregiver's movements may be detected using the motion sensor 410 or the imaging sensor 420. In addition, since the positional relationship between the person being assisted and the caregiver is important in transfer and mobility assistance, these sensors may detect the movements and posture of the person being assisted in addition to the caregiver's movements. If it is determined that the movements are inappropriate, in step S804, the support information output unit 112 may notify that the movements are inappropriate or may give specific instructions for appropriate movements.

[0177] If a lift is needed, in step S805, the support information output unit 112 controls the movement of the lift to the person being assisted. In step S806, the support information output unit 112 requests support information regarding the transfer using the lift. In step S807, the support information output unit 112 performs notification processing of the requested support information.

[0178] The support information output unit 112 may, for example, determine in step S806 whether the lift is being used appropriately. For example, the support information output unit 112 may acquire correct data, such as the state in which the sling is attached so that the person being assisted can be safely lifted, and compare the correct state with the actual state. If it is determined to be inappropriate, in step S807 the support information output unit 112 may notify that it is inappropriate, or it may give specific instructions on the appropriate attachment state.

[0179] After lifting the person being assisted with the lift, they may be seated in a wheelchair or moved as is. The support information output unit 112 may determine which to use, taking into account the availability of lifts, the destination, and the condition of the person being assisted, and notify the caregiver accordingly.

[0180] Figure 23 shows an example of a transfer / mobility assistance sequence, and various variations of the specific sequence are possible. For example, control may be added to change the height of the care bed 310 (for example, the height from the floor to the top of the bottom of the care bed should be 200mm to 500mm for non-assisted persons who can stand, a height slightly higher than the wheelchair (200mm to 500mm) for transfers to a wheelchair, and a height slightly lower than the wheelchair (200mm to 500mm) for transfers from a wheelchair to a bed, etc.) to a height suitable for transfer / mobility assistance. The specific control is the same as for assistance with excretion, etc., so a detailed explanation is omitted.

[0181] 3.2.7 Specific changes in assistance depending on the factors The above explains the specific sequences of assistance with eating, toileting, and transfer / mobility. Although not explained in Figures 21-23, each piece of supporting information may be determined based on the presence or absence of abnormal behavior and its contributing factors. For example, as described above using Figure 13, the presence or absence of abnormal behavior and the results of the determination of the contributing factors of abnormal behavior are used as input data when determining supporting information.

[0182] For example, if the factor determination unit 111 determines that a factor is related to dementia, the support information output unit 112 changes the output in each assistance sequence, assuming that dementia is progressing. For example, in meal assistance, the support information output unit 112 outputs information representing the amount of food provided per bite (amount provided with a spoon) and information representing the timing of providing each bite of food (timing of providing food with a spoon) as support information. In this case, if the behavior is determined to be abnormal behavior related to dementia, the support information output unit 112 may change at least one of the amount and timing of provision compared to when the behavior is determined not to be abnormal behavior related to dementia. In this way, it becomes possible to appropriately change the pace of meals for care recipients with dementia and care recipients without dementia. For example, the support information output unit 112 may reduce the amount provided or delay the timing of provision. In this way, it becomes possible to appropriately manage the pace of meals when care recipients are prone to choking due to dementia.

[0183] Furthermore, the support information output unit 112 outputs information as support information that identifies the timing of the excretion assistance, which is the timing at which excretion assistance should begin. In this case, if the behavior is determined to be abnormal behavior due to dementia, the timing of the excretion assistance may be changed compared to when the behavior is determined not to be abnormal behavior due to dementia. In this way, it becomes possible to start the excretion assistance sequence at an appropriate timing depending on whether or not the person has dementia. For example, the support information output unit 112 may start the excretion assistance earlier. By adjusting the timing of the excretion assistance in this way, it becomes easier to maintain a clean state even when it becomes difficult for the person being cared for to control their own excretion timing due to dementia.

[0184] In addition, if abnormal behavior is determined to be due to a dementia factor, the support information output unit 112 may change the support information to provide the following meal assistance. For example, if abnormal behavior is determined to be due to a dementia factor, the support information output unit 112 may set a higher priority for notification regarding the following. For example, the support information output unit 112 may provide notification regarding the following in the case of dementia factors, but may not provide notification regarding the following in the case of no abnormal behavior or in the case of factors other than dementia. Alternatively, the support information output unit 112 may provide notification regarding the following regardless of whether or not it is due to a dementia factor, but may implement control to make notification more likely when it is due to a dementia factor. For example, the support information output unit 112 may increase the notification frequency in the case of dementia factors, or relax the conditions for determining whether or not notification is necessary. • To focus on eating, make sure to relieve yourself before meals. • Notifies you if you are getting enough sleep and if you are feeling well. After serving the meal, observe without providing assistance to understand the situation for the day. • Create a comfortable environment, prepare calming tableware, and use tableware you cherish. • Adjust to a comfortable eating position. If the person is not eating well, adjust the feeding time accordingly. • To help them understand that it's a meal, speak to them and assist them with the first bite. • Offer to explain how to eat it. • Provide moisture to prevent dehydration. Adjust the amount of food on the spoon and the feeding speed to prevent choking. If your appetite is poor, increase your activity level and regulate your daily routine.

[0185] Furthermore, if abnormal behavior is determined to be due to dementia, the support information output unit 112 may change the support information to perform the following type of excretion assistance. • Notify us if you are getting enough sleep and if you are feeling well. • Select and use appropriate pants, diapers, or pads. • If using the toilet, check that the toilet has been flushed. • Take precautions against falls and slips, as the number of times you need to go to the toilet may increase. Observe the timing of urination and urge them to go to the toilet. • There is a possibility of smearing feces, so guide the patient to the toilet and change their diaper at the appropriate time. • Assign staff who are a good match for each other.

[0186] The support information output unit 112 may also output support information related to sleep assistance. If abnormal behavior is determined to be a factor in dementia, the support information output unit 112 may change the support information to perform the following sleep assistance. • Maintain a healthy autonomic nervous system by regulating your lifestyle. • Exercise to increase your daytime activity level. • Monitor for any unusual behavior at night.

[0187] When providing nighttime monitoring, for example, sensor information from bed exit sensors and monitoring sensors is used as input data. For example, the support information output unit 112 instructs caregivers who are not assisting with breakfast to install the sensor while the person being cared for is eating breakfast. Also, when performing exercise, the support information output unit 112 may notify caregivers, for example, after daytime toileting assistance, suggesting recreation or rehabilitation.

[0188] Furthermore, as described above with reference to Figure 6, the factor determination unit 111 may be able to determine whether the cause of the abnormal behavior is an environmental factor or an excretory disorder factor. For example, if the abnormal behavior is determined to be due to an excretory disorder factor, the support information output unit 112 may change the support information to provide assistance as follows. • Notify customers about the addition of laxatives to their dinner. • Changes to the contents of meals (applicable to breakfast, lunch, and dinner) • Instruct that fluids be provided after meals. • Suggest recreational activities or rehabilitation after daytime toileting assistance.

[0189] Furthermore, the support information output unit 112 may not only instruct the addition of laxatives, but may also suggest specific types of laxatives and administration times. For example, the support information output unit 112 may use information indicating how many consecutive days laxatives have been administered, information on the interval between bowel movements, etc., as input data to inform the caregiver of the type of laxative. In addition, if a person being cared for who was initially judged to have dementia-related issues is later judged to have excretory problems, the support information output unit 112 may instruct the caregiver to remove sensors other than the excretion sensors among the sensors installed to address dementia.

[0190] Furthermore, if the abnormal behavior is determined to be due to environmental factors, the support information output unit 112 may change the support information to provide assistance as follows. • Automatically control the rhythm of speakers and lighting to match the data before environmental influences occurred.

[0191] By creating an environment similar to the one before the abnormal behavior occurred, it becomes possible to regulate the care recipient's daily rhythm. Caregivers may also change settings to temporarily suspend or disable the application of automatic control. Furthermore, if a care recipient whose behavior was initially judged to be dementia-related is later determined to be environmentally related, the support information output unit 112 may instruct the caregiver to remove the sensors that were placed to address dementia.

[0192] Furthermore, the support information output unit 112 may increase the types of support information to be output when the behavior is determined to be a dementia factor compared to when it is not determined to be abnormal behavior. For example, the support information for "creating a suitable environment, providing calming tableware, and providing tableware that the person is attached to" mentioned above will be output when the behavior is determined to be a dementia factor, but will not be output when it is determined to be due to other factors. In this case, for example, input data from a temperature sensor, humidity sensor, illuminance sensor, or atmospheric pressure sensor may be used to determine a favorable environment for the person being cared for.

[0193] Therefore, the support information output unit 112 may increase the types of sensor information used when the behavior is determined to be a dementia factor, compared to when it is not determined to be abnormal behavior. In this way, the types of input data increase, making it possible to accurately obtain support information that is appropriate for caregiving in dementia.

[0194] The support information output unit 112 may also determine whether to add a new sensor based on information identifying one or more usable sensors and sensor information added when the behavior is determined to be a dementia factor. Here, the one or more usable sensors are specifically sensors placed in the target nursing care facility and are identified based on the third correspondence information 125 in Figure 16. As described above using Figures 14 to 16, depending on the type of sensor placed in the nursing care facility, it may not be possible to output the given support information with sufficient accuracy, and the support information may be set to "cannot be output". Therefore, depending on the nursing care facility, even if the factor determination unit 111 determines that it is a dementia factor, it may be difficult to output support information appropriate for dementia. The information processing device may, for example, determine whether to add a sensor and suggest adding the sensor that needs to be added or the device containing the sensor. In this way, it becomes possible to output support information that matches the factor appropriately.

[0195] As explained above, it is expected that appropriate assistance will vary depending on the presence or absence of abnormal behavior and the factors causing such abnormal behavior. According to the method of this embodiment, the results of the factor determination of the person being assisted are used when supporting the caregiver's assistance. As a result, it becomes possible to have the caregiver provide assistance that is more appropriate for the person being assisted.

[0196] Specifically, by digitizing the tacit knowledge of skilled caregivers, it becomes possible to enable less skilled caregivers to provide appropriate care. For example, less skilled caregivers can provide care equivalent to that of skilled caregivers, improving the reproducibility of care. Furthermore, variations in care skills are suppressed, and organizational management becomes easier, thus reducing incidents such as falls among those being cared for. As a result, it is possible to reduce the number of empty beds due to hospitalization and the amount of overtime required to prepare accident reports. Also, if incidents are suppressed, caregivers will not become overly sensitive to risks, thus reducing stress and, consequently, the turnover rate can be reduced. In addition, by improving the skills of caregivers and the working environment, it is possible to improve the satisfaction of those being cared for and their families, and improve their quality of life (QOL).

[0197] In this embodiment, the information processing system 10, server system 100, caregiver device 200, etc., may implement part or most of their processing using programs. In this case, the information processing system 10, etc., of this embodiment is implemented by a processor such as a CPU executing a program. Specifically, a program stored in a non-temporary information storage medium is read, and the CPU or other processor executes the read program. Here, the information storage medium (a medium readable by a computer) stores programs, data, etc., and its function can be realized by an optical disc, HDD, or memory (card-type memory, ROM, etc.). The CPU or other processor then performs various processing of this embodiment based on the program stored in the information storage medium. That is, the information storage medium stores programs that enable the computer to function as each part of this embodiment.

[0198] Furthermore, the method of this embodiment can be applied to an information processing method that determines whether the behavior of the person being cared for is abnormal behavior due to dementia factors, based on (1) information on the level of dementia of the person being cared for and (2) at least one of the environmental information, excretion information, and sleep information of the person being cared for, and outputs support information to support the caregiver's care of the person being cared for, based on the determination result and sensor information which is sensing result regarding the caregiver or the person being cared for.

[0199] 4. Variations <Parallel processing of multiple assistance sequences> The assistance sequences described above using Figures 21 to 23 may be executed sequentially. For example, a given caregiver, while in standby mode, responds with OK at step S506 in Figure 20, executes the sequence corresponding to one of Figures 21 to 23, and returns to standby mode after completion. Standby mode refers to a state in which the caregiver is not executing any assistance sequences. Then, by responding with OK again at step S506, the caregiver executes the sequence corresponding to one of Figures 21 to 23.

[0200] However, in nursing care facilities, one caregiver may assist multiple individuals simultaneously. For example, if individuals A and B are seated close together, one caregiver may simultaneously assist both individuals with meals. In this case, it would be inefficient to perform the meal assistance sequence shown in Figure 21 for individual B only after completing the sequence for individual A.

[0201] Therefore, the support information output unit 112 may be capable of executing multiple assistance sequences in parallel for a single caregiver. For example, in the above example, the support information output unit 112 would execute the meal assistance sequence for person A and the meal assistance sequence for person B in parallel. Although this example describes a 1:2 ratio of caregivers to people receiving assistance, one caregiver may be responsible for three or more people receiving assistance at the same time.

[0202] For example, in the meal assistance sequence for person A, the support information output unit 112 performs the processing in step S605 and notifies the result in step S606 in the format of "The minimum amount to be provided to person A is x grams." Similarly, in the meal assistance sequence for person B, the processing in step S605 is performed and the result is notified in step S606 in the format of "The minimum amount to be provided to person B is y grams." In this way, the support information output unit 112 performs the acquisition of input data for person A and the acquisition of input data for person B in parallel, and based on the respective input data, it outputs support information for person A and the output of support information for person B at the necessary timing. In this way, even if there is a one-to-many relationship between caregivers and people being cared for, it becomes possible to have the caregiver perform the necessary assistance for each person being cared for. Furthermore, by installing a wide-angle camera capable of simultaneously imaging multiple people being cared for, it is also possible to use the same input data for person A and person B.

[0203] However, since there is only one caregiver, it is not easy to respond to all of the support information even if multiple pieces of information are communicated at very close intervals. For example, if step S608 is communicated for person A, the caregiver will take the amount of food indicated in the communication on a spoon and bring it to person A's mouth. Even if step S608 is communicated for person B before this is completed, it will be difficult for the caregiver to take food for person B on a spoon and bring it to person B's mouth.

[0204] Alternatively, when assisting multiple people receiving care with meals, it is more efficient to have everyone gather in a dining area such as a cafeteria before feeding them. Therefore, even if it is determined that the caregiver and person receiving care A are in position (Yes in step S603), if person receiving care B is not in position, it may not be advisable to start the processes in steps S607-S609 for person receiving care A.

[0205] Considering these factors, the support information output unit 112 may perform processing that takes into account the relationships between multiple assistance sequences, rather than simply executing assistance sequences for multiple care recipients in parallel. For example, when the support information output unit 112 executes multiple assistance sequences in parallel for a given care recipient, it may control the execution and suspension (suspension) of each assistance sequence.

[0206] For example, if the support information output unit 112 issues the notification in step S608 for person A, it may suspend the meal assistance sequence for person B. Then, when the caregiver has finished giving person A one bite of food, it resumes the meal assistance sequence for person B. Since the support information output unit 112 has determined in step S607 that it is OK to provide food to person B, in step S608 it issues a notification to the caregiver to have person B take one bite of food. In this case, since the caregiver is performing an action for person B, the support information output unit 112 suspends the meal assistance sequence for caregiver A until that action is completed.

[0207] Alternatively, if the support information output unit 112 determines that person A is in position (Yes in step S603), it may suspend the meal assistance sequence for person A until it determines that all other persons being assisted by the same caregiver have also taken their positions.

[0208] Figure 24A is a state transition diagram illustrating the transitions in the assistance sequence for a given caregiver. For example, the support information output unit 112 executes two meal assistance sequences to support caregivers who provide meal assistance to persons A and B. In this case, the support information output unit 112 performs state transitions based on given conditions. For example, when the support information output unit 112 is executing a meal assistance sequence for person A, if it determines that one unit of assistance by the caregiver has been completed, it stops the meal assistance sequence for person A and transitions to a state where it executes a meal assistance sequence for person B.

[0209] Alternatively, the support information output unit 112 may determine the priority of the support information to be notified in each assistance sequence. For example, suppose the support information output unit 112 determines that person A has finished eating and therefore should be notified to record the eating result (step S610), and that person B has not finished eating and therefore should be notified to provide a bite of food (step S608). Recording the eating result can be done at any time as long as cleanup is completed, whereas providing a bite of food will not complete person B's meal until it is done. In this case, the support information output unit 112 may prioritize the execution of person B's meal assistance sequence and suspend person A's meal assistance sequence. Even in this way, it is possible to achieve appropriate state transitions between multiple assistance sequences targeting multiple people. State transitions between multiple assistance sequences can be thought of as other assistance sequences interrupting an assistance sequence that is currently running.

[0210] Furthermore, although the above example shows two meal assistance sequences being executed in parallel, the method of this embodiment is not limited to this. Figure 24B is another diagram illustrating the state transitions between assistance sequences in this embodiment.

[0211] As shown in Figure 24B, in this embodiment, various sequences such as meal assistance sequences, excretion assistance sequences, transfer assistance sequences, and abnormal response sequences may be executed in parallel. In this case, the support information output unit 112 may control the transitions between each assistance sequence shown in Figure 24B. Although Figure 24B shows an example where a standby state is passed through when transitioning from one type of assistance sequence to another, direct transitions may also occur between each assistance sequence. Also, as shown in Figure 24A, a meal assistance sequence may contain multiple assistance sequences. Similarly, other assistance sequences such as excretion assistance sequences may also contain multiple assistance sequences.

[0212] For example, suppose a caregiver is providing a meal to person A when person A becomes abnormal. An abnormal condition might be, for example, choking. In this case, the caregiver will stop assisting person A with eating and take action to address the abnormality. For example, the support information output unit 112 performs a background determination to start the abnormality response sequence, similar to steps S501 to S503 in Figure 20, and starts the abnormality response sequence when it detects an abnormality in person A. Although the processes in steps S505 to S506 in Figure 20 may be executed, the processes in steps S505 to S506 may be omitted, considering that the person in charge of person A is the same person in charge of meal assistance and that there is a high possibility of urgency.

[0213] This adds an abnormality response sequence to the assistance sequences to be executed. The support information output unit 112 then suspends the currently executing meal assistance sequence and starts executing the abnormality response sequence. If the abnormality is resolved by the abnormality response sequence, the support information output unit 112 resumes the suspended meal assistance sequence or transitions to another assistance sequence.

[0214] Alternatively, while a caregiver is providing a meal to person A, person A may need to go to the toilet. In this case, an excretion assistance sequence is added to the assistance sequence to be performed. Depending on person A's ADL and the location of the toilet, a transfer assistance sequence may also be necessary. For example, the support information output unit 112 suspends the meal assistance sequence, first executes a transfer assistance sequence to move person A to the toilet, then executes an excretion assistance sequence, and resumes the suspended meal assistance sequence after completion.

[0215] Various factors can cause changes in the required assistance, including the assistance recipient's initiative, their physical condition, illnesses such as dementia, medication, environment, season, external factors, and discrepancies between the day's care progress and the plan. For example, the support information output unit 112 may perform a process to detect these factors and determine the next assistance sequence based on the detected factors and the currently executing assistance sequence.

[0216] In this way, the support information output unit 112 can appropriately respond to various situations by executing multiple assistance sequences in parallel and controlling the state transitions between these assistance sequences. For example, as mentioned above, even in cases where there is one caregiver and many people receiving assistance, it can support the determination of which assistance to perform and in what order. This reduces the burden on the caregiver, thereby reducing the risk of incidents such as aspiration and falls for the person receiving assistance. Furthermore, even if other assistance is suddenly needed during the execution of a given assistance, the support information output unit 112 can appropriately support the caregiver in providing the assistance they should provide at that time, thus reducing the burden on the caregiver and the risk to the person receiving assistance.

[0217] <User-added data> In the above explanation, it was assumed that the learning unit 114 creates the NN122 for outputting support information. For example, the provider of the information processing device may pre-select a given nursing care facility, etc., for learning purposes and create the NN122 for outputting support information using data from that nursing care facility, etc. If a new nursing care facility is added to use the services provided by the information processing device, for example, an existing NN122 for outputting support information may be used in common.

[0218] However, the method of this embodiment is not limited thereto, and new training data may be added by users such as nursing care facilities, and additional machine learning may be performed using said training data.

[0219] For example, while maintaining the common NN122 for outputting support information across multiple care facilities, data from each facility can be aggregated and used for machine learning. In this case, since training data can be collected from multiple care facilities, there is the advantage of being able to easily increase the amount of training data.

[0220] Alternatively, additional machine learning may be performed for each nursing care facility. In this case, the NN122 for outputting support information will be updated for each nursing care facility. In other words, it becomes possible to make the NN122 for outputting support information specific to the target nursing care facility.

[0221] Figure 25A shows an example of a screen displayed on the display unit 214 of a mobile terminal device 210. Compared to Figure 17, object OB4 for adding data has been added. When the caregiver selects object OB4, the screen transitions to Figure 25B.

[0222] Figure 25B includes area RE1, which displays the name of the support information to which training data will be added, and area RE2, which can be used to input the person being cared for ID, caregiver ID, and output data. The person being cared for ID is information that identifies the person being cared for. The caregiver ID is information that identifies the caregiver. Output data is information that corresponds to the output of NN122 for outputting support information. In Figure 25B, since the target is the timing of diaper changes, an example is shown where time is used as the output data. However, the format of the output data can be modified in various ways depending on the type of support information, and may be an image, audio, numerical, binary data representing truth or false, or other format.

[0223] In the example in Figure 25B, a caregiver with caregiver ID abcde performed toileting assistance on a person with care recipient ID 12345, and determined that the time 2021 / MM / DD hh:mm:ss was appropriate for changing the diaper. In addition to the caregiver's actions, the input data corresponding to the diaper change timing is acquired at the care facility. That is, a dataset that associates the input data with the output data 2021 / MM / DD hh:mm:ss can be used as training data for the NN122, which outputs support information for diaper change timing.

[0224] However, in this embodiment, it is assumed that the tacit knowledge of skilled caregivers will be converted into data, and that appropriate care will be provided regardless of the caregiver's skill level. Therefore, even if the above dataset is obtained from a given caregiver's input, it is unclear whether it is positive or negative data. Positive data represents a dataset in which appropriate correct answer data is associated with the input data, and negative data represents a dataset in which inappropriate correct answer data is associated with the input data.

[0225] Therefore, the learning unit 114 may, for example, maintain correspondence information that associates caregiver IDs with the caregiver's skill level. Skill level may be manually entered by the care facility manager, or it may be automatically determined based on years of experience, qualifications held, past care history, etc. The learning unit 114 treats datasets from highly skilled caregivers as positive data and datasets from less skilled caregivers as negative data.

[0226] Alternatively, even with assistance provided by an expert, there are cases where assistance is performed according to the manual and cases where the assistance method is adjusted based on the expert's intuition. The tacit knowledge of an expert is highly likely to be used when that expert acts on intuition. Therefore, as shown in Figure 25B, the display screen area RE2 may allow input of whether or not intuition was used. For example, when determining the timing of diaper changes, the caregiver inputs whether or not intuition was used into area RE2. The learning unit 114 uses the dataset where the corresponding input is "yes" as positive data.

[0227] The learning process after acquiring the training data is the same as in the example described above using Figure 8, so a detailed explanation will be omitted. In the example of Figure 25B, updating the NN122 that outputs support information for diaper change timing makes it possible to output more accurate diaper change timing. Although the above explanation described an example related to diaper change timing, additional training data can be added for other support information in the same way.

[0228] <Customer Support Information> Furthermore, Figures 43 to 45 were used as examples of outputtable support information. However, as can be seen from the above explanation, the support required in caregiving is diverse, and the necessary support may differ depending on the care facility or the caregiver. Therefore, it is conceivable that types of support information not included in the existing support information may be required. Accordingly, in this embodiment, caregivers may be able to add their own custom support information.

[0229] For example, in Figure 25B, the name of the support information displayed in area RE1 is not fixed and can be edited arbitrarily by the caregiver. The caregiver inputs the name of the desired custom support information using text such as "Timing to do XXXX". "XXXX" is text that represents a specific assistance action performed by the caregiver. When the caregiver performs the assistance action corresponding to "XXXX", they also input the caregiver ID, the person being assisted ID, the output data, whether or not they used intuition, etc. As a result, the output data and information indicating whether the output data is positive or negative are obtained as part of the training data for the NN122 that outputs support information, which outputs "Timing to do XXXX".

[0230] Furthermore, the information processing device may control the display unit 214 of the mobile terminal device 210 to display a screen for identifying input data from the training data. Figure 25C shows an example of a display screen for identifying input data. The screen shown in Figure 25C includes an area RE3 for displaying the name of the custom support information, and an area RE4 for selecting the name of the device already installed in the target nursing care facility and the name of the input data that can be obtained by the device.

[0231] For example, a sleep scan is a sensing device 450, as shown in Figure 2D, which can detect heart rate, respiratory rate, and activity level. The caregiver selects the data they want to use as input data when requesting custom support information from the data obtainable by the device. Figure 25C shows an example where the caregiver has selected to use respiratory rate from the sleep scan and images from a bedside camera as input data, but has not used the output of the pulse oximeter as input data.

[0232] By using the screen shown in Figure 25C, the name of the custom support information is associated with the input data used to output that custom support information. This association information is transmitted to the server system 100 and stored in the storage unit 120.

[0233] In the storage unit 120 of the server system 100, time-series respiratory rates and time-series bedside camera images collected from the target nursing facility are stored. Therefore, the learning unit 114 extracts the respiratory rate and camera image corresponding to the output data obtained using FIG. 25B as input data. For example, the server system 100 holds the acquisition timing of the output data based on FIG. 25B, and reads out the respiratory rate and camera image for a predetermined period set based on the acquisition timing from the storage unit 120. Then, the learning unit 114 performs learning processing on the support information output NN122 for outputting custom support information based on the training data in which the read input data and output data are associated.

[0234] Also, as shown in FIG. 25C, an object OB5 for performing a learning start operation may be displayed on the display unit 214 of the portable terminal device 210. When it is detected that the caregiver has performed a selection operation on the object OB5, the learning unit 114 performs the above-described learning process. As a result, a support information output NN122 that outputs custom support information is newly created. Note that since the learning process is the same as the example described above, a detailed description is omitted. Also, in the machine learning of custom support information, it is sufficient that training data associating input data and output data can be obtained, and the user interface is not limited to that described above.

[0235] At this time, the structure of the NN can be variously modified. FIG. 26 is a diagram showing an example of the structure of a general-purpose NN. The NN shown in FIG. 26 includes a CNN1 that extracts feature amounts with image data as input, a CNN2 that extracts feature amounts with audio data as input, a vector conversion NN that extracts feature amounts with text data as input, and a CNN3 that extracts feature amounts with other sensor information as input. The NN in FIG. 26 also includes a DNN (Deep Neural Network) that receives the outputs from the CNN1, CNN2, vector conversion NN, and CNN3 and outputs custom support information.

[0236] NN shown in FIG. 26 can receive an image, voice, text, and other sensor information as inputs. The input data of the custom support information can have various patterns as shown in FIG. 25C for example, and in any pattern, it is possible to appropriately receive the input data. When image data is not selected as the input data, the input to CNN1 is treated as 0. The same applies when voice data, text data, or other sensor information is not selected as the input data, and the input to the corresponding NN among CNN2, vector conversion NN, and CNN3 becomes 0.

[0237] FIG. 25D is an example of a screen displayed on the display unit 214 of the mobile terminal device 210 after the machine learning is completed. The display screen of FIG. 25D displays, for example, the accuracy rate obtained using the validation data in the learning process. Also, in the example of FIG. 25D, based on this learning result, an assistant can select whether to output custom support information. For example, when the assistant selects yes in response to the question "Do you want to apply?", the output of custom support information becomes possible. For example, in the same way as the example described above in FIG. 17, by actively setting custom support information such as "the timing to do XXXX", the custom support information is output. On the other hand, when the assistant selects no, the output of custom support information is not performed.

[0238] Also, there may be a case where the assistant thinks that although the accuracy rate is low and it cannot be adopted as it is, the target custom support information is important and wants to use it. In this case, it may be possible to request analysis processing to the administrator or provider of the information processing device. For example, when the assistant selects yes in response to the question "Do you want to request analysis?", a change process of the support information output NN122 that outputs custom support information on the server system 100 side is executed.

[0239] The learning unit 114 of the server system 100 may, for example, try to improve the accuracy by changing the structure of the NN if the original accuracy is below a predetermined threshold. This is because the NN shown in Figure 26 has a configuration that takes generality into consideration, as described above, and there is a possibility that the accuracy will improve by making it a structure that is more specialized for custom support information. If the original accuracy rate exceeds a predetermined threshold, the learning unit 114 may skip the modification process for the support information output NN122.

[0240] For example, as shown in Figure 10, if there are multiple neural networks (NNs) with different structures, such as the NN122 for outputting support information, the learning unit 114 may classify these multiple NNs into several classes.

[0241] Figure 27 illustrates the classification process of the neural network (NN). For example, the learning unit 114 obtains n-dimensional features by performing text mining on the text representing the name of the output support information, and then performs clustering based on these n-dimensional features. For the sake of explanation, Figure 27 shows a two-dimensional feature plane, but n may be 3 or greater. For example, if we consider the NN 122 for outputting support information that outputs "diaper change timing," words such as "diaper," "change," and "timing" are extracted, and the n-dimensional features of the NN that outputs "diaper change timing" are obtained based on the extraction results.

[0242] Furthermore, the clustering method is not limited to text mining; the learning unit 114 may cluster multiple neural networks by performing analytical processing such as logistic regression analysis. Alternatively, the learning unit 114 may manually assign clustering results to some of the multiple neural networks shown in Figure 10 and then use those results to cluster the remaining neural networks. This approach makes it possible to improve the accuracy of the clustering process.

[0243] In the example shown in Figure 27, among the multiple neural networks (NNs) held by the server system 100, NN1 to NN3 were classified as Class 1, NN4 to NN7 as Class 2, and NN8 to NN10 as Class 3. The learning unit 114 also determines which class a network belongs to by similarly calculating n-dimensional features based on the names of the custom support information. For example, the learning unit 114 extracts words such as "XXXX" and "timing" from the name of the custom support information, "timing for doing XXXX," and calculates n-dimensional features corresponding to the custom support information based on the extraction results. The learning unit 114 determines the structure of the NN to be used for training based on the clustering results of the custom support information.

[0244] For example, as shown in Figure 27, suppose the custom support information is classified as Class 1. In this case, the learning unit 114 selects one of NN1 to NN3 and creates a NN for the custom support information using the structure of the selected NN and the training data for the custom support information described above. At this time, only the structure of the original NN may be used, and all the weights may be newly calculated. Alternatively, transfer learning may be performed using some of the weights of the original NN as they are. For example, the learning unit 114 performs machine learning using the structures of NN1 to NN3 and the training data for the custom support information to find the accuracy of the trained model. Then, the learning unit 114 presents the highest accuracy to the caregiver, as in Figure 25D, and asks for input whether to apply it or not. If the caregiver responds with yes, the corresponding support information output NN 122 is stored in the storage unit 120, and the custom support information can be output.

[0245] Furthermore, when performing additional machine learning, the relationship between the training data accumulation period—in other words, the acquisition period of the data to be analyzed—and the ADL (Activities of Daily Living) of the person being cared for becomes important. For example, suppose a person being cared for who was previously able to act independently suffers a fracture due to a fall and requires assistance in a wheelchair. When ADL changes significantly in this way, the appropriate care for that person will differ greatly before and after the change. Therefore, for example, learning results based on training data from before the change in ADL may not be useful after the change in ADL.

[0246] Therefore, although not shown in Figure 25C, for example, when performing a learning initiation operation, it may be possible to input not only the type of input data but also the analysis period. The caregiver specifies a period during which the ADL of the person being cared for is considered to be at the same level as it is now. In this way, the NN122 for outputting support information, which is the learning result, will contain content corresponding to the current ADL of the person being cared for, thus enabling appropriate care support. Furthermore, it is assumed that the server system 100 collects the ADL of the person being cared for as one of the input data. Therefore, when a learning initiation operation is performed, the learning unit 114 may acquire the time-series changes in the ADL of the person being cared for and automatically set the analysis period based on the time-series changes in the ADL.

[0247] Although this embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel aspects and effects of this embodiment. Therefore, all such modifications are included within the scope of this disclosure. For example, any term that appears at least once in the specification or drawings together with a broader or synonymous term may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of this embodiment and its modifications are also included within the scope of this disclosure. In addition, the configuration and operation of information processing systems, server systems, mobile terminal devices, etc., are not limited to those described in this embodiment, and various modifications are possible.

[0248] [Additional Notes] The information processing device according to this embodiment includes: a factor determination unit that determines whether the behavior of the person being cared for is abnormal behavior due to dementia factors, based on (1) dementia level information of the person being cared for and (2) at least one of environmental information, excretion information, and sleep information of the person being cared for; and a support information output unit that outputs support information to support the caregiver's care of the person being cared for, based on the determination result of the factor determination unit and sensor information which is sensing result regarding the caregiver or the person being cared for. [Explanation of Symbols]

[0249] 10...Information processing system, 100...Server system, 110...Processing unit, 111...Factor determination unit, 112...Support information output unit, 113...Setting unit, 114...Learning unit, 120...Storage unit, 121...NN for factor determination, 122...NN for support information output, 123...First correspondence information, 124...Second correspondence information, 125...Third correspondence information, 130...Communication unit, 200...Assistant device, 210...Mobile terminal device 211…Processing unit, 212…Storage unit, 213…Communication unit, 214…Display unit, 215…Operation unit, 220…Wearable device, 300…Care device, 310…Care bed, 320…Lift, 400…Sensor group, 410…Motion sensor, 420…Imaging sensor, 430…Odor sensor, 440,450…Sensing device, NW…Network, OB1-OB5…Object, RE1-RE4…Area

Claims

1. When the behavior of the person being cared for is determined to be abnormal, a factor determination unit determines whether the abnormal behavior is a dementia factor and whether it is a cause of excretory dysfunction, based on (1) the dementia level information of the person being cared for and (2) at least one of the environmental information, excretion information, and sleep information of the person being cared for. A support information output unit outputs support information to support the caregiver's assistance of the person being assisted, based on the determination result of the factor determination unit and sensor information which is a sensing result relating to the caregiver or the person being assisted. Information processing device including

2. In claim 1, The information processing device includes information that supports at least one of the following: meal assistance for assisting the person being assisted with eating, excretion assistance for assisting the person being assisted with excretion, and transfer / mobility assistance for assisting the person being assisted with transferring or moving.

Images