Information Processing Apparatus and Information Processing Method

Abstract

To provide an information processing device and an information processing method that appropriately support assistance of a care recipient assisted by a caregiver.SOLUTION: In an information processing system in which a server system 100 is connected to a device for a caregiver, a nursing care device, and a group of sensors via a network, the information processing device (server system 100) includes: a factor determination unit that determines whether or not a conduct of a care recipient is caused by an abnormal conduct resulting from a dementia factor based on dementia level information of the care recipient, and at least one of environment information, excretion information, and sleep information of the care recipient; and a support information output unit that outputs support information for supporting assistance of the care recipient by the caregiver based on the determination result of the factor determination unit and sensor information that is a sensing result regarding the caregiver who assists the care recipient or the care recipient.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and the like.

Background Art

[0002] Conventionally, systems used in medical sites, nursing facilities, etc. are known. For example, Patent Document 1 discloses a method for instructing an assistance method for moving a care recipient.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided are an information processing apparatus, an information processing method, and the like that appropriately support the assistance of a care recipient by a caregiver.

Means for Solving the Problems

[0005] The information processing apparatus according to the present embodiment includes a factor determination unit that determines whether or not the behavior of the care recipient is abnormal behavior due to dementia factors based on (1) the dementia level information of the care recipient and (2) at least one of the environmental information, excretion information, and sleep information of the care recipient, and a support information output unit that outputs support information for supporting the assistance of the care recipient by the caregiver based on the determination result of the factor determination unit and the sensor information that is the sensing result regarding the caregiver or the care recipient who provides assistance to the care recipient.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24A

Figure 24B

Figure 25A

Figure 25B

Figure 25C

Figure 25D

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Modes for Carrying Out the Invention

[0007] Hereinafter, this embodiment will be described with reference to the drawings. For the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted. Note that the embodiment described below does not unduly limit the content described in the claims. Also, not all of the configurations described in this embodiment are essential constituent elements.

[0008] 1. Example of System Configuration FIG. 1 is a configuration example of an information processing system 10 including an information processing apparatus according to the present embodiment. The information processing system 10 according to the present embodiment digitizes the "intuition" and "tacit knowledge" in the work performed by caregivers based on the "intuition" and "tacit knowledge" of caregivers in, for example, a nursing facility, so as to give instructions to caregivers so that appropriate assistance can be provided regardless of the proficiency of the caregivers. The information processing system 10 shown in FIG. 1 includes a server system 100, an assistant device 200, a nursing device 300, and a sensor group 400. However, the configuration of the information processing system 10 is not limited to FIG. 1, and various modifications such as omitting a part and adding other configurations are possible. Also, the same applies to FIGS. 3 and 4 described later in terms of the possibility of modification such as omission and addition of the configuration.

[0009] The information processing apparatus of the present embodiment corresponds to, for example, the server system 100. However, the method of the present embodiment is not limited to this, and the processing of the information processing apparatus may be executed by distributed processing using the server system 100 and other devices. For example, the information processing apparatus of the present embodiment may include the server system 100 and the assistant device 200. Hereinafter, an example in which the information processing apparatus is the server system 100 will be described.

[0010] The server system 100 is connected to the assistant device 200, the nursing device 300, and the sensor group 400 via, for example, a network NW. The network NW here is, for example, a public communication network such as the Internet, but may be a LAN (Local Area Network) or the like. For example, the assistant device 200, the nursing device 300, and the sensor group 400 are arranged in a nursing facility or the like. The server system 100 performs processing based on the information from the sensor group 400, and based on the processing result, outputs information to the assistant device 200 and remotely controls the nursing device 300.

[0011] In FIG. 1, an example is shown in which each of the assistant device 200, the care device 300, and the sensor group 400 can communicate with the server system 100 via the network NW, but the present invention is not limited to this. For example, a relay device (not shown) may be provided in a care facility or the like. The relay device is a device that can communicate with the server system 100 via the network NW. Information output by the sensor group 400 may be aggregated by the relay device using a LAN within the care facility, and the relay device may transmit the information to the server system 100. Further, information from the server system 100 may be transmitted to the relay device, and the relay device may transmit necessary information to the assistant device 200 or the care device 300. For example, in a care facility, it is assumed that a plurality of assistant devices 200 and a plurality of care devices 300 are used simultaneously. The relay device may perform a process of selecting the assistant device 200 or the care device 300 that is the transmission target of the information from the server system 100. Alternatively, the relay device may be an administrator terminal used by an administrator of the care facility and may operate based on an operation input of the administrator. For example, information from the server system 100 may be displayed on a display unit of the relay device, and the administrator who has viewed the display result may select the destination assistant device 200 or care device 300. As described above, the information processing device of the present embodiment can be variously modified, and for example, the above-described relay device may be included in the information processing device.

[0012] The server system 100 may be one server or may include a plurality of servers. For example, the server system 100 may include a database server and an application server. The database server stores various data to be described later with reference to FIG. 3. The application server performs processes to be described later with reference to FIGS. 7, 8, 18 to 23, etc. Here, the plurality of servers may be physical servers or virtual servers. When virtual servers are used, the virtual servers may be provided on one physical server or may be distributed and arranged on a plurality of physical servers. As described above, the specific configuration of the server system 100 in the present embodiment can be variously modified.

[0013] The caregiver device 200 is a device used by a caregiver who provides assistance to a care recipient (patient, resident) in a care facility or the like, and is a device used for presenting information to the caregiver or for inputting information by the caregiver. For example, the caregiver device 200 may be a device carried or worn by the caregiver. For example, the caregiver device 200 includes a portable terminal device 210 and a wearable device 220. The portable terminal device 210 is, for example, a smartphone, but may be other portable devices. The wearable device 220 is a device that can be worn by the caregiver, and includes, for example, earphones or headphones and a headset including a microphone. The wearable device 220 may be a glasses-type device, a wristwatch-type device, or a device of other shapes. The caregiver device 200 may also be other devices such as a PC (Personal Computer).

[0014] The care device 300 is a device used for providing care (including assistance) to a care recipient in a care facility or the like. While the caregiver device 200 is mainly a device for presenting information to the caregiver, the care device 300 is a device for directly performing the assistance of the care recipient. For example, the care device 300 may include a care bed 310 whose bottom (the bottom may be plate-shaped or mesh-shaped and the shape is not limited) can change the angle and height, and a lift 320 for transferring the care recipient from the care bed 310 to a wheelchair. The care device 300 may also include other devices such as a wheelchair, a walker, a rehabilitation device, and a food cart for serving meals.

[0015] Figure 2A shows an example of the care bed 310. The care bed 310 can change the height and angle of a plurality of bottoms respectively. Thereby, it is possible to flexibly change the posture of the care recipient lying on the care bed 310. Figure 2B shows an example of the lift 320. The lift 320 is a device used, for example, for transferring a care recipient with a low ADL (Activity of Daily Living) evaluation index and difficult to transfer manually.

[0016] The sensor group 400 includes a plurality of sensors arranged in a nursing facility or the like. The sensor group 400 may include a motion sensor 410, an imaging sensor 420, and an odor sensor 430. The motion sensor 410 may be an acceleration sensor, a gyro sensor, or other sensor capable of detecting movement. The motion sensor 410 may be a sensor for detecting the movement of the care recipient or a sensor for detecting the movement of the caregiver. The imaging sensor 420 is a sensor that converts a subject image formed through a lens into an electrical signal. The odor sensor 430 is a sensor that detects and quantifies odors. The sensor group 400 can also include various sensors such as a temperature sensor, a humidity sensor, an illuminance sensor, a magnetic sensor, a position sensor, and an atmospheric pressure sensor.

[0017] In FIG. 1, the caregiver device 200, the care device 300, and the sensor group 400 are described separately. For example, the sensors included in the sensor group 400 may be arranged in rooms, dining halls, corridors, stairs, etc. within the nursing facility. For example, cameras including the imaging sensor 420 are arranged at various locations within the nursing facility. Also, a sensing device for sensing information necessary for care may be used. By providing sensors at various locations in the nursing facility, not only can the necessary information be sensed, but the positions of the sensors can also be specified.

[0018] For example, FIG. 2C shows an example of a sensing device 440 arranged on the mattress of the care bed 310. The sensing device 440 shown in FIG. 2C includes, for example, an odor sensor 430 and detects whether the care recipient has excreted. Note that the sensing device 430 may also be able to determine whether the care recipient is sick from body odor or breath. Also, FIG. 2D shows an example of a sensing device 450 arranged under the mattress on the care bed 310 (arranged between the care bed 310 and the mattress). The sensing device 450 shown in FIG. 2D includes, for example, a pressure sensor and can detect the heart rate, respiratory rate, and activity level of the care recipient. Note that the sensing device 450 may also be able to determine whether the care recipient is in a sleeping state or has left or returned to the bed.

[0019] However, the method of this embodiment is not limited to the above examples, and the sensors included in the sensor group 400 may be provided in the assistant device 200 or the care device 300. For example, as the sensors included in the sensor group 400, a camera, an acceleration sensor, a gyro sensor, a GPS (Global Positioning System) sensor, etc. included in the mobile terminal device 210 may be used. Also, the care device 300 may be provided with a motion sensor for detecting the posture of the care device 300, a camera for imaging the care recipient or the assistant using the care device 300, and the like.

[0020] FIG. 3 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit 130.

[0021] The processing unit 110 of this embodiment is configured by the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be configured by one or more circuit devices mounted on a circuit board and one or more circuit elements. The one or more circuit devices are, for example, IC (Integrated Circuit), FPGA (field-programmable gate array), etc. The one or more circuit elements are, for example, resistors, capacitors, etc.

[0022] The processing unit 110 may also be implemented by the following processors. The server system 100 of this embodiment includes a memory for storing information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various types of data, etc. The processor includes hardware. The processor can use various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), etc. The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, or it may be a register, or it may be a magnetic storage device such as a hard disk drive (HDD), or it may be an optical storage device such as an optical disk drive. For example, the memory stores instructions readable by a computer, and when the processor executes the instructions, the functions of the processing unit 110 are realized as processing. The instructions here may be instructions in the instruction set that constitutes a program, or may be instructions that instruct the hardware circuit of the processor to operate.

[0023] The processing unit 110 includes a cause determination unit 111, a support information output unit 112, a setting unit 113, and a learning unit 114.

[0024] Based on an input including at least the dementia level information of the assisted person, the cause determination unit 111 determines whether the behavior of the assisted person is abnormal behavior due to dementia factors. For example, the cause determination unit 111 determines whether the behavior of the assisted person is abnormal behavior due to dementia factors based on (1) the dementia level information of the assisted person and (2) at least one of the environmental information, excretion information, and sleep information of the assisted person. Details of each piece of information will be described later.

[0025] The support information output unit 112 outputs support information for assisting the care recipient by the caregiver based on the determination result output by the cause determination unit 111 and the sensor information that is the sensing result regarding the caregiver who cares for the care recipient or the care recipient. Details of the support information will be described later.

[0026] The setting unit 113 performs setting processing when using the information processing system 10 according to the present embodiment. For example, the caregiver who is the user of the information processing system 10 may be able to set which of a number of support information to output. In this case, as will be described later with reference to FIGS. 17 and 18, the setting unit 113 performs processing for accepting a setting operation by the caregiver, processing for updating setting information, and the like. Further, the setting unit 113 may execute setting processing for adding user-specific custom support information to the output. Specific examples will be described later with reference to FIGS. 25A to 25D and the like.

[0027] The learning unit 114 outputs a learned model by performing machine learning based on training data. The machine learning here is, for example, supervised learning. The training data in supervised learning is a data set in which input data corresponding to the input of the model and correct data representing appropriate output data when the input data is input are associated with each other. The learning unit 114 may generate a learned model, for example, by performing machine learning using a neural network. Hereinafter, the neural network will be denoted as NN. For example, the learning unit 114 performs processing for generating a cause determination NN 121 and a support information output NN 122. Details of the processing in the learning unit 114 will be described later. However, machine learning is not essential in the present embodiment, and the learning unit 114 can be omitted. Further, even when machine learning is performed, the learning process can be executed in a learning device different from the server system 100, and in this case, the learning unit 114 can also be omitted.

[0028] The storage unit 120 is a work area of the processing unit 110 and stores various types of information. The storage unit 120 can be realized by various memories, and the memory may be a semiconductor memory such as SRAM, DRAM, ROM, or flash memory, or may be a register, a magnetic storage device, or an optical storage device.

[0029] The storage unit 120 stores information used in the processing by the cause determination unit 111 and information used in the processing by the support information output unit 112. For example, the storage unit 120 may store the cause determination NN 121 and the support information output NN 122 obtained by machine learning using an NN. Here, the cause determination NN 121 and the support information output NN 122 include, in addition to the information defining the structure of the NN, parameters used in the operations using the structure. Specifically, the parameters are weights whose values are determined by machine learning.

[0030] The storage unit 120 may also store the first association information 123, the second association information 124, and the third association information 125. The first association information 123 is information that associates an assistant with information indicating whether or not to output each support information to the assistant. The second association information 124 is information that associates support information with sensor information necessary for the output of the support information. The third association information 125 is information that associates a given care facility with sensor information that can be obtained in the care facility. Specific examples of each association information will be described later with reference to FIGS. 14 to 16. Also, the storage unit 120 may store other information.

[0031] The communication unit 130 is an interface for performing communication via the network NW, and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. The communication unit 130 may operate according to the control by the processing unit 110, or may include a processor for communication control different from the processing unit 110. The communication unit 130 is an interface for performing communication according to, for example, TCP / IP (Transmission Control Protocol / Internet Protocol). However, various modifications of the specific communication method are possible.

[0032] FIG. 4 is an example of the assistant device 200, and is a block diagram showing a detailed configuration example of the portable terminal device 210. The portable terminal device 210 includes, for example, a processing unit 211, a storage unit 212, a communication unit 213, a display unit 214, and an operation unit 215.

[0033] The processing unit 211 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. Further, the processing unit 211 may be realized by a processor. The processor can use various processors such as a CPU, a GPU, and a DSP. The function of the processing unit 211 is realized as processing by the processor executing instructions stored in the memory of the portable terminal device 210.

[0034] The storage unit 212 is a work area of the processing unit 211 and is realized by various memories such as SRAM, DRAM, and ROM.

[0035] The communication unit 213 is an interface for performing communication via the network NW, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 213 communicates with the server system 100 via, for example, the network NW.

[0036] The display unit 214 is an interface for displaying various information, which may be a liquid crystal display, an organic EL display, or a display of other types. The operation unit 215 is an interface for receiving user operations. The operation unit 215 may be buttons or the like provided on the portable terminal device 210. Further, the display unit 214 and the operation unit 215 may be a touch panel configured integrally.

[0037] Also, the portable terminal device 210 may include components not shown in FIG. 4, such as a light emitting unit, a vibration unit, and a sound output unit. The light emitting unit is, for example, an LED (light emitting diode) and performs notification by light emission. The vibration unit is, for example, a motor and performs notification by vibration. The sound output unit is, for example, a speaker and performs notification by sound. Also, as described above, the portable terminal device 210 may include sensors included in the sensor group 400.

[0038] 2. Factor determination and support information output The information processing apparatus according to the present embodiment performs a process of determining factors of the behavior of the person to be assisted and a process of outputting support information for supporting the assistance of the person to be assisted by the assistant. By doing so, it becomes possible to cause the assistant to provide appropriate assistance according to the person to be assisted, taking into account factors such as dementia. Hereinafter, machine learning will be described as a specific example of a method for performing factor determination and support information output processing. However, the method of the present embodiment is not limited to using machine learning, and various modifications can be made. Also, hereinafter, an example of using NN as machine learning will be described, but other methods such as SVM (support vector machine) may be used as machine learning, or methods developed from NN or SVM may be used.

[0039] 2.1 Brief explanation of NN Figure 5 shows a basic structural example of an NN. One circle in Figure 5 is called a node or a neuron. In the example of Figure 5, the NN has an input layer, two or more intermediate layers, and an output layer. The input layer is I, the intermediate layers are H1 and Hn, and the output layer is O. Also, in the example of Figure 5, the number of nodes in the input layer is 2, the number of nodes in each intermediate layer is 5, and the number of nodes in the output layer is 1. However, the number of intermediate layers and the number of nodes included in each layer can be variously modified. Also, Figure 5 shows an example in which each node included in a given layer is connected to all the nodes included in the next layer, but this configuration can also be variously modified.

[0040] The input layer receives input values and outputs them to the intermediate layer H1. In the example of Figure 5, the input layer I receives two types of input values. Note that each node in the input layer may perform some processing on the input value and output the value after the processing.

[0041] In an NN, a weight is set between two connected nodes. W1 in Figure 5 is the weight between the input layer I and the first intermediate layer H1. W1 represents a set of weights between a given node included in the input layer and a given node included in the first intermediate layer. For example, W1 in Figure 5 is information including 10 weights.

[0042] At each node in the first intermediate layer H1, an operation is performed in which the output of the node in the input layer I connected to the node is weighted and added using the weight W1, and a bias is further added. Further, at each node, the output of the node is obtained by applying an activation function, which is a non-linear function, to the addition result. The activation function may be a ReLU function, a sigmoid function, or another function.

[0043] Also, the same applies to the subsequent layers. That is, in a given layer, the output to the next layer is obtained by weighted addition of the output of the previous layer using the weight W, adding a bias, and then applying an activation function. The NN uses the output of the output layer as the output of the NN.

[0044] As can be understood from the above description, in order to obtain desired output data from input data using an NN, it is necessary to set appropriate weights and biases. In learning, training data is prepared by associating given input data with correct answer data representing the correct output data for the input data. The learning process of the NN is a process of obtaining the most probable weights based on the training data. Note that in the learning process of the NN, various learning methods such as the backpropagation method are known. In the present embodiment, since these learning methods can be widely applied, detailed descriptions thereof are omitted.

[0045] Also, the NN is not limited to the configuration shown in FIG. 5. For example, as the NN, a convolutional neural network (CNN) may be used. The CNN has a convolutional layer and a pooling layer. The convolutional layer performs a convolutional operation. Specifically, the convolutional operation here is a filter process. The pooling layer performs a process of reducing the vertical and horizontal sizes of the data. In the CNN, by performing a learning process using the backpropagation method or the like, the characteristics of the filter used in the convolutional operation are learned. That is, the weights in the NN include the filter characteristics in the CNN. Also, as the NN, a network with another configuration such as an RNN (Recurrent neural network) may be used.

[0046] 2.2 Factor determination FIG. 6 is a diagram illustrating input data and output data of the factor determination NN 121 used for factor determination. The input data in factor determination includes, for example, dementia level information. The input data also includes at least one of environmental information, sleep information, and excretion information. FIG. 6 shows an example in which the input data includes all of environmental information, sleep information, and excretion information. The input data may also include other information. For example, as shown in FIG. 6, the input data may include medication information and dietary water information. Also, the configuration of the factor determination NN 121 is not limited to FIG. 6, and various modifications can be made.

[0047] Dementia level information is information representing the degree of progression of dementia in the care recipient. For example, the dementia level information may be the score of the MMSE (Mini-Mental State Examination), or the score of the Revised Hasegawa Dementia Scale (HDS-R), or other information representing the results of a dementia examination. Further, the dementia level information may be information based on a brain image obtained using CT (Computed Tomography) or MRI (magnetic resonance imaging). For example, the dementia level information may be information representing the result of a diagnosis made by a doctor based on a brain image, or the brain image itself, or the result of performing some kind of image processing on the brain image.

[0048] Environmental information is information representing the living environment of the care recipient. The environmental information includes temperature information representing the temperature of the living environment of the care recipient, humidity information representing the humidity, illuminance information representing the illuminance, and atmospheric pressure information representing the atmospheric pressure. For example, temperature sensors, humidity sensors, illuminance sensors, and atmospheric pressure sensors are arranged in places regularly used by the care recipient, such as the care recipient's bedroom or cafeteria, and temperature information, humidity information, illuminance information, and atmospheric pressure information are obtained based on the output of each sensor.

[0049] Further, the environmental information may include information related to sound. For example, a microphone is arranged in a living environment such as a bedroom, and the information collected by the microphone is used as environmental information. The environmental information may be information related to sound pressure, or information representing the result of frequency analysis. Further, the environmental information may include information related to the time when a specific sound is generated, etc.

[0050] The environmental information may also include information about the care bed 310 used by the care recipient. The information about the care bed 310 may be information for identifying the model of the care bed 310, or may be information such as the type and firmness of the mattress used in conjunction with the care bed 310. Further, the information about the care bed 310 may include information representing the drive results of the care bed 310. For example, information about the angle and height of the bottom of the care bed 310, or information such as the time when the care bed 310 was driven may be used as environmental information.

[0051] The sleep information is information representing the sleep state of the care recipient. For example, the sleep information may be detected using a sensing device 450 shown in FIG. 2D or the like. Further, the sleep information may be detected using a wristwatch-type device including a photoelectric sensor or the like for detecting the pulse rate. The sleep information includes, for example, information such as the sleep start time, wake-up time, daily sleep duration, sleep depth, number and time of mid-sleep awakenings, heart rate, respiratory rate, and activity level during sleep.

[0052] The excretion information includes information representing the excretion state of the care recipient. For example, the excretion information may be detected using a sensing device 440 shown in FIG. 2C or the like. The sensing device 440 outputs, for example, information on the presence or absence of excretion of the care recipient, the type of excretion, and the timing at which excretion is determined to have occurred, based on an odor sensor 430 or the like. The excretion information includes, for example, information such as the number of excretions, excretion intervals, and types of excretion within a given span. Further, the excretion information may also include information such as a captured image of the diaper after excretion and comments added by the caregiver.

[0053] The medication information is information for identifying the medications administered to the care recipient. For example, the medication information is information representing the name, dosage, administration time, etc. of the medications taken by the care recipient. Further, the medication information may also include information such as the information on the prescription issued to the care recipient.

[0054] Diet and fluid intake information represents information on the diet and fluids consumed by the care recipient. For example, diet and fluid intake information includes the time of eating, the menu, and the actual amount eaten. Further, diet and fluid intake information may include information for specifying ease of eating, such as the firmness and size of the food ingredients. Also, diet and fluid intake information includes the time of fluid intake, the type of fluid (such as water or tea), and the intake amount.

[0055] In the learning stage, training data for creating the factor determination NN121 is obtained by associating correct answer data with the above input data over a predetermined period. The predetermined period here may be a fixed period such as one day. Alternatively, the predetermined period may be a period set based on the occurrence time of some abnormal behavior of the care recipient when the care recipient performs some abnormal behavior.

[0056] Also, the correct answer data may be provided by an expert having specialized knowledge such as a doctor. When the care recipient exhibits abnormal behavior, the expert diagnoses the care recipient and identifies the factors of the abnormal behavior. The correct answer data here is information representing the identified factors. For example, the correct answer data indicates whether the behavior is due to dementia factors, environmental factors, sleep disorder factors, or excretion disorder factors. For example, when the result of associating the input data corresponding to one period of one care recipient with the correct answer data is taken as one data set, training data including a large number of data sets is obtained by increasing the number of care recipients and the target period.

[0057] The learning unit 114 of the server system 100 obtains training data for factor determination and creates the factor determination NN121 by performing machine learning based on the training data.

[0058] FIG. 7 is a flowchart for explaining the learning process of generating the factor determination NN121. When this process is started, first, in step S101, the learning unit 114 obtains input data for learning. The input data here is as described above and includes, for example, dementia level information, environmental information, sleep information, and excretion information. The input data may also include other information such as medication information and diet and fluid intake information.

[0059] Also in step S102, the learning unit 114 acquires correct answer data associated with the input data. For example, the learning unit 114 executes the processes of steps S101 and S102 by reading out any one data set among the training data acquired in the learning stage.

[0060] In step S103, the learning unit 114 performs a process of updating the weights of the NN. Specifically, the learning unit 114 inputs the input data acquired in step S101 to the factor determination NN 121, and obtains output data by performing a forward operation using the weights at that stage. The learning unit 114 obtains an objective function based on the output data and the correct answer data. The objective function here is, for example, an error function based on the difference between the output data and the correct answer data, or a cross-entropy function based on the distribution of the output data and the distribution of the correct answer data.

[0061] For example, when the output layer of the factor determination NN 121 is a known softmax layer, the output of the output layer is probability data whose sum is 1. For example, the output layer includes four nodes, a first node to a fourth node. The output value of the first node represents "the probability that the behavior of the assisted person is a dementia factor". The output value of the second node represents "the probability that the behavior of the assisted person is an environmental factor". The output value of the third node represents "the probability that the behavior of the assisted person is a sleep disorder factor". The output value of the fourth node represents "the probability that the behavior of the assisted person is an excretion disorder factor". The correct answer data is data in which the value of the correct factor is 1 and the other values are 0. For example, when an expert determines it to be a dementia factor, data in which the probability of the dementia factor is 1 and the other three probabilities are 0 is used as the correct answer data.

[0062] The learning unit 114 updates the weights so that, for example, the error function decreases. As weight update methods, the error backpropagation method and the like described above are known, and these methods can also be widely applied in this embodiment.

[0063] In step S104, the learning unit 114 determines whether to end the learning process. For example, the plurality of data sets included in the training data may be divided into learning data and validation data. The learning unit 114 may end the learning process when the process of updating the weights using all the learning data has been performed, or may end the learning process when the correct answer rate based on the validation data exceeds a given threshold.

[0064] If the learning process is not ended, the learning unit 114 returns to step S101 to continue the process. That is, the learning unit 114 reads a new data set from the training data and performs a process of updating the weights based on the data set.

[0065] If the learning process is ended, the learning unit 114 stores the factor determination NN 121 at that stage as a learned model in the storage unit 120. Note that FIG. 7 is an example of the learning process, and the method of the present embodiment is not limited thereto. For example, in machine learning, methods such as batch learning are also widely known, and these methods can be widely applied in the present embodiment.

[0066] FIG. 8 is a flowchart for explaining the process of the factor determination unit 111 in the inference stage. When this process is started, first, in step S201, the factor determination unit 111 determines whether the assisted person has performed an abnormal behavior suspected of dementia. Note that the factor determination unit 111 may automatically determine whether the behavior of the assisted person is an abnormal behavior based on sensor information or the like regarding the assisted person. For example, the sensor group 400 includes a motion sensor 410, an imaging sensor 420, a microphone, etc., and the factor determination unit 111 determines the presence or absence of an abnormal behavior by detecting the movement and voice of the assisted person. Alternatively, the caregiver himself / herself may observe the movement of the assisted person and input the observation result using the caregiver device 200 or the like. In this case, the factor determination unit 111 executes the process of step S201 based on the input of the caregiver. If it is determined that the assisted person is not performing an abnormal behavior, the factor determination unit 111 ends the process without performing steps S202 and subsequent steps.

[0067] When it is determined that the assisted person has performed abnormal behavior, in step S202, the factor determination unit 111 acquires input data regarding the assisted person. For example, the storage unit 120 acquires and stores, via the communication unit 130, dementia level information regarding the assisted person, sensor information collected by the sensor group 400, and the like. The dementia level information may be acquired, for example, at a nursing facility or the like and transmitted from the device of the nursing facility or the like to the server system 100. The factor determination unit 111 performs a process of reading, as input data, data regarding the target assisted person among the collected data, such as dementia level information, environmental information, sleep information, and excretion information corresponding to a predetermined period.

[0068] In step S203, the factor determination unit 111 reads the factor determination NN 121 from the storage unit 120. Then, the input data acquired in step S202 is input to the factor determination NN 121, and output data is obtained by performing a forward calculation. The output data of the factor determination NN 121 is, for example, as described above, four probability values representing the likelihood of each factor. The factor determination unit 111 determines, for example, the factor with the maximum probability value as the factor of the abnormal behavior of the assisted person. For example, when the value representing the likelihood of the dementia factor is greater than the likelihoods of the other three factors, the factor determination unit 111 determines that the abnormal behavior is due to the dementia factor. Further, the output of the factor determination unit 111 is not limited to this, and may be the four probability values themselves, or a value calculated based on them.

[0069] The factor determination unit 111 periodically executes the process shown in FIG. 8, for example. The frequency of the process is arbitrary, but may be, for example, about once a day. In this way, it is possible to periodically determine the presence or absence of abnormal behavior of the assisted person and the factors when there is abnormal behavior. For example, the factor determination unit 111 may execute the process of FIG. 8 every morning and determine the assistance policy for that day based on the process result. Also, when abnormal behavior is recognized in the assisted person, various modifications can be made to the process of the factor determination unit 111, such as executing the process of FIG. 8 without waiting for the next processing timing.

[0070] In addition, as a process separate from the process using the factor determination NN 121, an example of determining whether or not the behavior of the person requiring assistance is abnormal has been described (see step S201 in FIG. 8). However, an NN for making a determination including whether or not the behavior is abnormal may be created.

[0071] For example, in addition to the input data shown in FIG. 6, sensor information or the like representing the behavior of the person requiring assistance may be input to the factor determination NN 121. The factor determination NN 121 may include a node that outputs the probability that there is no abnormality in the behavior of the person requiring assistance, in addition to the nodes that output the probabilities of the four factors shown in FIG. 6. In the learning stage, training data is created using data indicating the absence of abnormal behavior in addition to data indicating the presence of abnormal behavior. Specifically, the correct data associated with the input data includes data representing "no abnormal behavior". In this case, the factor determination unit 111 can estimate the presence or absence of abnormal behavior and the factors in the case of abnormal behavior by inputting the input data to the factor determination NN 121.

[0072] 2.3 Assistance Support 2.3.1 Input and Output FIG. 9 is a diagram illustrating schematic input data of the support information output NN 122 used for outputting support information. As shown in FIG. 9, the input data may include sensor information. The sensor information includes information sensing the person requiring assistance or information sensing the assistant. The sensor information is output from sensors included in the sensor group 400, for example.

[0073] In addition, the sensor information may include information sensing the living environment of the person requiring assistance. In this case, the sensor information corresponds to the environmental information described above, for example. For example, the sensor information may include outputs of a temperature sensor, a humidity sensor, an illuminance sensor, a barometric pressure sensor, a microphone, and the like.

[0074] The input data may also include the attribute data of the person requiring assistance and physical assessment data representing a physical assessment. The attribute data of the person requiring assistance includes information such as the age, gender, height, weight, medical history, and medication history of the person requiring assistance. The physical assessment data includes information such as the ADL assessment value, rehabilitation history, fall risk, and pressure ulcer risk.

[0075] The input data may also include the attribute data of the caregiver and data related to the care facility. The attribute data of the caregiver includes the age, gender, height, weight, caregiving experience, and qualifications held by the caregiver. The information related to the care facility includes information such as the care schedule at the care facility, the number and usage status of the care devices 300, and statistical data on the number of persons requiring assistance and the degree of care required.

[0076] Figures 28 to 42 are diagrams illustrating the details of the data used as input when supporting the assistance of a person requiring assistance by a caregiver in the present embodiment, and more specifically, are diagrams showing examples of the input data of the NN 122 for outputting support information. As shown in Figures 28 to 42, various types of information can be used for the input data in the present embodiment. Note that it is not essential that all of the input data shown in Figures 28 to 42 be acquired, and some information may be omitted. Also, other information (not shown) may be added to Figures 28 to 42.

[0077] The output data of the NN 122 for outputting support information is information used to support the execution of each caregiving action when the assistance of a person requiring assistance by a caregiver is subdivided into a plurality of caregiving actions. For example, the output data of the NN 122 for outputting support information is support information for determining the start timing of caregiving, the movements and vocalizations during caregiving, the types and amounts of items to be provided to the person requiring assistance, and the like.

[0078] Figures 43 to 45 are diagrams illustrating the details of the data used when supporting the assistance of a person requiring assistance by a caregiver in the present embodiment, and more specifically, are diagrams showing examples of the support information, which is the output data of the NN 122 for outputting support information.

[0079] Figure 43 shows an example of support information output in meal assistance for assisting the meal of a care recipient. For example, in meal assistance, the caregiver grasps the characteristics of the care recipient and explains them clearly to the care recipient himself / herself, so as to smooth the execution of the meal. For example, in the case of a care recipient with the characteristic of low chewing ability, if the caregiver grasps this, it is possible to take measures to prevent aspiration, and it is also useful to guide the care recipient to say, "The rice has been softened, so chew it well." The output data of Number 1 in Figure 43 is support information for "conveying the characteristics of the user" to the caregiver, which may be data representing the characteristics of the care recipient itself, or information converted to be easily understood by the caregiver. Also, as described above, the caregiver may convey the characteristics of the care recipient to the care recipient himself / herself, and the output data of Number 1 in Figure 43 may include data for this purpose. The same applies to Number 2 and subsequent numbers. The output data shown in Figure 43 includes information for supporting various actions of the caregiver in meal assistance.

[0080] Figure 44 shows an example of support information output in toileting assistance for assisting the excretion of a care recipient. Note that toileting assistance may be performed in the toilet or using diapers. Numbers 66 - 72 represent the output data when performing toileting assistance in the toilet, and Numbers 73 - 75 represent the output data when performing toileting assistance using diapers.

[0081] Figure 45 shows an example of support information output in transfer assistance and mobility assistance for assisting the transfer or movement of a care recipient. Note that transfer and mobility assistance vary in the presence or type of equipment depending on the condition of the care recipient and the availability of equipment such as lifts. In the example of Figure 45, Numbers 92 - 103 represent the output data when performing assistance using a wheelchair, Numbers 104 - 107 represent the output data when performing assistance using a cane, and Numbers 108 - 112 represent the output data when performing assistance using a lift.

[0082] As shown in FIGS. 43 to 45, the support information may include information that supports at least one of meal assistance, excretion assistance, and transfer / movement assistance. By doing so, it becomes possible to appropriately support highly necessary assistance in a nursing facility or the like. For example, by supporting meal assistance, it becomes possible to suppress incidents such as aspiration and improve the nutritional status of the care recipient. By supporting excretion assistance, it becomes possible to suppress excretion leakage, reduce the man-hours required for dealing with excretion leakage and the risks occurring at that time, reduce excretion disorders, and suppress the risk of falls. Also, by supporting transfer / movement assistance, it becomes possible to reduce the risk of falls and make prior preparations for necessary caregivers, etc.

[0083] 2.3.2 Configuration Example of NN for Support Information Output FIGS. 10 to 12 are diagrams showing more specific configuration examples of the NN 122 for support information output shown in FIG. 9. As shown in FIG. 10, the NN 122 for support information output may be a set of a plurality of NNs each of which outputs one piece of support information. The support information 1 in FIG. 10 corresponds to any one of the support information shown in FIGS. 43 to 45. The input data group 1 represents one or a plurality of input data necessary for the output of the support information 1 among the plurality of input data shown in FIGS. 28 to 42. The same applies to the support information 2 and subsequent ones.

[0084] Also, as shown in FIG. 11, the NN 122 for support information output may be a set of a plurality of NNs that can output a plurality of related output data together. In the example of FIG. 11, the NN 122 for support information output includes a NN for meal assistance support information output, a NN for excretion assistance support information output, and a NN for transfer assistance support information output.

[0085] For example, the NN for outputting meal assistance support information outputs a plurality of meal assistance support information. The output data of the NN for outputting meal assistance support information corresponds to the plurality of support information shown in FIG. 43. The input data of the NN for outputting meal assistance support information represents a plurality of input data necessary for outputting meal assistance support information among the plurality of input data shown in FIGS. 28 to 42. The output of the NN for outputting excretion assistance support information corresponds to the plurality of support information shown in FIG. 44. The output of the NN for outputting transfer assistance support information corresponds to the plurality of support information shown in FIG. 45.

[0086] Also, as shown in FIG. 12, the NN 122 for outputting support information may be one NN. The input data in the NN of FIG. 12 is a set of all the data shown in FIGS. 28 to 42, and the output data is a set of all the support information shown in FIGS. 43 to 45.

[0087] Also, the configuration of the NN 122 for outputting support information is not limited to FIGS. 10 to 12. For example, by dividing the NN for outputting meal assistance support information into several parts, an intermediate configuration between FIGS. 10 and 11 may be used. In addition, various modifications of the specific configuration of the NN 122 for outputting support information are possible.

[0088] FIG. 13 is a diagram showing an example of the relationship between the NN 121 for factor determination and the NN 122 for outputting support information. The input data for factor determination in FIG. 13 is the input in FIG. 6 and includes dementia level information and the like. Also, the input data of the support information output 122 is the input in FIG. 9, specifically, the data shown in FIGS. 28 to 42. Note that a part of the input data for factor determination and the input data for support information output may overlap.

[0089] In the example shown in FIG. 13, the output data of the factor determination NN 121 is used as part of the input data of the support information output NN 122. The output data of the factor determination NN 121 may be information that identifies one factor that is the determination result as described above, or may be information based on a plurality of probability values. As shown in FIG. 10 or FIG. 11, when the support information output NN 122 includes a plurality of NNs, the output data of the factor determination NN 121 may be input to all the NNs or may be input to some of the NNs. In this way, it becomes possible to output support information based on the result of factor determination in the factor determination unit 111. As a result, it becomes possible to make the caregiver understand the degree of progression of the dementia of the care recipient and to perform various types of care according to the degree of progression.

[0090] Note that when no abnormal behavior is observed in the care recipient, the output of the factor determination NN 121 may be treated as 0. Also, as described above, the factor determination NN 121 may be able to output information indicating "no abnormal behavior".

[0091] However, in the method of this embodiment, it is only necessary that the determination result by the factor determination unit 111 be used for outputting support information, and the specific method is not limited to the example in FIG. 13.

[0092] 2.3.3 Learning Process and Inference Process The flow of the learning process of the support information output NN 122 in the learning unit 114 is the same as the case of creating the factor determination NN 121. The training data for creating the support information output NN 122 includes a data set in which correct answer data representing the assistance result performed by a skilled caregiver using tacit knowledge is associated with the input data.

[0093] For example, when providing meal assistance to the care recipient, at least the sensors necessary for meal assistance among the sensor group 400 are turned on. As a result, among the input data shown in FIGS. 28 to 42, data related to meal assistance is acquired by the sensor group 400 and stored in the storage unit 120 of the server system 100. Also, data representing the assistance results of the caregiver, such as the posture (corresponding to Numbers 9-12 in FIG. 43, etc.) of the caregiver making the care recipient take, the timing of providing meals with a spoon (corresponding to Number 26 in FIG. 43), and the amount provided per mouthful (corresponding to Number 25 in FIG. 43), is stored in the storage unit 120 as correct answer data.

[0094] The learning unit 114 inputs the input data among the training data into the NN 122 for support information output, and performs a forward calculation using the weights at that time to obtain output data. Also, the learning unit 114 obtains an objective function (an error function such as a mean squared error function, etc.) based on the output data and the correct answer data, and updates the weights so as to reduce the error using the error backpropagation method or the like. The NN 122 for support information output at the end of learning is stored in the storage unit 120 as a learned model.

[0095] As shown in FIG. 13, the output of the NN 121 for factor determination may be included in the input of the NN 122 for support information output. In this case, the input data among the training data includes information representing the factors of the abnormal behavior of the care recipient. For example, as described in the learning process of the NN 121 for factor determination, the correct answer data given by an expert such as a doctor may be used as one of the input data in the training data. Alternatively, if the learning of the NN 121 for factor determination has been completed first, as shown in FIG. 8, an inference process using the NN 121 for factor determination may be performed, and the result may be used as one of the input data in the training data.

[0096] The correct data is information representing the assistance results obtained by a skilled assistant using tacit knowledge, similar to the example described above. A skilled assistant can naturally provide assistance suitable for the assisted person considering factors such as the degree of progression of the dementia of the assisted person. That is, by using the assistance results of a skilled assistant as the correct data, it is possible to machine-learn appropriate assistance according to the factors of abnormal behavior. The processing after the training data is acquired is the same in this case. That is, the learning unit 114 performs forward calculations using the input data among the training data, obtains an error function from the output data and the correct data, and updates the weights so as to minimize the error.

[0097] The support information output unit 112 of the server system 100 acquires the input data shown in FIGS. 28 to 42 at the inference stage. Here, the input data only needs to include data capable of outputting the desired support information, and it is not essential to acquire all the input data in FIGS. 28 to 42. Also, the support information output unit 112 acquires the determination result of the factor determination unit 111 as one of the input data. The support information output unit 112 reads out the learned support information output NN 122 from the storage unit 120 and inputs the input data to the support information output NN 122. In the case of using an NN capable of outputting a plurality of support information as shown in FIGS. 11 and 12, and only a part of the support information is required to be output, there is a possibility that a part of the input data has not been acquired. In this case, the support information output unit 112 may, for example, set the value of the unacquired input data to 0. The support information output unit 112 obtains support information as output data by performing forward calculations.

[0098] 3. Flow of Processing Next, the specific flow of processing when assisting the assisted person by an assistant in an assistance facility or the like will be described.

[0099] 3.1 Estimation of Behavior Factors First, apart from the process of starting and executing a specific assistance sequence, the server system 100 determines whether there is any abnormal behavior in the person to be assisted, and if so, what the cause of the abnormal behavior is.

[0100] For example, the cause determination unit 111 periodically performs the above-described process using FIG. 8. In this way, for each of a plurality of persons to be assisted, the presence or absence of abnormal behavior and the cause of the abnormal behavior are determined. Hereinafter, the description will be made on the assumption that the result of the cause determination by the cause determination unit 111 has been obtained.

[0101] 3.2 Assistance Support 3.2.1 User Settings Examples of the support information in this embodiment are as shown in FIGS. 43 to 45. The support information output unit 112 may output all of this support information. However, if the amount of information to be notified is too large, a less experienced caregiver may not be able to fully understand the content or may not be able to recognize the difference in importance for each process. Also, a caregiver with a certain degree of experience may find the notification of support information bothersome because they can appropriately execute the given assistance even without support. Therefore, in this embodiment, the user, who is the caregiver, may be able to set the support information to be output.

[0102] The storage unit 120 of the server system 100 may store the first association information 123. FIG. 14 is a specific example of the first association information. As shown in FIG. 14, the first association information 123 is information that associates a caregiver ID for identifying a caregiver, support information, and information representing the output setting of the support information.

[0103] The output setting includes active and inactive. When a given support information is set to active, the support information output unit 112 outputs the support information to the target caregiver. When a given support information is set to inactive, the support information output unit 112 does not output the support information to the target caregiver. In this way, it becomes possible to flexibly set the output support information for each caregiver.

[0104] However, as shown in FIGS. 28 to 42, since there are a very large number of types of input data assumed in this embodiment, it is not always possible for a nursing facility to acquire all the input data. For example, due to constraints such as budget and the structure of the nursing facility, there may be cases where sensors necessary to acquire a given input data cannot be arranged. In this case, there is a possibility that a given support information cannot be obtained with sufficient accuracy due to the lack of input data.

[0105] Therefore, the output setting of the support information may include non-outputtable in addition to active / inactive. Non-outputtable represents a setting in which the support information is not output because the necessary input data cannot be acquired. Inactive is different from non-outputtable because it represents a setting in which the necessary input data can be acquired but the support information is deliberately not output.

[0106] For example, the storage unit 120 of the server system 100 may store the second association information 124 and the third association information 125. FIG. 15 shows a specific example of the second association information 124. As shown in FIG. 15, the second association information 124 includes support information and an essential input data group essential for output of the support information. The support information is any one of a plurality of data shown in FIGS. 43 to 45. The essential input data group is one or more of the data shown in FIGS. 28 to 42. The essential input data group may be, for example, data specified by a user. Alternatively, for a plurality of candidate input data groups, support information output NNs 122 are created respectively, and the candidate input data group with the highest correct answer rate using the validation data may be selected as the essential input data group. Also, the essential input data group is not limited to one set, and a plurality of candidate input data groups with a correct answer rate equal to or higher than a predetermined threshold may be used as the essential input data group.

[0107] FIG. 16 shows a specific example of the third association information 125. The third association information 125 is information associating a nursing facility with input data that can be acquired at the nursing facility. For example, a person in charge of a nursing facility may select input data that can be acquired at the nursing facility and transmit the selection result to the server system 100. Alternatively, the sensor group 400 arranged in the nursing facility associates information specifying the nursing facility with sensor information and transmits it to the server system 100. The processing unit 110 of the server system 100 may create the third association information 125 based on the acquisition history of the sensor information.

[0108] The setting unit 113 of the server system 100 determines whether each support information can be output for each nursing facility based on the second association information 124 and the third association information 125. Specifically, the setting unit 113 determines whether the support information can be output depending on whether the essential input data group necessary for output of the support information is included in the input data group that can be acquired at the target nursing facility.

[0109] In addition, there is a correspondence relationship between the input data and the sensor used to acquire the input data. Therefore, the storage unit 120 may store fourth association information associating the input data with one or more sensors used to acquire the input data. By using the fourth association information in addition to the second association information 124 and the third association information 125, it becomes possible to determine whether support information can be output for each sensor. Alternatively, instead of separately providing the fourth association information, the input data of the second association information 124 and the third association information 125 may be replaced with the information of the sensor.

[0110] Also, the device including a given sensor is not limited to one. For example, when a motion sensor 410 and an imaging sensor 420 are required, a device such as a smartphone including both a camera and an acceleration sensor may be used, or two separate devices may be used. Also, among cameras, a plurality of models with different resolutions, magnifications, etc. can be used. Therefore, the storage unit 120 may store fifth association information associating the sensor with the device including the sensor. In this case, it becomes possible to manage data in units of devices. For example, if the device already introduced by the care facility side is specified, in the server system 100, the sensors included in the device and the input data that can be acquired using the sensors are specified. Since it is not necessary for the person in charge or the caregiver of the care facility to know the sensors included in the device and the input data that can be acquired by the device, it is possible to improve the convenience of the user.

[0111] So far, an example of determining whether support information can be output has been described based on a care facility as a unit (see, for example, FIG. 16). However, the method of this embodiment is not limited to this. For example, when one care facility has a first space for residents with a high degree of care need and a second space for residents with a low degree of care need, it is conceivable that a large number of sensors are arranged in the first space and few sensors are arranged in the second space. In this case, the server system 100 may separately manage the support information that can be output in the first space and the support information that can be output in the second space. In addition, various modifications are possible for specific methods such as managing whether support information can be output for each care recipient.

[0112] FIG. 17 is an example of a setting screen for setting support information to be output. The processing described below is realized, for example, by the storage unit 212 of the mobile terminal device 210 storing a web application program that communicates with the server system 100 and the processing unit 211 operating according to the web application program. For example, the display of the display screen and the reception of user operations are performed using the display unit 214 and the operation unit 215 according to the web application program. Also, the generation and update of the display screen, database control according to user operations, etc. are executed by the setting unit 113 of the server system 100. However, the method of this embodiment is not limited to using a web application program, and various modifications such as using a so-called native application are possible. Also, in FIG. 17, an example is shown in which the setting screen is displayed on the display unit 214 of the mobile terminal device 210, but the setting screen may be displayed on other caregiver devices 200.

[0113] For example, the setting screen is a screen on which active, inactive, and non-outputtable can be selected for each of a plurality of support information. FIG. 17 illustrates a setting screen including objects OB1 to OB3 corresponding to three pieces of support information: "Diaper change timing", which is support information for excretory assistance, "Amount provided with a spoon", and "Timing of providing food with a spoon", which are support information for meal assistance.

[0114] For example, when the corresponding support information is active, the object is displayed in the first mode. When the corresponding support information is inactive, the object is displayed in the second mode. When the corresponding support information cannot be output, the object is displayed in the third mode. Here, the display mode may be controlled using the size, shape, and color of the object, or may be controlled using the size, font, color, etc. of the text included in the object. In addition, various modifications of the specific display mode are possible.

[0115] In FIG. 17, objects OB1 to OB3 are buttons, and an example is shown in which the color of the button differs depending on whether it is active / inactive / unable to output. For example, "Diaper change timing" is active, "Amount provided with a spoon" is unable to be output, and "Meal provision timing with a spoon" is inactive. In this case, the support information output unit 112 outputs the support information representing "Diaper change timing" and does not output the support information representing "Meal provision timing with a spoon". Also, in the target care facility, it may be difficult to accurately obtain "Amount provided with a spoon" due to a shortage of sensors, so the output of "Amount provided with a spoon" is not allowed. By displaying objects OB1 to OB3 in different modes, it is possible to clearly present the current settings to the caregiver.

[0116] By the caregiver operating the operation unit 215 of the mobile terminal device 210, it is possible to switch between active / inactive. For example, when the caregiver performs an operation to select "Diaper change timing", information indicating that is transmitted to the server system 100. The setting unit 113 performs a process of updating the output setting corresponding to "Diaper change timing" of the target caregiver ID among the first association information 123 to inactive. Also, the setting unit 113 generates a display screen in which the corresponding object OB1 is displayed in the second mode corresponding to inactive, and transmits it to the mobile terminal device 210 via the communication unit 130. The display unit 214 displays the display screen.

[0117] Similarly, when a selection operation is performed on an object corresponding to non-active support information, the setting unit 113 actively updates the output settings corresponding to the target assistant and support information. Further, the display unit 214 changes the display mode of the selected object to the first mode.

[0118] On the other hand, even if a selection operation is performed on an object corresponding to support information that cannot be output, the display unit 214 maintains the display in the third mode indicating non-output. In this case, the setting unit 113 does not perform the update process of the first association information 123.

[0119] Also, when a selection operation is performed on an object corresponding to support information that cannot be output, a suggestion of input data required for output of the support information may be made. For example, the server system 100 may identify the required input data based on the second association information 124 and perform a process of displaying the input data on the display unit 214 of the portable terminal device 210. Also, as described above, the input data here may be replaced with a sensor or a device. For example, the setting unit 113 may identify a sensor or a device required for output of the support information selected by the user and perform a process of displaying the sensor or the device on the display unit 214 of the portable terminal device 210.

[0120] FIG. 18 is a flowchart for explaining the above setting process. First, the assistant uses his / her own assistant device 200 to execute a setting change operation. In step S301, the setting unit 113 of the server system 100 receives the setting change operation via the network NW.

[0121] In step S302, the setting unit 113 performs a process of displaying a setting screen on the assistant device 200 based on the first association information 123 at that time and the assistant ID representing the assistant who performed the setting change operation. The process of step S302 may be a process of creating an image corresponding to the setting screen and transmitting the image to the assistant device 200, or a process of transmitting information for generating the setting screen to the assistant device 200. The information for generating the setting screen may be an extraction result obtained by extracting a part of the data corresponding to the assistant ID from the first association information 123. Further, the information for generating the setting screen may be a processed result obtained by performing some kind of processing on the extraction result. Thereby, for example, a screen corresponding to FIG. 17 is displayed on the display unit 214 of the mobile terminal device 210.

[0122] In step S303, the setting unit 113 determines a user operation executed on the assistant device 200. If an operation for changing the setting is not detected, the setting unit 113 ends the process.

[0123] Also, when an operation for deactivating the support information that was active or an operation for activating the support information that was inactive is performed, in step S304, the setting unit 113 reflects the setting change. Specifically, the setting unit 113 performs a process of updating the first association information 123 based on the information from the assistant device 200.

[0124] Also, when a selection operation for support information that cannot be output is performed, in step S305, the setting unit 113 identifies the input data, sensor, or device that is insufficient for outputting the support information. In step S306, the setting unit 113 performs a process of presenting the identified input data, sensor, or device to the assistant. The process of step S306 may be a process of transmitting the display image itself, similar to the process of step S302, or a process of transmitting the information used for generating the display image. Further, the presentation here is not limited to display, and a presentation process using voice or the like may be performed.

[0125] 3.2.2 Output Processing of Support Information FIG. 19 is a flowchart for explaining the output processing of support information by the support information output unit 112. In step S401, the support information output unit 112 acquires input data corresponding to the support information that is the output target. Specifically, the storage unit 120 of the server system 100 stores one or a plurality of input data for outputting the target support information among the plurality of input data shown in FIGS. 28 to 42. The association between the support information and the input data is performed using, for example, the second association information 124 described above.

[0126] In step S402, the support information output unit 112 obtains the support information by inputting the necessary input data to the support information output NN 122. In step S403, the support information output unit 112 determines whether notification based on the support information is necessary. If notification is necessary, in step S404, the support information output unit 112 performs a notification process. The notification may be by voice using an earphone of the headset, or may be a display using the display unit 214 of the mobile terminal device 210, or may be another notification. If notification is not required, or after performing the notification process, the support information output unit 112 ends the process.

[0127] As described above, the number of support information to be output can be changed according to the types of sensors provided in the nursing facility and the settings of the caregiver. However, in any case, for each of the support information to be output, the processing flow of specifying the input data, performing calculations by the NN, and performing notification as necessary shown in FIG. 19 is common.

[0128] If there is room in the processing performance of the server system 100, the support information output unit 112 may always perform the processing shown in FIG. 19 for all the support information set as the output target, and appropriately execute the notification process for those determined to require notification.

[0129] Also, considering the reduction of processing load, the process shown in FIG. 19 may be executed only for the support information required at that time. For example, as shown in FIGS. 43 to 45, the support information can classify the required situations, such as the support information required for meal assistance and the support information required for excretion assistance. Therefore, the support information output unit 112 may specify the support information required in the current situation and execute the process shown in FIG. 19 for the specified support information. For example, the support information output unit 112 may determine whether to start assistance for each of meal assistance, excretion assistance, and transfer / movement assistance. The support information output unit 112 performs the process shown in FIG. 19 for the support information related to the assistance determined to be started. The start determination will be described later with reference to FIG. 20.

[0130] Also, in the case of meal assistance, it is possible to classify in time series such as assistance performed before a meal, assistance performed during a meal, and assistance performed after a meal. Therefore, the support information output unit 112 can define the order in which the processes shown in FIG. 19 are executed among the plurality of support information. Also, depending on the assistance, there may be cases where restrictions exist on the execution order and necessity between assistances, such as the second assistance being required only when the first assistance has been performed.

[0131] Therefore, the support for assistance by the information processing system 10 of the present embodiment may be performed according to an assistance sequence combining a plurality of assistances. Specifically, the support information output unit 112 sequentially outputs a plurality of support information according to the assistance sequence, thereby supporting the assistance of the assisted person by the assistant.

[0132] Hereinafter, examples of the assistance sequence will be described for each of meal assistance, excretion assistance, and transfer / movement assistance. Specifically, first, the start determination of each assistance sequence is performed, and then the specific flow of each assistance sequence is described.

[0133] Note that, as will be described later with reference to FIGS. 21 to 23, in the following assistance sequences, for convenience of explanation, only some of the support information in FIGS. 43 to 45 is targeted for output. However, it is easily understood by those skilled in the art that in each of the assistance sequences described later, it is possible to perform modified implementations such as omitting the output of some support information or adding the output of other support information shown in FIGS. 43 to 45.

[0134] 3.2.3 Start determination The assistance in this embodiment may include meal assistance, excretion assistance, and transfer / movement assistance. However, these types of assistance do not need to be performed constantly. When the assisted person requires such assistance and there is an assistant who can perform the assistance, a specific assistance sequence is executed. That is, in this embodiment, first, the start determination of the assistance sequence is performed, and depending on the determination result, the start or standby of the assistance sequence may be determined.

[0135] FIG. 20 is a flowchart for explaining the start determination. This process is executed periodically, for example, for each assisted person. First, in step S501, the support information output unit 112 acquires at least a part of the input data shown in FIGS. 28 to 42. In step S502, the support information output unit 112 obtains support information by inputting the acquired input data to the support information output neural network 122. The support information here is information that identifies at least one of the start timing of meal assistance, the start timing of excretion assistance, and the start timing of transfer / movement assistance. For example, the support information output unit 112 may determine whether to start each type of assistance at the timing when the process of FIG. 20 is performed. Alternatively, the support information output unit 112 may output information that identifies a specific time, such as after how many minutes to start each type of assistance.

[0136] In step S503, the support information output unit 112 determines whether the current timing is the start timing of the assistance sequence. If it is determined that it is not the start timing, the support information output unit 112 ends the process and waits until the process shown in FIG. 20 is performed again.

[0137] For example, in addition to the information on the meal schedule in the care facility, the support information output unit 112 determines the start timing of the meal assistance sequence by using, as input data, the individual color, body temperature, weight, medication, past meal history, excretion history, rehabilitation history, etc. of the care recipient.

[0138] Regarding excretion assistance as well, for example, a case where a rough schedule such as five times a day is determined can be considered. Therefore, the support information output unit 112 determines the start timing of the excretion assistance sequence by using, as input data, in addition to the information on the excretion assistance schedule in the care facility, the amount and timing of the care recipient's meals, the amount and timing of water intake, the presence or absence of laxative administration, the past excretion history, rehabilitation records, the state of pressure ulcers, etc.

[0139] Also, the support information output unit 112 determines the start timing of the assistance sequence regarding transfer and movement assistance by using, as input data, the care recipient's ADL, medical history, etc. in addition to the occurrence of events that require the care recipient to move, such as meals and recreation.

[0140] When it is determined that the current timing is the start timing of the assistance sequence, in step S504, the support information output unit 112 performs a process of determining an assistant who will assist the target care recipient. For example, the support information output unit 112 may hold information such as the work shifts of the assistants in the care facility and the assignment of the care recipient, and determine the assistant based on this information.

[0141] In step S505, the support information output unit 112 performs a notification process of instructing the start of the assistance sequence to the assistant device 200 of the determined assistant. For example, the support information output unit 112 may perform a process of playing a voice such as "Please start assisting Mr. A with his meal" on a wearable device 220 such as a headset. Also, the support information output unit 112 may perform a process of displaying the same text on the display unit 214 of the mobile terminal device 210.

[0142] In step S506, the support information output unit 112 determines the response of the assistant to the above notification process. For example, as responses by the assistant, three types, namely, "OK", "Later", and "transfer", may be set. The response by the assistant may be made by voice. For example, based on the detection result by the microphone of the headset, the response of the assistant may be acquired. Further, the response of the assistant may be realized by other modes such as text input.

[0143] "OK" is a response indicating that it is possible to start the instructed assistance sequence. In this case, in step S507, the support information output unit 112 proceeds to a specific assistance sequence. For example, the support information output unit 112 starts the processes of FIGS. 21, 22, 23, etc.

[0144] "Later" is a response indicating that it is not possible to start the assistance sequence immediately, but it is considered that it will be possible to start after a predetermined time has elapsed. For example, it corresponds to a case where another task is currently being performed, but the instructed assistance sequence can be started after the completion of the task. In this case, in step S508, the support information output unit 112 waits for a predetermined time, and after the waiting, returns to step S505 and executes the notification process for the same assistant again.

[0145] "transfer" is a response indicating that it is difficult to execute the assistance sequence and requesting a request to another assistant. In this case, the support information output unit 112 returns to step S504 and selects another assistant. The same applies to the processes after step S505.

[0146] However, the process when "transfer" is selected is not limited to this. For example, when a given assistant selects "transfer", the support information output unit 112 may notify a plurality of assistants all at once. Then, among the plurality of assistants, the assistant who responded with "OK" may be selected, and a specific assistance sequence may be started for the selected assistant.

[0147] 3.2.4 Meal assistance Figure 21 is a flowchart for explaining a specific assistance sequence when providing meal assistance. First, when the meal assistance sequence starts, in step S601, the support information output unit 112 controls to turn on the sensors necessary for meal assistance support among the sensor group 400 arranged in a nursing facility or the like. In step S601, the support information output unit 112 may remotely control the on / off of the sensors included in the sensor group 400. Alternatively, the support information output unit 112 may instruct the devices in the nursing facility, such as the caregiver device 200, to turn on the sensors and devices to be turned on, and the caregiver may perform an operation to turn on the sensors according to the instruction. Hereinafter, although not explicitly shown in the flowchart, the sensor group 400 periodically transmits sensor information to the server system 100, and it is assumed that the support information output unit 112 can acquire the input data necessary for outputting support information.

[0148] In step S602, the support information output unit 112 outputs support information for providing a meal according to the care recipient based on the support information output NN122. For example, in step S602, the support information output unit 112 outputs support information for instructing a meal according to the care recipient's allergies and medication according to the medical condition.

[0149] Next, in step S603, the support information output unit 112 determines whether the care recipient and the caregiver have moved to the position where they will have a meal. The meal may be taken in the care recipient's living room or in a cafeteria or the like. The process of step S603 is performed by using, for example, information such as a camera or RFID (radio frequency identifier) that can identify the positions of the care recipient and the caregiver as input data. Note that in the process of step S603, for example, when the care recipient and the meal are imaged on the screen of the camera carried by the caregiver, the support information output unit 112 may determine that the care recipient and the caregiver have moved to the position where they will have a meal.

[0150] When at least one of the assisted person and the caregiver is not located at the position, in step S604, after waiting for a certain period of time, the support information output unit 112 performs the process of step S603 again.

[0151] When the assisted person and the caregiver are located at the position, in step S605, the support information output unit 112 obtains the minimum provision amount of the meal. The minimum provision amount here may be an amount less than the served amount. In other words, the caregiver does not have to make the assisted person eat all the served meal, and once the minimum provision amount is reached, it is not necessary to force-feed more. The process of step S605 is performed, for example, by using the color of the assisted person's face, care records, changes in weight, meal schedule, etc. as input data.

[0152] In step S606, the support information output unit 112 notifies the caregiver of the obtained minimum provision amount. The notification may be by voice using earphones such as a headset, or may be a display using the display unit 214 of the mobile terminal device 210.

[0153] In step S607, the support information output unit 112 obtains the timing of providing a meal with a spoon and the amount provided with the spoon. The timing of providing a meal with a spoon represents the timing of putting a mouthful of food placed on the spoon into the mouth of the care recipient. The amount provided with the spoon represents the amount of a mouthful of food. The process of step S607 is performed, for example, by using input data related to the chewing state of the care recipient. The input data related to the chewing state is, for example, information regarding the state of the care recipient's mouth, throat, facial expression, complexion, posture, changes in the meal in response to an utterance, the timing of swallowing, the time the food is in the mouth, the meal rhythm, etc., and may be, for example, a captured image of the care recipient. Also, the input data related to the chewing state is information regarding the movement of the jaw, cheeks, the whole face, and the body, and may be, for example, the sensor information of the motion sensor 410. Further, the input data related to the chewing state may include voice data representing the voice quality and volume in response to an utterance during a meal, information representing differences in timing and amount during past meals, differences due to seasons, differences due to physical condition, etc. Note that the information representing the meal rhythm may be a captured image or the sensor information of the motion sensor 410. In addition, various modifications are possible for the sensors used when acquiring the above information.

[0154] In step S608, the support information output unit 112 notifies the caregiver of the obtained timing of providing a meal with a spoon and the amount provided with the spoon. For example, in step S607, the support information output unit 112 determines whether the current timing is the timing of providing a meal with a spoon. When it is determined that it is the providing timing, the support information output unit 112 notifies to that effect in step S608, and when it is determined that it is not the providing timing, no notification is made. Also, when it is determined that it is not the providing timing, the support information output unit 112 may notify that it is not the providing timing when the caregiver attempts to provide a meal to the care recipient.

[0155] Also, when it is determined that it is, for example, the provision timing, the support information output unit 112 may, in step S607, obtain the amount of provision with the spoon, and in step S608, notify the caregiver of the obtained amount of provision in grams or in stages such as more / normal / less. Alternatively, the support information output unit 112 may, in step S607, obtain support information indicating whether the amount of provision with the spoon is appropriate by using input data representing the amount of food actually placed on the spoon by the caregiver. The input data in this case includes, for example, the output of a camera that images the caregiver's hand. When the amount of food placed on the spoon is too much or too little, in step S608, the support information output unit 112 may issue a notification prompting a change in the amount of food placed on the spoon.

[0156] Note that the expression of the care recipient may be used to determine whether the assistance of the caregiver is correct. For example, the support information output unit 112 may not output a specific instruction as correct when the care recipient has a smiling face, and may output an instruction when the care recipient has an unhappy face. For example, the support information output unit 112 may use, as the input data in step S607, an image of the care recipient's face or the result of an expression determination process based on the image. In this way, it is possible to determine whether the pace of eating is appropriate based on the expression of the care recipient. Alternatively, the support information output unit 112 may determine the degree of relaxation from the analysis of the heartbeat (pulse). The support information output unit 112 may not output a specific instruction as correct when the degree of relaxation of the care recipient is high, and may output an instruction when the degree of relaxation of the care recipient is low. For example, the support information output unit 112 may use, as the input data in step S607, the heart rate, the pulse rate, or information representing the analysis result thereof. The fact that the expression and the degree of relaxation may be used to determine whether the assistance is appropriate also applies to steps other than S607 in FIG. 21 and FIGS. 22 and 23 described later.

[0157] In step S609, the support information output unit 112 determines whether or not the meal of the assisted person is finished. If not, the process returns to step S607. By repeating the processes of steps S607 and S608 in this way, it becomes possible to sequentially present to the caregiver the timing of providing one mouthful of food and the amount provided at that time. As a result, it becomes possible to make the assisted person have a meal at an appropriate pace.

[0158] If it is determined that the meal is finished, in step S610, the support information output unit 112 instructs the recording of the meal result. For example, as the meal result, an imaging image of the uneaten food is acquired. Note that the recording instruction may be an instruction to the caregiver to perform imaging using the mobile terminal device 210 or the like, or may be an automatic imaging by remotely controlling a camera arranged at an appropriate position.

[0159] In step S611, the support information output unit 112 requests support information indicating whether or not the assisted person needs to replenish water. If water replenishment is necessary, in step S612, the support information output unit 112 issues a notification instructing the caregiver to replenish water. Note that the support information output unit 112 may request support information indicating a specific replenishment amount in step S611 and notify the replenishment amount in step S612.

[0160] If water replenishment is not necessary, or after the process of step S612, the meal assistance sequence ends.

[0161] Note that FIG. 21 is an example of a meal assistance sequence, and the specific sequence can be implemented in various modified forms. For example, when performing meal assistance on the care bed 310, control for switching the care bed 310 to a meal mode suitable for eating (for example, a mode of raising the back bottom to an angle set in the range of 30 degrees to 90 degrees, a mode of raising the knee bottom to an angle set in the range of 0 degrees to 30 degrees, a mode of lowering the foot bottom to an angle set in the range of 0 degrees to 90 degrees, a mode of tilting the bed at an angle set in the range of 0 degrees to 20 degrees so that the head side is higher) may be added. For example, the support information output unit 112 may obtain support information indicating whether the person to be assisted and the served meal are in a state suitable for starting the meal by using the output of a camera or the like as input data. When it is determined that the person to be assisted and the meal are properly set, the support information output unit 112 performs a notification process of asking the caregiver whether the care bed 310 can be moved. When the caregiver answers "OK", the care bed 310 may be changed to the meal mode.

[0162] Also, the support information output unit 112 can output various support information related to meal assistance to the caregiver or the person in charge of cooking before the meal can be eaten at the meal location.

[0163] 3.2.5 Excretion Assistance FIG. 22 is a flowchart for explaining a specific assistance sequence when performing excretion assistance. When the excretion assistance sequence is started, in step S701, the support information output unit 112 controls to turn on the sensors necessary for the support of excretion assistance among the sensor group 400 arranged in the care facility or the like.

[0164] Next, in step S702, the support information output unit 112 determines whether the caregiver has moved to the position where toileting assistance is to be provided. For example, if the care recipient excretes into the diaper on the nursing bed 310, it is assumed that the toileting assistance will be provided in the care recipient's living room. In this case, the process of step S702 is performed by using, for example, the output of a camera arranged in the living room, or the output of a camera of the portable terminal device 210 carried by the caregiver, the output of an RFID, etc. as input data.

[0165] If the caregiver is not present, in step S703, the support information output unit 112 waits for a certain period of time and then performs the process of step S702 again. If the caregiver is present, in step S704, the support information output unit 112 requests support information regarding the removal of the diaper. In step S705, the support information output unit 112 performs a notification process of the requested support information.

[0166] For example, if the posture of the care recipient, the posture of the caregiver, the direction of pulling out the diaper, etc. are not appropriate when removing the diaper, feces may adhere to the care recipient's clothing or sheets, which is not preferable. Therefore, the support information output unit 112 may, for example, in step S704, determine whether the movement of the caregiver when removing the diaper is appropriate. For example, the support information output unit 112 may obtain the correct movement and compare the movement with the actual movement of the caregiver. If it is determined to be inappropriate, in step S705, the support information output unit 112 may notify that it is inappropriate, or may specifically instruct the appropriate movement.

[0167] In step S706, the support information output unit 112 requests support information regarding the wearing of the diaper. In step S707, the support information output unit 112 performs a notification process of the requested support information.

[0168] The support information output unit 112 may, for example, in step S706, determine whether the movements of the caregiver when putting on a new diaper are appropriate. For example, the support information output unit 112 may obtain the correct movements and compare the movements with the actual movements of the caregiver. If it is determined to be inappropriate, in step S707, the support information output unit 112 may notify that it is inappropriate, or may specifically instruct appropriate movements.

[0169] It is a problem that the sheets and the like get dirty when removing the diaper. However, since the caregiver is nearby, it is easy for the caregiver to recognize the dirt and the response is relatively easy. On the other hand, when the diaper is not properly worn and leakage occurs, the caregiver is not always nearby when the leakage occurs. Also, considering the burden on the caregiver, it is not easy to excessively increase the frequency of toileting assistance, and there is a risk that the leakage will be left unattended for a long time. In view of the above, the support information output unit 112 may set conditions so that the notification in the process of step S707 is easier to perform than the process of step S705. For example, when the degree of deviation between the correct movement and the actual movement exceeds a threshold value and the notifications in steps S705 and S707 are performed, the threshold value in step S707 is set smaller than the threshold value in step S705.

[0170] Alternatively, in step S706, by using the sensor information of the sensor provided in the diaper as input data, it may be possible to more precisely determine whether the wearing state is appropriate. Also in this case, it is possible to provide assistance that places more emphasis on wearing the diaper.

[0171] In step S708, the support information output unit 112 instructs to record the excretion state. Specifically, the support information output unit 112 gives instructions to image the state of urine and feces and to measure the weight of urine and feces. Note that the weight measurement may be the weight measurement of the diaper, or if the garbage bin is being transported, it may be the weight measurement of the garbage. Also, the instruction to record here may be to have the caregiver perform imaging and weight measurement, or may be to remotely control the camera and sensor.

[0172] Note that FIG. 22 shows an example of the excretion assistance sequence, and various modifications of the specific sequence are possible. For example, when performing excretion assistance with the nursing bed 310, control may be added to change the height of the nursing bed 310 to a height suitable for excretion assistance (for example, a height of 50 mm to 100 mm from the floor surface to the upper surface of the bottom, where the caregiver does not need to bend down). For example, when the caregiver responds with "OK" in step S506 of FIG. 20, by changing the height of the nursing bed 310, a state in which excretion assistance can be easily performed is realized when the caregiver arrives. When the nursing bed 310 has a speaker, the support information output unit 112 may perform control to output a voice for explaining the purpose to the care recipient before changing the height. The support information output unit 112 may also notify the caregiver that the height change of the nursing bed 310 has been completed.

[0173] 3.2.6 Transfer assistance or movement assistance FIG. 23 is a flowchart for explaining a specific assistance sequence when performing transfer assistance or movement assistance. When the transfer / movement assistance sequence is started, in step S801, the support information output unit 112 performs control to turn on the sensors necessary for supporting transfer / movement assistance among the sensor group 400 arranged in the nursing facility or the like.

[0174] Next, in step S802, the support information output unit 112 determines whether a lift is required for the transfer / movement assistance of the care recipient. The process of step S802 is performed by using as input data the physical difference between the caregiver and the care recipient, the ADL of the care recipient, the time required for transfer, the inventory of lifts in the nursing facility, and the like.

[0175] When a lift is not required, the caregiver manually transfers the care recipient to the wheelchair. In step S803, the support information output unit 112 requests support information regarding the manual transfer. In step S804, the support information output unit 112 performs a notification process for the requested support information. Note that the support information output unit 112 may notify the caregiver of whether wheelchair locking is necessary before the transfer. If the caregiver responds that it is necessary, the support information output unit 112 performs control to lock the wheelchair. Alternatively, the support information output unit 112 may automatically determine whether locking is necessary, and if it is determined to be necessary, may instruct the caregiver to lock the wheelchair before the process of step S803.

[0176] In step S803, for example, the support information output unit 112 may determine whether the caregiver's use of their body during the manual transfer is appropriate. For example, the support information output unit 112 may obtain the correct movements, such as the caregiver's posture and the position where the care recipient is placed on the feet, and compare these movements with the actual movements of the caregiver. The movements of the caregiver may be detected using the motion sensor 410 or may be detected using the imaging sensor 420. Also, since the positional relationship between the care recipient and the caregiver is important in transfer and movement assistance, in addition to the movements of the caregiver, these sensors may detect the movements and postures of the care recipient. If it is determined to be inappropriate, in step S804, the support information output unit 112 may notify that it is inappropriate or may specifically instruct the appropriate movements.

[0177] When a lift is required, in step S805, the support information output unit 112 performs control to move the lift to the location of the care recipient. In step S806, the support information output unit 112 requests support information regarding the transfer using the lift. In step S807, the support information output unit 112 performs a notification process for the requested support information.

[0178] The support information output unit 112 may, for example, determine in step S806 whether the usage of the lift is appropriate. For example, the support information output unit 112 may obtain correct data such as the wearing state of the sling that can safely lift the care recipient, and compare the correct state with the actual state. When it is determined to be inappropriate, in step S807, the support information output unit 112 may notify that it is inappropriate, or may specifically instruct the appropriate wearing state.

[0179] After lifting the care recipient with the lift, the care recipient may be seated in a wheelchair or may be moved as is. The support information output unit 112 may determine which to use in consideration of the lift inventory, the destination of movement, and the state of the care recipient, and notify the caregiver.

[0180] Note that FIG. 23 is an example of a transfer / movement assistance sequence, and various modifications of the specific sequence are possible. For example, control may be added to change the height of the nursing bed 310 (for example, the height of the nursing bed from the floor surface to the upper surface of the bottom is 200 mm to 500 mm, which is the height at which the feet can firmly touch when sitting on the bed for a non-care recipient who can stand, 200 mm to 500 mm, which is slightly higher than the wheelchair when transferring to a wheelchair, 200 mm to 500 mm, which is slightly lower than the wheelchair when transferring from a wheelchair to a bed, etc.) to a height suitable for transfer / movement assistance. Since the specific control is the same as that for excretion assistance, etc., detailed description is omitted.

[0181] 3.2.7 Specific changes in assistance according to factors The specific sequences of meal assistance, excretion assistance, and transfer / movement assistance have been described above. Although the description has been omitted in FIGS. 21 to 23, each support information may be obtained based on the presence or absence of abnormal behavior and factors. For example, as described above with reference to FIG. 13, the presence or absence of abnormal behavior and the determination result of the cause of abnormal behavior are used as input data when obtaining support information.

[0182] For example, when the cause determination unit 111 determines that it is a cause of dementia, the support information output unit 112 changes the output in each assistance sequence on the assumption that the dementia is progressing. For example, in meal assistance, the support information output unit 112 outputs, as support information, information representing the amount of food provided per mouthful (the amount provided with a spoon) and information representing the timing of providing a mouthful of food (the timing of providing food with a spoon). At this time, when the behavior is determined to be an abnormal behavior due to a dementia cause, the support information output unit 112 may change at least one of the amount provided and the timing of providing compared to the case where the behavior is determined not to be an abnormal behavior due to a dementia cause. In this way, it becomes possible to appropriately change the pace of eating between the care recipient with dementia and the care recipient without dementia. For example, the support information output unit 112 may reduce the amount provided or may delay the timing of providing. In this way, when the care recipient is prone to choking due to dementia, it becomes possible to appropriately manage the pace of eating.

[0183] Also, in excretion assistance, the support information output unit 112 outputs, as support information, information specifying the excretion assistance timing which is the timing to start excretion assistance. At this time, when the behavior is determined to be an abnormal behavior due to a dementia cause, the excretion assistance timing may be changed compared to the case where the behavior is determined not to be an abnormal behavior due to a dementia cause. In this way, it becomes possible to start the excretion assistance sequence at an appropriate timing according to whether it is dementia or not. For example, the support information output unit 112 may advance the excretion assistance timing. By adjusting the excretion assistance timing in this way, even when it becomes difficult for the care recipient to control the excretion timing due to dementia, it becomes possible to easily maintain a clean state.

[0184] In addition, when it is determined that the abnormal behavior is caused by dementia, the support information output unit 112 may change the support information so as to provide the following meal assistance. For example, when it is determined that the abnormal behavior is caused by dementia, the support information output unit 112 may set a higher priority for notifications regarding the following. For example, when it is determined that the cause is dementia, the support information output unit 112 may issue the following notifications, and may not issue the following notifications when there is no abnormal behavior or when the cause is other than dementia. Alternatively, the support information output unit 112 may issue the following notifications regardless of whether the cause is dementia, but may perform control to make the notifications more likely to be issued when the cause is dementia. For example, the support information output unit 112 may increase the notification frequency or relax the conditions for determining the necessity of notifications when the cause is dementia. · Urinate and defecate before meals so as to concentrate on eating. · Notify whether the sleep is sufficient or the physical condition is good. · After serving the meal, observe without assistance to understand the current situation. · Arrange the environment, prepare stable tableware, and prepare tableware with affection. · Adjust to a comfortable eating position. · Make adjustments such as shifting the time when the meal does not progress. · Speak to help the person understand that it is a meal and assist with the first bite. · Speak and teach the person how to eat. · Provide moisture to prevent dehydration. · Adjust the amount of the spoon and the feeding speed to prevent choking. · Increase the activity level and adjust the daily rhythm when the meal does not progress.

[0185] Also, when it is determined that the abnormal behavior is caused by dementia, the support information output unit 112 may change the support information so as to provide the following excretion assistance. · Notify whether the sleep is sufficient or the physical condition is good. · Select and use appropriate pants, diapers, pads, etc. · When urinating or defecating in the toilet, check if the water is flowing or check the toilet. · Take fall prevention measures as the number of trips to the toilet may increase. · Observe the excretion timing and prompt the person to go to the toilet. · Since there is a possibility of fecal play, observe the excretion timing and conduct toilet induction and diaper change. · Assign staff with good compatibility.

[0186] In addition, the support information output unit 112 may output support information regarding sleep assistance. When it is determined as abnormal behavior due to dementia factors, the support information output unit 112 may change the support information so as to perform the following sleep assistance. · Adjust the daily rhythm to keep the autonomic nerves normal. · Exercise to increase the activity level during the day. · Keep watch to see if there is any abnormal behavior at night.

[0187] When keeping watch at night, for example, use the sensor information of the getting-out-of-bed sensor or the watching sensor as input data. For example, the support information output unit 112 instructs the caregiver who does not assist with breakfast to introduce the sensor during breakfast for the person receiving care. Also, when exercising, the support information output unit 112 may notify the caregiver to propose recreation or rehabilitation, for example, after daytime excretion assistance.

[0188] Also, as described above with reference to FIG. 6, the factor determination unit 111 may be able to determine whether the factor of abnormal behavior is an environmental factor or an excretion disorder factor. For example, when it is determined as abnormal behavior due to excretion disorder factors, the support information output unit 112 may change the support information so as to perform the following assistance. · Notify the addition of a laxative to the dinner meal. · Change the content of the meal (applicable to breakfast, lunch, and dinner). · Instruct to provide water after meals. · Propose recreation or rehabilitation after daytime excretion assistance.

[0189] Note that the support information output unit 112 may not only simply instruct the addition of a laxative, but also propose a specific type of laxative and the dosing time. For example, the support information output unit 112 may use, as input data, information indicating how many consecutive days a laxative has been administered, information on the defecation interval, etc., to notify the type of laxative. Further, when a care recipient determined to be caused by dementia is subsequently determined to be caused by excretory disorder, the support information output unit 112 may instruct the caregiver to remove sensors other than the excretory sensor among the sensors arranged to cope with dementia.

[0190] Also, when it is determined that the abnormal behavior is due to environmental factors, the support information output unit 112 may change the support information so as to perform the following assistance. · Automatically control the rhythm of the speaker and lighting to be the same as the data before the environmental cause

[0191] By approaching the same environment as before the occurrence of abnormal behavior in this way, it becomes possible to adjust the daily rhythm of the care recipient. Note that the caregiver or the like may be able to change settings such as temporarily stopping the application of automatic control or not applying automatic control. Further, when a care recipient determined to be caused by dementia is subsequently determined to be caused by environmental factors, the support information output unit 112 may instruct the caregiver to remove the sensors arranged to cope with dementia.

[0192] Also, when the behavior is determined to be due to dementia factors, the support information output unit 112 may increase the types of support information to be output compared to the case where it is not determined to be abnormal behavior. For example, the support information for "arranging the environment, preparing calming tableware, and preparing favorite tableware" described above is output when it is determined to be due to dementia factors, but may not be an output target when it is determined to be due to other factors. In this case, for example, input data such as a temperature sensor, a humidity sensor, an illuminance sensor, and an atmospheric pressure sensor may be used to determine a preferable environment for the care recipient.

[0193] Therefore, when the behavior is determined to be a risk factor for dementia, the support information output unit 112 may increase the types of sensor information used as compared to the case where the behavior is not determined to be abnormal behavior. By doing so, since the types of input data increase, it becomes possible to accurately obtain support information for providing care suitable for dementia.

[0194] In addition, the support information output unit 112 may determine whether it is necessary to add a new sensor based on information identifying one or more available sensors and sensor information added when the behavior is determined to be a risk factor for dementia. Here, the one or more available sensors are specifically sensors arranged in the target care facility and identified based on the third association information 125 in FIG. 16. As described above with reference to FIGS. 14 to 16, depending on the type of sensor arranged in the care facility, there is a possibility that given support information cannot be output with sufficient accuracy, and the support information may be set to "unable to output". Therefore, depending on the care facility, even if the factor determination unit 111 determines that it is a risk factor for dementia, it may be difficult to output support information suitable for dementia. The information processing device may, for example, determine whether it is necessary to add a sensor, and propose the addition of a sensor that needs to be added or a device including the sensor. By doing so, it becomes possible to appropriately output support information according to the factor.

[0195] As described above, it is assumed that appropriate assistance changes depending on the presence or absence of abnormal behavior and the factors of the abnormal behavior. According to the method of the present embodiment, when supporting the assistance by the caregiver, the factor determination result of the behavior of the care recipient is used. As a result, it becomes possible to make the caregiver provide assistance more suitable for the care recipient.

[0196] Specifically, it is possible to digitize the tacit knowledge of skilled assistants and enable appropriate assistance even for less skilled assistants. For example, even less skilled assistants can provide assistance equivalent to that of skilled ones, improving the reproducibility of assistance. Also, the variation in care skills is suppressed, making organizational management easier, and thus suppressing the occurrence of incidents such as the fall of the care recipient. As a result, the occurrence of empty beds due to hospitalization and overtime due to the creation of accident reports can be suppressed. Also, if incidents are suppressed, the over-sensitivity of assistants to risks is also suppressed, enabling stress reduction and, as a result, suppressing the turnover rate. Also, by enabling the improvement of assistants' skills and the improvement of the working environment, it becomes possible to improve the satisfaction of the care recipient and their family and the Quality of Life (QOL).

[0197] Note that the information processing system 10, server system 100, assistant device 200, etc. of this embodiment may realize part or most of their processing by a program. In this case, the information processing system 10, etc. of this embodiment is realized by a processor such as a CPU executing the program. Specifically, the program stored in a non-temporary information storage medium is read out, and the read program is executed by a processor such as a CPU. Here, the information storage medium (a medium readable by a computer) stores programs, data, etc., and its function can be realized by an optical disk, HDD, or memory (card-type memory, ROM, etc.). And a processor such as a CPU performs various processes of this embodiment based on the program stored in the information storage medium. That is, the information storage medium stores a program for functioning a computer as each part of this embodiment.

[0198] Moreover, the method of this embodiment can be applied to an information processing method that determines whether the behavior of the person to be assisted is abnormal behavior due to dementia factors based on (1) the dementia level information of the person to be assisted and (2) at least one of the environmental information, excretion information, and sleep information of the person to be assisted, and outputs support information for supporting the assistance of the person to be assisted by the caregiver based on the determination result and the sensor information that is the sensing result regarding the caregiver or the person to be assisted who assists the person to be assisted.

[0199] 4. Modification Example <Parallel Processing of Multiple Assistance Sequences> Each of the above-described assistance sequences described with reference to FIGS. 21 to 23 may be executed sequentially. For example, a given caregiver responds with OK in step S506 of FIG. 20 while in the standby state, thereby executing a sequence corresponding to any of FIGS. 21 to 23, and returns to the standby state after completion. Note that the standby state represents a state in which the target caregiver is not executing any assistance sequence. Then, by responding with OK again in step S506, a sequence corresponding to any of FIGS. 21 to 23 is executed.

[0200] However, in a nursing facility or the like, there may be a case where one caregiver concurrently assists multiple persons to be assisted. For example, after seating the person to be assisted A and the person to be assisted B in close proximity, one caregiver simultaneously performs meal assistance for the person to be assisted A and the person to be assisted B. In this case, it is inefficient to execute the meal assistance sequence of FIG. 21 for the person to be assisted B after the execution of the meal assistance sequence of FIG. 21 for the person to be assisted A is completed.

[0201] Therefore, the support information output unit 112 may be capable of executing multiple assistance sequences in parallel for one caregiver. For example, in the above example, the support information output unit 112 executes the meal assistance sequence for the person to be assisted A and the meal assistance sequence for the person to be assisted B in parallel. Here, an example where the ratio of caregiver to person to be assisted is 1:2 is described, but the number of persons to be assisted concurrently served by one caregiver may be three or more.

[0202] For example, in the meal assistance sequence of the care recipient A, the support information output unit 112 performs the process of step S605 and notifies the result in the form of "The minimum provision for Mr. A is x grams" etc. in step S606. Similarly, in the meal assistance sequence of the care recipient B, the process of step S605 is performed and the result is notified in the form of "The minimum provision for Mr. B is y grams" etc. in step S606. In this way, the support information output unit 112 executes the acquisition of input data regarding the care recipient A and the acquisition of input data regarding the care recipient B in parallel, and based on each input data, outputs support information regarding the care recipient A and outputs support information regarding the care recipient B at the necessary timing. By doing so, even if the relationship between the caregiver and the care recipients is one-to-many, it becomes possible to have the caregiver perform the necessary assistance for each care recipient. In addition, by installing a wide-angle camera capable of imaging a plurality of care recipients simultaneously, it is also possible to make the input data regarding the care recipient A and the input data regarding the care recipient B common.

[0203] However, since there is only one caregiver, even if a plurality of support information is notified at very close timings, it is not easy to respond to all of them. For example, when the notification of step S608 is made for the care recipient A, the caregiver takes the amount of food according to the notification with a spoon and performs the operation of carrying it to the mouth of the care recipient A. Even if the notification of step S608 is made for the care recipient B before that is completed, it is difficult for the caregiver to take the food for the care recipient B with a spoon and carry it to the mouth of the care recipient B.

[0204] Alternatively, when performing meal assistance for a plurality of care recipients, it is efficient to let all of them eat after gathering at a meal place such as a cafeteria. Therefore, even if it is determined that the caregiver and the care recipient A have arrived at the location (Yes in step S603), if the care recipient B is not present at the location, it may not be preferable to start the processes of steps S607 - S609 etc. for the care recipient A.

[0205] Considering these points, the support information output unit 112 may execute processing that takes into account the relationships between multiple assistance sequences, rather than simply executing the assistance sequences for multiple assisted persons in parallel. For example, when the support information output unit 112 executes multiple assistance sequences in parallel for a given assisted person, it may control the execution and suspension of each assistance sequence.

[0206] For example, when the support information output unit 112 notifies the assisted person A in step S608, it may suspend the meal assistance sequence for the assisted person B. Then, when one mouthful of food for the assisted person A by the caregiver is completed, the meal assistance sequence for the assisted person B is resumed. Since the support information output unit 112 has determined in step S607 that it is acceptable to provide a meal for the assisted person B, in step S608, it executes a notification to the caregiver to have the assisted person B take one mouthful of food. In this case, since the caregiver is performing an operation for the assisted person B, the support information output unit 112 performs a process of suspending the meal assistance sequence for the assisted person A until the operation is completed.

[0207] Alternatively, when the support information output unit 112 determines that the assisted person A has arrived at the location (Yes in step S603), it may suspend the meal assistance sequence for the assisted person A until it is determined that all other assisted persons for whom the same caregiver is responsible for meal assistance have arrived at the location.

[0208] Figure 24A is a state transition diagram for explaining the transition of an assistance sequence for a given caregiver. For example, the support information output unit 112 executes two meal assistance sequences to support the caregiver who provides meal assistance to the assisted persons A and B. At this time, the support information output unit 112 performs a state transition based on a given condition. For example, when the support information output unit 112 determines that one unit of assistance by the caregiver has been completed while executing the meal assistance sequence for the assisted person A, it stops the meal assistance sequence for the assisted person A and transitions to a state where it executes the meal assistance sequence for the assisted person B.

[0209] Alternatively, the support information output unit 112 may determine the priority of the support information to be notified in each assistance sequence. For example, if the support information output unit 112 determines that the notified person A has completed eating and thus performs a notification for storing the eating result (step S610), and determines that the notified person B has not completed eating and thus performs a notification for providing one mouthful of food (step S608). The recording of the eating result can be executed at an arbitrary timing until the cleanup is performed, while the provision of one mouthful of food will not complete the eating of the notified person B unless it is performed. Therefore, in this case, the support information output unit 112 may prioritize the execution of the meal assistance sequence for the notified person B and suspend the meal assistance sequence for the notified person A. Even in this way, it is possible to realize an appropriate state transition between a plurality of assistance sequences for a plurality of notified persons. Note that the state transition between a plurality of assistance sequences may be considered as another assistance sequence interrupting the currently executing assistance sequence.

[0210] Also, although an example in which two meal assistance sequences are executed in parallel has been shown above, the method of this embodiment is not limited to this. FIG. 24B is another diagram for explaining the state transition between assistance sequences in this embodiment.

[0211] As shown in FIG. 24B, in this embodiment, various sequences such as a meal assistance sequence, an excretion assistance sequence, a transfer assistance sequence, and an abnormality handling sequence may be executed in parallel. In this case, the support information output unit 112 may control the transition between the respective assistance sequences shown in FIG. 24B. Note that in FIG. 24B, an example of passing through a standby state when transitioning from a given type of assistance sequence to another type of assistance sequence is shown, but a direct transition may be made between each assistance sequence. Also, as shown in FIG. 24A, a plurality of assistance sequences may be included in the meal assistance sequence. Similarly, other assistance sequences such as the excretion assistance sequence may also include a plurality of assistance sequences.

[0212] For example, when a given caregiver is providing a meal to care recipient A and care recipient A enters an abnormal state. The abnormal state may be, for example, a choking situation. In this case, the caregiver will stop the meal assistance for care recipient A and respond to the abnormality. For example, the support information output unit 112 executes the start determination of the abnormality response sequence in the background in the same manner as steps S501 to S503 in FIG. 20, and starts the abnormality response sequence when an abnormality of care recipient A is detected. Although the processes of steps S505 to S506 in FIG. 20 may be executed, considering that the person in charge of care recipient A is the same as the person in charge of meal assistance and there may be a high degree of urgency, the processes of steps S505 to S506 may be omitted.

[0213] As a result, an abnormality response sequence is added to the assistance sequence to be executed. Then, the support information output unit 112 suspends the currently executing meal assistance sequence and starts the execution of the abnormality response sequence. When it is confirmed by the abnormality response sequence that the abnormality has been resolved, the support information output unit 112 makes a transition to another assistance sequence, such as resuming the suspended meal assistance sequence.

[0214] Alternatively, when a given caregiver is providing a meal to care recipient A, care recipient A may want to go to the toilet. In this case, an excretion assistance sequence is added to the assistance sequence to be executed. Also, depending on the ADL of care recipient A and the location of the toilet, etc., a transfer assistance sequence may be required. For example, the support information output unit 112 suspends the meal assistance sequence, first executes a transfer assistance sequence to move to the toilet, then executes an excretion assistance sequence, and after completion, resumes the suspended meal assistance sequence.

[0215] The factors that change the necessary assistance include those due to the initiative of the person being assisted, those due to the physical condition of the person being assisted, those due to diseases such as dementia, those due to medications, those due to the environment, those due to seasons, those due to external factors, those due to the deviation between the care progress and the schedule on that day, etc. Various factors can be considered. For example, the support information output unit 112 may perform a process of detecting these factors and perform a process of determining the transition destination assistance sequence based on the detected factors and the currently executed assistance sequence.

[0216] In this way, by the support information output unit 112 executing a plurality of assistance sequences in parallel and controlling the state transition between the plurality of assistance sequences, it is possible to appropriately respond to various situations. For example, as described above, even when there is a one-to-many relationship between the caregiver and the person being cared for, it is possible to support the determination of the assistance to be executed and its order. Since it is possible to reduce the burden on the caregiver, it is possible to reduce the risk of incidents such as aspiration and falls of the person being cared for. Also, when other assistance suddenly becomes necessary during the execution of a given assistance, since the caregiver can appropriately support the assistance to be performed at that time, it is possible to reduce the burden on the caregiver and the risk of the person being cared for.

[0217] <Data addition by user> Above, the description has been made assuming that the support information output NN 122 is created by the learning unit 114. For example, the provider of the information processing device may select a given care facility, etc. in advance for learning and create the support information output NN 122 using the data from the care facility, etc. When a care facility that newly uses the service provided by the information processing device is added, for example, the existing support information output NN 122 is commonly used.

[0218] However, the method of this embodiment is not limited to this, and new training data may be added by users of care facilities, etc., and additional machine learning may be executed using the training data.

[0219] For example, while maintaining the state where the NN122 for support information output is common among multiple care facilities, data from each care facility may be collected and used for machine learning as a whole. In this case, since training data can be collected from multiple care facilities, there is an advantage that it is easy to increase the number of training data.

[0220] Alternatively, additional machine learning may be executed for each care facility. In this case, the NN122 for support information output is updated for each care facility. That is, it becomes possible to make the NN122 for support information output specialized for the target care facility.

[0221] FIG. 25A is an example of a screen displayed on the display unit 214 of the mobile terminal device 210, for example. When compared with FIG. 17, an object OB4 for data addition is added. When the caregiver performs a selection operation on the object OB4, the screen transitions to the screen of FIG. 25B.

[0222] In FIG. 25B, it includes a region RE1 for displaying the name of the support information for which training data is to be added, and a region RE2 where the ID of the care recipient, the ID of the caregiver, and output data can be input. The ID of the care recipient is information for identifying the care recipient. The ID of the caregiver is information for identifying the caregiver. The output data is information corresponding to the output of the NN122 for support information output. Since FIG. 25B targets the diaper changing timing, an example where time is used as the output data is shown. However, the format of the output data can be variously modified according to the type of support information, and it may be an image, may be audio, may be a numerical value, may be binary data representing true or false, or may be other formats.

[0223] In the example of FIG. 25B, it represents that when a caregiver with a caregiver ID of abcde performed toileting assistance for a care recipient with a care recipient ID of 12345, it was determined that the time of 2021 / MM / DD hh:mm:ss was appropriate as the diaper change timing. Also, separately from the caregiver's operation, the input data corresponding to the diaper change timing has been acquired at the care facility. That is, the data set associating the input data with the output data of 2021 / MM / DD hh:mm:ss can be the training data for the support information output unit NN122 that outputs the diaper change timing.

[0224] However, in this embodiment, it is assumed that the tacit knowledge of skilled caregivers is digitized and appropriate assistance is provided regardless of the caregiver's proficiency level. Therefore, even if the above data set is acquired by the input of a given caregiver, it is unclear whether it is positive data or negative data. Positive data represents a data set in which appropriate correct data is associated with the input data, and negative data represents a data set in which inappropriate correct data is associated with the input data.

[0225] Thus, the learning unit 114 may hold, for example, association information that associates the caregiver ID with the caregiver's proficiency level. The proficiency level may be manually input by the administrator of the care facility or automatically determined based on the number of years of experience, held qualifications, past care history, etc. The learning unit 114 regards the data set by a caregiver with a high proficiency level as positive data and the data set by a caregiver with a low proficiency level as negative data.

[0226] Alternatively, even with the assistance of a skilled person, there are two possibilities: assisting in accordance with the manual and adjusting the way of assistance based on one's own intuition. The tacit knowledge of a skilled person is highly likely to be used when the skilled person acts according to intuition. Therefore, as shown in FIG. 25B, the area RE2 of the display screen may be able to input whether intuition has been used or not. When determining, for example, the timing of diaper change, the caregiver inputs whether intuition has been used or not into the area RE2. The learning unit 114 uses the dataset when the corresponding input is "yes" as positive data.

[0227] The learning process after acquiring the training data is the same as the example described above with reference to FIG. 8, so detailed description is omitted. In the example of FIG. 25B, by updating the support information outputting NN122 that outputs the diaper change timing, it becomes possible to output a more accurate change timing. Although the example regarding the diaper change timing has been described above, addition of training data is similarly possible for other support information.

[0228] <Custom support information> Also, FIGS. 43 to 45 have been exemplified as the support information that can be output above. However, as can be understood from the above description, the support required in assistance is diverse, and depending on the nursing facility or the caregiver, the required support may be different. Therefore, there may be cases where support information of a type not included in the existing support information is required. Thus, in this embodiment, the caregiver may be able to add arbitrary custom support information.

[0229] For example, in FIG. 25B, the name of the support information displayed in the area RE1 is not fixed and may be arbitrarily editable by the caregiver. The caregiver uses text such as "the timing of doing XXXX" to input the name of the desired custom support information. "XXXX" is text representing, for example, a specific caregiving action performed by the caregiver. Also, when the caregiver performs the caregiving action corresponding to "XXXX", the caregiver inputs the caregiver ID, the care recipient ID, output data, whether intuition was used, etc. Thereby, as part of the training data of the support information output NN122 that outputs "the timing of doing XXXX", the output data and information indicating whether the output data is positive data or negative data are acquired.

[0230] Furthermore, the information processing apparatus may perform control to display, on the display unit 214 of the mobile terminal device 210, a screen for specifying input data among the training data. FIG. 25C is an example of a display screen for specifying input data. The screen shown in FIG. 25C includes an area RE3 that displays the name of the custom support information, and an area RE4 where the name of the device already arranged in the target care facility and the name of the input data that can be acquired by the device are selectable.

[0231] For example, the sleep scan is, for example, the sensing device 450 shown in FIG. 2D, which can detect the heart rate, respiratory rate, and activity level. The caregiver selects the data that he / she wants to use as the input data when obtaining the custom support information from among the data that can be acquired by the device. FIG. 25C shows an example where the caregiver selects the respiratory rate by the sleep scan and the images of the camera provided beside the bed as the input data, and does not use the output of the pulse oximeter as the input data.

[0232] By using the screen shown in FIG. 25C, the name of the custom support information and the input data used for the output of the custom support information are associated with each other. The association information representing this association is transmitted to the server system 100 and stored in the storage unit 120.

[0233] In the storage unit 120 of the server system 100, time-series respiration rates and time-series camera images on the bedside collected from the target nursing facility are stored. Therefore, the learning unit 114 extracts the respiration rate and camera image corresponding to the output data obtained using FIG. 25B as input data. For example, the server system 100 holds the acquisition timing of the output data based on FIG. 25B, and reads out the respiration rate and camera image for a predetermined period set based on the acquisition timing from the storage unit 120. Then, the learning unit 114 performs a learning process of the support information output NN 122 for outputting custom support information based on the training data in which the read input data and output data are associated.

[0234] Also, as shown in FIG. 25C, an object OB5 for performing a learning start operation may be displayed on the display unit 214 of the mobile terminal device 210. When it is detected that the caregiver has performed a selection operation on the object OB5, the learning unit 114 performs the above learning process. As a result, a support information output NN 122 for outputting custom support information is newly created. Since the learning process is the same as the above-described example, detailed description is omitted. Also, in the machine learning of custom support information, it is sufficient that training data associating input data and output data can be obtained, and the user interface is not limited to the one described above.

[0235] At this time, the structure of the NN can be variously modified. FIG. 26 is a diagram showing an example of the structure of a general-purpose NN. The NN shown in FIG. 26 includes a CNN1 that extracts feature amounts using image data as input, a CNN2 that extracts feature amounts using audio data as input, a vector conversion NN that extracts feature amounts using text data as input, and a CNN3 that extracts feature amounts using other sensor information as input. Also, the NN in FIG. 26 includes a DNN (Deep Neural Network) that receives the outputs from the CNN1, CNN2, vector conversion NN, and CNN3 and outputs custom support information.

[0236] NN shown in FIG. 26 can receive images, audio, text, and other sensor information as inputs. Input data of custom support information can have various patterns as shown in, for example, FIG. 25C, and any pattern can appropriately receive the input data. If image data is not selected as input data, the input to CNN1 is treated as 0. The same applies when audio data, text data, or other sensor information is not selected as input data, and the input to the corresponding NN among CNN2, vector conversion NN, and CNN3 becomes 0.

[0237] FIG. 25D is an example of a screen displayed on the display unit 214 of the mobile terminal device 210 after the machine learning is completed. The display screen of FIG. 25D displays, for example, the accuracy rate obtained using validation data in the learning process. Also, in the example of FIG. 25D, based on this learning result, the assistant can select whether to output custom support information. For example, when the assistant selects yes in response to the question "Do you want to apply?", the output of custom support information becomes possible. For example, similar to the example described above in FIG. 17, by actively setting custom support information such as "the timing to do XXXX", the custom support information is output. On the other hand, when the assistant selects no, the output of custom support information is not performed.

[0238] Also, although the accuracy rate is low and it cannot be adopted as it is, there may be a possibility that the assistant thinks that the target custom support information is important and wants to use it. In this case, it may be possible to request the administrator or provider of the information processing device to perform analysis processing. For example, when the assistant selects yes in response to the question "Do you want to request analysis?", the modification process of the support information output NN122 that outputs custom support information is executed on the server system 100 side.

[0239] For example, when the original correct rate is equal to or lower than a predetermined threshold, the learning unit 114 of the server system 100 may attempt to improve the correct rate by changing the structure of the NN. Since the NN shown in FIG. 26 has a configuration considering versatility as described above, the correct rate may be improved by adopting a structure more specialized for custom support information. When the original correct rate exceeds the predetermined threshold, the learning unit 114 may skip the change process of the NN 122 for outputting support information.

[0240] For example, when there are a plurality of NNs with different structures from each other as the NN 122 for outputting support information as shown in FIG. 10, the learning unit 114 may classify the plurality of NNs into several classes.

[0241] FIG. 27 is a diagram for explaining the classification process of the NN. For example, the learning unit 114 obtains an n-dimensional feature amount by performing text mining processing using text representing the name of the output support information, and performs clustering based on the n-dimensional feature amount. For the sake of convenience of explanation, FIG. 27 shows a two-dimensional feature amount plane, but n may be 3 or more. For example, when targeting the NN that outputs "diaper changing timing" among the NNs 122 for outputting support information, words such as "diaper", "changing", and "timing" are extracted, and the n-dimensional feature amount of the NN that outputs "diaper changing timing" is obtained based on the extraction result.

[0242] Also, the clustering method is not limited to text mining processing, and the learning unit 114 may cluster a plurality of NNs by performing analysis processing such as logistic regression analysis. Further, the learning unit 114 may manually assign clustering results to some of the plurality of NNs shown in FIG. 10 and use the results to cluster the remaining NNs. In this way, it is possible to improve the accuracy of the clustering process.

[0243] In the example of FIG. 27, among the plurality of NNs held by the server system 100, NN1 to NN3 are classified into class 1, NN4 to NN7 are classified into class 2, and NN8 to NN10 are classified into class 3. Also, the learning unit 114 determines to which class it belongs by similarly obtaining n-dimensional feature amounts based on the name of the custom support information. For example, the learning unit 114 extracts words such as "XXXX" and "timing" from the name of the custom support information "the timing of doing XXXX", and obtains the n-dimensional feature amount corresponding to the custom support information based on the extraction result. The learning unit 114 determines the structure of the NN used for learning based on the clustering result of the custom support information.

[0244] For example, as shown in FIG. 27, assume that the custom support information is classified into class 1. In this case, the learning unit 114 selects any one of NN1 to NN3, and creates an NN for custom support information using the structure of the selected NN and the training data for the custom support information described above. At this time, only the structure of the original NN may be used, and all weights may be newly calculated. Alternatively, transfer learning may be performed using some of the weights of the original NN as they are. For example, the learning unit 114 performs machine learning using the structures of NN1 to NN3 respectively and the training data for the custom support information, and obtains the accuracy rate of the learned model. Then, the learning unit 114 presents the highest accuracy rate to the assistant in the same manner as in FIG. 25D, and asks the assistant to input whether to apply it. When the assistant responds with "yes", the corresponding support information output NN122 is stored in the storage unit 120, and the output of the custom support information becomes possible.

[0245] Note that when additional machine learning is performed, the relationship between the accumulation period of the training data, in other words, the acquisition period of the data to be analyzed, and the ADL of the assisted person becomes important. For example, assume that an assisted person who was able to move independently fractured a bone due to a fall and requires wheelchair assistance. When the ADL changes significantly in this way, the appropriate assistance for the assisted person before and after the change is very different. Therefore, for example, the learning result based on the training data before the ADL change may not be useful after the ADL change.

[0246] Therefore, although not shown in FIG. 25C, for example, when performing the learning start operation, not only the input of the type of input data but also the input of the analysis period may be possible. The caregiver designates a period in which the ADL of the target care recipient is considered to be at the same level as the current one. By doing so, the support information output NN 122, which is the learning result, will have content corresponding to the current ADL of the care recipient, enabling appropriate assistance support. Also, the server system 100 is assumed to collect, for example, the ADL of the care recipient as one of the input data. Therefore, when the learning start operation is performed, the learning unit 114 may acquire the time-series change of the ADL of the target care recipient and automatically set the analysis period based on the time-series change of the ADL.

[0247] Although the present embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications are possible without substantially departing from the novel matters and effects of the present embodiment. Therefore, all such modified examples are included in the scope of the present disclosure. For example, in the specification or drawings, a term described at least once together with a broader or synonymous different term can be replaced with the different term anywhere in the specification or drawings. Also, all combinations of the present embodiment and modified examples are included in the scope of the present disclosure. Also, the configurations and operations of the information processing system, server system, mobile terminal device, etc. are not limited to those described in the present embodiment, and various modifications can be made.

Explanation of Reference Numerals

[0248] 10… Information processing system, 100… Server system, 110… Processing unit, 111… Cause determination unit, 112… Support information output unit, 113… Setting unit, 114… Learning unit, 120… Memory unit, 121… Neural network for cause determination, 122… Neural network for support information output, 123… First association information, 124… Second association information, 125… Third association information, 130… Communication unit, 200… Device for caregiver, 210… Portable terminal device, 211… Processing unit, 212… Memory unit, 213… Communication unit, 214… Display unit, 215… Operation unit, 220… Wearable device, 300… Nursing care device, 310… Nursing care bed, 320… Lift, 400… Sensor group, 410… Motion sensor, 420… Imaging sensor, 430… Odor sensor, 440, 450… Sensing device, NW… Network, OB1 - OB5… Object, RE1 - RE4… Region

Claims

1. When it is determined that the behavior of the person in need of assistance is abnormal behavior, based on (1) the dementia level information of the person in need of assistance and (2) at least one of the environmental information, excretion information, and sleep information of the person in need of assistance, a factor determination unit that determines whether the abnormal behavior is a dementia factor and whether it is a sleep disorder factor; A support information output unit that outputs support information for supporting the assistance of the person in need of assistance by the caregiver based on the determination result of the factor determination unit and sensor information that is the sensing result regarding the caregiver who assists the person in need of assistance or the person in need of assistance; An information processing apparatus including the above.

2. In Claim 1, The support information includes information for supporting at least one of meal assistance for assisting the meal of the person in need of assistance, excretion assistance for assisting the excretion of the person in need of assistance, and transfer / movement assistance for assisting the transfer or movement of the person in need of assistance. An information processing apparatus.

3. In Claim 2, The support information output unit In the meal assistance, outputs information indicating the amount of meal provided per bite and information indicating the timing of providing a bite of meal as the support information, When the behavior is determined to be the abnormal behavior due to the dementia factor, an information processing apparatus that changes at least one of the provided amount and the provided timing compared to when the behavior is determined not to be the abnormal behavior due to the dementia factor.

4. In Claim 2, The support information output unit Outputs information for specifying the excretion assistance timing, which is the timing to start the excretion assistance, as the support information, When the behavior is determined to be the abnormal behavior due to the dementia factor, an information processing apparatus that changes the excretion assistance timing compared to when the behavior is determined not to be the abnormal behavior due to the dementia factor.

5. In any one of Claims 1 to 4, The support information output unit When the behavior is determined to be the dementia factor, an information processing apparatus that increases the types of the sensor information used compared to when the abnormal behavior is not determined.

6. In Claim 5, The support information output unit An information processing apparatus that determines whether it is necessary to add a new sensor based on information for specifying one or more available sensors and the sensor information added when the behavior is determined to be the dementia factor.

7. When it is determined that the behavior of the person in need of assistance is abnormal behavior, based on (1) the dementia level information of the person in need of assistance and (2) at least one of the environmental information, excretion information, and sleep information of the person in need of assistance, determine whether the abnormal behavior is a dementia factor and whether it is a sleep disorder factor. Output support information for supporting the assistance of the person in need of assistance by the caregiver based on the determination result and the sensor information, which is the sensing result regarding the caregiver who assists the person in need of assistance or the person in need of assistance. Information processing method.

Images