Information processing apparatus and information processing method

Abstract

To provide an information processing device and an information processing method, etc. for properly estimating a main factor of peripheral symptoms.SOLUTION: An information processing device includes: an acquisition part for acquiring input data including the sensing data acquired using a sensor; and a factor estimation part for estimating, on the basis of the input data, the main factor causing a state in which a care recipient has been put in a state where a peripheral symptom of dementia is observed, is any one of a plurality of factors including an environmental factor caused by surrounding environment of the care recipient, a first core factor caused by the executive function disorder of the care recipient, a second core factor caused by the disorientation of the care recipient and a psychological factor caused by the psychological state of the care recipient.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and the like.

Background Art

[0002] Conventionally, a system used in a scene where a caregiver assists a care recipient is known. Patent Document 1 discloses a method of arranging sensors in a living space and generating provision information regarding the state of a resident living in the living space based on the time change of detection information acquired by the sensors.

[0003] Also, Patent Document 2 discloses a method of evaluating an instability risk based on biological information acquired from a sensor worn by a user and notifying a countermeasure method.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Provided are an information processing apparatus, an information processing method, and the like that appropriately support a caregiver in assisting a care recipient.

Means for Solving the Problems

[0006] One aspect of the present disclosure relates to an information processing apparatus including: an acquisition unit that acquires input data including sensing data acquired using a sensor; and a factor estimation unit that estimates which of a plurality of factors including an environmental factor resulting from the surrounding environment of the care recipient, a first core factor resulting from the executive function disorder of the care recipient, a second core factor resulting from the disorientation disorder of the care recipient, and a psychological factor resulting from the psychology of the care recipient is the main factor that has caused a peripheral symptom state in which the care recipient receiving care from a caregiver shows peripheral symptoms of dementia, based on the input data.

[0007] Another aspect of the present disclosure relates to an information processing method of acquiring input data including sensing data acquired using a sensor, and estimating which of a plurality of factors including an environmental factor resulting from the surrounding environment of the care recipient, a first core factor resulting from the executive function disorder of the care recipient, a second core factor resulting from the disorientation disorder of the care recipient, and a psychological factor resulting from the psychology of the care recipient is the factor that has caused a peripheral symptom state in which the care recipient receiving care from a caregiver shows peripheral symptoms of dementia, based on the input data.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17A

Figure 17B

Figure 17C

Figure 17D

Figure 17E

Figure 17F

Figure 17G

Figure 18

Figure 19

Figure 20A

Figure 20B

Mode for Carrying Out the Invention

[0009] Hereinafter, this embodiment will be described with reference to the drawings. Regarding the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted. Note that the embodiment described below does not unduly limit the content described in the claims. Also, not all of the configurations described in this embodiment are essential constituent elements of the present disclosure.

[0010] 1. System configuration example FIG. 1 is a configuration example of an information processing apparatus 20 according to this embodiment. The information processing apparatus 20 includes an acquisition unit 21 and a factor estimation unit 22. Details of each part of the information processing apparatus 20 will be described later. The information processing apparatus 20 in this embodiment outputs information for supporting assistance to an assistant who assists a person to be assisted, for example, in a medical facility or a nursing facility.

[0011] The assistant here may be a nursing staff member in a nursing facility, or a nurse or a practical nurse in a medical facility such as a hospital. That is, the assistance in this embodiment includes various actions for supporting the person to be assisted, and may include nursing care or actions related to medical treatment such as injections. Also, the person to be assisted here is a person who receives assistance from the assistant, and may be a resident of a nursing facility or a patient who is hospitalized or visits a hospital. Further, the person to be assisted may be, for example, a person to be assisted who may have dementia.

[0012] Also, the assistance in this embodiment may be performed at home. For example, the person to be assisted in this embodiment may be a person requiring home care who receives home care, or a patient who receives home medical treatment. Also, the assistant may be a family member of a person requiring home care or a patient, etc., or a visiting helper, etc.

[0013] Figure 2 is a schematic diagram explaining the symptoms and causes in dementia. For dementia patients receiving assistance, various symptoms called peripheral symptoms can be observed. The peripheral symptoms here include restless behavior and delirium. Restless behavior refers to a state where the behavior is excessive and restless. Delirium is a disorder of mental function accompanied by a decline in attention and thinking ability. More specifically, the peripheral symptoms may include various symptoms such as delusions, depression, insomnia, excitement, wandering, hallucinations, etc., as shown in Figure 2. Also, the peripheral symptoms are not limited to the examples shown in Figure 2 and may include various symptoms not seen in the normal state, such as anxiety, misidentification, hyperactivity, unclean behavior, abusive language, violence, etc. Moreover, the peripheral symptoms include mental symptoms and behavioral symptoms and are also called BPSD (Behavioral and Psychological Symptom of Dementia). In the above examples, depression, delusions, hallucinations, etc. correspond to mental symptoms, and wandering, violence, etc. correspond to behavioral symptoms. The peripheral symptoms not only reduce the quality of life of the patient receiving assistance but also increase the burden on the caregiver.

[0014] In dementia, the symptoms directly caused by the decline in brain function are called core symptoms. As shown in Figure 2, the core symptoms include executive function disorder, disorientation, memory disorder, aphasia, apraxia, etc.

[0015] Executive function disorder refers to a disorder that makes it difficult to perform things in an orderly manner. When suffering from executive function disorder, for example, when performing a series of actions combining multiple actions, even if each action can be performed, it becomes difficult to efficiently proceed with multiple actions.

[0016] Disorientation refers to a disorder that makes it difficult to grasp the situation one is in. When suffering from disorientation, for example, the patient receiving assistance cannot grasp what time it is now, where they are, what they are doing, etc.

[0017] Memory impairment refers to a disorder in which it becomes difficult to remember new things or to forget things that were previously remembered. Aphasia refers to a state in which words cannot be understood or thoughts cannot be expressed as words. Apraxia refers to a state in which actions that were routinely performed can no longer be carried out.

[0018] When peripheral symptoms are observed in a dementia patient, these core symptoms may be the cause. However, as shown in FIG. 2, it is known that peripheral symptoms occur as a result of the interaction between core symptoms and factors such as the environment and psychology of the care recipient, and it has not been easy to identify the main factor.

[0019] Therefore, in the method of the present embodiment, information related to at least one of the care recipient who may have dementia and the caregiver who cares for the care recipient is acquired as input data, and a process of estimating the main factor of the peripheral symptoms is performed based on the input data.

[0020] As shown in FIG. 1, the information processing apparatus 20 according to the present embodiment includes an acquisition unit 21 and a factor estimation unit 22. The acquisition unit 21 acquires input data including sensing data which is data acquired using sensors. The factor estimation unit 22 estimates which of a plurality of factors is the main factor for the care recipient who receives care from the caregiver to be in a peripheral symptom state in which peripheral symptoms of dementia are observed. Here, the plurality of factors include environmental factors resulting from the care recipient's surrounding environment, a first core factor resulting from the care recipient's executive function disorder, a second core factor resulting from the care recipient's disorientation disorder, and psychological factors resulting from the care recipient's psychology. Note that the configuration of the information processing apparatus 20 is not limited to that in FIG. 1, and various modifications such as adding other configurations or omitting some configurations are possible. Also, the same applies to FIGS. 3 to 5 and the like described later in terms of the possibility of modifications such as omitting or adding configurations.

[0021] According to the method of this embodiment, when peripheral symptoms are observed in the care recipient, the main cause can be appropriately estimated. As described above, since peripheral symptoms occur with multiple factors related to each other, it has not been easy for the caregiver to determine the main cause. However, according to the method of this embodiment, it is possible to appropriately identify the main cause. For example, since it becomes possible to prompt the caregiver to take measures according to the main cause, it becomes possible to improve the quality of life of the care recipient with dementia and to reduce the burden on the caregiver. Note that in the method of this embodiment, the cause estimation unit 22 may estimate whether the care recipient is in a peripheral symptom state based on the input data. In this way, it also becomes possible to execute the determination of whether or not peripheral symptoms are observed in the information processing apparatus 20.

[0022] Hereinafter, an example of the information processing system 10 including the information processing apparatus 20 will be described with reference to FIGS. 3 to 5.

[0023] FIG. 3 is a diagram showing a configuration example of the information processing system 10. As shown in FIG. 3, the information processing system 10 includes a server system 100, a terminal device 200, a management terminal device 300, and a sensing device 400. However, the configuration of the information processing system 10 is not limited to FIG. 3. For example, in FIG. 3, as the sensing device 400, a bedside sensor 420, a detection device 430, and a swallowing mucus detection device 460 are illustrated. However, as will be described later, other devices may be used as the sensing device 400. For example, as the sensing device 400, the devices described later with reference to FIGS. 6 to 10 may be used. In the following, when it is not necessary to distinguish between a plurality of sensing devices 400 from each other, they are simply referred to as the sensing device 400.

[0024] The information processing apparatus 20 of this embodiment corresponds to, for example, the server system 100. However, the method of this embodiment is not limited to this, and the processing of the information processing apparatus 20 may be executed by distributed processing using the server system 100 and other devices. For example, the information processing apparatus 20 of this embodiment may include the server system 100 and the terminal device 200. Hereinafter, an example in which the information processing apparatus 20 is the server system 100 will be described.

[0025] The server system 100 is connected to the terminal device 200, the management terminal device 300, and the sensing device 400 via a network, for example. The network here is a public communication network such as the Internet, for example. However, the network is not limited to a public communication network and may be a LAN (Local Area Network) or the like. For example, the server system 100 may perform communication according to the IEEE802.11 standard. However, various modifications can be made to the communication method between the devices.

[0026] The server system 100 may be a single server or may include a plurality of servers. For example, the server system 100 may include a database server and an application server. The database server stores various data to be described later with reference to FIG. 4. The application server performs the processes to be described later with reference to FIGS. 13 to 14 and the like. Here, the plurality of servers may be physical servers or virtual servers. When virtual servers are used, the virtual servers may be provided on one physical server or may be distributed and arranged on a plurality of physical servers. As described above, the specific configuration of the server system 100 in the present embodiment can be variously modified.

[0027] The terminal device 200 is a device used, for example, by an assistant who assists a person in need of assistance. The terminal device 200 here is a portable terminal device such as a smartphone or a tablet terminal, for example. However, the terminal device 200 may be other devices such as a PC (Personal Computer), a headset, or a wearable device such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. Also, one assistant may use a plurality of terminal devices 200. For example, the assistant may use both a smartphone and a headset.

[0028] The management terminal device 300 is a device used to manage information of care recipients who are residents in, for example, a nursing facility or the like. The management terminal device 300 is, for example, a PC, but other devices may also be used. The management terminal device 300 has, for example, care software installed therein and performs management of care recipients and schedule management of caregivers (staff in a nursing facility), etc. For example, the management terminal device 300 stores information regarding the attributes of care recipients. The attributes here include age, gender, height, weight, medical history, medication history, etc.

[0029] The sensing device 400 has various sensors and acquires sensing data based on the sensors. The sensing data in the following may be the sensor output itself or information obtained by arithmetic processing based on the sensor output.

[0030] In the information processing system 10 shown in FIG. 3, for example, the management terminal device 300 transmits information regarding care recipients to the server system 100. Also, the sensing device 400 transmits sensing data to the server system 100. The server system 100 performs a process of estimating the presence or absence of peripheral symptoms and the main factors when peripheral symptoms are observed, using the data transmitted from the management terminal device 300 and the sensing device 400 as input data. The server system 100 transmits information to the terminal device 200 based on the estimated main factors. For example, the server system 100 may transmit information representing the main factors or information representing specific countermeasures obtained based on the main factors. Also, the transmission destination of the information by the server system 100 is not limited to the terminal device 200. For example, the server system 100 may transmit the estimation result of the main factors to the sensing device 400 or other controlled devices (not shown in FIG. 3). The sensing device 400 and the controlled devices execute a process of changing, for example, the operation mode based on the transmitted main factors. Details of the process will be described later.

[0031] FIG. 4 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit 130.

[0032] The processing unit 110 of the present embodiment is configured by the following hardware. The hardware can include at least one of a circuit that processes digital signals and a circuit that processes analog signals. For example, the hardware can be configured by one or more circuit devices mounted on a circuit board or one or more circuit elements. The one or more circuit devices are, for example, IC (Integrated Circuit), FPGA (field-programmable gate array), etc. The one or more circuit elements are, for example, resistors, capacitors, etc.

[0033] Also, the processing unit 110 may be implemented by the following processors. The server system 100 of this embodiment includes a memory for storing information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various types of data, etc. The memory may be the storage unit 120 or another memory. The processor includes hardware. The processor can use various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor). The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, or a register, or a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores instructions readable by a computer, and when the processor executes the instructions, the functions of the processing unit 110 are realized as processing. The instructions here may be instructions in the instruction set that constitutes a program or instructions that instruct the hardware circuit of the processor to operate.

[0034] The processing unit 110 includes, for example, an acquisition unit 111, a cause estimation unit 112, a countermeasure determination unit 113, a communication processing unit 114, a learning unit 115, and a presentation processing unit 116.

[0035] The acquisition unit 111 acquires input data for identifying the main cause when peripheral symptoms are observed in the assisted person. For example, the acquisition unit 111 acquires input data from the management terminal device 300 or the sensing device 400 via the communication unit 130. Details of the input data will be described later.

[0036] Based on the input data, the main cause estimation unit 112 performs a process of estimating the main cause. The server system 100 of the present embodiment may store, for example, information associating the input data with the main cause. The information associating the input data with the main cause is, for example, the learned model 124. However, the information associating the input data with the main cause may be table data or an algorithm for obtaining the main cause based on the input data. The main cause estimation unit 112 estimates the main cause based on the information associating the input data with the main cause and the input data acquired by the acquisition unit 111.

[0037] Based on the estimated main cause, the countermeasure determination unit 113 obtains countermeasure information representing a countermeasure recommended for the peripheral symptoms of the care recipient. The server system 100 of the present embodiment may store, for example, a countermeasure table 125 associating the main cause with the countermeasure information. The countermeasure determination unit 113 obtains a countermeasure corresponding to the main cause based on the estimated main cause and the countermeasure table 125.

[0038] The communication processing unit 114 controls communication using the communication unit 130. For example, the communication processing unit 114 executes a process of creating data to be transmitted, such as a MAC frame in the data link layer. Also, the communication processing unit 114 may perform a process of interpreting the frame structure and the like on the data received by the communication unit 130, extracting necessary data, and outputting it to an upper layer such as an application.

[0039] The learning unit 115 performs a process of obtaining the learned model 124 by performing machine learning based on the training data. The machine learning here may use a neural network (hereinafter referred to as NN), an SVM, or other learning methods. Also, the learning unit 115 may perform a process of updating the existing learned model 124 by performing machine learning based on the training data for update.

[0040] The prompting processing unit 116 performs processing for prompting the information required in the server system 100 on other devices such as the terminal device 200. For example, the prompting processing unit 116 may perform processing of transmitting information representing the main cause and information representing the recommended countermeasures to the terminal device 200 or the like via the communication unit 130. Further, the prompting processing unit 116 may transmit information specifying the prompting mode in the terminal device 200 or the like. The prompting here may be display of an image or text, output of voice, or prompting using light emission, vibration, or the like.

[0041] The storage unit 120 is a work area of the processing unit 110 and stores various information. The storage unit 120 can be realized by various memories, and the memory may be a semiconductor memory such as SRAM, DRAM, ROM, or flash memory, a register, a magnetic storage device, or an optical storage device.

[0042] The storage unit 120 may store user information 121, device information 122, log data 123, learned model 124, and countermeasure table 125.

[0043] The user information 121 is information for managing the users of the information processing system 10 and includes information such as a user ID and a user name that uniquely identify the user. The users here include both the assisted person and the caregiver. Also, the users here may include the relatives of the assisted person. The relatives are persons who have daily contact with the assisted person and may include family members, close friends, and facility-related persons.

[0044] Device information 122 is information for managing various devices included in the information processing system 10, and includes a device ID that uniquely identifies a device, a device name, a device type ID representing the type of the device, a vendor, etc. Here, the device may be the terminal device 200, the management terminal device 300, or the sensing device 400. Further, the device information 122 may include a user ID that identifies a user who uses the target device. The user of the terminal device 200 is, for example, a caregiver. The user of the sensing device 400 may be the care recipient, the caregiver in charge of the care recipient, or both. Further, the device information 122 may include information on a facility where the sensing device 400 is used, etc.

[0045] Log data 123 is a log of the input data acquired by the acquisition unit 111. For example, the log data 123 is information in which time-series input data and the acquisition timing of the input data are associated with each other. Further, the log data 123 may include information identifying the device that is the source of the input data, the target care recipient, etc.

[0046] The learned model 124 is information acquired by the learning process of the learning unit 115, and is a model that identifies the main factors of peripheral symptoms based on input data. Details of the learned model 124 will be described later.

[0047] The countermeasure table 125 is information in which main factors and recommended countermeasures are associated with each other. Specific associations will be described later.

[0048] The communication unit 130 is an interface for performing communication via a network. When the server system 100 performs wireless communication, it includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. However, the server system 100 may perform wired communication. In this case, the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 operates according to the control by the communication processing unit 114. However, it is not precluded that the communication unit 130 includes a processor for communication control different from the communication processing unit 114. The communication unit 130 may perform communication according to a method defined in, for example, IEEE802.11 or IEEE802.3. However, various modifications of the specific communication method are possible.

[0049] FIG. 5 is a block diagram showing a detailed configuration example of the terminal device 200. The terminal device 200 includes, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250. The configuration of the terminal device 200 is not limited to FIG. 5, and modifications such as omitting some configurations and adding other configurations are possible. For example, the terminal device 200 may have various sensors according to the terminal device 200, such as a motion sensor such as an acceleration sensor or a gyro sensor, an imaging sensor, a pressure sensor, and a GPS (Global Positioning System) sensor. Also, as described above, the terminal device 200 can use devices in various modes and may have a configuration specific to the device not shown in FIG. 5.

[0050] The processing unit 210 is constituted by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. Also, the processing unit 210 may be realized by a processor. The processor can use various processors such as a CPU, a GPU, and a DSP. The function of the processing unit 210 is realized as processing by the processor executing instructions stored in the memory of the terminal device 200.

[0051] The memory unit 220 is a work area of the processing unit 210 and is implemented by various memories such as SRAM, DRAM, and ROM.

[0052] The communication unit 230 is an interface for performing communication via a network and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100 via, for example, a network. The communication unit 230 may perform wireless communication compliant with the IEEE802.11 standard with the server system 100.

[0053] The display unit 240 is an interface for displaying various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface for receiving user operations. The operation unit 250 may be a button or the like provided in the terminal device 200. Also, the display unit 240 and the operation unit 250 may be a touch panel configured integrally.

[0054] Also, the terminal device 200 may include components not shown in FIG. 5, such as a light emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light emitting unit is, for example, an LED (light emitting diode) and performs notification by light emission. The vibration unit is, for example, a motor and performs notification by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and performs notification by sound.

[0055] Also, part or all of the processing performed by the information processing apparatus 20 of the present embodiment may be realized by a program. The processing performed by the information processing apparatus 20 is, for example, the processing performed by the processing unit 110 of the server system 100.

[0056] The program according to this embodiment can be stored in a non-transitory information storage device (information storage medium), which is a computer-readable medium, for example. The information storage device can be realized by, for example, an optical disk, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 etc. perform various processes of this embodiment based on the program stored in the information storage device. That is, the information storage device stores a program for causing a computer to function as the processing unit 110 etc. A computer is a device including an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for causing a computer to execute each step described later using FIGS. 13 to 14 etc.

[0057] Also, the method of this embodiment can be applied to an information processing method including the following steps. The information processing method includes a step of acquiring input data including sensing data acquired using a sensor, and a step of estimating which of a plurality of factors is the factor that has caused the state of the care recipient who receives assistance from the caregiver to be in a peripheral symptom state in which peripheral symptoms of dementia are observed. As described above, the plurality of factors include an environmental factor caused by the peripheral environment of the care recipient, a first core factor caused by the executive function disorder of the care recipient, a second core factor caused by the disorientation disorder of the care recipient, and a psychological factor caused by the psychology of the care recipient.

[0058] 2. Details of the processing Next, the processing according to this embodiment will be described in detail. First, for each of the plurality of factors, examples of data related to the factor will be described. The data related to the factor is used, for example, as a part of the input data. Then, the processing in the server system 100 will be described in detail. Furthermore, taking meal assistance as an example, a specific example of a countermeasure for adding a device to be used will be described. Also, an example of the data structure when transmitting the estimated main factor to the terminal device 200 or other devices will be described.

[0059] 2.1 Examples of data related to factors 2.1.1 Environmental factors As described above with reference to FIG. 2, the environment around the care recipient is known as one of the factors of peripheral symptoms. Data related to environmental factors includes data identifying the environment around the care recipient. Hereinafter, data related to environmental factors is referred to as environmental information. Environmental information includes information regarding brightness, temperature, humidity, smell, sound, video, and season.

[0060] For example, information representing brightness may be information regarding lighting equipment arranged in the living environment of the care recipient. The information on lighting equipment may be brightness information expressed in units such as lumens, or may be color temperature information. It may also be information specifying the time when the lighting is turned on during the day. Such information may be automatically acquired from, for example, the operation log of the lighting equipment, or may be input by a caregiver or the like.

[0061] Also, information representing temperature and humidity may be acquired using sensors such as a thermometer and a hygrometer. Also, information representing temperature and humidity may be information regarding air conditioning equipment arranged in the living environment of the care recipient. When the air conditioning equipment includes a temperature sensor and a humidity sensor, information representing temperature and humidity may be acquired based on the output of the sensors. Also, information representing temperature and humidity may be information such as the set temperature of the air conditioning equipment, the operation mode (cooling, heating, dehumidifying, etc.), and the operation time. Such information may be automatically acquired from, for example, the operation log of the air conditioning equipment, or may be input by a caregiver or the like.

[0062] Information representing smell is, for example, information regarding the smell present around the care recipient, and includes information such as the type of smell and the duration for which the smell is felt. For example, when flowers or air fresheners are arranged in the care recipient's living room, the type of smell may be specified based on the variety of flowers, the manufacturer and product model number of the air freshener, etc. Also, information regarding detergents and fabric softeners may be used as information representing the smell of the care recipient's clothing. Also, information representing smell may include information representing the body odor of the care recipient himself or herself or the caregiver. Information representing smell may be automatically acquired based on the output of an odor sensor, or may be input by a caregiver or the like.

[0063] The information representing sound includes information representing the type of music output from a music player or the like, the time zone when the music is output, and the like. The type of music may be information representing a genre such as pop or classical. The information representing music may be automatically acquired based on the operation log of the music player, or may be input by an assistant or the like. Also, the information representing sound may be information acquired using a sound collection device such as a microphone. For example, information such as the frequency, magnitude, and duration of the sound may be used as the information representing the sound.

[0064] The information representing video includes information representing the type of video displayed using a display device such as a television, the time zone when the video is output, and the like. The type of video may be information representing a genre such as a movie, drama, news, etc., and may include more detailed genres such as an action movie or a comedy movie. The information representing video may be automatically acquired based on the operation log of the television, or may be input by an assistant or the like.

[0065] The information representing the season is, for example, information specifying which of spring, summer, autumn, or winter the current season corresponds to. For example, the information representing the season may be 2-bit data that discriminates among the four states of spring, summer, autumn, and winter. Also, the information representing the season may include information as to whether it is the time of the changing of the seasons, or may include information on a specific date. The information representing the season may be automatically acquired using calendar software or the like operating in the management terminal device 300 or the like, or may be input by an assistant or the like.

[0066] 2.1.2 The First Core Factor (Executive Function Disorder) As described above, executive function disorder, apraxia, etc. are known as core symptoms, and executive function disorder and apraxia represent a state in which it becomes difficult to perform daily behaviors. Therefore, the data related to the first core factor may be data representing the ability to perform daily activities. The data representing the ability here may be an index value of ADL (Activities of Daily Living). Also, the ADL here may be iADL (Instrumental Activities of Daily Living).

[0067] For example, for care recipients who exhibit executive functional impairments or incontinence, daily activities such as getting up, eating, and excretion become difficult to perform properly. Here, getting up includes, for example, rising to a standing position or getting out of bed. In this embodiment, sensing data from sensing devices 400 such as a sensing device 400 for detecting getting up, a sensing device 400 for detecting eating, and a sensing device 400 for detecting incontinence may be output to the server system 100 as data related to the first core factor. Hereinafter, specific examples of each sensing device 400 will be described.

[0068] <Getting up> FIG. 6 is a diagram showing an example of an imaging device 410 which is a sensing device 400 for detecting getting up, and an example of an output image IM1 of the imaging device 410. The imaging device 410 has an image sensor that outputs a captured image as sensing data. The imaging device 410 may be arranged in a place where a large number of people gather and move around, such as a living area or a hall in a nursing facility. In the example of FIG. 6, the imaging device 410 is arranged on top of a television set. The place where the imaging device 410 is arranged is not limited to this, and it may be arranged in other places such as a living room. For example, although a bedside sensor 420, a detection device 430, and getting up movements in bed are detected using FIG. 7, an imaging device 410 arranged in a living room may be used instead of these sensing devices 400.

[0069] The imaging device 410 may perform a process of detecting the start of movement of a person based on the captured image. For example, by operating according to an application installed in the imaging device 410, the imaging device 410 acquires the captured image as input data, and executes a process of detecting a person from the captured image and a process of determining whether there is a start of movement of the detected person. For example, the imaging device 410 outputs the time-series captured images to the server system 100 as sensing data. Alternatively, the imaging device 410 may determine the presence or absence of a start of movement by executing the processes described below based on the time-series captured images, and output the determination result to the server system 100 as sensing data. Further, the imaging device 410 may determine the presence or absence of a start of movement, and when a start of movement is detected, output the captured images for a predetermined period before and after the start of movement to the server system 100 as sensing data. In this case, information particularly useful for determining the starting-up ability can be output as sensing data. In addition, various modifications are possible for specific examples of the sensing data.

[0070] For example, the imaging device 410 performs a face recognition process of recognizing the face of a person based on the captured image. For example, the imaging device 410 stores a face image of a person to be detected, and may perform a face recognition process based on a matching process using the face image as a template. Also, various face recognition methods are known, and they can be widely applied in this embodiment. For example, when the movement of the detected face region is in a state where it is equal to or less than a given threshold for a certain period of time, the imaging device 410 sets the position of the face region in this state as a reference position. Then, the imaging device 410 sets a detection region at a position a predetermined distance away from the reference position, and when the face region reaches the detection region, it may be determined that there is a start of movement. For example, when a starting-up operation is performed, since it is assumed that the position of the face moves relatively upward, the detection region may be a region set at a position a predetermined distance above the reference position. In this case, when the position of the face region on the image moves upward by a predetermined distance or more with respect to the reference position, a start of movement is detected. Here, the detection region is, for example, a linear region, but other shaped regions may be set.

[0071] In addition, the imaging device 410 can identify the target care recipient through face recognition processing. Therefore, the imaging device 410 may perform movement detection for a specific care recipient and omit movement detection for other care recipients. The specific care recipient here may be, for example, a care recipient suspected of having dementia or a care recipient whose ADL index value is below a predetermined level. For example, the imaging device 410 may perform movement detection for a care recipient with a high risk of falling when getting up.

[0072] Also, the movement detection process is not limited to the above method. For example, the imaging device 410 may perform skeleton tracking processing based on the captured image. As a method of skeleton tracking based on an image, various methods are known, such as "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" (https: / / arxiv.org / pdf / 1611.08050.pdf), OpenPose disclosed by Zhe Cao et al., and these can be widely applied in this embodiment.

[0073] Also, OpenPose discloses a method of performing skeleton tracking for each of a plurality of people captured in an image and displaying the results. In the example of FIG. 6, the image sensor outputs a captured image including three care recipients, and the imaging device 410 determines the presence or absence of movement for each of the three care recipients.

[0074] For example, a care recipient whose ability has declined and who has difficulty getting around may fall even when taking a standing-up posture. Therefore, the imaging device 410 may determine whether the care recipient is taking a standing-up posture by skeleton tracking. For example, when the imaging device 410 determines that the care recipient has bent forward with their hands on their knees or the seat surface of a chair from a sitting state, it may determine that the care recipient is in a standing-up posture and notify the caregiver of the fall risk. For example, when the imaging device 410 detects that the distance between the position of the hand and the position of the knee is equal to or less than a predetermined value, or that the position of the shoulder has moved downward by a predetermined value or more, from the skeleton tracking result, it may determine that the care recipient is taking a standing-up posture.

[0075] Alternatively, the imaging device 410 may divide the processing target data into windows in units of several seconds, and determine that a posture change such as standing up has occurred when a specific position such as the head or neck has moved by a predetermined threshold or more within each window. Note that the part to be detected for movement may be other than the head or neck. Also, the movement direction may be vertical, horizontal, or diagonal. Also, the threshold used for detection may be changed according to the part to be detected. Alternatively, the imaging device 410 may obtain a region that encloses the feature points detected by skeleton tracking for a stationary care recipient, and determine that there has been a starting movement such as standing up when a predetermined number or more of feature points have moved outside the region. In addition, various modifications of the method for detecting the starting movement using the imaging device 410 are possible.

[0076] IM1 in FIG. 6 is an example of an output image output using the imaging device 410. Hereinafter, an example in which the imaging device 410 generates an output image will be described, but the server system 100 may generate the output image IM1 based on the sensing data.

[0077] For example, the imaging device 410 may superimpose and display some display object on the captured image. In the example of FIG. 6, an object including an "!" mark is displayed in association with the assisted person whose movement is detected. In this way, it becomes possible to easily notify the caregiver of the assisted person whose movement is detected. For example, the imaging device 410 may transmit the output image IM1 to the server system 100 as part of the sensing data. The server system 100 outputs the output image IM1 to the terminal device 200 used by the caregiver. However, the output of the imaging device 410 may be information for identifying the assisted person whose movement is detected (for example, the ID of the assisted person), and various modifications of the specific mode are possible. For example, although an example of notifying the assisted person whose movement is detected is shown here, information for stopping the movement of the assisted person may be output. For example, the imaging device 410 may identify the assisted person who has started moving and output voice data, video data, etc. of the family members, etc. of the assisted person. Particularly in the case of dementia patients, the response to calls becomes dull, but in many cases, they remember the faces and voices of family members, etc., which is effective for stopping the movement. By stopping the movement of the assisted person in this way, it is possible to gain time until the caregiver intervenes.

[0078] FIG. 7 is a diagram for explaining an example of the bedside sensor 420 and the detection device 430 arranged at the bottom of the bed 610. The bedside sensor 420 and the detection device 430 are sheet-shaped or plate-shaped devices provided, for example, as shown in FIG. 7, between the bottom of the bed 610 and the mattress 620.

[0079] The bedside sensor 420 includes a pressure sensor that outputs a pressure value as sensing data, and is disposed on the bottom side where the caregiver uses it to get on and off the bed. In the example of FIG. 7, the caregiver gets on and off the bed using the front side of the bed 610. At this time, as shown in FIG. 7, a fall prevention fence is disposed on the front side of the bed 610, and the bedside sensor 420 may be disposed at a position where the fence is not provided. In this way, the user getting on and off the bed 610 once performs an operation of sitting on the bedside sensor 420. The bedside sensor 420 may output time-series pressure data as sensing data to the server system 100. Alternatively, the bedside sensor 420 may determine the presence or absence of a start of movement by executing the process described below, and output the determination result to the server system 100 as sensing data.

[0080] The bedside sensor 420 operates according to, for example, an application installed in the bedside sensor 420 to acquire a pressure value as input data, and executes a process of determining the movement of the care recipient on the bed 610 from the pressure value.

[0081] For example, when the care recipient stands up from the bed 610, the care recipient shifts from a lying position on the bed to a sitting position on the bedside (hereinafter referred to as an end sitting position), and further applies force by putting hands on the knees and the bottom surface to perform a standing up operation. It is assumed. The pressure value detected by the bedside sensor 420 increases in the order of the lying position, the end sitting position, and the standing up operation. For example, the bedside sensor 420 may determine that a start of movement is detected when a change from the end sitting position to the standing up operation is detected based on a comparison process between the pressure value and a given threshold value. Alternatively, from the viewpoint of detecting the standing up operation at a faster stage, the bedside sensor 420 may determine that a start of movement is detected when a change from the lying position to the end sitting position is detected based on a comparison process between the pressure value and a given threshold value.

[0082] Alternatively, when the rising operation continues, the buttocks of the assisted person lift off the bottom surface, so the pressure value output from the pressure sensor decreases significantly. Therefore, the bedside sensor 420 may determine that a rising operation has been performed when the pressure value decreases to or below a second threshold value that is smaller than the first threshold value after increasing to or above the first threshold value based on the time-series change of the pressure value. In addition, various modifications can be made to the specific processing content of the start-of-movement determination.

[0083] Also, the detection device 430 shown in FIG. 7 is a sensing device 400 that senses information related to the sleep of the assisted person. The detection device 430 includes a pressure sensor that outputs a pressure value.

[0084] When the user lies down on the bed, the detection device 430 detects the body vibration (body movement, vibration) of the user via the mattress 620. Based on the body vibration detected by the detection device 430, information regarding the respiration rate, heart rate, activity level, posture, wakefulness / sleep, getting out of bed / being in bed is obtained. Also, the detection device 430 may determine non-REM sleep and REM sleep, and the depth of sleep. For example, by analyzing the periodicity of body movements, the respiration rate and heart rate may be calculated from the peak frequency. The analysis of periodicity is, for example, Fourier transform or the like. The respiration rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time is, for example, 1 minute. Also, by detecting body vibration per sampling unit time, the number of detected body vibrations may be calculated as the activity level. Also, when the user gets out of bed, since the pressure value detected is lower than when in bed, it is possible to determine getting out of bed / being in bed based on the pressure value and its time-series change.

[0085] For example, the detection device 430 may output the output of the pressure sensor to the server system 100 as sensing data. Alternatively, the detection device 430 may output the information regarding the respiration rate, heart rate, activity level, posture, wakefulness / sleep, getting out of bed / being in bed described above to the server system 100 as sensing data.

[0086] For example, when the assisted person transitions from the in-bed state to the out-of-bed state, the detection device 430 determines that there is movement. Also, from the perspective of detecting signs of movement at an earlier stage, the detection device 430 may determine that there is movement when the assisted person transitions from the sleeping state to the waking state. The detection device 430 may output information regarding waking / sleeping and out-of-bed / in-bed as sensing data to the server system 100, or may output the detection result of movement as sensing data to the server system 100.

[0087] <Meal> FIG. 8 is a diagram illustrating a swallowing muscle detection device 460, which is a sensing device 400 used in a meal scene. As shown in FIG. 8, the swallowing muscle detection device 460 includes a throat microphone 461 worn around the neck of the assisted person and a terminal device 462 having a camera. Note that instead of the terminal device 462, other devices having a camera may be used. The throat microphone 461 outputs voice data due to swallowing, coughing, etc. of the assisted person. The camera of the terminal device 462 outputs a captured image of the assisted person's meal situation. The terminal device 462 is, for example, a smartphone, a tablet PC, etc. placed on the table where the assisted person eats. The throat microphone 461 is connected to the terminal device 462 using Bluetooth (registered trademark), etc., and the terminal device 462 is connected to the server system 100 via a network. However, both the throat microphone 461 and the terminal device 462 may be directly connectable to the server system 100, and various modifications of the specific connection mode are possible.

[0088] The swallowing muscle detection device 460 outputs the time-series voice data detected by the throat microphone 461 and the time-series captured images captured by the camera of the terminal device 462 as sensing data to the server system 100. Alternatively, the swallowing muscle detection device 460 may obtain various information regarding the meal by executing the processes described below based on the voice data and the captured images, and output the information as sensing data to the server system 100.

[0089] The swallowing cough detection device 460 determines the cough and swallowing of the care recipient based on the voice data of the throat microphone 461. A device that detects swallowing using a microphone worn around the neck is described, for example, in U.S. Patent Application No. 16 / 276,768 filed on February 15, 2019, titled “Swallowing action measurement device and swallowing action support system”. This patent application is hereby incorporated by reference in its entirety into the present specification. The processor can detect the number of coughs, the time of coughs (occurrence time, duration, etc.), and whether swallowing has occurred based on the voice data.

[0090] Also, the camera of the terminal device 462 can detect the mouth, eyes of the care recipient, and utensils such as chopsticks and spoons used by the care recipient by imaging the care recipient from the front direction, as shown in FIG. 8 for example. Various methods for detecting these facial parts and objects based on image processing are known, and in this embodiment, known methods can be widely applied.

[0091] For example, the swallowing cough detection device 460 can determine whether the mouth of the care recipient is open, whether food is coming out of the mouth, and whether the food is being chewed based on the captured image of the camera. Also, the swallowing cough detection device 460 can determine whether the eyes of the care recipient are open based on the captured image of the camera. Further, the swallowing cough detection device 460 can determine whether chopsticks, spoons, etc. are near the tableware, whether the care recipient is holding them, and whether the food is being spilled based on the captured image of the camera.

[0092] In the method of this embodiment, based on this information, the situation regarding the swallowing and cough of the care recipient is estimated. For example, the swallowing cough detection device 460 may obtain information regarding the meal based on the detection results of coughs and swallowing and the determination result of the opening and closing of the mouth of the care recipient.

[0093] For example, the swallowing cough detection device 460 may determine whether coughing occurs frequently based on the number and duration of coughs, and output the determination result. For example, the swallowing cough detection device 460 may determine that coughing occurs frequently when the number of coughs per unit time exceeds a threshold value. In this way, the situation regarding coughing can be automatically determined.

[0094] Further, the swallowing cough detection device 460 may obtain the swallowing time from when the assisted person opens their mouth until swallowing based on the swallowing detection result and the determination result of the opening and closing of the assisted person's mouth. In this way, for example, when the number of swallows has decreased, it is possible to determine specific situations such as whether the action of putting food into the mouth itself has not been performed, or whether swallowing has not occurred after putting food into the mouth. For example, the swallowing cough detection device 460 may start counting up the timer when the mouth transitions from a closed state to an open state based on the captured image of the terminal device 462, and stop measuring the timer when swallowing is detected by the throat microphone 461. The time at the stop represents the swallowing time. In this way, it is possible to accurately determine whether the risk of aspiration during meals is high and whether the caregiver should perform some action.

[0095] Further, the swallowing cough detection device 460 may determine the pace of eating based on the swallowing time. Also, the swallowing cough detection device 460 may determine whether the swallowing time is long based on the change in the swallowing time during one meal (for example, the increase amount or ratio relative to the swallowing time at the beginning). Alternatively, the processor may obtain the average swallowing time for each of multiple meals for the same assisted person, and determine whether the swallowing time has become longer based on the change in the average swallowing time.

[0096] Moreover, by using the determination result of the opening and closing of the mouth based on the captured image of the terminal device 462, it is possible to determine whether the situation is such that the assisted person does not open their mouth even when the caregiver approaches with a spoon or the like. In this way, in a situation where the assisted person has difficulty opening their mouth, if the swallowing time becomes long, it can be presumed that the situation is such that choking has occurred because food remains in the mouth. Also, by using the captured image to determine whether food is coming out of the mouth, whether the food is being chewed, or the recognition result of the mouth, it is possible to determine whether the assisted person is in a situation where they cannot chew the food properly. For example, if the number of chewing times is normal but the swallowing time is long, it is presumed that the situation is such that the food cannot be chewed properly. Also, if it is determined using the captured image that the eyes are closed, it is possible to determine whether the assisted person seems sleepy.

[0097] Also, by performing recognition processing of chopsticks, spoons, etc. using the captured image, it may be determined whether the situation is such that the person is playing with food, unable to hold utensils, doing nothing, etc. For example, if an object such as a spoon overlaps with the hand of the assisted person but the time until the object is brought to the mouth is equal to or longer than a predetermined threshold value, it is determined that the person is unable to hold the eating utensil or is playing with food. Also, if an object such as a spoon does not overlap with the hand of the assisted person and the time during which the assisted person's line of sight is directed towards the meal is equal to or longer than a predetermined threshold value, it is determined that the person is just watching the meal without doing anything.

[0098] <Incontinence> FIG. 9 is an example of an incontinence detection device 450 which is an example of a sensing device 400 used for the detection of incontinence. The incontinence detection device 450 is placed, for example, on the mattress 620 of the bed 610.

[0099] The incontinence detection device 450 shown in FIG. 9 includes, for example, an odor sensor and detects whether the assisted person has had an incontinence episode. The odor sensor here may be of a semiconductor type, a crystal oscillator type, or another type. For example, the incontinence detection device 450 determines that incontinence has occurred when the numerical data representing the odor output by the odor sensor is equal to or greater than a given threshold value.

[0100] The incontinence detection device 450 may output the output data of the odor sensor as sensing data to the server system 100, or may determine the presence or absence of incontinence based on the output data and output the determination result as sensing data to the server system 100. Further, the incontinence detection device 450 may output the incontinence rate as sensing data. Here, the incontinence rate may be the ratio of the number of days on which incontinence is detected to the total number of days in the target period, or may be the ratio of the number of excretion times determined to be incontinence to the total number of excretion times in a unit period such as one day, or may be other ratios.

[0101] Moreover, the incontinence detection device 450 is not limited to those including an odor sensor. For example, the incontinence detection device 450 may include a capacitance sensor and may detect the presence or absence of incontinence based on the change in capacitance due to the presence or absence of liquid (urine). The incontinence detection device 450 outputs at least one of the output data of the capacitance sensor and the detection result of incontinence (which may be the presence or absence of incontinence or the incontinence rate) to the server system 100 as sensing data.

[0102] In the above example, for example, as shown in FIG. 9, incontinence is detected using the incontinence detection device 450 disposed on the mattress 620. Therefore, when the care recipient is not wearing a diaper, urine reaches the vicinity of the incontinence detection device 450, so it is possible to appropriately detect changes in odor and capacitance due to incontinence. On the other hand, when the care recipient is wearing a diaper, if the incontinence is such that there is leakage from the diaper, the change in odor and capacitance is large, but if the incontinence is an amount that can be absorbed within the diaper, the change in odor and capacitance is small, and it may be difficult to detect incontinence within the diaper.

[0103] Therefore, in this embodiment, the incontinence detection device 450 may change the content of the incontinence determination according to the state of the care recipient, specifically, according to whether the care recipient is wearing a diaper. For example, when the care recipient is not wearing a diaper, the incontinence detection device 450 sets a first threshold as the threshold for determining the presence or absence of incontinence, and when the care recipient is wearing a diaper, the incontinence detection device 450 sets a second threshold smaller than the first threshold as the threshold for determining the presence or absence of incontinence. In this way, since the sensitivity when wearing a diaper is increased, it becomes possible to accurately detect incontinence. Further, the determination regarding incontinence may be executed in the server system 100. For example, the processing unit 110 may perform determination using a plurality of different thresholds according to whether a diaper is worn.

[0104] Also, the incontinence detection device 450 shown in FIG. 9 is a device that detects incontinence on the bed 610, but the sensing device 400 used for detecting incontinence is not limited to this. For example, the sensing device 400 may include a capacitance sensor and be a portable device worn on the undergarment or diaper of the care recipient. In this way, it becomes possible to acquire sensing data related to incontinence even at locations other than the bed 610.

[0105] Further, in this embodiment, both the incontinence detection device 450 disposed on the bed and the portable device worn on the clothing of the care recipient may be used as the sensing device 400 related to incontinence. The incontinence detection device 450 and the server system 100 may obtain the determination of the presence or absence of incontinence, the incontinence rate, etc. by combining the sensing data from these sensing devices 400. Alternatively, when the care recipient is not wearing a diaper, the incontinence detection device 450 disposed on the bed is used, and when the care recipient is wearing a diaper, the sensing device 400 to be used may be switched according to the situation, such as using both the incontinence detection device 450 and the portable device.

[0106] <Other Modification Examples> Further, based on the sitting posture maintaining ability and walking ability, the estimation of the user's ADL and the like may be performed. The sitting posture maintaining ability represents the ability to maintain the sitting state. The walking ability represents the ability to walk without falling.

[0107] FIG. 10 is an example of a sensing device 400 that outputs sensing data for estimating the sitting posture maintaining ability or walking ability, and shows, for example, a seat surface sensor 440 disposed on the seat surface of a wheelchair 630. The seat surface sensor 440 includes a pressure sensor that outputs a pressure value, and based on the pressure value, determines which of a plurality of postures including normal, forward shift, and lateral shift is the posture (hereinafter also referred to as the sitting posture) when the care recipient sits on the wheelchair 630. Forward shift represents a state where the center of gravity of the user is shifted forward more than normal, and lateral shift represents a state where the center of gravity of the user is shifted to either the left or right more than normal. Both forward shift and lateral shift correspond to states where the risk of falling from the seat surface is relatively high. Note that the seat surface sensor 440 may be a sensor device disposed on a normal chair, or may be a sensor device that determines the posture of a user sitting on a bed or the like. Further, the seat surface sensor 440 may perform a fall possibility determination to determine whether there is a possibility that the care recipient will fall from the seat surface.

[0108] In the example of FIG. 10, four pressure sensors Se1 to Se4 are disposed on the back side of a cushion 441 disposed on the seat surface of the wheelchair 630. The pressure sensor Se1 is a sensor disposed in the front, the pressure sensor Se2 is a sensor disposed in the rear, the pressure sensor Se3 is a sensor disposed on the right, and the pressure sensor Se4 is a sensor disposed on the left. Here, the front, rear, left, and right represent the directions as seen from the care recipient when the care recipient is sitting on the wheelchair 630.

[0109] As shown in FIG. 10, the pressure sensors Se1 to Se4 are connected to a control box 442. The control box 442 includes a processor that controls the pressure sensors Se1 to Se4 and a memory that serves as a work area for the processor. The processor detects a pressure value by operating the pressure sensors Se1 to Se4.

[0110] The care recipient sitting in the wheelchair 630 may feel pain in the buttocks and may shift the position of the buttocks. For example, the state where the buttocks are shifted forward more than usual is called forward displacement, and the state where they are shifted laterally is called lateral displacement. Also, forward displacement and lateral displacement may occur simultaneously, causing the center of gravity to shift obliquely. By using the pressure sensors arranged on the cushion 441 as shown in FIG. 10, it is possible to appropriately detect changes in the position of the buttocks, so it becomes possible to accurately detect forward displacement and lateral displacement.

[0111] For example, first, the timing when the care recipient transfers to the wheelchair 630 and assumes a normal posture is set as the initial state. In the initial state, since the care recipient sits deeply on the seat surface of the wheelchair 630, it is assumed that the value of the rear pressure sensor Se2 is relatively large. On the other hand, when forward displacement occurs, since the position of the buttocks moves forward, the value of the front pressure sensor Se1 increases. For example, the processor of the control box 442 may determine that forward displacement has occurred when the value of the pressure sensor Se1 has increased by a predetermined amount or more compared to the initial state. Also, instead of using the value of the pressure sensor Se1 alone, processing may be performed using the relationship between the values of the pressure sensor Se2 and the pressure sensor Se1. For example, the difference in voltage values, which are the outputs of the pressure sensor Se2 and the pressure sensor Se1, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used. Also, when the value of the pressure sensor Se1 exceeds a certain threshold, it is determined that the care recipient is sitting on the wheelchair 630, and it may be determined that forward displacement has occurred only based on the change in the value of the pressure sensor Se2 without comparing it with the pressure sensor Se1.

[0112] Similarly, when lateral displacement occurs, the position of the buttocks moves in either the left or right direction. Thus, if it is a left displacement, the value of the pressure sensor Se4 increases, and if it is a right displacement, the value of the pressure sensor Se3 increases. Therefore, when the value of the pressure sensor Se4 increases by a predetermined amount or more compared to the initial state, the processor may determine that a left displacement has occurred, and when the value of the pressure sensor Se3 increases by a predetermined amount or more compared to the initial state, the processor may determine that a right displacement has occurred. Alternatively, the processor may determine right and left displacements using the relationship between the values of the pressure sensor Se4 and the pressure sensor Se3. Similar to the example of forward displacement, the difference in voltage values, which are the outputs of the pressure sensor Se4 and the pressure sensor Se3, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used.

[0113] The seat surface sensor 440 may output the pressure values, which are the outputs of the pressure sensors Se1 to Se4, to the server system 100 as sensing data, or may output the determination results of forward and lateral displacements, the determination result of the possibility of falling, etc. to the server system 100 as sensing data. Further, the control box 442 may include a light emitting unit or the like, and notification to the caregiver may be performed using the light emitting unit. In this way, since the change in the sitting posture in the wheelchair 630 or the like can be clearly notified to the caregiver, it is possible to suppress the fall of the care recipient.

[0114] Moreover, the sensing device 400 of this embodiment may include a device that outputs sensing data for estimating walking ability. The sensing device 400 here is, for example, a foot pressure sensor or an acceleration sensor. The foot pressure sensor may include, for example, pressure sensors provided at multiple locations on the insole. The sensing data of the foot pressure sensor may be the time-series change of the pressure value itself, or the time-series change of the center-of-gravity position, etc. Also, the acceleration sensor may be included in a smartphone or the like carried by the care recipient, or may be included in a wearable device (which may be, for example, a wristwatch-type device or a device attached to clothing or the skin) worn by the care recipient. The sensing data of the acceleration sensor may be the time-series change of the acceleration value itself, or information regarding the period or amplitude of the acceleration value. For example, for a care recipient with high walking ability, parameters such as the center-of-gravity position, the period, and the amplitude of the acceleration are stable, but as the walking ability decreases, the variation in these parameters increases due to foot entanglement, wobbling, falling, etc. Therefore, the sensing data of the foot pressure sensor and the acceleration sensor becomes information representing walking ability.

[0115] Also, a device for determining the dementia level based on the ratio of the number of long presses to the number of presses in the operation of an electronic device is described, for example, in PCT / JP2018 / 038075 entitled "Dementia Determination System". This patent application is hereby incorporated by reference in its entirety into the present specification. Similar to this method, the sensing device 400 of this embodiment may include a device that outputs, as sensing data, the long-press ratio of the device, etc.

[0116] 2.1.3 Second Core Factor (Spatial Disorientation) As described above, cognitive impairment is known as a core symptom. Cognitive impairment refers to a state in which it becomes difficult to recognize the situation (such as time and place) one is in. For example, care recipients with cognitive impairment tend to wander more at night. Also, care recipients with cognitive impairment may experience more incontinence because they do not know the location of the toilet. In addition, care recipients with cognitive impairment may move to a place different from the place where they should originally move because they do not know the location. For example, they may enter the living quarters or bed assigned to other care recipients, or may excrete in a place other than the toilet.

[0117] Therefore, in this embodiment, information including at least one of wandering information regarding the wandering of the care recipient and incontinence information regarding the incontinence of the care recipient may be transmitted to the server system 100 as data related to the second core factor. Note that the wandering here may be wandering at night. Also, the wandering may be moving to a place different from the place where one should originally move, regardless of day or night. Also, as described above, incontinence may occur due to moving to a place different from the place where one should originally move, so it is not precluded that a part of the wandering information overlaps with the incontinence information.

[0118] The detection device 430 shown in FIG. 7 can determine the in-bed / out-of-bed state as described above. Therefore, in this embodiment, the detection device 430 may be used as the sensing device 400 for obtaining wandering information. For example, the detection device 430 may transmit at least one of the output of the pressure sensor and the in-bed / out-of-bed result to the server system 100 as sensing data. Alternatively, by associating with time information, the timing of out-of-bed at night, the duration of out-of-bed, the number of out-of-bed times at night per unit time, etc. may be obtained. These out-of-bed at night can be used as wandering information representing wandering at night. The detection device 430 may transmit wandering information regarding out-of-bed at night to the server system 100 as sensing data. It is also possible to detect out-of-bed using the bedside sensor 420 in FIG. 7. Therefore, it is not precluded that the bedside sensor 420 is used as a device for obtaining wandering information.

[0119] As a device for acquiring wandering information, an RFID (radio frequency identifier) reader and an IC tag may be used. For example, RFID readers are installed in advance in rooms, beds, dining halls, toilets, etc. in a nursing facility, and IC tags are installed on the clothing of the care recipient or on terminal devices carried by the care recipient. In this way, based on the reading result of the RFID reader, it becomes possible to obtain, as wandering information, the history of which places such as a room, a dining hall, and a toilet the target care recipient has moved to. For example, the RFID reader may output the reading result as wandering information to the server system 100. Alternatively, in the RFID reader or another device connected to the RFID reader, it may be determined whether the care recipient has entered another person's room or bed, and the determination result may be output as wandering information. For example, the frequency (e.g., the number of times per month) that the care recipient has entered another person's room or bed may be used as wandering information.

[0120] Also, when excretion occurs outside the toilet, the location is often fixed to a certain extent. Therefore, an RFID reader may be installed in advance at a location where the target care recipient may misidentify as a toilet, and the reading result at that location or the determination result of whether the care recipient has moved to that location may be output as wandering information. For example, the frequency (e.g., the number of times per month) that the IC tag of the target care recipient is read by the target RFID reader may be used as wandering information or incontinence information.

[0121] Incontinence information may also be obtained using the incontinence detection device 450 described above with reference to FIG. 9, or a capacitance sensor or the like attached to the clothing of the care recipient. Similar to the example of the first core factor, the incontinence information here may be at least one of output data from a capacitance sensor or the like and a detection result of incontinence (which may be the presence or absence of incontinence or the incontinence rate). Further, the incontinence information may include the output of the detection device 430. For example, the incontinence information may be information including the number of times of getting out of bed and the time out of bed before and after incontinence is recorded, or the time-series changes thereof. In this way, it becomes possible to distinguish whether incontinence occurred in bed without noticing the urge to urinate, or whether incontinence occurred because the location of the toilet was unknown although the care recipient got out of bed.

[0122] Incontinence information may also include information regarding the location where incontinence occurred. Information regarding the location may be obtained, for example, using the RFID reader and IC tag described above, or by other methods. By including location information in the incontinence information, it becomes possible to distinguish whether incontinence occurred on the way to the toilet or in bed. In the case of incontinence on the way to the toilet, it is highly likely that the presence or absence of the urge to urinate and the place to excrete can be recognized, and the severity is relatively low. On the other hand, incontinence in bed may indicate that the urge to urinate was not recognized or that the current location is not recognized as the toilet, so the severity is relatively high. By including incontinence location information in the incontinence information, these cases can be distinguished. For example, when a given care recipient changes from a state where incontinence is likely to occur on the way to the toilet to a state where incontinence is likely to occur in bed, it can be determined that the condition of the care recipient may be deteriorating.

[0123] As described above, PCT / JP2018 / 038075 describes a device for determining the dementia level. In the present embodiment, as the sensing device 400 that outputs sensing data related to the second core factor, a device similar to this method may be included.

[0124] 2.1.4 Psychological factors In the case of dementia, when symptoms such as being unable to perform daily activities well, not being able to understand what has been heard, and not being able to remember occur, there is a possibility that the symptoms may worsen due to being scolded by family members or other caregivers. Also, when there is scolding from caregivers, there are cases where, fearing further scolding, the person tries to hide the fact that they were unable to do something well. That is, not only at the timing of being scolded, but also in the subsequent period, the care recipient may develop psychological anxiety and worsen the symptoms.

[0125] As described above, the psychological state of the care recipient is greatly related to the state of the caregivers who have daily contact with the care recipient. Therefore, in the present embodiment, the biometric information of the caregivers of the care recipient may be acquired as data related to the psychological factors of the care recipient. For example, a caregiver who becomes highly excited about the surrounding symptoms of the care recipient may experience an increase in heart rate, activity level, a decrease in sleep time due to stress, etc. Therefore, based on the biometric information of the caregivers of the care recipient, it is possible to estimate the presence or absence of a caregiver who affects the psychology of the care recipient.

[0126] The biometric information here is, for example, information such as the heart rate, activity level, and sleep time described above. The biometric information may also include other information representing the biological activities of the caregiver. The biometric information of the caregiver may be detected, for example, using the detection device 430 described above with reference to FIG. 7. Also, as a device for detecting biometric information such as heart rate, a method using a wearable device such as a wristwatch type including various sensors such as an acceleration sensor and a photoelectric sensor is known, and the biometric information of the caregiver may be acquired using these methods. For example, these devices may output the output of the sensor as biometric information to the server system 100, or may transmit the heart rate, etc. obtained based on the output of the sensor to the server system 100 as biometric information. Also, the determination as to whether the caregiver has become highly excited may be made based on whether the biometric information has changed significantly compared to the normal state. That is, the biometric information here may be data representing a time-series change.

[0127] 2.2 Details of Processing in the Server System Next, an example of the processing executed in the server system 100 will be described. Hereinafter, the learning process for creating the learned model 124, the inference process for estimating the main factors using the learned model 124, and the update process for updating the learned model 124 will be described respectively. Note that the method of the present embodiment is not limited to using the learned model 124, and the main factors may be specified by other processes using input data.

[0128] 2.2.1 Learning Process In the present embodiment, machine learning using a neural network (NN) may be performed. However, machine learning is not limited to NN, and other methods such as SVM (support vector machine) and k-means method may be used, or methods developed from these may be used. Also, in the following, supervised learning will be exemplified, but other machine learning such as unsupervised learning may be used.

[0129] FIG. 11 is a basic structural example of an NN. One circle in FIG. 11 is called a node or a neuron. In the example of FIG. 11, the NN has an input layer, two or more intermediate layers, and an output layer. The input layer is I, the intermediate layers are H1 and Hn, and the output layer is O. Also, in the example of FIG. 11, the number of nodes in the input layer is 2, the number of nodes in each intermediate layer is 5, and the number of nodes in the output layer is 1. However, the number of intermediate layers and the number of nodes included in each layer can be variously modified. Also, FIG. 11 shows an example in which each node included in a given layer is connected to all nodes included in the next layer, but this configuration can also be variously modified.

[0130] The input layer receives input values and outputs them to the intermediate layer H1. In the example of FIG. 11, the input layer I receives two types of input values. Note that each node in the input layer may perform some processing on the input value and output the value after the processing.

[0131] In an NN, a weight is set between two connected nodes. W1 in FIG. 11 is the weight between the input layer I and the first intermediate layer H1. W1 represents a set of weights between a given node in the input layer and a given node in the first intermediate layer. For example, W1 in FIG. 11 is information including 10 weights.

[0132] At each node in the first intermediate layer H1, an operation is performed in which the outputs of the nodes in the input layer I connected to the node are weighted and added using the weight W1, and a bias is further added. Further, at each node, the output of the node is obtained by applying an activation function, which is a non-linear function, to the addition result. The activation function may be a ReLU function, a sigmoid function, or other functions. Also, the activation function may be a softmax function that normalizes the sum of a plurality of outputs to 1. For example, the softmax function may be used as the activation function at the last node.

[0133] The same applies to the subsequent layers. That is, in a given layer, the output to the next layer is obtained by weighted addition of the output of the previous layer using the weight W, adding a bias, and then applying an activation function. The NN uses the output of the output layer as the output of the NN.

[0134] As can be seen from the above description, in order to obtain desired output data from input data using an NN, it is necessary to set appropriate weights and biases. In learning, training data is prepared in which given input data is associated with correct answer data representing the correct output data for the input data. The learning process of the NN is a process of obtaining the most probable weights based on the training data. Note that various learning methods such as the backpropagation method are known in the learning process of the NN. In the present embodiment, since those learning methods can be widely applied, detailed description thereof is omitted.

[0135] Also, the structure of the NN in this embodiment is not limited to the example in FIG. 11. For example, the NN in this embodiment may be an RNN (Recurrent Neural Network). An RNN is an NN in which the input at a certain point in time affects the subsequent output and is suitable for processing time-series data. For example, the NN may be an LSTM (Long Short Term Memory).

[0136] FIG. 12 is a diagram illustrating the input data and output data of the NN in this embodiment. As shown in FIG. 12, the input data includes the biological information of the care recipient, the biological information of the care recipient's relatives, the information of the electronic medical record and the care software, the sensing data, and the environmental information. Also, the input data may include other data (not shown in FIG. 12).

[0137] The biological information of the care recipient is information representing the state of the biological activities of the care recipient and includes, for example, information such as heart rate, activity level, and sleep time. Also, the biological information may include various information such as body temperature, blood pressure value, blood oxygen saturation, electrocardiogram, and electroencephalogram measurement results. These information can be obtained using various devices in the same manner as in the example of obtaining the biological information of the relatives. Since the biological information is closely related to the state of the care recipient, there may be a difference in the biological information between the state in which peripheral symptoms are likely to occur and the state in which they are less likely to occur. Therefore, it is considered useful to use the biological information in estimating the peripheral symptoms.

[0138] Regarding the biological information of the relatives, it is as described above. That is, the input data used for estimating the main factors in this embodiment may include the biological information of the care recipient's relatives. And the biological information of the relatives is data related to psychological factors as described above. In this way, since the information related to psychological factors can be included in the input data, it becomes possible to appropriately estimate the contribution degree of the psychological factors to the peripheral symptoms.

[0139] Here, an example of using the biometric information of a person related to the care recipient has been described in the determination of whether the main cause of the peripheral symptoms is a psychological factor. However, the method of using the biometric information of a person related to the care recipient, who is a different person from the care recipient himself / herself, to estimate the psychological state of the care recipient is applicable in other situations as well. For example, the method of this embodiment can be applied to an information processing apparatus including an acquisition unit (corresponding to, for example, acquisition unit 111) that acquires the biometric information of the person related to the care recipient, and an estimation unit that estimates the psychological state of the care recipient based on the biometric information of the person related to the care recipient. As described above, the relationship between the care recipient and the person related to him / her strongly affects the psychological state of the care recipient. Therefore, by deliberately using the information of the person related to the care recipient instead of the information of the care recipient himself / herself, it becomes possible to appropriately estimate the psychological state of the care recipient.

[0140] The information in the electronic medical record is information provided from a hospital or the like where the care recipient has received a medical examination, and includes information such as medical history and medication history. The information of the care support software is information accumulated by software used for the management of the care recipient and the caregiver in a care facility. The information of the care support software includes, for example, information representing attributes such as the age of the care recipient, the occurrence history of the peripheral symptoms (e.g., unruly behavior) of the care recipient, the diet content and intake amount, the assigned caregiver, the participation status in activities implemented in the care facility, and various information representing the life history of the care recipient in the care facility.

[0141] For example, when suffering from a specific disease or receiving medication for a specific drug, there is a possibility that peripheral symptoms are likely to occur. Therefore, it is useful to use the information in the electronic medical record in the estimation regarding peripheral symptoms. Also, since the life of the care recipient in the care facility may change depending on the presence or absence and the likelihood of occurrence of peripheral symptoms, the information of the care support software is also related to the peripheral symptoms. Therefore, it is useful to use the information of the care support software in the estimation regarding peripheral symptoms.

[0142] The sensing data is the output of the sensing device 400. Here, the sensing device 400 may include an imaging device 410 for detecting the start of movement, a bedside sensor 420 and a detection device 430, a swallowing mucus detection device 460 for performing processing related to meals, and the like. The sensing device 400 may also include an incontinence detection device 450, a seat surface sensor 440, an acceleration sensor, a foot pressure sensor, and the like. The sensing data of these sensing devices 400 is information representing the abilities of the care recipient in daily activities, such as the ability to stand up, swallowing ability, seat holding ability, walking ability, and the like. As described above, due to executive dysfunction or apraxia, it becomes difficult for the care recipient to perform daily activities well. That is, these sensing data are information related to the first core factor.

[0143] That is, the input data of the present embodiment may include ability information representing the abilities of the care recipient in daily activities as data related to the first core factor. In this way, since it becomes possible to include data related to the first core factor in the input data, it becomes possible to accurately determine the contribution degree of the first core factor to the peripheral symptoms.

[0144] Here, the ability information is, as described above, sensing data representing the sensing results in at least one of the start of movement, meals, seat holding, and walking of the care recipient. In this way, it is possible to obtain, as ability information, information indicating to what extent the movements in daily life can be performed normally. Since it is possible to obtain information with a high degree of relevance to executive dysfunction or apraxia, it becomes possible to accurately determine the contribution degree of the first core factor to the peripheral symptoms.

[0145] In addition, the sensing device 400 that outputs sensing data may include a detection device 430, an RFID reader and an IC tag, an incontinence detection device 450, and the like. The detection device 430, the RFID reader and the IC tag output wandering information as described above. The incontinence detection device 450 outputs incontinence information as described above. As described above, due to the cognitive impairment, it becomes difficult for the care recipient to recognize their current situation, and wandering, incontinence, etc. increase. That is, these sensing data are information related to the second core factor.

[0146] That is, the input data of the present embodiment may include at least one of incontinence information regarding the incontinence of the caregiver and wandering information regarding the wandering of the care recipient as data related to the second core factor. In this way, since it becomes possible to include data related to the second core factor in the input data, it becomes possible to accurately obtain the contribution degree of the second core factor to the peripheral symptoms.

[0147] In addition, the input data may include environmental information. As described above, the environmental information includes information regarding brightness, temperature, humidity, smell, sound, video, and season. In this way, since it becomes possible to include data related to environmental factors in the input data, it becomes possible to accurately obtain the contribution degree of environmental factors to the peripheral symptoms.

[0148] Note that as described above, the NN is an RNN, and the input data of the present embodiment may be time-series data. That is, each data included in the input data described above is not limited to data at a single timing and may be time-series data.

[0149] In addition, as shown in FIG. 12, the output data of the NN may include the contribution degree of environmental factors to the peripheral symptoms, the contribution degree of the first core factor, the contribution degree of the second core factor, and the contribution degree of psychological factors. Here, the contribution degree may be information indicating the probability that the peripheral symptoms are caused by the target factor.

[0150] For example, NN may output five pieces of data: a peripheral symptom score representing the probability of the occurrence of peripheral symptoms, an environmental factor score representing the probability of the occurrence of peripheral symptoms due to environmental factors, a first core factor score representing the probability of the occurrence of peripheral symptoms due to the first core factor, a second core factor score representing the probability of the occurrence of peripheral symptoms due to the second core factor, and a psychological factor score representing the probability of the occurrence of peripheral symptoms due to psychological factors. For example, each score may be information normalized so as to have a value between 0 and 1. In this case, the peripheral symptom score represents the probability of the occurrence of peripheral symptoms. Also, each factor score represents the degree of contribution of the corresponding factor to the peripheral symptoms. In this way, based on the output data, it becomes possible to determine the presence or absence of peripheral symptoms and the degree of contribution of each factor when the peripheral symptoms occur.

[0151] However, as shown in FIG. 12, a configuration in which the peripheral symptom score is not directly output may also be used. For example, the sum of the environmental factor score, the first core factor score, the second core factor score, and the psychological factor score may be used as the peripheral symptom score. Also in this case, based on the output data of NN, it becomes possible to determine the presence or absence of peripheral symptoms and the degree of contribution of each factor when the peripheral symptoms occur.

[0152] Note that, here, an NN that performs both the determination of the presence or absence of peripheral symptoms and the determination of the degree of contribution of each factor is exemplified, but it is not limited to this. For example, the determination of the presence or absence of peripheral symptoms and the determination of the degree of contribution of each factor may be executed using different NNs.

[0153] Also, NN may not only identify any one of the environmental factor, the first core factor, the second core factor, and the psychological factor as the main factor, but also output information for identifying more detailed factors. For example, the environmental factor may be divided into more detailed factors such as a brightness factor, a temperature factor, a humidity factor, and a season factor. The first core factor can be divided into factors such as a starting factor, a diet factor, a sitting factor, and a walking factor. The second core factor can be divided into factors such as a nocturnal wandering factor, an incontinence factor, and an intrusion factor into an inappropriate place. In addition, various modifications can be made to the configuration of NN.

[0154] FIG. 13 is a flowchart for explaining a learning process in which the learning unit 115 creates a learned model 124. First, in step S101, the learning unit 115 acquires learning input data. Examples of the input data are as described above with reference to FIG. 12, and data having a similar configuration can also be used for the learning input data.

[0155] Next, in step S102, the learning unit 115 acquires correct answer data. The correct answer data here includes at least information for identifying the main factors of the peripheral symptoms. Also, in the above-described example, the correct answer data may include information for identifying the presence or absence of the peripheral symptoms. The correct answer data here is, for example, data input by an expert having knowledge about the symptoms and factors of dementia, such as a doctor or a skilled caregiver. For example, an expert who has examined or assisted the care recipient estimates the presence or absence and main factors of the peripheral symptoms and inputs the estimation result. For example, the learning unit 115 displays an annotation screen on a PC of an expert connected to the server system 100 and prompts the input of the estimation result on the annotation screen. Then, the learning unit 115 associates the input annotation result with the input data regarding the target care recipient as the correct answer data. Thereby, training data in which the correct answer data is associated with the learning input data is acquired.

[0156] In step S103, the learning unit 115 performs machine learning based on the training data. Specifically, the learning unit 115 inputs the learning input data into the NN and obtains output data by performing a forward operation using the weights at that stage. The learning unit 115 obtains an objective function based on the output data and the correct answer data. The objective function here is, for example, an error function based on the difference between the output data and the correct answer data, or a cross-entropy function based on the distribution of the output data and the distribution of the correct answer data. The learning unit 115 updates the weights based on the objective function. As the weight update method, the error backpropagation method and the like described above are known, and those methods can also be widely applied in this embodiment.

[0157] In step S104, the learning unit 115 determines whether to end the learning process. For example, the learning unit 115 ends the learning process when a given condition is satisfied. For example, the training data may be divided into learning data and validation data. The learning unit 115 may end the learning process when the accuracy rate based on the validation data exceeds a given threshold. After the learning process ends, the NN including the weights at that stage is stored in the storage unit 120 as the learned model 124. Note that the learning process is not limited to being executed by the server system 100 and may be executed by an external device. The server system 100 may acquire the learned model from the external device.

[0158] 2.2.2 Inference Processing The factor estimation unit 112 may perform a process of obtaining the main factor based on the learned model 124 that takes the input data as input and outputs a peripheral symptom score representing the probability that the assisted person is in a peripheral symptom state and a factor score for each of the plurality of factors. As described above, the peripheral symptom score is not limited to being directly output and may be obtained as the sum of a plurality of factor scores or the like. Specifically, the factor estimation unit 112 obtains the main factor by using the learned model 124 acquired by the above-described learning process. In this way, since machine learning can be applied to the estimation of the main factor, it becomes possible to accurately obtain the main factor.

[0159] FIG. 14 is a flowchart for explaining an inference process of estimating the main factor of peripheral symptoms based on input data. The factor estimation unit 112 may periodically execute the inference process for example for all assisted persons using the information processing system 10. Alternatively, the factor estimation unit 112 may execute the inference process for assisted persons suffering from dementia or assisted persons suspected of having dementia. Also, when an assistant or a related person determines that a peripheral symptom is observed in the assisted person, the factor estimation unit 112 may execute the inference process triggered by an operation input from the assistant or the like. In addition, various modifications are possible for the target and timing of executing the inference process.

[0160] First, in step S201, the acquisition unit 111 acquires input data. The input data is as described above with reference to FIG. 12. For example, the acquisition unit 111 acquires electronic medical records and care software information from the management terminal device 300, and acquires sensing data from the sensing device 400. The environmental information is stored in, for example, care software and transmitted from the management terminal device 300 to the acquisition unit 111. However, the environmental information may be directly transmitted from the device that detects the environmental information to the acquisition unit 111. The biometric information of the care recipient and related persons is transmitted from the device that detects the biometric information to the acquisition unit 111. However, the biometric information may be transmitted to the acquisition unit 111 via another device such as the terminal device used by the related person or the management terminal device 300.

[0161] In step S202, the factor estimation unit 112 estimates the main factors of the peripheral symptoms based on the input data. For example, the factor estimation unit 112 may obtain a factor score representing the degree of contribution to the peripheral symptom state for each of the environmental factor, the first core factor, the second core factor, and the psychological factor based on the input data which is time-series data. The factor score here corresponds to each score described above with reference to FIG. 12. For example, the factor estimation unit 112 reads out the learned model 124 which is an RNN or LSTM from the storage unit 120, and inputs the input data which is time-series data to the learned model 124 to obtain the peripheral symptom score and the factor scores of each factor. When the peripheral symptoms occur, not only the events that occurred most recently but also the events that occurred in the past and their accumulation may affect the care recipient. For example, although the second core factor was suspected when looking only at a single timing, there are cases where it is determined that the contribution of the first core factor is large when observed over a longer span. In that regard, by using the input data which is time-series data, it becomes possible to accurately estimate the main factors. In step S202, the factor estimation unit 112 may obtain a peripheral symptom score representing the probability that the peripheral symptoms occur. The peripheral symptom score may be one of the outputs of the NN as described above, or may be the total value of the factor scores related to a plurality of factors, or may be obtained from other outputs.

[0162] In step S203, the cause estimation unit 112 determines whether the peripheral symptom score is equal to or greater than a given threshold. If the peripheral symptom score is less than the threshold (S203: No), it is estimated that no peripheral symptom has occurred. Therefore, the processes of steps S204 - S206 are not performed, and the inference process shown in FIG. 14 ends.

[0163] If the peripheral symptom score is equal to or greater than the threshold (S203: Yes), the probability that a peripheral symptom has occurred is high. Therefore, in step S204, the cause estimation unit 112 estimates the main cause. For example, the cause estimation unit 112 estimates the main cause based on the degree of change in the cause score. For example, the cause estimation unit 112 obtains the degree of increase in the cause score over a predetermined period. Here, the degree of increase may be the ratio of the cause score at the end point of the period to the cause score at the start point of the period. Also, the degree of increase may be the ratio of the maximum value to the minimum value of the cause score within the period. Other methods for obtaining the degree of increase can be variously modified. The cause estimation unit 112 determines as the main cause the cause with the largest degree of increase in the cause score.

[0164] Also, in step S204, when the increase rate of the cause score of any of the environmental factor, the first core factor, the second core factor, and the psychological factor is equal to or less than a predetermined threshold, the cause estimation unit 112 may determine that the main cause is the psychological factor. As described above, although it is possible to use the biometric information of the related person as information related to the psychological factor, there may be a person or event that burdens the psychology of the care recipient other than the related person who is the measurement target of the biometric information. In this case, although it is actually a psychological factor, the cause score of the psychological factor may not change. Therefore, when the increase rate of any of the cause scores is small, the cause estimation unit 112 may determine that the main cause is the psychological factor.

[0165] Note that there may be individual differences in the factor scores of each factor in this embodiment. For example, since the ability to perform daily operations well is related not only to cognitive functions but also to motor functions, the level of ability represented by the ability information varies depending on the motor function of the care recipient. Therefore, even if there are care recipients in whom the executive function disorder contributes equally to the peripheral symptoms, depending on the level of motor function, there are care recipients with a high factor score value for the first core factor and those with a low factor score value. The same applies to other factors. The magnitude relationship of the factor scores of each of the environmental factor, the first core factor, the second core factor, and the psychological factor may vary for each care recipient. When simply comparing the values (for example, when assuming that the factor with the maximum factor score is the main factor), the estimation accuracy may not be sufficient. In that regard, by using the degree of variation of the factor scores, the judgment is made based on the past values of the target care recipient. As a result, the influence of individual differences can be suppressed, and the main factor can be estimated accurately. However, in this embodiment, it is not precluded from determining the main factor using the magnitude of the factor score itself.

[0166] Also, as described above in the learning process, the NN is not limited to estimating the factor scores of the environmental factor, the first core factor, the second core factor, and the psychological factor, and the factor scores of factors obtained by further dividing each factor may be obtained. That is, when the factor estimation unit 112 uses, as sub-factors, factors obtained by dividing at least one of the environmental factor, the first core factor, the second core factor, and the psychological factor, the main factor may be estimated from among the plurality of sub-factors. Alternatively, after the factor estimation unit 112 estimates the main factor from among the four factors using the learned model 124, the process of estimating the factor with the highest contribution degree among the sub-factors included in the main factor may be separately executed. For example, the factor estimation unit 112 estimates the sub-factor with a high contribution degree according to which data among the input data related to the main factor has the largest variation. For example, when the main factor is the first core factor, if the variation in the pressure value at the time of rising is large, the factor estimation unit 112 may determine that the rising factor is dominant, and if the variation in various information measured during meals is large, the factor estimation unit 112 may determine that the meal factor is dominant.

[0167] In addition, the information processing apparatus 20 of this embodiment may further include a countermeasure determination unit that determines a countermeasure for the peripheral symptom state according to the main factor estimated by the factor estimation unit 22. For example, as shown in FIG. 4, the server system 100 includes a countermeasure determination unit 113 that determines a countermeasure for the peripheral symptom state according to the main factor estimated by the factor estimation unit 112. In this way, not only can the main factor be estimated, but also a countermeasure corresponding to the main factor can be determined, which can reduce the burden on the caregiver. However, only the main factor may be notified to professionals such as nursing staff, and the countermeasure may be notified to non-professionals such as the family members of the care recipient, and the notification content may be changed according to the caregiver notified.

[0168] For example, in step S205, the countermeasure determination unit 113 performs a process of obtaining a countermeasure recommended for the caregiver or the related person based on the estimated main factor. For example, as shown in FIG. 4, the storage unit 120 may store a countermeasure table 125. The countermeasure table 125 is information in which the main factor and the recommended countermeasure are associated. The countermeasure determination unit 113 obtains a countermeasure based on the estimated main factor and the countermeasure table 125. Hereinafter, the information representing the countermeasure obtained by the countermeasure determination unit 113 is described as countermeasure information.

[0169] For example, when it is determined that the main factor is an environmental factor, the countermeasure determination unit 113 outputs countermeasure information representing a countermeasure for aligning the environment of the care recipient with the reference environment. The reference environment here is an environment preferable for the care recipient, for example, the environment before a large environmental change occurs. For example, when a peripheral symptom in which the environmental factor is estimated as the main factor occurs after a care recipient who was living at home enters a facility or the like, the reference environment corresponds to the home environment of the care recipient. Alternatively, when there is a change in the living room or a pattern change in a nursing facility, the environment before the change may be used as the reference environment. Also, based on the log of the occurrence or frequency of occurrence of the peripheral symptoms of the care recipient, a process of estimating an environment in which the target care recipient is less likely to develop peripheral symptoms may be performed. In this case, the environment in which the peripheral symptoms are less likely to occur is used as the reference environment.

[0170] For example, based on the log data of household appliances such as lighting equipment, air conditioning equipment, music players, and televisions in a reference environment, information for identifying the reference environment (hereinafter referred to as reference environment information) may be obtained. These household appliances may also be controlled by an application such as a smartphone, and the reference environment information may be obtained by acquiring the setting information of each device using the application. At this time, an application having a function of sharing the information of the home environment with other devices such as the server system 100 may be provided. Further, the reference environment information may be obtained by performing measurements using a photoelectric sensor, an odor sensor, a microphone, etc. In the above, information such as brightness, temperature, humidity, scent, sound, and video has been exemplified as the reference environment information, but the reference environment information is not limited thereto. For example, the type of assistive device being used and the information on the operation history may be used as the reference environment information. In this way, it becomes possible to match the assistive devices used by the care recipient and their operation settings to the reference environment. Here, the assistive device may be a nursing bed whose height and bottom angle can be changed, a reclining wheelchair whose back angle can be changed, a walker, or other devices. The operation setting may be a setting such as the above-mentioned height, bottom angle, and back angle.

[0171] The countermeasure determination unit 113, for example, acquires the reference environment information and determines, as a recommended countermeasure, a countermeasure for matching the environment of the care recipient to the reference environment. Although an example where the countermeasure determination unit 113 acquires the reference environment information has been shown here, it is not limited thereto. For example, the countermeasure determination unit 113 outputs information indicating that the acquisition of the reference environment information is recommended, and the reference environment information may be acquired by other devices that have acquired the information.

[0172] Also, as described above, the input data of the present embodiment may include information for specifying seasons as data related to environmental factors. Then, the factor estimation unit 112 may determine whether the main factor is a seasonal factor resulting from seasons among environmental factors. When such determination is possible, the countermeasure determination unit 113 may vary the countermeasures depending on whether the main factor is estimated to be a seasonal factor or an environmental factor other than the seasonal factor.

[0173] For example, when an environmental factor other than the seasonal factor is the main factor, the countermeasure determination unit 113 recommends a countermeasure of adjusting to the reference environment as described above. On the other hand, there are cases where peripheral symptoms are likely to appear in patients with dementia when a specific season arrives or at the turn of seasons. In such cases of peripheral symptoms due to seasonal factors, even if adjusted to the home environment or the like, the effect of suppressing peripheral symptoms may be low. Also, if it is a seasonal factor, there is a possibility that the peripheral symptoms will naturally subside after the target season has passed. Therefore, when it is determined to be a seasonal factor, the countermeasure determination unit 113 may recommend a countermeasure that suppresses proactive responses. In this way, unnecessary countermeasures can be suppressed. Also, if it is known that it is a seasonal factor, since it only needs to get through that season, a period with a heavy burden can be estimated to some extent. Therefore, it is possible to reduce the mental burden on the caregiver.

[0174] Also, when it is estimated that the main factor is the first core factor, the countermeasure determination unit 113 may perform a process of determining, as a countermeasure to be recommended, to use an assistance device that supports daily operations. As described above, when the first core factor is the main factor, it means that the inability to perform daily operations is the cause of the peripheral symptoms. By introducing an assistance device that supports daily operations, it becomes easier for the person to be assisted to perform daily operations. Here, the use of the assistance device is not limited to the introduction of a new device and may include changing the operation mode of an existing device.

[0175] Note that the countermeasure decision unit 113 may recommend countermeasures for communicating with the care recipient or caregiver before introducing the assistive device. For example, a process of using a communication robot or a chatbot to interview the care recipient or caregiver as to whether operation support is necessary may be performed. If a response indicating that operation support is necessary is obtained in the interview, the countermeasure decision unit 113 recommends introducing the assistive device. For example, the countermeasure decision unit 113 may perform a process of specifying the specific device type, vendor, model number, etc. Alternatively, the countermeasure decision unit 113 may perform a process of specifying an EC (Electric Commerce) site or the like where the assistive device can be purchased. Note that the countermeasure decision unit 113 outputs information indicating that these processes are recommended, and the actual processes may be executed by other devices that have received the information.

[0176] Also, when it is estimated that the main factor is the second core factor, the countermeasure decision unit 113 may perform a process of determining, as a countermeasure to be recommended, to intervene in the care recipient's risk of falling. When the factor of spatial disorientation is large, the care recipient cannot grasp the situation in which he / she is placed, so the risk of falling tends to be high. Therefore, by intervening in the risk of falling, the occurrence of incidents due to falling can be suppressed.

[0177] For example, the countermeasure determination unit 113 may output a countermeasure for using a portable airbag that can be worn on the waist or the like of the care recipient. By doing so, even when the caregiver is not near the care recipient or when there is a shortage of manpower, injuries caused by falls can be suppressed. Further, the countermeasure determination unit 113 may recommend a countermeasure of performing a detection process of the start of movement using the imaging device 410, the bedside sensor 420, the acceleration sensor, etc. described above with reference to FIG. 6 or the like. For example, when the start of movement is detected, the caregiver is notified to that effect using the output image IM1 or the like in FIG. 6. By doing so, since the caregiver can be notified that there is a care recipient with a high risk of falling, the fall risk can be suppressed. Also, the start of movement may be detected using the detection device 430 in FIG. 7. For example, the detection device 430 determines that there is a start of movement of the care recipient when shifting from the in-bed state to the out-of-bed state, or when shifting from the sleep state to the waking state. For example, by presenting the detection result of the start of movement to the caregiver, the level of the fall risk can be presented, so that it becomes possible to clearly present to the caregiver whether intervention is necessary.

[0178] Further, when it is estimated that the main factor is a psychological factor, the countermeasure determination unit 113 may perform a process of determining, as a recommended countermeasure, identifying a person related to the care recipient who has a high degree of influence on the psychology of the care recipient. As described above, a person related to the care recipient who has an influence on the psychology of the care recipient has a high probability of showing a change in biological information. However, the person related to the care recipient does not always live with the care recipient, and there is also a possibility that the biological information changes for reasons unrelated to the care recipient. That is, even if there is a person related to the care recipient whose biological information has changed, the person related to the care recipient does not necessarily affect the psychology of the care recipient.

[0179] Therefore, the countermeasure decision unit 113 may recommend a countermeasure for determining the strength of the involvement of the relevant person. For example, the assisted person and the relevant person may each carry a communication device, and based on a beacon signal or the like transmitted and received by the communication device, a process for obtaining the degree of involvement of the relevant person may be performed. For example, it is determined that the longer the time the relevant person is located within a predetermined distance from the assisted person, the higher the strength of the involvement with the assisted person. Here, the beacon signal may be a Bluetooth (registered trademark) advertisement packet, an SSID broadcast in IEEE 802.11, or a beacon signal of another communication method. If there is a relevant person whose degree of involvement is equal to or greater than a predetermined value and the change in biometric information is large, it is determined that there is a high possibility that the relevant person is the cause of the peripheral symptoms.

[0180] In this case, the countermeasure decision unit 113 may recommend a countermeasure of having a conversation with the assisted person using a communication robot or a chat robot. For example, in the conversation, among the environmental factors, the first core factor, and the second core factor, questions regarding the factor may be asked to the assisted person in descending order of the increase rate of the factor score. For example, in a case where the assisted person is scolded by the relevant person due to the factor of not being able to do something well, it is useful to identify the specific situation of which action by which factor makes the relevant person angry. In that regard, since the higher the increase rate of the factor score, the more likely it is to be the cause of the peripheral symptoms, the reason why the relevant person scolds the caregiver (that is, the reason why the assisted person feels stress) can be efficiently identified by asking questions in that order. At this time, the conversation may be texturized, and by performing text mining processing on the text, a situation in which the assisted person feels a psychological burden may be identified.

[0181] When a situation in which the assisted person feels a psychological burden is identified, the countermeasure decision unit 113 may further recommend a countermeasure of presenting a summary of the conversation to the relevant person in question. This may be a summary of the text or a summary of the recorded data obtained by recording the conversation. By doing so, it is possible to notify the relevant person of the situation and specific words and actions that have imposed a burden on the assisted person, thereby suppressing the recurrence of the same situation.

[0182] The information processing apparatus 20 of the present embodiment may include a presentation processing unit that presents at least one of the main factors estimated by the factor estimation unit 22 and the countermeasures determined by the countermeasure determination unit. For example, as shown in FIG. 4, the server system 100 includes a presentation processing unit 116 that presents at least one of the main factors estimated by the factor estimation unit 112 and the countermeasures determined by the countermeasure determination unit 113.

[0183] For example, in step S206, the presentation processing unit 116 performs a process of presenting the main factors and countermeasures. For example, the presentation processing unit 116 presents the main factors and countermeasures to the terminal device 200 used by the assistant via the communication processing unit 114 and the communication unit 130. The presentation here may be a display of an image or text, or an output of voice using the speaker of the headset.

[0184] Note that the presentation of the countermeasures is to present the devices, operations, etc. necessary for the execution of the countermeasures, and the specific operations, etc. may be entrusted to the assistant. However, the method of the present embodiment is not limited to this, and based on the countermeasures determined by the countermeasure determination unit 113, the server system 100 may perform automatic control of the control target device to be controlled. For example, the control of the nursing bed and the reclining wheelchair for adapting to the reference environment may be automatically executed based on an instruction from the server system 100. Also, when a conversation with the person to be assisted using a communication robot or the like is performed, the activation of the communication robot and the operations for starting the conversation may be automatically executed based on a control signal from the server system 100. In addition, various modifications are possible for the specific methods of executing the recommended countermeasures.

[0185] Also, in FIG. 12, an example was described in which various input data are input to the NN, and output data including factor scores corresponding to a plurality of factors is output based on the input data. However, since the processing load increases when a large number of parameters are input to the NN, the factor estimation unit 112 may divide the processing. For example, the factor estimation unit 112 makes a first determination for determining a main factor based on the first input data, and when the main factor is not estimated in the first determination, a second determination for determining the main factor is made based on second input data including data not included in the first input data. For example, the factor estimation unit 112 first sets the input data related to the environmental factor and the first core factor as the first input data, and performs a process of obtaining the factor scores of the environmental factor and the first core factor. When the degree of variation of any one of them is equal to or greater than a predetermined value, the factor is estimated as the main factor, and the estimation process of the main factor is terminated. When the degrees of variation of the factor scores of the environmental factor and the first core factor are both less than the threshold value, the factor estimation unit 112 may set the input data related to the second core factor and the psychological factor as the second input data, and obtain the factor scores of the second core factor and the psychological factor. In this case, in the previous stage of the process, the input of the wandering information, which is data related to the second core factor, and the biometric information of the related person, which is data related to the psychological factor, can be omitted. Also, in the subsequent stage of the process, the input of the environmental information related to the environmental factor and the ability information, which is data related to the first core factor, can be omitted. As a result, the processing load in the factor estimation unit 112 can be reduced.

[0186] Also, the second input data may be data including the first input data. For example, the first input data may be a part of the input data shown in FIG. 12, and the second input data may be all of the input data shown in FIG. 12. In this way, first, the first determination with a relatively low load can be tried. If the main factor is estimated in the first determination, the second determination becomes unnecessary, so the processing load can be reduced.

[0187] Also, when using data related to any one of the environmental factor, the first core factor, the second core factor, and the psychological factor and not related to any of the other three as the specific input data, the second input data may include at least one specific input data not included in the first input data. For example, in the example described above with reference to FIG. 12, the information related to incontinence is not specific input data because it is related to both the first core factor and the second core factor. On the other hand, the information such as brightness, temperature, and humidity included in the environmental information has a low degree of association with the first core factor, the second core factor, and the psychological factor, and is specific input data.

[0188] In this way, it becomes possible to use, as the first input data, data obtained by excluding at least one or more specific input data from the second input data. For example, as the first determination, processing may be performed centering on data such as the biometric information of the care recipient, the electronic medical record, and the information of the care support software, the degree of association of which is not biased toward a specific factor. In this way, even if the types of data included in the first input data are reduced, it is possible to suppress a determination biased toward a specific factor (the factor score of a specific factor is likely to increase). Also, since the specific input data is included in the second input data, it is possible to accurately obtain the factor score of each factor. By varying the presence or absence and the number of specific input data between the first input data and the second input data as described above, it becomes possible to combine two determinations with different characteristics.

[0189] Note that the method for estimating the main factor based on the learned model 124 has been described above, but the method of this embodiment is not limited thereto. For example, the factor estimation unit 112 may obtain the degree of temporal variation for each of the plurality of input data. For example, the factor estimation unit 112 may identify the input data with the largest degree of variation and determine the factor corresponding to the input data as the main factor. The correspondence between each input data and the factor is as described above. Further, the factor estimation unit 112 may obtain an evaluation value for each factor by obtaining the degree of variation of the plurality of input data. For example, the evaluation value may be numerical data that increases as the degree of variation of the input data corresponding to the target factor increases. For example, when a plurality of input data are associated with a given factor, the evaluation value of the given factor is calculated based on the degree of variation of each of the plurality of input data. The factor estimation unit 112 may estimate the factor with the largest evaluation value among the plurality of factors as the main factor. In addition, various modifications are possible for the details of the process of estimating the main factor.

[0190] 2.2.3 Update Process Also, in this embodiment, an update process of the learned model 124 may be executed. For example, when the presentation process of the factor information representing the main factor is performed, the learning unit 115 may perform a process of updating the learned model 124 based on the factor information, the first peripheral symptom score which is the peripheral symptom score before the presentation of the factor information, and the second peripheral symptom score which is the peripheral symptom score after the presentation of the factor information. Here, the first peripheral symptom score is not limited to the peripheral symptom score at a single timing, and may be a statistic in a predetermined period before the presentation of the factor information. For example, the first peripheral symptom score may be an average value, a maximum value, or a minimum value in a predetermined period before the presentation of the factor information. Further, the first peripheral symptom score may be the number of times the peripheral symptom score exceeds the threshold corresponding to the presence of the peripheral symptom. The second peripheral symptom score is the same as the first peripheral symptom score except that the target period is the period after the presentation of the factor information.

[0191] By doing so, it becomes possible to compare the degree of occurrence of peripheral symptoms before and after presenting the main factor. For example, if it is determined that the degree of occurrence of peripheral symptoms has decreased based on the first peripheral symptom score and the second peripheral symptom score, it indicates that the presented main factor is appropriate and that the condition of the care recipient has improved due to the countermeasures based on the main factor. In this case, the learned model 124 is updated so that the same main factor is likely to be selected in the same situation. For example, the learning unit 115 performs a weight update process using the training data associating the input data with the estimated result of the main factor as positive training data.

[0192] On the other hand, if it is determined that the degree of occurrence of peripheral symptoms has increased based on the first peripheral symptom score and the second peripheral symptom score, it means that the presentation of the main factor has not led to the improvement of peripheral symptoms. That is, there may be an error in the estimated result of the main factor. In this case, the learned model 124 is updated so that the same main factor is less likely to be selected in the same situation. For example, the learning unit 115 performs a weight update process using the training data associating the input data with the estimated result of the main factor as negative training data.

[0193] Also, when the presentation process of the factor information and the countermeasure information representing the countermeasure is performed, the learning unit 115 may perform a process of updating the learned model based on the factor information, the countermeasure information, the first peripheral symptom score, and the second peripheral symptom score. In this case, it becomes possible to update the learned model 124 from the perspective of whether the presentation of the countermeasure contributes to the improvement of peripheral symptoms in addition to the presentation of the main factor.

[0194] 2.3 Specific Examples in Meal Assistance Taking meal assistance as an example, specific examples of countermeasures executed based on the estimated result of the main factor will be described. For example, consider a case where a nursing facility has hitherto left the meal to the care recipient himself / herself and used a swallowing cough detection device 460 for monitoring related to meals.

[0195] In this case, since the peripheral symptom score has exceeded the threshold from a certain point in time, the process of the factor estimation unit 112 identifying the main factor may be executed. Note that the sensing data of the swallowing mucus detection device 460 may be used as input data when calculating the peripheral symptom score from the beginning. Alternatively, based on the fact that the peripheral symptom score calculated without using the sensing data of the swallowing mucus detection device 460 has exceeded the threshold, the sensing data of the swallowing mucus detection device 460, which is the specific input data, may be added to the input data. In any case, the factor estimation unit 112 estimates that the first core factor is the main factor based on the input data, and more specifically, it is determined that the contribution of "watching a meal without doing anything" is high.

[0196] In this case, the countermeasure decision unit 113 determines the use of an assistance device for supporting meals as the recommended countermeasure, and the presentation processing unit 116 presents the countermeasure to the terminal device 200 of the caregiver. Note that the presentation processing unit 116 may first output a voice such as "Is meal support necessary?" from the terminal device 462 of the swallowing mucus detection device 460, and when the care recipient responds "yes" or makes a nodding motion in response, present the above countermeasure to the terminal device 200 of the caregiver.

[0197] The assisting device here may be an automatic feeding device 470 that automatically delivers food to the mouth of the person being assisted, as shown in FIG. 15. The automatic feeding device 470 includes a container 472 that contains a plurality of dishes into which food is placed, and an arm 471 that scoops up a predetermined amount of the food placed in the container 472 and transports it to the mouth of the person being assisted. A tab 473 may be provided on the container 472. The tab 473 will be described later with reference to FIGS. 17A to 17G. The arm 471 includes, for example, a plurality of frames connected by joints and an end effector provided at the tip. The end effector here may be, for example, a hand that grips a spoon or the spoon itself. The automatic feeding device 470 may also include a scale 474 for measuring weight and a camera 475. In the example of FIG. 15, the scale 474 is arranged adjacent to the container 472, but it is not limited to this. For example, a scale may be built into the spoon, or a scale may be provided on the arm 471 to measure the weight based on the change in the weight of the spoon. Also, in the example of FIG. 15, the camera 475 is arranged near the tip of the arm 471 and images, for example, the contents of the spoon or the face of the person being assisted. For example, a device for automatically feeding a patient is described in U.S. Patent Application No. 15 / 094,800, filed on April 8, 2016, entitled "APPARATUS AND METHOD FOR FOOD CAPTURE". This patent application is hereby incorporated by reference in its entirety into the present specification. The automatic feeding device 470 in the present embodiment may be, for example, the feeding device disclosed in U.S. Patent Application No. 15 / 094,800, or a device having a similar mechanism.

[0198] FIG. 16 is a diagram for explaining an example of the connection between the swallowing mucus detection device 460 and the automatic feeding device 470. For example, the automatic feeding device 470 is connected to the terminal device 462 of the swallowing mucus detection device 460. Also, as described above, the terminal device 462 is connected to the throat microphone 461. These connections may be wired connections using a cable such as a USB (Universal Serial Bus) cable, or wireless connections using WiFi (registered trademark), Bluetooth, or the like.

[0199] In addition, various specific modes of countermeasures recommending the use of the automatic supply device 470 can be considered. For example, the presentation processing unit 116 may present information recommending the use of the automatic supply device 470 as countermeasure information representing the recommended countermeasures. In this case, the determination of whether to use the automatic supply device 470, the arrangement of the automatic supply device 470 when using it, and the connection shown in FIG. 16 are executed by an assistant or the like. Further, the presentation processing unit 116 may perform a process of displaying the connection method between the automatic supply device 470 and the swallowing mucus detection device 460 on the terminal device 200 as countermeasure information representing the recommended countermeasures. In this way, it becomes possible to smoothly introduce the automatic supply device 470. Further, the countermeasure determination unit 113 may first confirm the necessity of support by communicating with at least one of the assisted person and the assistant using a communication robot or the like. When an answer indicating that support is necessary is obtained, the countermeasure determination unit 113 may output countermeasure information indicating that it will perform automatic conveyance of the automatic supply device 470 and automatic connection between the automatic supply device 470 and the terminal device 462. Note that the devices performing automatic conveyance and automatic connection are not limited to the server system 100, and may be other devices that have received the countermeasure information. In this case, a wireless connection such as WiFi may be used for the connection between the automatic supply device 470 and the terminal device 462. In this way, since the arrangement and connection of the automatic supply device 470 can be automated, it becomes possible to reduce the burden on the assistant.

[0200] Teaching may be performed on the automatic supply device 470 before actual use. In teaching, for example, the assistant actually moves the arm 471 to cause the automatic supply device 470 to store the movement of the arm 471 until the spoon is carried to the mouth of the assisted person. Note that since teaching is a method known in the robot field, a detailed description thereof is omitted.

[0201] After the connection and teaching are completed, meal assistance in which the swallowing muscle detection device 460 and the automatic feeder 470 cooperate is executed. For example, the automatic feeder 470 may determine the amount of food placed on the spoon at one time and the pace of carrying the spoon to the mouth based on the determination result of the swallowing muscle detection device 460. For example, the swallowing muscle detection device 460 may pre-learn these parameters based on the tacit knowledge of an expert. Further, the swallowing muscle detection device 460 (narrowly, the terminal device 462) may store application software that executes processing corresponding to tacit knowledge, and perform processing of transmitting the application software to the automatic feeder 470. In this way, at least a part of the processing (processing corresponding to tacit knowledge) in the swallowing muscle detection device 460 described below can be executed in the automatic feeder 470.

[0202] For example, the automatic feeder 470 puts food for one spoonful, and obtains the amount of food based on the scale 474 or image processing. The terminal device 462 determines whether the amount of food placed on the spoon is an appropriate amount for the assisted person based on the above-described tacit knowledge.

[0203] When it is determined that the amount is appropriate, the automatic feeder 470 carries the spoon to the mouth of the assisted person based on the teaching result. Then, when it is determined that the mouth of the assisted person is open based on the captured image, the arm 471 is controlled to insert the spoon into the mouth. Here, the captured image may be captured by the camera of the terminal device 462. Alternatively, the captured image may be captured by a camera 475 provided near the tip of the arm 471.

[0204] The terminal device 462 determines whether swallowing is detected based on the voice data from the throat microphone 461. When swallowing is detected, it determines whether there is any food remaining in the mouth (referred to as "food remaining in the mouth"). For example, it is known that the louder the swallowing sound, the larger the amount of food being swallowed. Therefore, the terminal device 462 can estimate the magnitude of the swallowing sound when there is no food remaining in the mouth based on the amount of food taken on the spoon. If the swallowing sound detected by the throat microphone 461 is smaller than the expected swallowing sound, the terminal device 462 waits until swallowing occurs again, assuming there is food remaining in the mouth. Also, when it is determined that the swallowing sound is sufficiently loud and there is little food remaining in the mouth, the terminal device 462 permits the automatic feeder 470 to perform the operation for the next mouthful. As a result, the automatic feeder 470 repeats the above operation. By repeating this loop process, it becomes possible to appropriately support a care recipient who is having difficulty eating properly.

[0205] In the above loop process, the swallowing choking detection device 460 may determine whether situations such as the mouth not opening, the time until swallowing becoming longer, or the person looking sleepy have occurred. For example, the terminal device 462 determines whether the mouth has stopped opening based on the captured image of the camera. Also, the terminal device 462 determines the time from when the mouth opens until swallowing based on the captured image and the voice data from the throat microphone 461. Further, when the inclination of the posture is detected based on the seat surface sensor 440, the terminal device 462 determines that the person looks sleepy. Also, when eating is taking place in bed, the detection device 430 may be used to determine whether the person looks sleepy.

[0206] When it is determined that situations such as the mouth not opening, the time until swallowing becoming longer, or the person looking sleepy have occurred, the terminal device 462 prompts the care recipient to respond by outputting voice data. After outputting the voice data or the like, the terminal device 462 determines whether the situation has improved. If it has improved, it returns to the above-described loop process. If the situation does not improve, the terminal device 462 may output the voice data again. Alternatively, the terminal device 462 may request intervention from the caregiver.

[0207] Note that, as described above, in meal assistance, there is an appropriate amount of food suitable for the person being assisted, and this amount may vary depending on the target person being assisted. Therefore, when using the automatic supply device 470, it is desirable that the amount of food placed on the spoon can be finely adjusted. Thus, in the present embodiment, the height of the tab 473 provided on the container 472 may be adjusted. By doing so, fine adjustment of the amount of food becomes possible. In particular, when the amount scooped up by the spoon is too large, it becomes easier to reduce it to an appropriate amount. Hereinafter, specific examples of the tab 473 will be described. Note that the tab 473 here is a member having one end connected to the periphery of the dish portion where the food is placed and the other end inclined in the direction toward the center of the dish portion. By providing the tab 473, when scooping up food using a spoon, it is possible to prevent the food from spilling out of the dish. Further, by providing the tab 473, the food is pushed to the back side of the spoon by the tab 473, making it possible to easily scoop up the food.

[0208] Figs. 17A to 17G are diagrams showing a configuration example of the tab 473 in the present embodiment. Fig. 17A is a view observing the dish portion of the container 472 and the tab 473 from vertically above. As described above, one end of the tab 473 is connected to the periphery of the dish portion where the food is placed, and the other end is inclined in the direction toward the center of the dish portion.

[0209] Figs. 17B to 17D are diagrams exemplifying the cross-sectional structures of the tab 473 at three locations A-A, B-B, and C-C in Fig. 17A, respectively. Fig. 17B is the cross-sectional structure at A-A. The tab 473 has a surface 473a inclined in the direction toward the center of the dish portion and a member 473b connected to the surface 473a and having a surface in the horizontal direction. For example, the member 473b is a member having a substantially triangular cross-sectional shape as shown in Fig. 17B.

[0210] Similarly, FIG. 17C is an example of a cross-sectional structure at B-B, and FIG. 17D is an example of a cross-sectional structure at C-C. The point that the tab 473 has a surface 473a on which it inclines and a member 473b having a substantially triangular cross-sectional shape is the same as in FIG. 17A. However, as can be seen from FIGS. 17B to 17D, the height of the member 473b may vary depending on the position of the tab 473. For example, when the height up to the member 473b in FIG. 17B is L1, the height up to the member 473b in FIG. 17C is L2, and the height up to the member 473b in FIG. 17D is L3, then L1 < L2 < L3.

[0211] For example, when the automatic feeder 470 moves the spoon along the trajectory indicated by the arrow in FIG. 17E, the food in the spoon collides with the member 473b at a relatively low position (FIG. 17B). As a result, when the tip of the spoon is pulled upward along the tab 473, a relatively large amount of food falls onto the dish portion, and the amount of food remaining in the spoon decreases.

[0212] On the other hand, when the automatic feeder 470 moves the spoon along the trajectory indicated by the arrow in FIG. 17G, the food in the spoon collides with the member 473b at a relatively high position (FIG. 17D). As a result, when the tip of the spoon is pulled upward along the tab 473, the degree of contact between the food and the tab 473 relatively decreases, so the amount of food falling onto the dish portion decreases, and the amount of food remaining in the spoon increases.

[0213] Also, when the automatic feeder 470 moves the spoon along the trajectory indicated by the arrow in FIG. 17F, an intermediate amount of food between the case of FIG. 17E and the case of FIG. 17G remains in the spoon.

[0214] In this way, by providing tabs 473 with different heights according to the position, as shown in FIGS. 17E to 17G, by adjusting the trajectory of the spoon, the amount of food for one mouthful can be easily adjusted. For example, if it is determined that the amount of food scooped up by the spoon on the trajectory of FIG. 17G is too much as the amount for one mouthful as a result of weighing 474 or image processing measurement, it is possible to easily reduce the amount by changing the trajectory to FIG. 17F or FIG. 17E. At this time, it is possible to perform the scooping operation again after returning all the food already scooped up by the spoon to the container 472, but this is not essential. For example, by an easy operation of moving the spoon along a new trajectory with the food remaining in the spoon, an appropriate amount of food collides with the tab 473 and falls into the container 472, so it is also possible to perform fine adjustment in the direction of reducing the amount. As a result, it becomes possible to automatically supply an amount of food suitable for the assisted person. Here, an example of providing tabs 473 with different heights according to the position has been described, but the method of this embodiment is not limited to this. For example, a movable tab 473 may be provided, and the amount of food for one mouthful may be adjusted by adjusting the angle or height of the tab 473.

[0215] When the automatic supply device 470 is used, the operation log of the automatic supply device 470 and the operation log of the terminal device 462 that controls the automatic supply device 470 may be automatically recorded. In this way, various information can be automatically collected, such as at what pace the meal was taken, what situations occurred during the assisted person's meal, how the situation was handled, and whether the handling was effective.

[0216] Also, the food intake amount may be automatically recorded. For example, in a nursing facility, it is often the case that a memo with a barcode is provided along with the meal. Therefore, when the above-described loop process ends, a process of reading the barcode using a camera may be performed. The camera here may be the camera 475 provided on the arm 471, or may be the camera of the terminal device 462. Thereby, the target care recipient can be identified. Also, as described above, the food intake amount is obtained as the total amount of food scooped up with a spoon. Also, since it is known which dish on the container 472 has which kind of dish served, the intake content can also be specified based on the operation log of the arm 471. Also, it may be determined which dish of food has been consumed using RFID or the like, and the process of specifying the intake content can be variously modified. Therefore, in this embodiment, a process of associating and storing the intake amount and the intake content with the care recipient corresponding to the barcode may be performed. Note that in this embodiment, it may be possible to select in what order a plurality of dishes are eaten. For example, the automatic supply device 470. It may be controlled to rotate each dish in order, or may be controlled to intensively feed a specific dish. And in the automatic recording of the intake amount, the order of eating the dishes may be recorded.

[0217] Also, in meal assistance, a case where a small number (narrowly, one) of caregivers assist a plurality of care recipients simultaneously is also conceivable. In this case, a swallowing / muscle detection device 460 may be provided for each, and information regarding swallowing and muscle may be monitored. Also, as described above, the automatic supply device 470 may be used for a care recipient who has difficulty eating. In this case, the swallowing / muscle detection device 460 and the automatic supply device 470 used by a plurality of care recipients are not prevented from operating independently. However, for a plurality of care recipients assisted by the same caregiver, the plurality of swallowing / muscle detection devices 460 and the automatic supply devices 470 may operate in cooperation.

[0218] For example, assume that N swallowing apnea detection devices 460 and N automatic supply devices 470 are used by N care recipients (N is an integer of 2 or more). In this case, one of the N swallowing apnea detection devices 460 may operate as a master, and the other N - 1 devices may operate as slaves. For example, the slave swallowing apnea detection device 460 may transmit detection results such as swallowing or apnea to the master swallowing apnea detection device 460. Then, the master terminal device 462 may perform a process of collectively displaying the outputs of the N swallowing apnea detection devices 460.

[0219] FIG. 18 shows an example of a screen displayed on the display unit of the master terminal device 462. In the example of FIG. 18, information regarding six care recipients is displayed. Information regarding each care recipient includes an audio waveform that is the output of the throat microphone 461 and a determination result as to whether the care recipient is normal, requires attention, or is abnormal based on the swallowing apnea determination. The display mode of the determination result is arbitrary. For example, as shown in FIG. 18, the determination result may be displayed using the lighting / extinguishing of three buttons corresponding to normal, requires attention, and abnormal, respectively. Normal, for example, represents a state in which no apnea is detected. Requires attention represents, for example, a state in which apnea is detected but the degree of risk is low. Abnormal represents a state in which a highly risky apnea is detected. In this way, it becomes possible to display information on a plurality of care recipients in an easy-to-understand manner. In particular, in the example of FIG. 18, since the amount of information per care recipient is limited, the caregiver can easily determine whether intervention is necessary.

[0220] Also, it may be possible to control N automatic supply devices 470 using the terminal device 462 as the master. For example, as shown in FIG. 18, a stop button for emergently stopping the automatic supply device 470 may be displayed in the display area for each care recipient. When a selection operation of the stop button is performed, the terminal device 462 as the master stops the operation of the automatic supply device 470 used by the target care recipient. For example, when the stop button of the care recipient using the terminal device 462 as the master is pressed, the terminal device 462 directly outputs a control signal for stopping the automatic supply device 470. When the stop button of the care recipient using a terminal device 462 other than the master is pressed, the terminal device 462 as the master transmits information instructing the stop to the slave terminal device 462 used by the target care recipient, and the slave terminal device 462 performs control to stop the target automatic supply device 470. Note that the swallowing muscle detection device 460 (terminal device 462) as the master and the swallowing muscle detection device 460 (terminal device 462) as the slave may be directly connected or may be connected via the server system 100.

[0221] Also, instead of any of the plurality of swallowing muscle detection devices 460 operating as the master, the terminal device 200 of the caregiver may control the plurality of swallowing muscle detection devices 460. For example, the screen shown in FIG. 18 may be displayed on the display unit 240 of the terminal device 200 of the caregiver. Also, when a selection operation of any stop button is performed, the terminal device 200 may output a signal instructing the stop of the automatic supply device 470 to the target terminal device 462 directly or via the server system 100.

[0222] Also, as can be understood from the above description, the processing content of the swallowing muscle detection device 460 changes depending on whether the automatic supply device 470 is used or not. In other words, the operation mode of the swallowing muscle detection device 460 changes when the main factor is determined to be the first core factor. Thus, in the present embodiment, the operation mode of the device used for assistance may be changed based on the estimation result of the main factor.

[0223] For example, an information processing apparatus 20 (e.g., a server system 100) is used for assisting a care recipient and includes a communication unit 130 that communicates with a device operating in any of a plurality of operation modes. The device here may be, for example, a sensing device 400 or other device. Then, the communication unit 130 may transmit information representing the main factor estimated by the factor estimation unit 112 to the above device as information for determining in which of the plurality of operation modes to operate. In this way, the operation mode of each device can be automatically changed based on the estimation result of the main factor. As a result, at least a part of the response to peripheral symptoms can be automated, making it possible to further reduce the burden on the caregiver.

[0224] For example, the device here may include a first device that operates in a plurality of operation modes including a first mode for performing normal processing for detecting at least one of swallowing and choking during the meal of the care recipient, and a second mode for performing processing for automatically delivering food to the mouth of the care recipient in addition to the normal processing. The first device here is, for example, a swallowing and choking detection device 460. When it is estimated that the main factor is the first core factor, the communication unit (e.g., the communication unit 130 of the server system 100) of the information processing apparatus 20 transmits information representing the main factor to the first device as information for instructing the transition from the first mode to the second mode. When the first device receives information indicating that the main factor is the first core factor from the information processing apparatus 20, it changes the operation mode from the first mode to the second mode. In this way, not only the monitoring of swallowing and choking but also the automation of meals can be realized. Furthermore, since the operation mode change is automated, the burden on the caregiver can be reduced.

[0225] 2.4 Data Transmission from Server System to Assistance Device As described above by taking the swallowing choking detection device 460 as an example, the sensing device 400 in the present embodiment includes a plurality of operation modes and may operate in any of these operation modes. For example, the sensing device 400 includes a plurality of applications that each execute processing corresponding to the tacit knowledge of a skilled person, and the operation mode may be changed by controlling the activation / inactivation of each application. That is, the tacit knowledge used is switched by switching the operation mode.

[0226] For example, each application may be a learned model generated using machine learning. Machine learning here represents learning based on training data in which input data (sensing data) and correct answer data corresponding to the input data are associated. The correct answer data is data input by a skilled person and is, for example, information representing the determination result of a skilled person in assistance. In this way, it becomes possible to appropriately digitize the tacit knowledge of a skilled person. Since the tacit knowledge of a skilled person is digitized, even an assistant with a low level of proficiency can perform assistance similar to that of a skilled person. For example, each process in each sensing device 400 described above with reference to FIGS. 6 to 10 may correspond to tacit knowledge one by one. Also, a device capable of executing a plurality of processes corresponding to tacit knowledge may be a device different from the sensing device 400.

[0227] For example, the switching of tacit knowledge may be executed based on the peripheral symptom score and the estimated result of the main cause as described above. In this way, since the tacit knowledge used can be switched according to the state of dementia of the person being assisted, it is possible to execute assistance suitable for dealing with and suppressing the occurrence of peripheral symptoms.

[0228] Also, the switching of tacit knowledge may be executed using the ability information of the care recipient. The ability information here is information representing the activity ability of the care recipient, and for example, it is information obtained as a result of the sensing device 400 performing processing using some tacit knowledge. The ability information may be the same information as the input data related to the first core factor, or may be different information. The ability information may be, for example, an index value representing the degree of ADL, or may be information related to the level of risk that may occur in care. The risks here include various risks such as the risk of falling related to falling, the risk of falling related to falling from a wheelchair, bed, etc., the risk of aspiration related to aspiration pneumonia, the risk of pressure ulcers related to pressure ulcers, etc. For example, when a specific risk is low, there is little need to use the tacit knowledge corresponding to the risk. For example, when the risk of aspiration is low, there is little need to use the tacit knowledge for determining the dangerous choking described above. Conversely, when a specific risk is high, it is desirable to actively use the tacit knowledge corresponding to the risk. Therefore, by considering the ability information, the switching of tacit knowledge can be appropriately executed.

[0229] Also, for the switching of tacit knowledge, scene information for specifying the care scene of the care recipient may be used. The scene information may be information for specifying the type of care to be executed, such as meal assistance, excretion assistance, transfer assistance, etc. Also, the scene information may be information related to the caregiver, such as the number and proficiency of the caregivers who execute the care for the care recipient. Also, the scene information may be information related to the care recipient, such as the attributes of the care recipient. In this way, it becomes possible to use tacit knowledge suitable for the type of care, the situation of the caregiver (such as whether they can handle risks with ease), and the attributes of the care recipient.

[0230] For the switching of tacit knowledge, device type information representing the type of the sensing device 400 used in combination may be used. The device type here represents a rough classification such as a wheelchair or a bed, and may be information that does not distinguish between vendors. In this way, it is possible to determine the tacit knowledge used in consideration of the sensing device 400 used in combination. For example, it becomes possible to use tacit knowledge suitable for the cooperation of a plurality of sensing devices 400.

[0231] Considering these, it is desirable for the server system 100 of the present embodiment to notify each sensing device 400 of ability information, scene information, device type information, and an estimated result of a main factor as information for specifying an operation mode. In the following, an example of notifying the estimated result of the main factor will be described, but the peripheral symptom score may be notified. Further, both the peripheral symptom score and the estimated result of the main factor may be notified.

[0232] However, there is a possibility that products of various vendors are mixed in the above-described sensing device 400. At this time, if the data structures of the data for transmitting the above ability information and the like are different for each vendor, the processing load on the server system 100 becomes large and the generality of communication control becomes low. Therefore, the server system 100 of the present embodiment may unify the data structure regardless of the vendor, model number, type, etc. of the sensing device 400. For example, the server system 100 makes the bit assignment of the MAC frame used in the data link layer of the communication with the sensing device 400 common regardless of the destination sensing device 400.

[0233] FIG. 19 is an example of the format of the MAC frame. Note that the format example shown in FIG. 19 is also applicable to management frames and control frames, but in the following, the data frame will be described as an example. As shown in FIG. 19, the MAC frame includes a MAC header, a frame body, and a trailer.

[0234] The MAC header includes fields such as Frame Control, Duration ID, Address 1, Address 2, Address 3, Sequence Control, Address 4, QoS Control, and HT Control. Also, some of these may be omitted.

[0235] Frame Control includes a type field for discriminating whether the target MAC frame is a data frame, a management frame, or a control frame. Frame Control may also include a subtype field for specifying a more detailed type. Information representing the scheduled period of using radio waves is stored in Duration / ID. The scheduled time of using radio waves may also be referred to as the time required for frame transmission. Duration / ID is used for RTS (Request To Send) / CTS (Clear to Send), etc.

[0236] Address 1 to Address 4 store information representing the addresses of the destination and source devices. For example, Address 1 corresponds to the destination address, and Address 2 corresponds to the source address. Data corresponding to the frame usage is stored in Address 3 and Address 4.

[0237] Sequence Control corresponds to the sequence number of the data to be transmitted. QoS Control stores information used for QoS control. QoS control represents control for performing transmission considering the priority of the frame. HT Control is a field used, for example, in management frames.

[0238] The configuration of the frame body will be described later with reference to FIGS. 20A and 20B. Also, the trailer is, for example, FCS (Frame Check Sequence). FCS is information used for error detection of the frame and is, for example, a checksum code. FSC is, for example, CRC (Cyclic Redundancy Code).

[0239] Figures 20A and 20B are diagrams for explaining an example of bit allocation of a frame body of a data frame transmitted by the server system 100. As shown in Figure 20A, the frame body may include fields of User ADL, scene flag, device type ID, primary factor, data type ID, Instruction length, and contents.

[0240] User ADL is a field for storing the ability information of the assisted person. For example, the ability information is numerical data indicating which stage the ability of the assisted person belongs to when the degree of ability is divided into a predetermined number of stages. For example, if the number of stages is 8 or less, User ADL is a 3-bit field. When the number of stages is 9 or more, User ADL may be a 4-bit or more field. Since the required number of bits depends on the definition of the ability information and is known, User ADL may be a fixed-length field. Also, the ability information in this embodiment is not limited to the index value of ADL, and may be more detailed information such as information representing the way of getting up, seat holding ability, swallowing ability, walking ability, etc. Therefore, User ADL may be a field having the number of bits capable of expressing each of these abilities. Also, User ADL may include an ID for identifying the assisted person.

[0241] The Scene flag is a field for storing scene information. For example, the scene information may include a bit indicating whether the number of assistants is equal to or greater than a predetermined number. When the bit has a first value (e.g., 0), it indicates that the number of assistants is equal to or greater than the predetermined number, and when it has a second value (e.g., 1), it indicates that the number is less than the predetermined number. The scene information may also include bits for specifying the type of assistance. For example, when identifying four types of assistance, namely, meal assistance, excretion assistance, transfer assistance, and others, the scene information includes 2 bits as bits for specifying the type of assistance. For example, when the 2 bits are 00, it represents meal assistance; when they are 01, it represents excretion assistance; when they are 10, it represents transfer assistance; and when they are 11, it represents others. The scene information is not limited to these, and other information may be used. Therefore, various modifications can be made to the specific number of bits and meanings of the Scene flag. However, since it is known what scene information is used and the number of bits required for representing the scene information is also known, the Scene flag may be a fixed-length field.

[0242] The device type ID is a field for storing device type information. The device type information may be different for each sensing device 400 described above with reference to FIGS. 6 to 10. Alternatively, for a sensing device 400 corresponding to a fall risk, the same device type ID may be assigned to the imaging device 410 in FIG. 6 and the bedside sensor 420 in FIG. 7. Alternatively, the device type ID may be assigned based on the type of sensor or the like that the sensing device 400 has. Since the number of target device type IDs is known, the device type ID may be a fixed-length field.

[0243] The "primary factor" is a field representing the estimation result of the main factor. For example, when the main factor is selected from any of the environmental factor, the first core factor, the second core factor, and the psychological factor, the "primary factor" may be 2-bit data identifying these four. However, as described above, the factor estimation unit 112 may estimate more detailed factors obtained by subdividing the above four factors, and the data structure of the "primary factor" can be variously modified and implemented.

[0244] Also, although the "primary factor" is exemplified here, instead of the "primary factor" field, a field storing information indicating the presence or absence of peripheral symptoms may be used. For example, a peripheral symptom score may be stored in the field.

[0245] The data type ID, instruction length, and contents shown in FIG. 20A are fields used for transmitting a control signal to a device to be controlled. The device to be controlled here may be the terminal device 200 used by the caregiver, or the sensing device 400, or another device used for the response represented by the response information. The data type ID is a field storing information representing the type of instruction output to the device to be controlled. The instructions here may include four types: "notification (alarm)", "movement / transportation", "control", and "recommendation, etc.". In this case, the data type ID is a 2-bit fixed-length field capable of identifying the four instructions.

[0246] "Notification" refers to the information used when notifying the processing result in the sensing device 400 to the device to be controlled. For example, "notification" may be an instruction used when notifying, to the terminal device 200 or the like, that a fall risk has been detected in a device for determining the fall risk. "Movement / transportation" refers to an instruction to move a movable device to be controlled, such as a reclining wheelchair or a walker. For example, the instruction for movement / transportation may be used for control to move a walker or the like closer so that the target care recipient can be caught when a fall risk is detected. "Control" broadly includes controls other than "movement / transportation" for operating the device to be controlled, and includes changing the angle of the back part of a reclining wheelchair, changing the bottom angle of a care bed, etc. "Recommendation, etc." includes, for example, recommendations for purchasing products used for improving the quality of assistance. For example, a recommendation may represent an instruction to output, to a device to be controlled, such as the terminal device 200 of the caregiver, that a determination has been made to recommend the use of a cushion on the bed 610 or the wheelchair 630. Also, "recommendation, etc." may include news distribution, etc. For example, "recommendation, etc." may be used for introducing popular sensing devices 400.

[0247] contents is a field that stores information specifying the content of a specific instruction. For example, in the contents field, information such as the notification content in "notification", the current position and the target position in "movement / transportation", the part to be controlled and the driving amount in "control", and the recommended product in "recommendation" is stored. Note that the length of the contents field varies according to the data type ID and the specific instruction content. Therefore, as shown in FIG. 20A, the frame body may include an instruction length field for storing the length of the contents field before the contents field. Since the maximum length of the contents field is considered to be known, the instruction length may be, for example, a fixed-length field.

[0248] Although the present embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications are possible without substantially departing from the novel matters and effects of the present embodiment. Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, in the specification or drawings, a term described at least once together with a broader or synonymous different term can be replaced with the different term at any location in the specification or drawings. Also, all combinations of the present embodiment and its modifications are included in the scope of the present disclosure. Further, the configurations and operations of the information processing system, information processing device, server system, terminal device, sensing device, etc. are not limited to those described in the present embodiment, and various modifications can be made.

Explanation of Signs

[0249] 10… Information processing system, 20… Information processing device, 21… Acquisition unit, 22… Factor estimation unit, 100… Server system, 110… Processing unit, 111… Acquisition unit, 112… Factor estimation unit, 113… Countermeasure determination unit, 114… Communication processing unit, 115… Learning unit, 116… Presentation processing unit, 120… Storage unit, 121… User information, 122… Device information, 123… Log data, 124… Learned model, 125… Countermeasure table, 130… Communication unit, 200… Terminal device, 210… Processing unit, 220… Storage unit, 230… Communication unit, 240… Display unit, 250… Operation unit, 300… Management terminal device, 400… Sensing device, 410… Imaging device, 420… Bedside sensor, 430… Detection device, 440… Seat surface sensor, 441… Cushion, 442… Control box, 450… Incontinence detection device, 460… Swallowing mucus detection device, 461… Throat microphone, 462… Terminal device, 470… Automatic supply device, 471… Arm, 472… Container, 473… Tab, 473a… Surface, 473b… Member, 474… Scale, 475… Camera, 510… Reclining wheelchair, 520… Nursing bed, 610… Bed, 620… Mattress, 630… Wheelchair, IM1… Output image, Se1~Se4… Pressure sensor

Claims

1. In a sensing device having a plurality of operation modes, an acquisition unit that acquires input data including sensing data acquired using a sensor; Based on the input data, a factor estimation unit that estimates which of a plurality of factors including an environmental factor caused by the surrounding environment of the care recipient, a first core factor caused by the executive function disorder of the care recipient, a second core factor caused by the disorientation of the care recipient, and a psychological factor caused by the psychology of the care recipient is the main factor for the care recipient receiving assistance from a caregiver to be in a peripheral symptom state where peripheral symptoms of dementia are observed; A communication unit that communicates with the sensing device; Including; The factor estimation unit is A learned model generated by a learning process based on training data in which correct data for specifying the main factor of the peripheral symptom is associated with the input data, and based on the learned model that takes the input data as input and outputs a factor score representing the degree of contribution to the peripheral symptom state for each of the environmental factor, the first core factor, the second core factor, and the psychological factor, estimates the main factor; The communication unit is An information processing device that causes the sensing device to switch to any of the plurality of operation modes by outputting the estimation result of the main factor to the sensing device.

2. In Claim 1, The factor estimation unit is Based on the input data that is time series data, for each of the environmental factor, the first core factor, the second core factor, and the psychological factor, by comparing the factor scores at two different timings, obtains the degree of variation of the factor score, and based on the degree of variation of the factor score, estimates the main factor. An information processing device.

3. In Claim 1, An information processing device further including a countermeasure determination unit that determines a countermeasure for the peripheral symptom state based on information in which the main factor estimated by the factor estimation unit and a countermeasure recommended for the main factor are associated.

4. In Claim 3, The countermeasure determination unit is An information processing device that performs a process of determining, as the countermeasure, to identify a person related to the care recipient who has a high degree of influence on the psychology of the care recipient when the main factor is estimated to be the psychological factor.

5. In Claim 3, The input data is As data related to the environmental factor, includes information for specifying a season. The cause estimation unit determines whether the main cause is a seasonal factor resulting from the season among the environmental factors by determining whether the peripheral symptoms are more likely to occur in a specific season than in other seasons, The countermeasure determination unit is an information processing apparatus that differentiates the countermeasures depending on whether the main cause is estimated to be the seasonal factor or an environmental factor other than the seasonal factor.

6. In any one of Claims 1 to 5 The input data is an information processing apparatus including ability information representing the ability of the care recipient in daily operations as data regarding the first core cause.

7. In any one of Claims 1 to 5 The input data is an information processing apparatus including wandering information representing the wandering of the care recipient as data regarding the second core cause.

8. In any one of Claims 1 to 5 The input data is an information processing apparatus including biometric information of a relative of the care recipient as data regarding the psychological factor.

9. In Claim 2 The cause estimation unit when the input data includes a plurality of the sensing data of different types from each other, performs a first determination for determining the main cause based on first input data that is part of the plurality of the sensing data, and when the main cause is not estimated in the first determination, performs a second determination for determining the main cause based on second input data including data not included in the first input data among the plurality of the sensing data.

10. In Claim 9 a degree of association with the environmental factor, the first core cause, the second core cause, and the psychological factor is set for each of the plurality of the sensing data, and when data determined to have a degree of association biased by a predetermined amount or more toward any one of the environmental factor, the first core cause, the second core cause, and the psychological factor among the plurality of the sensing data is defined as specific input data, the second input data includes at least one piece of the specific input data not included in the first input data.

11. In any one of Claims 1 to 5 The learned model is generated by a learning process based on training data in which correct data including information for identifying the main cause of the peripheral symptom and information indicating whether the assisted person is in the peripheral symptom state is associated with the input data. The factor estimation unit An information processing device that estimates whether the assisted person is in the peripheral symptom state based on the input data and the learned model.

12. In claim 11, The communication unit Communicates with a device used for assisting the assisted person and operating in any of a plurality of operation modes, An information processing device that transmits at least one of information indicating whether the state is the peripheral symptom state estimated by the factor estimation unit and information representing the main cause to the device as information for determining in which of the plurality of operation modes to operate.

13. In claim 12, The device Includes a first device that operates in the plurality of operation modes including a first mode that performs a normal process of detecting at least one of swallowing and choking in the meal of the assisted person and a second mode that performs a process of automatically transporting food to the mouth of the assisted person in addition to the normal process. The communication unit An information processing device that transmits information representing the main cause to the first device as information for instructing a transition from the first mode to the second mode when the main cause is estimated to be the first core cause.

14. In claim 2, The learned model is generated by a learning process based on training data in which correct data including information for identifying the main cause of the peripheral symptom and information indicating whether the assisted person is in the peripheral symptom state is associated with the input data. The factor estimation unit An information processing device that obtains the main cause based on the input data and a learned model that outputs a peripheral symptom score representing the probability that the assisted person is in the peripheral symptom state and a factor score for each of the plurality of factors.

15. In claim 14, An information processing device including a learning unit that performs a process of updating the learned model based on the factor information, a first peripheral symptom score that is the peripheral symptom score before the presentation of the factor information, and a second peripheral symptom score that is the peripheral symptom score after the presentation of the factor information when the presentation process of the factor information representing the main cause is performed.

16. In claim 15, A countermeasure determination unit that determines a countermeasure for the peripheral symptom state is further included based on the main factor estimated by the factor estimation unit and information in which a countermeasure recommended for the main factor is associated, The learning unit, When the presentation process of the factor information and the countermeasure information representing the countermeasure is performed, an information processing apparatus that performs a process of updating the learned model by determining whether or not the peripheral symptom state of the assisted person has improved after the presentation of the countermeasure information based on the first peripheral symptom score and the second peripheral symptom score.

17. In the information processing apparatus according to any one of claims 1 to 5, The communication unit, In the data link layer of communication with the sensing device, a data frame including a MAC (Media Access Control) header, a frame body, and a trailer, and an information processing apparatus that transmits the data frame in which a fixed-length region including a first region for storing an estimation result of the main factor is included in the frame body.

18. In the information processing apparatus according to any one of claims 1 to 5, The communication unit, An information processing apparatus that causes the sensing device to switch to any of the plurality of operation modes by outputting ability information representing the activity ability of the assisted person, scene information specifying an assistance scene of the assisted person, and device type information representing a type of a device used in combination with the sensing device to the sensing device.

19. In the information processing apparatus according to claim 18, The communication unit, In the data link layer of communication with the sensing device, a data frame including a MAC (Media Access Control) header, a frame body, and a trailer, and a fixed-length region including a first region for storing an estimation result of the main factor, a second region for storing the ability information, a third region for storing the scene information, and a fourth region for storing the device type information is included in the frame body, and an information processing apparatus that transmits the data frame.

20. An information processing apparatus, Obtains input data including sensing data acquired using a sensor in a sensing device having a plurality of operation modes, Based on the input data, estimate which of a plurality of factors is the main factor that causes the care recipient who receives assistance from a caregiver to enter a peripheral symptom state in which peripheral symptoms of dementia are observed. The plurality of factors include an environmental factor caused by the peripheral environment of the care recipient, a first core factor caused by the executive function disorder of the care recipient, a second core factor caused by the disorientation disorder of the care recipient, and a psychological factor caused by the psychology of the care recipient. By outputting the estimation result of the main factor to the sensing device, cause the sensing device to switch which of the plurality of operation modes it operates in. In the estimation of the main factor It is a learned model generated by a learning process based on training data in which correct answer data for specifying the main factor of the peripheral symptom is associated with the input data. Based on the learned model that takes the input data as input and outputs a factor score representing the degree of contribution to the peripheral symptom state for each of the environmental factor, the first core factor, the second core factor, and the psychological factor, estimate the main factor. Information processing method.

Images