Information processing system, information processing apparatus, and information processing method

Abstract

To provide an information processing system, an information processing apparatus, and an information processing method for appropriately supporting a caregiver's assistance for a person to be cared.SOLUTION: An information processing system includes: a device including an application that executes processing corresponding to tacit knowledge of a skilled person; and a server system that transmits a data frame including a media access control (MAC) header, a frame body, and a trailer in a data link layer of communication with the device. The server system transmits the data frame in which an area with a fixed length including a first area storing ability information, a second area storing scene information, and a third area storing device type information, is included in the frame body. The device determines active or non-active of the application based on at least one of the ability information, scene information, and device type information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and the like. [Background technology]

[0002] A system for use when a caregiver provides care to a care recipient has been known. Patent Document 1 discloses a method for generating information to be provided regarding the condition of a resident in a living space based on time-varying changes in detected information acquired by a sensor placed in the living space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-18760 Summary of the Invention [Problem to be solved by the invention]

[0004] An information processing system, an information processing device, an information processing method, etc. are provided that appropriately support a caregiver in providing assistance to a person being assisted. [Means for solving the problem]

[0005] One aspect of the present disclosure includes a device including an application that performs processing corresponding to the tacit knowledge of an expert, and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer at a data link layer for communication with the device, wherein the server system transmits the data frame, the frame body containing a fixed-length field including a first field that stores ability information representing the activity ability of a person being assisted, and the device is related to an information processing system that determines whether the application is active or inactive based on the ability information. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 2] FIG. 10 is a diagram illustrating an example of a relationship between devices and tacit knowledge. [Figure 3] FIG. 1 illustrates an example of the configuration of a server system. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a device. [Figure 5] FIG. 10 is a diagram showing an example of the relationship between the ability of a person being assisted and expected risks. [Figure 6] FIG. 10 is a sequence diagram illustrating processing of the information processing system. [Figure 7] FIG. 2 is a sequence diagram illustrating processing of the information processing system. [Figure 8] FIG. 10 shows a specific example of a device related to fall risk. [Figure 9] FIG. 10 shows a specific example of a device related to fall risk. [Figure 10] FIG. 10 shows a specific example of a device related to fall risk. [Figure 11] FIG. 10 shows a specific example of a device related to fall risk. [Figure 12] FIG. 10 shows a specific example of a device related to aspiration risk. [Figure 13] FIG. 1 shows a specific example of a device related to pressure ulcer risk. [Figure 14] FIG. 1 shows a specific example of a device related to pressure ulcer risk. [Figure 15A] 10 is an example of a screen used for adjusting the bed position. [Figure 15B] 10 is an example of a screen used for adjusting the bed position. [Figure 16] This is an example of a screen used in end-of-life care. [Figure 17] FIG. 10 is a diagram illustrating an example of operation modes of a device according to its capabilities. [Figure 18] FIG. 2 is a sequence diagram illustrating processing of the information processing system. [Figure 19]10 is a flowchart illustrating a process for determining an operation mode in a device. [Figure 20] 10 is a flowchart illustrating a process for determining an operation mode in a device. [Figure 21] FIG. 10 is a diagram illustrating an example of cooperation between the choking hazard detection device and another device. [Figure 22] 10 is a flowchart illustrating a process for determining an operation mode in a device. [Figure 23] FIG. 10 is a diagram illustrating a specific example of a device to be controlled. [Figure 24] FIG. 10 is a diagram illustrating a specific example of a device to be controlled. [Figure 25] FIG. 10 is a diagram illustrating a configuration example of an information processing system when serverless communication is performed. [Figure 26] FIG. 2 is a diagram illustrating an example of the configuration of a communication processing unit and a communication unit. [Figure 27] FIG. 10 is a diagram illustrating an example of the configuration of a MAC frame. [Figure 28A] FIG. 10 is a diagram illustrating an example of the configuration of a frame body. [Figure 28B] FIG. 10 is a diagram illustrating an example of the relationship between data type ID and contents. [Figure 29A] FIG. 10 is a diagram illustrating an example of the configuration of a frame body. [Figure 29B] FIG. 10 is a diagram illustrating an example of the relationship between data type ID and contents. [Figure 30] FIG. 10 is a diagram illustrating an example of parameters according to access categories. [Figure 31] FIG. 10 is a diagram illustrating an example of allocation of access categories. [Figure 32] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system for home care. [Figure 33] 10 is an example of a screen displayed on a terminal device of a caregiver. [Figure 34] 10 is an example of a screen displayed on a terminal device of a caregiver. [Figure 35] 10 is an example of a screen displayed on a terminal device of a caregiver. [Figure 36A]10 is an example of a screen displayed on a terminal device of a care manager. [Figure 36B] 10 is an example of a screen displayed on a terminal device of a care manager. [Figure 36C] 10 is an example of a screen displayed on a terminal device of a care manager. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, the present embodiment will be described with reference to the drawings. In the drawings, identical or equivalent elements are designated by the same reference numerals, and duplicate explanations will be omitted. Note that the present embodiment described below does not unduly limit the content described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

[0008] 1. System configuration example 1 shows an example of the configuration of an information processing system 10 according to this embodiment. The information processing system 10 according to this embodiment is for providing instructions to caregivers so that they can provide appropriate care regardless of their level of skill by digitizing the "intuition" and "tacit knowledge" of caregivers who perform tasks that are normally performed by the "intuition" and "tacit knowledge" of caregivers in, for example, medical facilities or nursing care facilities.

[0009] The caregiver here may be a caregiver at a nursing facility, or a nurse or licensed practical nurse at a medical facility such as a hospital. That is, assistance in this embodiment includes various actions to support the person being assisted, and may include nursing care or medical actions such as injections. The person being assisted here is a person receiving assistance from the caregiver, and may be a resident of a nursing facility or a patient who is hospitalized or visiting a hospital.

[0010] In addition, the assistance in this embodiment may be provided at home. For example, the person being assisted in this embodiment may be a person requiring care who receives home care, or a patient receiving home medical care. The caregiver may be a family member of the person requiring care or the patient, or a visiting helper, etc.

[0011] The information processing system 10 shown in FIG. 1 includes a server system 100, devices 200, and a gateway 300. However, the configuration of the information processing system 10 is not limited to that shown in FIG. 1 and various modifications are possible, such as omitting some components or adding other components. For example, FIG. 1 illustrates devices 200 as examples of a tablet-type terminal device such as a smartphone, a seat sensor 440 (described later using FIG. 10) placed on a wheelchair 630, and a detection device 430 (described later using FIG. 9) placed on a bed 610, but the number and types of devices 200 are not limited thereto. For example, the information processing system 10 may include various devices 200, which will be described later using FIG. 8 to FIG. 14. Furthermore, the information processing system 10 may include devices other than the devices 200 shown in FIG. 8 to FIG. 14. Note that, hereinafter, when it is not necessary to distinguish between multiple devices 200, they will be simply referred to as devices 200. Furthermore, modifications such as omissions and additions of components are possible, as in FIG. 3, FIG. 4, and so forth, which will be described later.

[0012] The information processing device of this embodiment corresponds to, for example, the server system 100. However, the method of this embodiment is not limited to this, and the processing of the information processing device may be executed by distributed processing using the server system 100 and other devices. For example, the information processing device of this embodiment may include the server system 100 and the device 200. An example in which the information processing device is the server system 100 will be described below.

[0013] The server system 100 is connected to the device 200 via, for example, a network. For example, the server system 100 is connected to a gateway 300 via a public communication network such as the Internet, and the gateway 300 is connected to the device 200 using a LAN (Local Area Network) or the like. For example, the gateway 300 may be an access point (AP) that performs communication in accordance with the IEEE802.11 standard, and the device 200 may be a station (STA) that performs communication in accordance with the IEEE802.11 standard. However, various modifications can be made to the communication method between the devices.

[0014] The server system 100 may be one server, or may include multiple servers. For example, the server system 100 may include a database server and an application server. The database server stores various data, which will be described later with reference to FIG. 3. The application server performs processing, which will be described later with reference to FIGS. 6 to 7, etc. The multiple servers here may be physical servers or virtual servers. If a virtual server is used, the virtual server may be provided on one physical server, or may be distributed across multiple physical servers. As described above, the specific configuration of the server system 100 in this embodiment can be modified in various ways.

[0015] The device 200 has, for example, various sensors and performs processing based on data sensed by the sensors (hereinafter referred to as sensing data). The digitization of the expert's tacit knowledge described above may be performed, for example, by the vendor of the device 200. For example, assume that the expert has the tacit knowledge to determine whether the expert is shifting forward or to the side based on the posture of the person being assisted in a wheelchair. In this case, the tacit knowledge can be digitized by collecting sensing data corresponding to the posture of the person being assisted detected by the seat sensor 440 and creating an application that determines whether the expert is shifting forward or to the side based on the sensing data. For example, a vendor provides the seat sensor 440 and the above application, so that an unskilled person (e.g., a new employee) can utilize the same forward slippage / lateral slippage determination as an expert.

[0016] Furthermore, the amount of tacit knowledge digitized in one device 200 is not limited to one. For example, the tacit knowledge digitized using the seat sensor 440 is not limited to the determination of forward or lateral slippage, and may include a determination of the possibility of a fall or a combination of both the determination of forward or lateral slippage and the possibility of a fall. The determination of forward or lateral slippage corresponds to determining whether the assisted person's posture is good or bad, and the determination of the possibility of a fall corresponds to determining whether the assisted person has slipped off the seat. Furthermore, in the determination of forward or lateral slippage, multiple tacit knowledge may exist that differ in the determination criteria for determining the degree of slippage as a forward or lateral slippage, the determination process contents, etc. Therefore, each device 200 may be capable of executing processes corresponding to one or more pieces of tacit knowledge and may switch between executing and not executing processes corresponding to each piece of tacit knowledge. For example, a vendor may implement tacit knowledge as application software (hereinafter simply referred to as an application) and register it in the server system 100. Each device 200 executes processes corresponding to the tacit knowledge by downloading and installing registered applications that are executable.

[0017] FIG. 2 is a diagram showing an example of the relationship between devices 200 and tacit knowledge (applications). FIG. 2 illustrates five devices 200a to 200e as devices 200 connected to the server system 100. In the example of FIG. 2, two tacit knowledges, tacit knowledge 1 and tacit knowledge 2, are associated with device 200a. For example, the applications of tacit knowledge 1 and tacit knowledge 2 are already installed in device 200a. Device 200a may be, for example, a device related to the risk of falling, which will be described later. Specifically, device 200a may be a seat sensor 440, and tacit knowledge 1 may be tacit knowledge related to determining whether a device is slipping forward or sideways, and tacit knowledge 2 may be tacit knowledge related to determining the possibility of a fall. Processing results corresponding to tacit knowledge 1 and tacit knowledge 2 are transmitted to, for example, the server system 100.

[0018] Devices 200b and 200c are, for example, devices related to aspiration risk, which will be described later, and each is associated with multiple pieces of tacit knowledge for dealing with aspiration risk, etc. Devices 200d and 200e are, for example, devices related to pressure ulcer risk, which will be described later, and each is associated with multiple pieces of tacit knowledge for dealing with pressure ulcer risk, etc. In this way, it is possible to digitize a variety of tacit knowledge using a variety of devices. Although two pieces of tacit knowledge are illustrated for one device 200 here, the number of pieces of tacit knowledge associated with one device 200 is not limited to this.

[0019] In the method of this embodiment, it is possible to switch the tacit knowledge to be used depending on the situation. For example, it is not necessary to use all of the tacit knowledge 1 to tacit knowledge 10 shown in FIG. 2 , and use / non-use of these may be switched as needed. Switching of tacit knowledge may be achieved by switching the device 200 to be used. For example, when the target is a care recipient with a high risk of falling but a low risk of aspiration and bedsores, device 200a is used, and devices 200b-200e are not used. In this way, at least one of tacit knowledge 1 and tacit knowledge 2 is used, and the other tacit knowledge is not used, making it possible to appropriately use tacit knowledge that is highly necessary for the care recipient. Furthermore, when the risk of aspiration of the care recipient increases, the tacit knowledge to be used can be switched by using device 200b or device 200c. Furthermore, when addressing the risk of aspiration, it is also possible to switch between cases where only device 200b is used, only device 200c is used, or both devices 200b and 200c are used. In this way, for example, when the risk of aspiration is high, necessary tacit knowledge can be used from tacit knowledge 3 to tacit knowledge 6 in FIG. 2 . The same applies when the risk of bedsores increases, and the tacit knowledge to be used can be switched by using device 200d or device 200e. Also, when the risk of falling decreases, it is possible to switch, such as by stopping the use of device 200a.

[0020] Furthermore, switching of tacit knowledge may be realized by switching the tacit knowledge used within the device 200 while maintaining the device 200 in use. For example, in the device 200a, switching may be made between a case where only tacit knowledge 1 is used, a case where only tacit knowledge 2 is used, and a case where both tacit knowledge 1 and tacit knowledge 2 are used. This processing can be realized, for example, by controlling the activation / inactivation of applications corresponding to each tacit knowledge.

[0021] The switching of tacit knowledge may be performed using ability information of the person being assisted. The ability information here is information that indicates the activity ability of the person being assisted, and is, for example, information obtained as a result of the device 200 performing some kind of processing using tacit knowledge. The ability information may be, for example, information related to the level of risk of falling, risk of aspiration, or risk of bedsore. Details of the ability information will be described later.

[0022] For example, when the processing result of tacit knowledge is obtained in a certain device 200, a process of switching the tacit knowledge to be used may be performed in the same device 200. For example, based on the processing result of tacit knowledge 1 in device 200a in Fig. 2, the activation / inactivation of an application corresponding to tacit knowledge 1 and an application corresponding to tacit knowledge 2 may be switched.

[0023] Furthermore, when a processing result of tacit knowledge is obtained in one device 200, the processing result may affect another device 200. For example, based on the processing result of tacit knowledge 1 in device 200a, the tacit knowledge used in device 200b may be switched. For example, a switch may be made from a state in which device 200b is not in use to a state in which device 200b is in use. Alternatively, tacit knowledge 3 and tacit knowledge 4 associated with device 200b may be individually switched between active and inactive.

[0024] Furthermore, the switching of tacit knowledge is not limited to being based on the ability information, but may be performed based on the assistance situation of the person being assisted, the combined use of multiple devices 200, etc. The information used for switching tacit knowledge and the process of switching tacit knowledge will be described in detail later.

[0025] The correspondence between the devices 200 and the tacit knowledge can be flexibly changed. For example, processing corresponding to one piece of tacit knowledge may be executed based on sensing data acquired using multiple devices 200. For example, processing corresponding to tacit knowledge 1 may be executed based on sensing data acquired by device 200a and sensing data acquired by device 200b. This increases the types of sensing data used in processing, thereby improving processing accuracy, etc. In the above example, the processing corresponding to tacit knowledge 1 may be executed by device 200a or device 200b, or may be realized by distributed processing between device 200a and device 200b. Furthermore, the processing corresponding to tacit knowledge is not limited to being executed by device 200 that acquires sensing data. For example, processing corresponding to tacit knowledge 1 may be executed by a device 200 other than device 200a or device 200b based on sensing data acquired by device 200a and sensing data acquired by device 200b. For example, in FIG. 1, the device 200, which is a smartphone, may execute processing corresponding to tacit knowledge based on sensing data acquired by the seat sensor 440 and sensing data acquired by the detection device 430.

[0026] Although the above describes an example in which tacit knowledge is digitized by the vendor of the device 200, the present invention is not limited to this. For example, a caregiver who uses the device 200 may also digitize his or her own tacit knowledge. For example, the caregiver may create an application corresponding to the tacit knowledge and register the application in the server system 100. In this way, it is possible to promote the digitization and use of tacit knowledge.

[0027] 3 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit .

[0028] The processing unit 110 of this embodiment is configured by the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be configured by one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices are, for example, an integrated circuit (IC), a field-programmable gate array (FPGA), etc. The one or more circuit elements are, for example, a resistor, a capacitor, etc.

[0029] The processing unit 110 may also be implemented by the following processor. The server system 100 of this embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information may be, for example, a program and various data. The memory may be the storage unit 120 or another memory. The processor includes hardware. Various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP), may be used. The memory may be a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, or may be a register, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores computer-readable instructions, and the processor executes the instructions to realize the functions of the processing unit 110. The instructions may be instructions from an instruction set that constitutes a program, or instructions that instruct the hardware circuitry of the processor to operate.

[0030] The processing unit 110 includes, for example, a capability information acquisition unit 111 , a scene information acquisition unit 112 , a device type information acquisition unit 113 , and a communication processing unit 114 .

[0031] The ability information acquisition unit 111 performs a process of acquiring ability information that indicates the activity ability of the person being assisted. For example, the ability information acquiring unit 111 acquires sensing data from the device 200. The sensing data here may be, for example, log data for a predetermined period of time. The ability information acquiring unit 111 determines time-series changes in the condition of the person being assisted based on the log data and estimates the ability information based on the changes. The ability information here may be index information related to activities of daily living (ADL). Various methods for evaluating ADL, such as the Barthel Index, are known, and these can be widely applied in this embodiment. Furthermore, the ability information may use a nine-level index disclosed as the Clinical Frailty Scale in "Frailty and the potential kidney transplant recipient: time for a more holistic assessment?" by Henry H.L. Wu et al. For example, the ability information acquiring unit 111 in this embodiment determines which of the nine levels the person being assisted belongs to based on the sensing data.

[0032] The capability information may be obtained in the device 200 or the like. The capability information acquisition unit 111 may execute a process of acquiring the capability information from the device 200 or the like via the communication unit 130.

[0033] The scene information acquisition unit 112 determines a scene in which assistance is provided to the person being assisted. The scene information here may be information specifying the type of assistance to be provided, such as meal assistance, excretion assistance, or movement / transfer assistance. The scene information may also be information about the assistants providing assistance to the person being assisted, such as the number and skill level of the assistants. The scene information may also be information about the person being assisted, such as the attributes of the person being assisted. For example, the scene information acquisition unit 112 performs processing to obtain scene information based on user input, the schedules of the assistants, etc.

[0034] The device type information acquisition unit 113 acquires information that identifies the type of device 200 that operates together with the target device 200. The device type here represents a rough classification such as a wheelchair or a bed, and may be information that does not distinguish between vendors. For example, a wheelchair from a first vendor and a wheelchair from a second vendor different from the first vendor may have the same device type. For example, the device type information acquisition unit 113 may identify other devices 200 used by the person being assisted who uses the target device 200 (or an assistant assisting the person being assisted), and may perform processing to acquire information indicating the type of the other devices 200 as device type information.

[0035] The communication processing unit 114 controls communication using the communication unit 130. For example, the communication processing unit 114 executes processing to create data to be transmitted, such as a MAC frame in the data link layer. The communication processing unit 114 may also perform processing such as interpreting the frame structure of data received by the communication unit 130, extracting necessary data, and outputting it to a higher layer such as an application.

[0036] The storage unit 120 is a work area for the processing unit 110 and stores various types of information. The storage unit 120 can be realized by various types of memory, and the memory may be a semiconductor memory such as an SRAM, a DRAM, a ROM, or a flash memory, or may be a register, a magnetic storage device, or an optical storage device.

[0037] The storage unit 120 may store user information 121 , device information 122 , and application information 123 .

[0038] In the method of this embodiment, for example, tacit knowledge is registered as an application in the server system 100. Each user who uses the tacit knowledge may register a device 200 in the system and then download the necessary application to the device 200 for use.

[0039] The user information 121 includes information such as a user ID and a user name that uniquely identify a user of the information processing system 10, and a device ID that is information that uniquely identifies a device 200 used by the user.

[0040] The device information 122 is information relating to the device 200, and includes a device ID, a device type ID indicating the type of the device 200, a vendor, an application ID of an installed application, etc. The application ID is information that uniquely identifies an application.

[0041] The application information 123 is information about an application, and includes an application ID, an application name, a creator, and the like. The application information may also include information that identifies the specific processing content of the application. The information that identifies the processing content may be the source code of a program or an executable file. If the application corresponds to a trained model, the application information may also include information about the structure of the trained model. For example, if the trained model is a neural network (hereinafter referred to as NN), the structure of the trained model includes the number of layers of the NN, the number of nodes included in each layer, the connection relationships between nodes, weights, activation functions, and the like.

[0042] By using the user information 121, it becomes possible to appropriately manage users who use the information processing system 10. Furthermore, by referring to the device information 122, it becomes possible to check details of the device 200 used by each user. Furthermore, by referring to the application information 123, it becomes possible to check details of each application registered in the server system 100.

[0043] The communication unit 130 is an interface for communicating via a network, and when the server system 100 performs wireless communication, it includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. However, the server system 100 may also perform wired communication, in which case the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 operates under the control of the communication processing unit 114. However, the communication unit 130 may also include a processor for communication control different from the communication processing unit 114. The communication unit 130 may perform communication according to a method specified in, for example, IEEE802.11 or IEEE802.3. However, the specific communication method can be modified in various ways.

[0044] FIG. 4 is a block diagram showing a detailed configuration example of the device 200. The device 200 includes, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250. As will be described later with reference to FIGS. 8 to 14, various aspects of the device 200 can be used in the method of this embodiment. The configuration of each device 200 is not limited to that shown in FIG. 4, and modifications such as omitting some components or adding other components are possible. For example, the device 200 may have various sensors appropriate for the device 200, such as a motion sensor such as an acceleration sensor or a gyro sensor, an imaging sensor, a pressure sensor, a GPS (Global Positioning System) sensor, etc.

[0045] The processing unit 210 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 210 may also be realized by a processor. Various types of processors, such as a CPU, a GPU, or a DSP, can be used as the processor. The processor executes instructions stored in the memory of the device 200, thereby realizing the functions of the processing unit 210 as processing.

[0046] The storage unit 220 is a work area for the processing unit 210, and is realized by various types of memory such as SRAM, DRAM, and ROM.

[0047] The communication unit 230 is an interface for communicating via a network, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100, for example, via the network. The communication unit 230 may perform wireless communication with the gateway 300 in accordance with, for example, the IEEE 802.11 standard, and communicate with the server system 100 via the gateway 300.

[0048] The display unit 240 is an interface that displays various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface that accepts user operations. The operation unit 250 may be a button or the like provided on the device 200. The display unit 240 and the operation unit 250 may also be a touch panel that is integrally configured.

[0049] Device 200 may also include components not shown in FIG. 4, such as a light-emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light-emitting unit is, for example, an LED (light emitting diode) and provides notification by emitting light. The vibration unit is, for example, a motor and provides notification by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and provides notification by sound.

[0050] 2. Device control based on capability information As shown in FIG. 1, the information processing system 10 of this embodiment includes a server system 100 and a device 200. The device 200 may operate in any of a plurality of operation modes. The operation mode in this embodiment may be determined by the combination of tacit knowledge (applications) to be used. For example, as described above, the device 200 can install multiple applications corresponding to different tacit knowledge, and each of the multiple applications can be switched between active and inactive. In this way, it becomes possible to appropriately switch the tacit knowledge to be used depending on the situation.

[0051] In particular, the server system 100 (capability information acquisition unit 111) may obtain capability information representing the activity capability of the person being assisted based on sensing data transmitted from the device. The server system 100 then transmits the obtained capability information to the device 200. The device 200 determines in which of a plurality of operation modes to operate based on the received capability information. Since the assistance to be performed changes in accordance with changes in the capability information of the person being assisted, the desired operation of the device 200 may change. However, according to the method of this embodiment, it is possible to appropriately switch the operation mode in accordance with the capability information. For example, the device 200 can appropriately switch between active and inactive states of an application (use / non-use of tacit knowledge) in accordance with the capability information. Processing based on the capability information will be described below.

[0052] 2.1 Overview Figure 5 shows an example of the relationship between the ability information of a person receiving care and the expected risks. For example, if the ability of a person receiving care is sufficiently high, they can perform daily activities without the assistance of others, so the risk in daily life is not high. However, as their abilities begin to decline, the person receiving care will first have difficulty, for example, performing daily activities. Daily activities refer to the actions of standing up and sitting down. In this case, it becomes difficult to maintain balance when starting to move, including getting up and down, increasing the risk of the person receiving care falling. Starting to move refers to starting to move from a state of small movement (or, in the narrow sense, a stationary state). On the other hand, at this stage, it is expected that there will be little difficulty in daily activities other than starting to move. For example, the person receiving care can maintain a sitting position for long periods of time, walk using a walker after standing up, and eat with some freedom.

[0053] As their abilities decline further, for example, the person receiving care may find it difficult to walk and may need assistance moving around using a wheelchair or other device. In this case, the risk of falling when starting to move is high, just as in the example above, but the person's ability to maintain a sitting position also declines, so the risk of falling must also be considered. For example, even when sitting in a bed or wheelchair, the person receiving care at this stage may lose their balance and fall off the bed mattress or wheelchair seat.

[0054] If the person's abilities decline further, for example, they may find it difficult to eat properly. For example, their swallowing ability may decline, increasing the risk of aspiration. The risk of aspiration means that there is a higher likelihood of developing aspiration pneumonia, for example. At this stage, the person receiving care is considered to be able to move around, so the high risk of falling and tripping described above remains, and the risk of aspiration pneumonia must also be considered.

[0055] As abilities decline further, for example, the person receiving care may become bedridden, requiring assistance with most daily activities. In this case, the risk of falls is still high, as diaper changes may be required in bed and the person may need to move around in a wheelchair. Furthermore, the risk of aspiration pneumonia is also high, as oral feeding will continue unless there are special circumstances such as a gastrostomy. Furthermore, the person will spend a lot of time in bed, and turning over on their own will become difficult, increasing the risk of bedsores. On the other hand, since the person receiving care is not expected to move on their own when they are bedridden, the risk of falling when they do so is low.

[0056] As shown in FIG. 5, the expected risks vary depending on the ability of the person being assisted, and therefore the assistance that the caregiver should provide also changes. Therefore, when using digitized tacit knowledge using device 200, the tacit knowledge to be used also changes depending on the ability. In this regard, in this embodiment, the operation mode of device 200 can be set according to the ability information, so processing can be performed according to changes in ability. For example, when providing assistance to approximately 10 people being assisted as a unit in a nursing home, it is conceivable that the people being assisted have different abilities. However, according to the method of this embodiment, the caregiver does not need to manually set the necessary assistance for each person being assisted. In other words, even when caring for multiple people being assisted, it is possible to set an appropriate operation mode without increasing the burden on the caregiver.

[0057] Although Figure 5 illustrates an example in which ability declines from top to bottom, the direction of ability change is not limited to this. For example, ability may recover due to the cure or remission of a disease, rehabilitation, etc. The method of this embodiment sets the operation mode according to ability, so it can flexibly respond to cases in which ability recovers. Furthermore, Figure 5 illustrates four stages: "unable to sit up and down," "unable to walk," "unable to eat properly," and "bedridden." However, the ability stages represented by the ability information are not limited to these; some stages may be omitted, or other stages may be added. Furthermore, a state in which the risk of falling is low because the patient can walk, but the risk of aspiration is high due to a decline in swallowing ability may also be considered. In other words, the above four stages are not limited to those that change in the order described above, and more complex combinations may be considered.

[0058] FIG. 6 is a sequence diagram illustrating the operations of the server system 100 and the device 200, and is a diagram illustrating pre-processing that is executed before the device 200 executes processing corresponding to tacit knowledge.

[0059] First, in step S101, the server system 100 performs a process of accepting application registration in advance. For example, as described above, each application corresponds to tacit knowledge and is created by the vendor of the device 200 or the like. The creator of the application logs in to the information processing system 10 of this embodiment using, for example, any terminal device (such as a PC or a smartphone), and then performs a process of registering the application in the server system 100 using a vendor screen (not shown) displayed on the display unit of the terminal device. The processing unit 110 of the server system 100 stores information about the registered application in the storage unit 120 as application information 123. Here, an example is shown in which vendor applications 1 to 3, which are applications created by vendors, are registered.

[0060] In step S102, a registration request for the device 200 is transmitted to the server system 100 based on an operation by a user who uses the device 200. The user here may be a caregiver who uses tacit knowledge, or may be an administrator of a nursing facility, for example. For example, the user executes the process of step S102 when introducing a new device 200 into his or her environment. For example, the user logs in to the information processing system 10 using the operation unit of the device 200 or the operation unit of a terminal device connected to the device 200, and then performs a process of registering the device 200 in the server system 100 using a user screen (not shown). The registration request includes, for example, a user ID that identifies the user, and information such as the vendor and model number of the device 200.

[0061] In step S103, the processing unit 110 of the server system 100 executes processing based on the registration request. For example, the processing unit 110 assigns a device ID that uniquely identifies the device 200 to the target device 200 and transmits the device ID to the device 200. The processing unit 110 may also execute processing to associate the logged-in user with the device 200 for which the registration request has been made. For example, the processing unit 110 may perform processing to add the device ID of the device 200 for which the registration request has been made to the user information 121 of the logged-in user. The processing unit 110 may also store the device ID of the device 200 for which the registration request has been made, in association with the device type ID of the device 200, etc., in the device information 122. The device type ID can be identified based on information such as the vendor and model number included in the registration request. Through the above processing, the newly introduced device 200 is registered in the information processing system 10.

[0062] Next, in step S104, an application to be used by the device 200 is selected based on an operation by the user of the device 200. For example, when a registered device 200 accesses the server system 100, the server system 100 may return a screen listing applications that can be used by the device 200. The user performs a user operation to select an application to be used from the listed applications. Here, consider an example in which a list of applications including vendor app 1 to vendor app 3 that were registered in the processing shown in step S101 is displayed, and the user performs an operation to select vendor app 1 to vendor app 3.

[0063] In step S105, the server system 100 permits the download of the selected application, and the device 200 downloads the selected application. The server system 100 may also perform a process of associating the device 200 with the application downloaded to the device 200. For example, the processing unit 110 performs a process of adding application IDs corresponding to vendor applications 1 to 3 to the device information 122 related to the device 200.

[0064] In step S106, the device 200 executes a process of installing the downloaded vendor applications 1 to 3. This enables the device 200 to operate in one of a plurality of operation modes. For example, the device 200 may switch between active and inactive for each of the vendor applications 1 to 3. In this case, the device 200 3= 8 possible operation modes can be selected. In this embodiment, a state in which all of vendor applications 1 to 3 are inactive is also considered to be one operation mode. The relationship between applications and operation modes is not limited to this. For example, multiple applications may operate exclusively. In the above example, device 200 may be able to set four operation modes: a mode in which all vendor applications are inactive, a mode in which only vendor application 1 is active, a mode in which only vendor application 2 is active, and a mode in which only vendor application 3 is active.

[0065] FIG. 7 is a sequence diagram illustrating the operations of the server system 100 and the device 200, and illustrates an example in which the operation mode of the device 200 changes based on the ability information of the person being assisted.

[0066] First, in step S201, the server system 100 performs processing to transmit data including capability information to the device 200. Fig. 7 shows an example in which data with an ADL index value of 2 is transmitted.

[0067] In step S202, the device 200 controls the activation / inactivation of installed vendor applications based on the acquired capability information. For example, the storage unit 220 of the device 200 may store information associating capability information with operation modes. The processing unit 210 of the device 200 performs processing to determine the operation mode based on the information and the capability information acquired from the server system 100. For example, the storage unit 220 may store table data associating ADL index values ​​with the activation / inactivation of each application. The processing unit 210 extracts records from the table data that match the received ADL index value, thereby determining the activation / inactivation of each application. Here, an operation mode is set in which, of vendor applications 1 to 3, vendor applications 1 and 2 are activated and vendor application 3 is deactivated. However, the process of determining the operation mode based on the capability information is not limited to the above example, and various modifications are possible.

[0068] After the process of step S202, the device 200 executes a process according to the vendor application 1 and a process according to the vendor application 2. Specifically, the processing unit 210 of the device 200 acquires sensing data using a sensor, and obtains a processing result by executing a process defined in the application using the sensing data as input. In step S203, the device 200 transmits the processing result to the server system 100. The processing result here corresponds to the result of a judgment made using the tacit knowledge of an expert. The information transmitted to the server system 100 here is not limited to the processing result, and may include information such as a log of the sensing data.

[0069] In step S204, the server system 100 executes control based on the processing result received from the device 200. For example, the processing unit 110 may perform processing to identify a device to be controlled and transmit a control signal to operate the device to be controlled. The device to be controlled here may be a reclining wheelchair 510, which will be described later using FIG. 23, or a nursing care bed 520, which will be described later using FIG. 24. In this case, the control signal may be a signal that instructs the device to be controlled to change the angle of the back of the reclining wheelchair 510 or the angle of the bottom of the nursing care bed 520. The control signal may also be a signal that instructs the device to be controlled to issue an alert. For example, the device to be controlled is a device that includes an alert unit such as a display unit or a light-emitting unit, and the control signal is a signal that instructs the device to be controlled to issue an alert using image display, light emission, etc.

[0070] Furthermore, the process of determining the control target device and the control content based on the processing result of the device 200 may be performed by the processing unit 110 of the server system 100 or by the processing unit 210 of the device 200. In the latter case, in step S203, information identifying the control target device and the control content may be transmitted in addition to the processing result. Alternatively, the device 200 or the server system 100 may only identify the control target device, and the specific control content may be determined in the control target device. In this case, in steps S203 and S204, processes of transmitting the processing results are respectively performed. In addition, various modifications are possible regarding the control target device, the control signal, and the like. For example, the control target device may be the device 200. Furthermore, in step S204, the server system 100 may perform a process of storing a log of the sensing data transmitted from the device 200 in the storage unit 120.

[0071] Furthermore, in step S205, the server system 100 executes a process to update the ability information of the person being assisted. For example, the ability information acquisition unit 111 may obtain the ability information based on the log of the sensing data. For example, the storage unit 120 of the server system 100 may store information associating the sensing data with the ability information. The processing unit 110 executes a process to obtain the ability information based on the stored information and the sensing data transmitted from the device 200. Here, the information associating the sensing data with the ability information may be a trained model. The training data for generating the trained model here is, for example, data in which the ability information of the person being assisted, determined by a person with specialized knowledge (e.g., a doctor or an experienced caregiver), is assigned as correct answer data to the sensing data related to the person being assisted. As described above, the correct answer data may be an index value representing the ability, or may be a collection of information representing the presence or absence or level of individual abilities (such as the ability to maintain a sitting position and the ability to swallow, which will be described later). The processing unit 110 obtains the ability information by inputting the sensing data into the trained model. Alternatively, the information stored in the memory unit 120 may be reference data, which is sensing data whose correspondence with the ability information is known. The processing unit 110 may determine the similarity between the acquired sensing data and the reference data and calculate the ability information based on the similarity. The reference data may be, for example, sensing data acquired from a highly capable person receiving care. In this case, if the similarity with the reference data is high, the ability value is determined to be high, and if the similarity is low, the ability value is determined to be low. The reference data may also be other information, such as sensing data acquired from a less capable person receiving care. The processing unit 110 may also use current ability information for processing. For example, the processing unit 110 may calculate a change in the ability information based on the sensing data and calculate updated ability information based on the change and the current ability information. A specific example of the process of calculating ability information will be described later. In the process of step S205, information other than sensing data, such as a report entered by a caregiver or examination results by a doctor, may be used.

[0072] In step S206, the server system 100 performs processing to transmit data including the updated capability information to the device 200. In Fig. 7, an example is shown in which data with an ADL index value of 3 is transmitted.

[0073] In step S207, the device 200 controls the activation / inactivation of installed vendor applications based on the acquired capability information. For example, as described above, the device 200 determines the activation / inactivation of each vendor application based on table data. In the example of FIG. 7, vendor applications 1 and 2 are maintained in an active state, and vendor application 3 is changed from inactive to active. As a result, from step S207 onwards, the device 200 transitions to a state in which it operates in an operation mode in which all of vendor applications 1 to 3 are active. The operations from step S207 onwards are similar to, for example, steps S203 to S206.

[0074] The method of the present embodiment is not limited to application to the information processing system 10 including the server system 100 and the device 200, but may also be applied to an information processing device. The information processing device here refers to the server system 100 in a narrow sense. The information processing device includes a communication unit (corresponding to the communication unit 130 in FIG. 3 ) that operates in one of a plurality of operation modes and communicates with the device 200 used to assist the person being assisted, and a processing unit (corresponding to the processing unit 110 in FIG. 3 , or more specifically, the ability information acquisition unit 111) that performs processing to obtain ability information representing the activity ability of the person being assisted based on sensing data transmitted from the device 200. The processing unit of the information processing device then performs processing to transmit the ability information to the device 200 via the communication unit as information for determining in which of a plurality of operation modes the device 200 will operate. In this way, it is possible to estimate the transition of the ability information of the person being assisted based on information collected from the device 200, and to operate the device 200 in accordance with the ability information.

[0075] Furthermore, part or all of the processing performed by the information processing system of this embodiment may be realized by a program. The processing performed by the information processing system includes, for example, at least one of processing performed by the processing unit 110 of the server system 100 and processing performed by the processing unit 210 of the device 200. Similarly, part or all of the processing performed by the information processing device of this embodiment may be realized by a program.

[0076] The program according to this embodiment can be stored in, for example, a non-transitory information storage device (information storage medium), which is a medium readable by a computer. The information storage device can be realized, for example, by an optical disc, a memory card, a HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 and the like perform various processes of this embodiment based on the programs stored in the information storage device. In other words, the information storage device stores programs for causing a computer to function as the processing unit 110 and the like. A computer is a device equipped with an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for causing a computer to execute each of the steps described above using Figures 6, 7, etc.

[0077] The technique of this embodiment can also be applied to an information processing method in an information processing system 10 including a device 200 that operates in any of a plurality of operation modes and is used to assist a person being assisted, and a server system 100 that is connected to the device 200 via a network. The information processing method includes the steps of: determining ability information that represents the activity ability of the person being assisted based on sensing data acquired by the device 200; and determining in which of a plurality of operation modes the device 200 will operate based on the determined ability information.

[0078] In the following, a specific device 200 and the processing executed in the device 200 will be described using the steps shown in FIG. 5 as examples.

[0079] 2.2 Unable to move around <Example of device and operation: Fall risk assessment> First, a description will be given of device 200 for dealing with the risk of falling when starting to move in a state where getting up and down becomes difficult. Figures 8 and 9 show an example of device 200 used to determine the risk of falling when starting to move.

[0080] Fig. 8 is a diagram showing an example of an imaging device 410 that captures an image of a person being assisted, and an example of an output image IM1 of the imaging device 410. In addition to the components shown in Fig. 4, the imaging device 410 has an image sensor that outputs the captured image as sensing data. The imaging device 410 may be placed in a place where many people gather together to engage in activities, such as a living room or hall in a care facility. In the example of Fig. 8, the imaging device 410 is placed on top of a television set.

[0081] The processing unit 210 of the imaging device 410 may perform processing to detect whether a person has started to move based on the captured image. The processing unit 210 operates, for example, in accordance with an application installed in the imaging device 410, to acquire the captured image as input data, and executes processing to detect a person from the captured image and processing to determine whether the detected person has started to move.

[0082] For example, the image capture device 410 performs face recognition processing to recognize a person's face based on the captured image. For example, the storage unit 220 of the image capture device 410 may store a facial image of a person to be detected, and the processing unit 210 may perform face recognition processing based on a matching process using the facial image as a template. Various techniques for face recognition processing are known, and a wide range of such techniques can be applied in this embodiment. For example, when the movement of a detected face area continues to be equal to or less than a given threshold for a certain period of time, the image capture device 410 sets the position of the face area in that state as the reference position. The image capture device 410 may then set a detection area at a position a predetermined distance away from the reference position, and determine that movement has occurred when the face area reaches the detection area. For example, when a person stands up, it is expected that the face position will move relatively upward, so the detection area may be set at a position a predetermined distance above the reference position. In this case, movement is detected when the position of the face area on the image moves upward by more than the predetermined distance from the reference position. Note that the detection area here is, for example, a linear area, but an area of ​​another shape may also be set.

[0083] Furthermore, the image capture device 410 can identify the target person being assisted by face recognition processing. Therefore, the image capture device 410 may perform movement detection for a person being assisted whose ability represented by the ability information is equal to or lower than a predetermined threshold (corresponding to being unable to perform daily activities), and may omit movement detection for a person being assisted whose ability is higher than the threshold.

[0084] The process of detecting the start of movement is not limited to the above method. For example, the image capture device 410 may perform skeletal tracking based on captured images. Note that various image-based skeletal tracking methods are known, such as "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" (https: / / arxiv.org / pdf / 1611.08050.pdf) and OpenPose disclosed by Zhe Cao et al., and these methods can be widely applied in this embodiment.

[0085] OpenPose also discloses a method for performing skeletal tracking on each of multiple people captured in an image and displaying the results. In the example of Fig. 8, the image sensor outputs a captured image including three people being assisted, and the image capture device 410 determines whether each of the three people being assisted has started to move.

[0086] For example, a person receiving care whose mobility has declined and who finds it difficult to get up and down may fall even when attempting to stand up. Therefore, the image capture device 410 may use skeletal tracking to determine whether the person is assuming a standing up posture. For example, if the image capture device 410 determines that the person has leaned forward from a sitting position, placing their hands on their knees or the seat of a chair, the image capture device 410 determines that the person is assuming a standing up posture and notifies the caregiver of the risk of falling. For example, the image capture device 410 may determine that the person receiving care is assuming a standing up posture when it detects from the skeletal tracking results that the distance between the positions of the hands and knees is less than a predetermined value, or that the position of the shoulders has moved downward by more than a predetermined value, etc.

[0087] Alternatively, the image capture device 410 may divide the data to be processed into windows of several seconds each, and determine that a posture change, such as standing up, has occurred when a specific position, such as the head or neck, moves within each window by more than a predetermined threshold. The target body part for movement detection may be other than the head or neck. The direction of movement may be vertical, horizontal, or diagonal. The threshold used for detection may be changed depending on the target body part. Alternatively, the image capture device 410 may determine an area containing feature points detected by skeletal tracking of a stationary person receiving care, and determine that a movement, such as standing up, has occurred when a predetermined number of feature points or more fall outside the area. Various other variations are possible for the method of movement detection using the image capture device 410.

[0088] IM1 in FIG. 8 is an example of an output image from the imaging device 410. The imaging device 410 may superimpose some kind of display object on the captured image. In the example of FIG. 8, an object including an exclamation mark is displayed in association with the person being assisted whose movement has been detected. This makes it possible to clearly notify the caregiver of the person being assisted whose movement has been detected. For example, in step S203 of FIG. 7, the imaging device 410 transmits the output image IM1 to the server system 100. In step S204, the server system 100 outputs the output image IM1 to a smartphone or the like used by the caregiver. However, the output of the imaging device 410 may be information identifying the person being assisted whose movement has been detected (e.g., the ID of the person being assisted), and various modifications are possible. For example, although an example of notifying the person being assisted whose movement has been detected is shown here, information for stopping the movement of the person being assisted may also be output. For example, the imaging device 410 may identify the person being assisted who has started moving and output audio data, video data, etc. of the person being assisted's family, etc. In particular, dementia patients tend to respond poorly to calls, but they often remember the faces and voices of family members, making stopping their movements an effective way to stop them. By stopping the movements of the person being assisted in this way, it is possible to buy time for the caregiver to intervene. An example of outputting family voice and video data will be described later in connection with device control based on scene information.

[0089] FIG. 9 is a diagram illustrating an example of a bedside sensor 420 and a detection device 430 disposed at the bottom of a bed 610. The bedside sensor 420 and the detection device 430 are, for example, as shown in FIG. 9, sheet- or plate-shaped devices 200 provided between the bottom of the bed 610 and the mattress 620. Although FIG. 9 illustrates both the bedside sensor 420 and the detection device 430, only one of them may be used. Furthermore, as will be described below, the bedside sensor 420 and the detection device 430 have in common the fact that they include a pressure sensor, so the bedside sensor 420 may also function as the detection device 430, or the detection device 430 may also function as the bedside sensor 420. Various other modifications are possible with regard to the specific aspects.

[0090] The bedside sensor 420 includes a pressure sensor that outputs a pressure value as sensing data, and is placed on the side of the bottom that the caregiver uses to get in and out of the bed. In the example of FIG. 9, the caregiver gets in and out of the bed using the front side of the bed 610. In this case, as shown in FIG. 9, a railing to prevent falls may be placed on the front side of the bed 610, and the bedside sensor 420 may be placed in a position where the railing is not provided. In this way, the user getting in and out of the bed 610 first sits on the bedside sensor 420.

[0091] The processing unit 210 of the bedside sensor 420 operates, for example, according to an application installed on the bedside sensor 420, to acquire pressure values ​​as input data and perform processing to determine the movement of the person being assisted on the bed 610 from the pressure values.

[0092] For example, when a person being assisted stands up from bed 610, it is assumed that the person being assisted transitions from a lying position on the bed to a sitting position at the side of the bed (hereinafter referred to as "edge-sitting position"), and then performs a standing-up motion by placing their hands on their knees or the bedside to apply force. The pressure values ​​detected by the bedside sensor 420 increase in the order of lying position, edge-sitting position, and standing-up motion. For example, the bedside sensor 420 may determine that the start of movement has been detected when it detects a change from an edge-sitting position to a standing-up motion based on a comparison process between the pressure value and a given threshold value. Alternatively, from the perspective of detecting a standing-up motion at an earlier stage, the bedside sensor 420 may determine that the start of movement has been detected when it detects a change from a lying position to an edge-sitting position based on a comparison process between the pressure value and a given threshold value.

[0093] Alternatively, as the standing-up motion continues, the buttocks of the person being assisted rise from the bottom surface, causing a large decrease in the pressure value output from the pressure sensor. Therefore, the processing unit 210 may determine that the standing-up motion has been performed when the pressure value increases above a first threshold and then decreases below a second threshold that is smaller than the first threshold, based on the time-series change in the pressure value. In addition, various modifications can be made to the specific processing content of the movement start determination.

[0094] When the bedside sensor 420 detects that the person being assisted has started to move, it transmits information indicating this to the server system 100. The server system 100 then transmits the information to, for example, a smartphone used by the caregiver, and a notification process is executed on the smartphone. In this way, it becomes possible to clearly notify the caregiver that the person being assisted has started to move.

[0095] 9 is the device 200 that senses information related to the sleep of the person being assisted. The detection device 430 includes a pressure sensor that outputs a pressure value.

[0096] When the user gets into bed, the detection device 430 detects the user's body vibrations (body movements, vibrations) through the mattress 620. Based on the body vibrations detected by the detection device 430, information on the breathing rate, heart rate, activity level, posture, wakefulness / asleep, and whether the user is out of bed or in bed can be obtained. The detection device 430 may also determine whether the user is in non-REM sleep or REM sleep, and the depth of sleep. For example, the periodicity of body movements may be analyzed, and the breathing rate and heart rate may be calculated from the peak frequency. The periodicity may be analyzed using, for example, a Fourier transform. The breathing rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time is, for example, one minute. Body vibrations may also be detected per sampling unit time, and the number of detected body vibrations may be calculated as the amount of activity. When the user gets out of bed, the detected pressure value decreases compared to when the user is in bed, so it is possible to determine whether the user is out of bed or in bed based on the pressure value and its time-series changes.

[0097] For example, the processing unit 210 of the detection device 430 may determine that the start of movement has been detected when the person being assisted transitions from being in bed to being out of bed, based on the result of the bed-out / bed-in determination.

[0098] When it is detected that the person being assisted has started to move, the detection device 430 transmits information indicating this to the server system 100. The server system 100 transmits the information to, for example, a smartphone used by the caregiver, and a notification process is executed on the smartphone. In this way, it is possible to notify the caregiver in an easy-to-understand manner that the person being assisted has started to move.

[0099] For example, each device 200 shown in FIGS. 8 and 9 is set to inactive operation mode 0 when the ability of the person being assisted, as represented by the ability information, is equal to or greater than a predetermined level, and is set to active operation mode 1 when the ability is less than the predetermined level. In this case, less than the predetermined level of ability means that the person is unable to get up and down. In this way, the above-mentioned movement start determination operation can be performed in an appropriate situation. As a result, when there is a person being assisted who is at high risk of falling, the risk of the person being assisted falling can be appropriately reduced.

[0100] <Ability information update> 7, the capability information acquisition unit 111 of the server system 100 may perform processing to update the capability information based on the sensing data. For example, the capability information acquisition unit 111 determines a change in the status related to living and / or drinking based on the sensing data.

[0101] For example, the ability information acquisition unit 111 may determine the manner of standing up based on sensing data. When standing up, as described above, the person places their hands on the platform or the like from a sitting position on the edge of the chair, then puts their weight on their feet, straightens their knees and straightens their spine. At this time, if the weight is not transferred sufficiently to the feet, the center of gravity will shift relatively backward, causing the person to fall onto the platform. If the weight is transferred excessively to the feet, the center of gravity will shift forward, causing the person to fall forward. Furthermore, if the person is sitting too shallowly on the edge of the chair, the buttocks may fall off the platform.

[0102] Therefore, the ability information acquisition unit 111 may determine whether the body movement of the person being assisted when standing up is appropriate based on the log of skeletal tracking by the imaging device 410 and the log of pressure values ​​from the bedside sensor 420 and the detection device 430. For example, the ability is determined to be higher when the movement when standing up is closer to a normal state, and the ability is determined to be lower when the deviation of the center of gravity or the position of the buttocks when sitting on the edge of the bed deviates from the normal state.

[0103] Alternatively, the ability information acquiring unit 111 may obtain the ability information based on the number of times the person stands up within a predetermined period. For example, the more times the person stands up, the higher the ability is determined to be, and the fewer times the person stands up, the lower the ability is determined to be.

[0104] The ability information acquisition unit 111 may also obtain ability information based on the elapsed time from sitting on the edge of the bed to standing up. Whether the person is lying down, sitting on the edge of the bed, or standing can be determined based on the relative positions of feature points in skeletal tracking and time-series changes in pressure values. For example, the shorter the time that elapsed from sitting on the edge of the bed to standing up, the higher the ability is determined to be, and the longer the time that elapsed, the lower the ability is determined to be.

[0105] Furthermore, the ability information acquiring unit 111 may obtain the ability information based on the amount of activity in bed 610. The amount of activity is detected by, for example, the detection device 430 as described above. The amount of activity may also be obtained based on the results of skeletal tracking or the output of the bedside sensor 420. For example, the greater the amount of activity, the higher the ability is determined to be, and the smaller the amount of activity, the lower the ability is determined to be.

[0106] In the method of this embodiment, the ability information may be determined using any one of the above methods, or a combination of two or more of them. Furthermore, as described above, the ability information may be determined based on a combination of sensing data and data other than sensing data. The method of this embodiment reduces the risk of a care recipient who is unable to sit up or down, and also makes it possible to appropriately determine changes in the ability information of the care recipient.

[0107] 2.3 Unable to walk <Example of device and action: Fall risk> Next, a description will be given of device 200 that operates to deal with the risk of falling from wheelchair 630 or the like when walking becomes difficult. Figures 10 and 11 show an example of device 200 used to determine the risk of falling.

[0108] FIG. 10 is a diagram showing a seat sensor 440, which is a device 200 placed on the seat of a wheelchair 630, for example. The seat sensor 440 includes a pressure sensor that outputs a pressure value, and determines, based on the pressure value, whether the posture of the person being assisted when sitting on the wheelchair 630 (hereinafter also referred to as the sitting posture) is one of a plurality of postures, including normal, forward shift, side shift, etc. Forward shift refers to a state in which the user's center of gravity is shifted further forward than normal, and side shift refers to a state in which the user's center of gravity is shifted to either the left or right than normal. Both forward shift and side shift correspond to states in which the risk of slipping is relatively high. The seat sensor 440 may be a sensor device placed on an ordinary chair, or may be a sensor device that determines the posture of a user sitting on a bed, etc. The seat sensor 440 may also perform a fall possibility determination, which determines whether or not the person being assisted is likely to fall from the seat.

[0109] 10, four pressure sensors Se1 to Se4 are arranged on the rear side of a cushion 441 that is placed on the seat of a wheelchair 630. Pressure sensor Se1 is a sensor that is arranged in the front, pressure sensor Se2 is a sensor that is arranged in the rear, pressure sensor Se3 is a sensor that is arranged on the right, and pressure sensor Se4 is a sensor that is arranged on the left. Note that front, back, left, and right here refer to directions as seen from the perspective of the person being assisted when the person is sitting in the wheelchair 630.

[0110] 10, the pressure sensors Se1 to Se4 are connected to a control box 442. The control box 442 includes a processor that controls the pressure sensors Se1 to Se4 and a memory that serves as a work area for the processor. For example, the processor corresponds to the processing unit 210, and the memory corresponds to the storage unit 220. The processor detects pressure values ​​by operating the pressure sensors Se1 to Se4.

[0111] A person being assisted sitting in wheelchair 630 may feel pain in the buttocks and may shift the position of the buttocks. For example, forward slippage occurs when the buttocks are shifted further forward than normal, and lateral slippage occurs when the buttocks are shifted to the left or right. In addition, forward slippage and lateral slippage may occur simultaneously, causing the center of gravity to shift diagonally. As shown in FIG. 10, by using a pressure sensor arranged on cushion 441, it is possible to appropriately detect changes in the position of the buttocks, and therefore it is possible to accurately detect forward or lateral slippage.

[0112] For example, the initial state is the timing when the person being assisted transfers to the wheelchair 630 and assumes a normal posture. In the initial state, the person being assisted sits deep in the seat of the wheelchair 630, so it is expected that the value of the rear pressure sensor Se2 will be relatively large. On the other hand, when forward slippage occurs, the position of the buttocks moves forward, and the value of the front pressure sensor Se1 will increase. For example, the processor of the control box 442 may determine that forward slippage has occurred when the value of the pressure sensor Se1 increases by a predetermined amount compared to the initial state. When the value of the pressure sensor Se1 exceeds a certain threshold, it may be determined that the person being assisted is in the wheelchair 630, and it may be determined that forward slippage has occurred based only on the change in the value of the pressure sensor Se2 without comparing it with the pressure sensor Se1. Furthermore, instead of using the value of the pressure sensor Se1 alone, processing may be performed using the relationship between the values ​​of the pressure sensors Se2 and Se1. For example, the difference in voltage values ​​output by the pressure sensors Se2 and Se1 may be used, or the ratio of the voltage values ​​may be used, or the rate of change of the difference or ratio relative to the initial state may be used.

[0113] Similarly, when a lateral slippage occurs, the position of the buttocks moves to either the left or the right, so if it is a left slippage, the value of pressure sensor Se4 increases, and if it is a right slippage, the value of pressure sensor Se3 increases. Therefore, the processor may determine that a left slippage has occurred when the value of pressure sensor Se4 increases by a predetermined amount compared to the initial state, and may determine that a right slippage has occurred when the value of pressure sensor Se3 increases by a predetermined amount compared to the initial state. Alternatively, the processor may determine whether a right slippage or a left slippage has occurred using the relationship between the values ​​of pressure sensors Se4 and Se3. As with the example of forward shift, the difference in voltage values ​​output from pressure sensor Se4 and pressure sensor Se3 may be used, the ratio of the voltage values ​​may be used, or the rate of change of the difference or ratio relative to the initial state may be used.

[0114] If a forward or lateral slippage is detected, the seat sensor 440 transmits information indicating this to the server system 100. The server system 100 then transmits the information to, for example, a smartphone used by the caregiver, and a notification process is executed on the smartphone. The control box 442 may also include a light-emitting unit, which may be used to notify the caregiver. In this way, the caregiver can be notified of changes in the seating posture of the wheelchair 630 or the like in an easy-to-understand manner, making it possible to prevent the person being assisted from falling.

[0115] FIG. 11 is a diagram showing a terminal device 450, which is a device 200 used to assist in adjusting a wheelchair position. The wheelchair position is information related to the position and posture of the person being assisted in the wheelchair 630. The wheelchair position may represent the seated posture described above, or may represent information including the placement of cushions, etc. As shown in FIG. 11 , in adjusting the wheelchair position, a terminal device 450 having a camera and fixed at a height that allows the camera to capture an image of at least the upper body of the person being assisted sitting in the wheelchair 630 may be used. Note that the terminal device 450 may be capable of capturing an image of a wider range of the person being assisted, for example, up to the knees, or the entire body. The terminal device 450 is placed in a predetermined position in, for example, a nursing facility, and the caregiver transfers the person being assisted into the wheelchair 630, moves the person to the front of the terminal device 450, and then adjusts the wheelchair position.

[0116] The terminal device 450 includes a display unit and displays the comparison result between the image captured by the camera and the training data. The training data here is, for example, image data of the person being assisted, captured by a highly skilled caregiver while the person is seated in the wheelchair 630 in an appropriate posture. For example, the training data is registered in advance using the terminal device 450 or the like. The training data may also be registered in the storage unit 120 of the server system 100. The training data is not limited to image data itself that represents an appropriate posture, but may also be data to which additional information has been added. The additional information here may be information indicating points that a highly skilled caregiver considers important. The training data may also be the results of skeletal tracking.

[0117] The terminal device 450 (processing unit 210) may output an image in which the teacher data that has been subjected to transparency processing is superimposed on the actual captured image. Furthermore, the terminal device 450 may determine whether the position of the person being assisted in the wheelchair 630 is appropriate based on a comparison process between the teacher data and the actual captured image, and output the determination result. Furthermore, if the position of the person being assisted is not appropriate, the terminal device 450 may present points that need to be corrected. Points that need to be corrected are, for example, areas where the difference between the teacher data and the actual captured image is greater than a predetermined value.

[0118] For example, each device 200 shown in FIGS. 10 and 11 is set to inactive operation mode 0 when the ability of the person being assisted, as represented by the ability information, is equal to or greater than a predetermined level, and is set to active operation mode 1 when the ability is less than the predetermined level. In this case, less than the predetermined level of ability means that the person is unable to walk. In this way, the position determination for the wheelchair 630 or the like described above can be performed in an appropriate situation. As a result, when there is a person being assisted who is at high risk of falling from the wheelchair 630 or bed 610, the risk of the person being assisted falling can be appropriately reduced.

[0119] <Device behavior regarding fall risk> 5, even if a care recipient who cannot walk is not bedridden, there is a possibility that the care recipient may start to move, such as standing up, and therefore the risk of falling is high. Even when a care recipient who cannot walk is the target, it is desirable to continue detecting the start of movement by each device 200 shown in FIGS. 8 and 9.

[0120] In this case, between a state where a person cannot sit up but can walk and a state where a person cannot walk, the latter may have lower ability and a higher risk of falling. Therefore, each device 200 shown in FIGS. 8 and 9 may operate in operation mode 1 when a person cannot sit up but can walk, and may operate in operation mode 2, which is different from operation mode 1, when a person cannot walk. For example, when a detection device 430 is used as device 200, processing unit 210 of detection device 430 may detect the start of movement based on the result of determining whether the person is out of bed or in bed in operation mode 1, and may detect the start of movement based on the result of determining whether the person is awake or asleep in operation mode 2. For example, in operation mode 2, processing unit 210 determines that there is a possibility of the person starting to move when the person transitions from a sleeping state to an awake state. In this way, in operation mode 2, the start of movement can be detected at an earlier stage than in operation mode 1, making it possible to further reduce the risk of falling.

[0121] <Ability information update> 7, the ability information acquisition unit 111 of the server system 100 may perform processing to update the ability information based on the sensing data. For example, the ability information acquisition unit 111 determines a change in sitting ability, which is the ability to maintain a sitting position, based on the sensing data.

[0122] For example, the ability information acquisition unit 111 may determine the number of forward or lateral shifts based on the sensing data. For example, the fewer the number of forward or lateral shifts, the higher the ability to maintain sitting posture is determined to be, and vice versa.

[0123] The ability information acquisition unit 111 may also calculate the sitting ability based on the time the person can maintain a certain posture (hereinafter referred to as the posture maintenance time). The posture maintenance time is, for example, the length of a period of time beginning when the person assumes a given reference posture and ending when the person's posture changes by a predetermined amount from the reference posture. The posture maintenance time may be, for example, the time from when the caregiver corrects the person's posture using the terminal device 450 until the seat sensor 440 determines that the person has shifted forward or to the side. For example, the longer the posture maintenance time, the higher the ability to maintain a certain posture is determined to be, and the shorter the posture maintenance time, the lower the ability to maintain a certain posture is determined to be. Furthermore, if the person's ability to maintain a certain posture is high, the same amount of pressure will be applied to each pressure sensor continuously. On the other hand, if the person's ability to maintain a certain posture decreases, the person will frequently tilt or correct their posture, even if it is not determined that the person has shifted forward or to the side. As a result, the pressure value is likely to decrease (pressure loss). Therefore, the ability information acquiring unit 111 may estimate the sitting ability based on the number of times such pressure is released, the frequency, the degree of decrease in value, the direction of the release (which of the pressure sensors Se1 to Se4 has the value decreased), etc.

[0124] In the method of this embodiment, the ability information may be determined using any one of the above methods, or a combination of two or more of them. Furthermore, as described above, the ability information may be determined based on a combination of sensing data and data other than sensing data. The method of this embodiment reduces the risk of a fall for a care recipient who cannot walk, and makes it possible to appropriately determine changes in the ability information of the care recipient.

[0125] 2.4 Difficulty eating <Example of device and action: Aspiration risk> Next, a device 200 that operates to address the risk of aspiration (or, more narrowly, the risk of aspiration pneumonia) when a person is unable to eat properly will be described. Fig. 12 shows an example of device 200 used to determine the risk of aspiration during eating.

[0126] FIG. 12 is a diagram illustrating a choking detection device 460, which is a device 200 used in a mealtime. As shown in FIG. 12, the choking detection device 460 includes a throat microphone 461 attached around the neck of the person being assisted and a terminal device 462 equipped with a camera. Note that another device equipped with a camera may be used instead of the terminal device 462. The throat microphone 461 outputs audio data resulting from the person being assisted swallowing, coughing, etc. The camera of the terminal device 462 outputs captured images of the person being assisted eating. The terminal device 462 is, for example, a smartphone placed on a table where the person being assisted eats. The throat microphone 461 is connected to the terminal device 462 using Bluetooth (registered trademark) or the like, and the terminal device 462 is connected to the server system 100 via the gateway 300. However, both the throat microphone 461 and the terminal device 462 may be connectable to the gateway 300, and various modifications of the specific connection configuration are possible.

[0127] For example, a processor included in the choking detection device 460 acquires audio data from the throat microphone 461 and an image captured using a camera. The processor here corresponds to the processing unit 210 and may be a processor included in the terminal device 462, for example.

[0128] The processor determines whether the person being assisted choked or swallowed based on the audio data from the throat microphone 461. A device for detecting swallowing using a microphone worn around the neck is described, for example, in U.S. Patent Application No. 16 / 276,768, filed February 15, 2019, entitled "Swallowing action measurement device and swallowing action support system." This patent application is incorporated by reference in its entirety. Based on the audio data, the processor can detect the number of choking episodes, the duration of the choking (time of occurrence, duration, etc.), and whether or not the person swallowed.

[0129] The camera of the terminal device 462 can detect the mouth, eyes, and chopsticks and spoons used by the person being assisted by capturing an image of the person being assisted from the front, as shown in Fig. 12. There are various known methods for detecting these facial features and objects based on image processing, and a wide range of known methods can be applied in this embodiment.

[0130] For example, the processor can determine, based on the image captured by the camera, whether the person being assisted has their mouth open, food coming out of their mouth, and whether they are chewing. The processor can also determine, based on the image captured by the camera, whether the person being assisted has their eyes open. The processor can also determine, based on the image captured by the camera, whether chopsticks, a spoon, or the like are near tableware, whether the person being assisted can hold them, and whether they are spilling food.

[0131] In the method of the present embodiment, the swallowing or choking status of the person being assisted is estimated based on this information. For example, the processor may perform processing based on the detection results of choking and swallowing and the determination result of whether the person being assisted is opening or closing their mouth.

[0132] For example, the processor may determine whether choking is occurring frequently based on the number of times or duration of choking and output the determination result. For example, the processor may determine that choking is occurring frequently when the number of times choking per unit time exceeds a threshold. In this way, choking situations can be automatically determined.

[0133] The processor may also calculate the swallowing time, measured from when the person being assisted opens their mouth to when they swallow, based on the swallowing detection results and the results of determining whether the person being assisted has opened or closed their mouth. In this way, even if it is determined that the number of swallows has decreased, it is possible to determine specific circumstances, such as whether the person has not even put food in their mouth, or whether the person has put food in their mouth but has not swallowed it. For example, the processor may start counting up a timer when the mouth transitions from a closed state to an open state based on the captured image of the terminal device 462, and stop counting the timer when swallowing is detected by the throat microphone 461. The time when the timer stops represents the swallowing time. This allows for accurate determination of whether the risk of aspiration is high during meals and whether the caregiver should take some kind of action, thereby making it possible to appropriately utilize the tacit knowledge of experts.

[0134] For example, the choking detection device 460 may output the swallowing time as a processing result to the server system 100. The processor may also determine the pace of eating based on the swallowing time. The processor may also determine whether the swallowing time is long based on changes in the swallowing time during one meal (for example, the increase or ratio compared to the swallowing time at the beginning). Alternatively, the processor may calculate the average swallowing time for each of multiple meals for the same person receiving care, and determine whether the swallowing time has become long based on changes in the average swallowing time.

[0135] Furthermore, by using the results of determining whether the mouth is open or closed based on the captured image of the terminal device 462, it can be determined whether the person no longer opens their mouth even when the caregiver approaches them with a spoon or the like. In this way, if the swallowing time becomes longer when the person being assisted is reluctant to open their mouth, it can be estimated that food is remaining in the mouth and becoming stagnant. Furthermore, by using the captured image to determine whether food is coming out of the mouth and whether the person being assisted is chewing the food, it can be determined whether the person being assisted is no longer able to chew the food. For example, if the number of chews is normal but the swallowing time is long, it can be estimated that the person being assisted is no longer able to chew the food. Furthermore, if it is determined that the eyes are closed based on the captured image, it can be determined whether the person being assisted is becoming sleepy.

[0136] Furthermore, by performing recognition processing of chopsticks, spoons, etc. using the captured image, it may be determined whether the child is playing with food, unable to hold a bowl, spilling food, etc.

[0137] As described above, various mealtime situations can be determined by using the choking detection device 460. In the choking detection device 460 for input data according to this embodiment, each of these determinations may be implemented as an application corresponding to tacit knowledge. For example, the choking detection device 460 may include an application for detecting choking and swallowing and calculating swallowing time, an application for detecting frequent choking, an application for detecting dangerous choking, an application for determining whether the patient is sleepy, and an application for determining whether the patient is playing with food. In this embodiment, the activation / inactivation of each application is controlled based on the ability information, etc. In other words, the execution / inactivation of the various processes described above may be flexibly settable. If the choking detection device 460 determines that a predetermined situation exists, it transmits information indicating this to the server system 100. The server system 100 may, for example, cause a terminal device used by the caregiver to execute a notification process based on the information. The server system 100 may also output information indicating an appropriate action according to the eating situation detected by the choking detection device 460 to a device used by the caregiver, and the device may then present an action based on the information. Presentation to the caregiver may be, for example, audio output to a headset, a display on a display unit of a smartphone or the like, or presentation using other methods.

[0138] <Device behavior regarding fall risk> Furthermore, as shown in Fig. 5, even if a person receiving care has difficulty eating, there is a possibility that the person may start to move, such as standing up, and therefore the risk of falling is high. Therefore, even when a person receiving care has difficulty eating, it is desirable to continue detecting the start of movement by each device 200 shown in Figs. 8 and 9. Each device 200 may operate, for example, in the above-mentioned operation mode 2. Furthermore, each device 200 may operate in operation mode 3, which is capable of detecting the start of movement at an earlier stage than operation mode 2.

[0139] <Device behavior regarding fall risk> Furthermore, even if a person receiving care who has difficulty eating is likely to use a wheelchair 630 or the like, the risk of falling is high. Therefore, even when a person receiving care who has difficulty eating is being treated as the target, it is desirable to continue the processing related to the wheelchair position by each device 200 shown in FIGS. 10 to 11. For example, each device 200 operates in operation mode 1, just as when a person receiving care who cannot walk is being treated as the target. Furthermore, each device 200 may operate in a mode different from when a person receiving care who cannot walk is being treated as the target. For example, when a person receiving care who cannot walk is being treated as the target, the device may operate in an operation mode that only performs a forward or lateral slippage determination, whereas when a person receiving care who has difficulty eating is being treated as the target, the device may operate in an operation mode that can perform a fall possibility determination in addition to a forward or lateral slippage determination.

[0140] <Ability information update> 7, the ability information acquiring unit 111 of the server system 100 may perform processing to update the ability information based on the sensing data. For example, the ability information acquiring unit 111 determines a change in swallowing ability based on the sensing data.

[0141] For example, the ability information acquisition unit 111 may determine that the shorter the swallowing time from when the person being assisted opens their mouth to when they swallow, the higher their swallowing ability is, and that the longer the swallowing time, the lower their swallowing ability is. For example, the ability information acquisition unit 111 may determine that the swallowing ability has declined when the most recent average swallowing time of the person being assisted becomes longer than the average swallowing time of past people being assisted by a threshold or more, determine that the swallowing ability has not declined (is maintained) when the average swallowing time does not exceed the threshold, and determine that the swallowing ability has recovered when the average swallowing time has shortened. Furthermore, if the change in the average swallowing time before and after changing the food texture is equal to or less than a threshold, the swallowing ability may be determined to have recovered. Furthermore, the ability information acquisition unit 111 may obtain ability information based on a combination of swallowing ability and ability to maintain a sitting position. For example, the ability information acquisition unit 111 may obtain ability information based on the duration or frequency of use of the wheelchair 630. For example, a person receiving care whose ability to maintain a sitting position has declined may transition to eating in bed, and therefore, the ability to use a wheelchair 630 indicates a high level of ability to maintain a sitting position. For example, the more frequently the wheelchair is used, the higher the ability is determined to be, and the less frequently the wheelchair is used, the lower the ability is determined to be. As described above, the choking detection device 460 may execute various applications that perform different processes. For example, the choking detection device 460 may first activate an application that detects choking and swallowing and calculates swallowing time, and the ability information acquisition unit 111 may obtain ability information based on the output of that application. When the ability information changes, other applications, such as an application that detects frequent choking, are changed from inactive to active. This allows the processing content executed by the choking detection device 460 to be appropriately controlled based on the ability information. Furthermore, when the number of active applications increases, the ability information acquisition unit 111 may combine the output of each application to obtain swallowing ability based on more sensing data.

[0142] 2.5 Bedridden <Examples of devices and actions: Pressure ulcer risk> Next, a device 200 that operates to address the risk of bedsores when the care recipient's abilities have further declined and he or she has become bedridden will be described. Figures 13 and 14 show an example of device 200 used to determine the risk of bedsores.

[0143] FIG. 13 is a diagram illustrating a bed position detection device 470, which is a device 200 arranged around a bed 610. As shown in FIG. 13, the bed position detection device 470 includes a first terminal device 471 fixed to the footboard side of the bed 610, a second terminal device 472 fixed to a side rail of the bed 610, and a display 473 fixed on the opposite side of the second terminal device 472. The display 473 is not limited to being fixed to the bed 610, and may be arranged in another position where it can be viewed naturally by a caregiver adjusting the bed position. For example, the display 473 may be fixed to a wall surface or a stand that stands on the floor. The display 473 can be omitted. Alternatively, either the first terminal device 471 or the second terminal device 472 may be omitted. For example, the following describes an example in which the first terminal device 471 is used to adjust the bed position and the second terminal device 472 is used to change diapers, but the present invention is not limited to this example.

[0144] The first terminal device 471 and the second terminal device 472 are devices such as smartphones equipped with cameras. The first terminal device 471 transmits captured images directly to the server system 100. The second terminal device 472 transmits captured images of the camera to the server system 100 directly or via the first terminal device 471. The display 473 receives images transmitted from the server system 100 directly or via another device such as the first terminal device 471, and displays the received images. Note that the first terminal device 471 and the second terminal device 472 may have a depth sensor instead of or in addition to a camera. That is, these devices may output depth images.

[0145] For example, in adjusting a bed position, a process of registering training data and a process of adjusting a position using the training data may be performed. The training data is information registered, for example, by an experienced caregiver. When adjusting a bed position, an inexperienced caregiver selects training data and adjusts the bed position so that the actual state of the person being assisted matches the training data. For example, the first terminal device 471 acquires a captured image of the person being assisted lying on the bed (including the state of cushions, etc.), and the display 473 displays an image representing the comparison result between the captured image and the training data. This enables caregivers of all skill levels to perform position adjustments similar to those performed by an experienced caregiver. For example, the experienced caregiver lies the person being assisted on the bed 610, positions the person in a position suitable for preventing bedsores, etc., and then uses the first terminal device 471 to capture an image of the person being assisted. After confirming that the bed position is appropriate, the experienced caregiver selects the registration button. As a result, the still image displayed when the registration button was pressed is transmitted to the server system 100 as training data. In this way, it becomes possible to register positions that an expert considers to be preferable as training data. At this time, additional information may be added, as in the case of the wheelchair position example described above.

[0146] Furthermore, when the caregiver actually adjusts the bed position, first, the first terminal device 471 is activated and image capturing begins. For example, the caregiver activates the first terminal device 471 by voice, and the display 473 displays the moving image captured by the first terminal device 471. The bed position detection device 470 may also accept a selection process of teacher data by the caregiver. The processing unit 110 determines the teacher data based on a user operation and performs control to display the teacher data on the display 473.

[0147] Alternatively, the processing unit 110 may automatically select the training data based on a similarity determination between the attributes of the person being assisted whose bed position is to be adjusted and the attributes of the person being assisted captured in the training data. The attributes here include information such as the age, sex, height, weight, medical history, and medication history of the person being assisted.

[0148] Alternatively, the bed position detection device 470 may automatically select training data based on a comparison between the attributes of the person being assisted whose bed position is to be adjusted and additional information contained in the training data. For example, the additional information in the training data may include text such as "For a person being assisted who shows a tendency of XX, it is recommended to adjust the left shoulder so that it is YY." In this case, if the person being adjusted corresponds to XX, the training data is likely to be selected. For example, a caregiver who adjusts the bed position may input information identifying the person being assisted into the first terminal device 471, and the bed position detection device 470 may identify the attributes of the person being assisted based on the information.

[0149] The bed position detection device 470 may output an image in which the teacher data that has been subjected to transparency processing is superimposed on a real-time image captured by the first terminal device 471, for example. At this time, additional information about the teacher data may be displayed in a recognizable manner. For example, when it is detected that the caregiver has uttered "Tell me the key points" using a microphone or the like of a headset, the bed position detection device 470 may output the text as voice from the headset via the server system 100, for example.

[0150] The bed position detection device 470 may determine whether the image captured during position adjustment is OK or NG based on the degree of similarity between the image and the training data, and output the determination result. The determination result is displayed on the display 473 directly or via the server system 100. The bed position detection device 470 may also perform processing to display specific points determined to be NG. For example, the server system 100 or the bed position detection device 470 may perform processing to compare the image captured by the first terminal device 471 with the training data and highlight areas determined to have a large difference.

[0151] In this way, it becomes possible to present a comparison between the bed position of the person being assisted and the ideal bed position, and to present information for realizing the ideal bed position.

[0152] The bed position detection device 470 may also be used to assist in changing diapers. It has been found that experts place importance on the following points as tacit knowledge in changing diapers. A.Is the patient in a lateral position? B. Is the diaper in the correct position? C. Check if the padding is coming out of the diaper D. Was the diaper properly fitted?

[0153] Therefore, the processing unit 210 of the bed position detection device 470 determines whether the above points A to D are satisfied, and transmits the determination result to the server system 100. The processing unit 210 here corresponds to, for example, the processor of the second terminal device 472. This makes it possible to change the diaper appropriately regardless of the skill level of the caregiver.

[0154] For example, the second terminal device 472 performs skeletal tracking processing on each image constituting a moving image of the person being assisted captured using a camera, and outputs an image in which the skeletal tracking results are superimposed on the original image as the processing result. The processing result may be displayed on the display 473, for example. In this way, the caregiver can check the display 473 in a natural posture while changing the diaper of the person being assisted.

[0155] In consideration of the case where diaper changes are performed at night, the second terminal device 472 may include a lighting unit. Also, in consideration of the privacy of the person being assisted, a depth sensor or the like may be used instead of a camera. The depth sensor may be a sensor using a ToF (Time of Flight) method, a sensor using structured illumination, or a sensor using another method.

[0156] 15A and 15B are examples of images displayed on the display 473 when a diaper is being changed. As described above, each image includes the person being assisted and the results of skeletal tracking of the person being assisted.

[0157] In the state shown in Fig. 15A, the person being assisted is stable in a lateral position, and the camera of the second terminal device 472 captures an image of the person being assisted directly from behind. For example, in Fig. 15A, there is a small difference between the front-to-back direction of the person being assisted's body and the direction of the camera's optical axis. As a result, as shown in Fig. 15A, many points that can be detected by skeletal tracking are detected.

[0158] 15B, the person being assisted is less stable than in FIG. 15A, and appears to be on the verge of falling onto his back. The camera of the second terminal device 472 captures the image of the person being assisted from diagonally behind, so the number of points detected by skeletal tracking is reduced. For example, the point corresponding to the waist is not detected because it is hidden by a diaper or the like.

[0159] Therefore, the second terminal device 472 may determine whether or not the subject is in the lateral position shown in A above based on the results of skeletal tracking. For example, the second terminal device 472 may determine that the subject is in the lateral position when a point corresponding to a specific part such as the waist is detected by skeletal tracking. However, the determination of the lateral position may also be based on whether or not a point other than the waist is detected, or the relationship between multiple points, and the specific method is not limited to this.

[0160] The bed position detection device 470 also performs object tracking processing based on moving images captured by the camera of the second terminal device 472 to continuously detect the diaper area in the image. Object tracking is well known, so a detailed description will be omitted. For example, in Figures 15A and 15B, the diaper area ReD is detected.

[0161] The bed position detection device 470 may determine whether the diaper position shown in B above is appropriate based on, for example, the relationship between the results of skeletal tracking and the diaper area ReD detected by object tracking. For example, taking into account the position where the diaper is worn, it determines whether the waist position detected by skeletal tracking and the diaper area ReD have a predetermined positional relationship. For example, the processing unit 210 may determine that the diaper position is appropriate if a line including two points corresponding to the pelvis passes through the diaper area ReD. Alternatively, the results of skeletal tracking and the detection results of the diaper area ReD may be extracted as features from training data by an expert, and machine learning may be performed using the features as input data. The trained model is a model that, for example, outputs the likelihood that the diaper position is appropriate when it receives the results of skeletal tracking and the detection results of the diaper area ReD.

[0162] The bed position detection device 470 may also determine whether the pads are protruding from the diaper shown in C above based on the horizontal length of the diaper area ReD. Since the pads are usually contained within the diaper, the length of the diaper area ReD on the image corresponds to the length of the diaper itself. The expected size of the diaper area ReD can be estimated based on the type and size of the diaper and the optical characteristics of the camera of the second terminal device 472. On the other hand, if the pad is protruding, the length of the diaper area ReD on the image will be longer by that amount. Therefore, if the length of the diaper area ReD detected from the image is longer than the expected length by a predetermined threshold or more, the bed position detection device 470 determines that the diaper is protruding from the pad and is inappropriate.

[0163] The bed position detection device 470 may also determine whether the diaper described above in D is properly put on by detecting the tape used to secure the diaper in place. Typically, the tape is a different color from the diaper itself. For example, the diaper itself is white and the tape is blue. The way the tape should be secured to properly put on the diaper is known from the diaper's structure. Therefore, the processing unit 210 can detect the tape area in the image based on the color and determine whether the diaper is properly put on based on the relationship between the tape area and the diaper area ReD or the relationship between the tape area and the position of the waist or other parts detected by skeletal tracking. When multiple diapers from different manufacturers or types are used, the bed position detection device 470 may acquire information identifying the diapers and determine whether the diapers are properly put on based on the identified type of diaper, etc.

[0164] For example, the bed position detection device 470 judges whether each of the above A to D is OK or NG, and transmits the judgment results to the server system 100. The server system 100 transmits the judgment results to the display 473 or the like. Furthermore, if the result is NG, the bed position detection device 470 may highlight the part that is significantly different from the correct data.

[0165] This approach not only ensures that the patient assumes a position that reduces the risk of bedsores, but also allows the caregiver to appropriately perform diaper changes by appropriately utilizing tacit knowledge in diaper changing. Note that the diaper change requires the patient to move. For example, the caregiver may first place the patient in a lateral position to facilitate diaper placement, or may raise the patient's legs to put on the diaper. Therefore, when the diaper change is complete, the patient may not be in a position suitable for lying down in bed. Therefore, the bed position detection device 470 may automatically execute the above-described process to adjust the bed position when it detects that the diaper change is complete. For example, the first terminal device 471 begins capturing an image of the patient, and the display 473 displays the training data, which has been subjected to transparency processing, superimposed on the real-time image captured by the first terminal device 471.

[0166] In the above, an example has been described in which the first terminal device 471 and the second terminal device 472 such as a smartphone are used for bed position adjustment support and diaper change support, but the specific device 200 is not limited to this.

[0167] FIG. 14 is a diagram illustrating a glasses-type device 480 such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses, which is another example of the device 200 used for bed position adjustment assistance or diaper change assistance. The glasses-type device 480 has, for example, a camera that captures an image of an area corresponding to the user's field of view. The glasses-type device 480 has a display in part or all of its lens portion, and is capable of allowing the user to visually recognize the situation in the outside world by transmitting light from the outside world or by displaying an image corresponding to the user's field of view captured by the camera. Furthermore, the glasses-type device 480 uses a display to additionally display some information in the user's field of view.

[0168] Even when the glasses-type device 480 is used, it is possible to acquire an image of the user on the bed 610. Therefore, the glasses-type device 480 may perform processing to assist in bed position adjustment by superimposing the captured image and the teacher data as described above. The glasses-type device 480 may also perform processing to assist in diaper changing by making a determination regarding the diaper area ReD. Similarly, when the glasses-type device 480 is used, bed position adjustment may be started when diaper changing is completed. For example, when completion of diaper adjustment is detected, correct answer data related to bed position adjustment may be displayed transparently on the display unit of the glasses-type device 480.

[0169] The glasses-type device 480 may also perform processing to automatically detect the presence and extent of a bedsore based on captured images of the skin of the person being assisted. This makes it possible to determine whether a bedsore actually exists and, if so, what state the bedsore is in. For example, a trained model is generated by machine learning based on training data that associates images of the person being assisted with correct answer data that identifies the bedsore area provided by a doctor or other expert. The glasses-type device 480 performs processing based on the trained model to determine whether or not a bedsore exists.

[0170] Mattresses and pillows capable of pressure detection and automatic position change are also known. Pressure detection is performed using a pressure sensor, similar to the bedside sensor 420 and the detection device 430. The mattress and pillow may encourage the care recipient to turn over by adjusting the height (thickness) of each part using air or other means. The device 200 that detects pressure and the device 200 that encourages automatic position change may be different. For example, if the detection device 430 determines that the same posture continues based on the pressure value, it transmits information indicating this to the server system 100. The server system 100 may control the mattress or pillow based on this information to encourage the care recipient to change position. This can also reduce the risk of bedsores in bedridden care recipients. The control method for these mattresses and pillows is not limited to one. For example, even for the same bedridden person, detailed ability information (such as ADLs) may be determined, and the encouragement to change position may be changed depending on the level of the ability information. For example, if a care recipient has extremely low abilities, it may be difficult to maintain posture, and the posture may change excessively when prompted to change position. Therefore, mattresses and pillows may have multiple operation modes that differ in the frequency of prompting for position changes, the amount of air used when prompting for position changes, etc., and the operation mode may be controlled according to the level of ability information. Furthermore, mattresses and pillows may operate in a mode that improves sleep conditions (for example, suppresses snoring by encouraging the care recipient to lie on their side) while the ability is high, and in a mode that reduces the risk of bedsores if the care recipient becomes bedridden. Operation mode control according to ability information may also be performed for conditions other than being bedridden.

[0171] Furthermore, when targeting a bedridden person receiving care, tacit knowledge regarding end-of-life care may be used. The device 200 on which an application corresponding to the tacit knowledge runs may be a terminal device such as a smartphone. For example, the processing unit 210 of the device 200 acquires five types of information as input data: the amount or intake ratio of each type of meal (for example, main dish, side dish, or each ingredient such as meat or fish), the amount of water intake, the timing of intake, information regarding illness, and weight (or BMI). These input data may be entered by a caregiver. Alternatively, some of the input data may be acquired automatically by the device 200 by using an automatic recording device for food intake.

[0172] Based on the input data, the processing unit 210 outputs output data indicating whether end-of-life care should be started after a predetermined period of time, or whether it is time to change the content of care after end-of-life care has started. For example, machine learning may be performed on the input data based on training data to which expert-provided correct answer data has been added. In this case, the processing unit 210 obtains output data by inputting the input data into a trained model. Other machine learning methods, such as SVM, may also be used, or methods other than machine learning may also be used.

[0173] End-of-life care here refers to assistance for individuals who are considered likely to die in the near future. End-of-life care differs from standard care in that it emphasizes the alleviation of physical and mental pain and the support of the individual living a dignified life. Furthermore, as the individual's condition changes over time, the appropriate care may change accordingly. In other words, by indicating the timing for initiating end-of-life care and the timing for changes in the care content during end-of-life care, it becomes possible to provide appropriate care to the individual until the end. For example, experienced caregivers possess tacit knowledge to estimate the timing and content of end-of-life care needed based on various factors such as the amount of food consumed. By digitizing this tacit knowledge, other caregivers can also provide appropriate end-of-life care.

[0174] When the ability represented by the ability information has declined to a state indicating bedriddenness, the processing unit 210 of the device 200 starts a judgment based on the input data and outputs an analysis result screen showing the judgment result to the server system 100. The analysis result screen may be displayed on the display unit 240 of the device 200.

[0175] FIG. 16 illustrates an example of an analysis result screen. The analysis result screen may include time-series changes in feature values ​​calculated based on input data and a determination result of whether end-of-life care should be initiated after a predetermined period of time. The feature values ​​may be important input information, such as a moving average of food intake, or information calculated based on the five pieces of input information. For example, when a neural network (NN) is used, the feature values ​​may be the output of a given intermediate layer or output layer. For example, the input data may include actual measurements of main dish intake, water content, and BMI up to February 13, 2020. The processing unit 210 may estimate trends in main dish intake, water content, and BMI from February 14, 2020 onward based on the trained model. The analysis screen may also include graphs showing time-series changes in the actual and estimated values ​​for these three items. Note that FIG. 16 illustrates a graph of the moving average of these values ​​over a seven-day period. This allows caregivers to easily understand the trends in important items in end-of-life care. As mentioned above, the input data may include other items, and the information displayed on the analysis result screen is not limited to the example of FIG.

[0176] The analysis result screen may also display a period during which end-of-life care may be provided. In the example of Figure 16, the text "End-of-life care may be provided from 2020-03-14" is displayed, and the corresponding period in the graph is displayed in a distinctive background color different from the other periods. This clearly indicates the timing and period during which end-of-life care is estimated to be necessary, making it possible to appropriately present information about end-of-life care to the user.

[0177] <Device behavior regarding fall risk> 5, a bedridden caregiver is unlikely to spontaneously start moving, such as standing up, and therefore the risk of falling is assumedly low. Therefore, when a bedridden care recipient is the target, each device 200 shown in FIGS. 8 and 9 may be set to operation mode 0, which is inactive.

[0178] <Device behavior regarding fall risk> On the other hand, even for bedridden care recipients, the risk of falling is high because they may use a wheelchair 630 or the like. Therefore, even when targeting a bedridden care recipient, it is desirable to continue the fall risk processing by each device 200 shown in FIGS. 10 and 11. In this case, the care recipient has difficulty changing their posture voluntarily, increasing the risk of bedsores. Therefore, the seat sensor 440 shown in FIG. 10 may operate in operation mode 3, which determines the level of the risk of bedsores in the wheelchair 630, in addition to determining forward and lateral slippage and falls. For example, the seat sensor 440 may determine that there is a risk of bedsores when a state in which there is little change in pressure value has continued for a predetermined period of time or more. Similarly, the terminal device 450 shown in FIG. 11 may operate in an operation mode that presents the caregiver with a position to prevent bedsores.

[0179] <Device behavior regarding aspiration risk> Even bedridden individuals receiving care may continue to eat orally. Therefore, the choking detection device 460 shown in FIG. 12 also operates when the target is a bedridden individual receiving care. The choking detection device 460 may operate in the same operating mode when the target is a bedridden individual receiving care who is not bedridden but has difficulty eating, and when the target is a bedridden individual receiving care. Furthermore, the choking detection device 460 may operate in an operating mode that can further reduce the risk of aspiration when the target is a bedridden individual receiving care, compared to when the target is a bedridden individual receiving care who has difficulty eating. For example, the choking detection device 460 may more easily detect the risk of aspiration by lowering the threshold used for comparison with the swallowing time. Alternatively, when the target is a bedridden individual receiving care, the choking detection device 460 may operate to reduce the risk of aspiration by adding a food type determination. Alternatively, when targeting a bedridden person receiving care, the choking detection device 460 may perform an operation to reduce the risk of aspiration by adding a determination as to whether or not there is a dangerous choking, in addition to the usual processes of detecting choking and calculating swallowing time, etc. Furthermore, the choking detection device 460 may add a process to determine whether choking occurs frequently, or a process to determine whether or not the person receiving care appears sleepy.

[0180] Furthermore, as described above in the end-of-life care, when the target is a bedridden person receiving care, the amount of food eaten may be automatically recorded. For example, the terminal device 462 of the choking detection device 460 may capture images of the meal before and after the meal and automatically record the amount of food eaten based on the difference between the images. That is, when the target is a bedridden person receiving care, the choking detection device 460 may operate in an operation mode that includes automatic recording of the amount of food eaten. Furthermore, a device other than the choking detection device 460 may be used as the device 200 that automatically records the amount of food eaten.

[0181] <Ability information update> 7, the capability information acquisition unit 111 of the server system 100 may perform processing to update the capability information based on the sensing data. For example, the capability information acquisition unit 111 performs an evaluation related to a bedsore based on the sensing data, and determines a change in capability based on the evaluation result.

[0182] For example, the capability information acquisition unit 111 determines the degree of dispersion of pressure values ​​using a detection device 430 or the like, and if pressure is applied to a specific location for a predetermined period of time or longer, it determines that the posture change required to prevent bedsores has not been achieved and that the capability is low.

[0183] Furthermore, the ability information acquiring unit 111 may determine the number of times that the positioning adjustment was judged as NG or the number of times that the diaper position was inappropriate based on information from the bed position detecting device 470, and obtain the ability information based on these numbers. For example, the fewer the number of times that the positioning adjustment was judged as NG or the fewer the number of times that the diaper position was inappropriate, the higher the ability is judged to be, and the more the number of times, the lower the ability is judged to be.

[0184] Furthermore, the ability information acquiring unit 111 may update the ability information based on the presence or absence of a bedsore output from the eyeglass-type device 480 such as MR glasses. For example, if there is no bedsore, the ability is determined to be higher than if there is a bedsore.

[0185] 2.6 Summary 8 to 14, etc., specific examples and operation examples of device 200 have been described. As described above, each device 200 switches its operation mode between at least an inactive operation mode 0 and an active operation mode 1 according to capability information. Also, as described above, the operation mode when active is not limited to one, and multiple operation modes with different processing contents may be set.

[0186] Fig. 17 shows an example of the operation mode of each device 200. As shown in Fig. 17, the imaging device 410 for detecting the risk of falling operates in operation mode 1 for a person being assisted who cannot sit up, in operation mode 2 for a person being assisted who cannot walk or who cannot eat well, and in inactive operation mode 0 for a person being assisted who is bedridden.

[0187] For example, operation mode 1 may be a mode in which it is determined that movement has occurred when edge-sitting is detected, and operation mode 2 may be a mode in which it is determined that movement has occurred when awakening is detected. Operation mode 2 detects movement at an earlier stage than operation mode 1, and therefore can further reduce the risk of falling.

[0188] In addition, the seat sensor 440 that detects the risk of falling operates in an inactive operation mode 0 for an assisted person who cannot stand up, in operation mode 1 for an assisted person who cannot walk, in operation mode 2 for an assisted person who cannot eat properly, and in operation mode 3 for an assisted person who is bedridden.

[0189] For example, operation mode 1 may be a mode that only performs forward and lateral slippage, operation mode 2 may be a mode that adds a fall risk assessment, and operation mode 3 may be a mode that adds a pressure ulcer assessment in a wheelchair. In this example, additional assessment targets are added as ability declines, making it possible to appropriately respond to increased risk associated with changes in ability.

[0190] In addition, the choking detection device 460, which detects the risk of aspiration, operates in inactive operation mode 0 for assisted persons who cannot sit up or walk, operates in operation mode 1 for assisted persons who cannot eat properly, and operates in operation mode 2 for assisted persons who are bedridden.

[0191] For example, operation mode 1 may be a mode that makes a judgment based on swallowing time, and operation mode 2 may be a mode that adds options such as food type to operation mode 1. In this example, additional judgment targets are added as ability declines, making it possible to appropriately respond to increased risks associated with changes in ability.

[0192] In addition, the bed position detection device 470, which detects the risk of bedsores, operates in an inactive operating mode 0 for assisted persons who are unable to sit up, walk, or eat properly, and operates in operating mode 1 for assisted persons who are bedridden.

[0193] As described above, the method of this embodiment makes it possible to take into consideration the relationship between capabilities and various risks, and to appropriately determine the presence or absence of risks in situations where it is highly necessary, and to perform processing to reduce the risks. Note that Fig. 17 shows an example of the relationship between the device 200, capability information, and operation mode, and the method of this embodiment is not limited to this.

[0194] When setting the operating mode of device 200 based on the ability information, it is expected that the functions of device 200 will be added to compensate for the decline in the ability of the person being assisted. However, a first device among devices 200 may be set to an operating mode in which more functions are used as the ability value represented by the ability information declines within a range above a predetermined threshold, and may be set to an operating mode in which fewer functions are used within a range below the predetermined threshold compared to a range above the predetermined threshold. That is, some of the devices in this embodiment may operate by increasing functions to compensate for a certain degree of decline in ability, but may shift to reducing functions when a certain level of decline in ability is observed. In this way, it is possible to appropriately consider the necessity of each tacit knowledge according to ability and reduce the processing load by deactivating applications corresponding to less necessary tacit knowledge.

[0195] The first device here may be a device 200 used to determine the risk of a fall when the care recipient begins to move. For example, the first device may be at least one of the imaging device 410 shown in FIG. 8, the bedside sensor 420 shown in FIG. 9, and the detection device 430 shown in FIG. 9. In the example of FIG. 17, within the ranges of "unable to sit up," "unable to walk," and "unable to eat properly," the operation modes of these devices 200 are changed to add processing content, while for the lower ability level of "bedridden," an operation mode corresponding to inactive is set. In this way, the continuation of the operation of a device whose necessity has decreased can be suppressed. Note that the detection device 430 may be used to determine end-of-life care for "bedridden," and it is not necessary for all of the devices 200 used to determine the risk of a fall when the care recipient begins to move to reduce their functions when the ability level is low.

[0196] 3. Device control based on scene information and device type information Furthermore, in the method of this embodiment, information other than capability information may be used to set the operation mode of the device 200. Scene information and device type information will be described below.

[0197] 3.1 Overview FIG. 18 is a sequence diagram illustrating the operation of the server system 100 and the device 200, and illustrates an example in which the operation mode of the device 200 changes based on the ability information of the person being assisted, the scene information, and the device type information.

[0198] First, in step S301, the server system 100 performs processing to transmit data including capability information, scene information, and device type information to the device 200. Note that either the scene information or the device type information may be omitted. Fig. 18 shows an example in which data is transmitted in which the ADL index value is 2, the scene ID that identifies the scene is 0, and the device type ID that identifies the device type is xx.

[0199] In step S302, the device 200 controls the activation / inactivation of installed vendor applications based on the acquired capability information, scene information, and device type information. For example, the storage unit 220 of the device 200 may store table data in which ADL index values, scene IDs, device type IDs, and the activation / inactivation of each application are associated with each other. The processing unit 210 of the device 200 determines the activation / inactivation of each application by extracting records from the table data that match the received data. Furthermore, as will be described later with reference to FIGS. 19, 20, and 22, each device 200 may store an algorithm for determining an operation mode based on the capability information, scene information, and device type information. FIG. 18 illustrates an example in which an operation mode is set in which, of vendor applications 1 to 3, vendor applications 1 and 2 are activated and vendor application 3 is deactivated.

[0200] After the process of step S302, the device 200 executes a process according to the vendor application 1 and a process according to the vendor application 2. In step S303, the device 200 transmits a process result to the server system 100. The process result here corresponds to a result of a judgment made using the tacit knowledge of an expert. The process result here may also include a log of sensing data detected by the device 200.

[0201] In step S304, the server system 100 executes control based on the processing result received from the device 200. For example, the processing unit 110 may identify a device to be controlled and transmit a control signal to operate the device to be controlled.

[0202] In step S305, the server system 100 executes a process of updating the ability information, scene information, and device type information of the person being assisted. The ability information updating process is as described above. The scene information acquisition unit 112 obtains scene information based on at least one of the log of sensing data acquired from the device 200, information about the person being assisted such as attributes, and information about the caregiver such as schedule. The device type information acquisition unit 113 acquires, as device type information, information indicating the types of other devices 200 used together with the target device 200. A specific example of the process will be described later.

[0203] In step S306, the server system 100 performs processing to transmit data including the updated capability information, scene information, and device type information to the device 200. In Fig. 18, an example is shown in which the scene ID has been changed from 0 to 1.

[0204] In step S307, the device 200 controls the activation / inactivation of installed vendor applications based on the acquired data. For example, as described above, the device 200 determines the activation / inactivation of each vendor application based on table data. In the example of FIG. 18, vendor applications 1 and 2 are maintained in an active state, and vendor application 3 is changed from inactive to active. As a result, from step S307 onwards, the device 200 transitions to a state in which it operates in an operation mode in which all of vendor applications 1 to 3 are active. The operations from step S307 onwards are similar to, for example, steps S303 to S306.

[0205] An example of setting an operation mode based on scene information and an example of setting an operation mode using device type information will be described below.

[0206] 3.2 Example of processing based on scene information The server system 100 may obtain scene information identifying an assistance scene for the person being assisted and transmit the obtained scene information to the device 200. The device 200 then determines which of a plurality of operation modes to operate in based on the capability information and the scene information. For example, the storage unit 220 of the device 200 stores information associating the capability information, the scene information, and the operation mode. The processing unit 210 determines the operation mode based on the stored information and the capability information and scene information acquired from the server system 100. The information associating the capability information, the scene information, and the operation mode may be, for example, table data associating the capability information value, the scene information value, and the active / inactive status of each application. Alternatively, the information associating the capability information, the scene information, and the operation mode may be an algorithm that determines the active / inactive status of each application based on the capability information and the scene information. This makes it possible to operate the device 200 appropriately according to the assistance scene, in other words, to perform assistance using appropriate tacit knowledge.

[0207] For example, the scene information acquisition unit 112 may obtain, as scene information, information about the caregivers who will be assisting the person being assisted. Specifically, the scene information may be information about the number of caregivers who can assist the person being assisted, or information about years of service, skill level, qualifications, etc. An example using the number of caregivers will be described below. In this case, the device 200 determines the operation mode depending on the number of caregivers.

[0208] The device 200 here may be, for example, the imaging device 410 described above with reference to FIG. 8. For example, when there are a certain number of caregivers in a living room or other area where care recipients tend to be active together, the caregivers can follow each other, allowing the caregivers to visually determine when the care recipient begins to move, or to intervene appropriately if they determine that the care recipient is at risk of falling. Intervention here refers to, for example, moving close to the care recipient who is about to start moving and supporting the body as needed to support the start of movement. In this case, there is a relatively low need for the imaging device 410 to be active.

[0209] On the other hand, when the number of caregivers in the target space is small, each caregiver has to perform many tasks, making it difficult to observe the movements of the person being assisted in detail and intervening at the appropriate time. In this case, the need for the imaging device 410 to be active becomes relatively high.

[0210] In view of the above, by determining the operation mode of the image capture device 410 using the scene information in addition to the capability information, it becomes possible to operate the image capture device 410 appropriately.

[0211] 19 is a flowchart illustrating the process of determining the operation mode in the image capturing device 410. First, in step S401, the processing unit 210 of the image capturing device 410 performs a process of identifying the target person being assisted. For example, the processing unit 210 identifies the person being assisted by performing a face recognition process based on the captured image.

[0212] In step S402, the processing unit 210 acquires the ability information of the person being assisted. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 in Fig. 18 from the server system 100.

[0213] In step S403, the processing unit 210 determines, based on the ability information, whether the ability of the person being assisted has declined to the point where the person is unable to sit up or walk. If the ability has not declined (step S403: No), the risk of falling is low. Therefore, in step S404, the processing unit 210 sets the operation mode of the imaging device 410 to mode 0, which corresponds to inactivity. Note that the ability information in this embodiment is not limited to the ADL index value but may be more detailed information, such as information indicating the above-mentioned standing-up method, sitting ability, swallowing ability, walking ability, etc. Furthermore, various sensing data can be used to determine sitting ability, etc., and various modifications can be made to the algorithm used to determine sitting ability, etc. from the sensing data. In determining whether the ability of the person being assisted has declined to the point where the person is unable to sit up or walk, as described above, processing using detailed ability information, which has a larger amount of data than the ADL index value, may be performed. In this case, the processing load of step S403 is large, so this processing may be performed in the server system 100.

[0214] If the ability of the person being assisted has deteriorated to the point where the person is unable to move around (step S403: Yes), in step S405, the processing unit 210 identifies the number of caregivers based on the scene information. For example, the scene information acquisition unit 112 of the server system 100 may determine the number of caregivers based on a captured image, or may determine the number of caregivers based on an assistance schedule (e.g., a roster) in a nursing care facility or the like. Alternatively, information indicating the number of caregivers in the target space may be obtained by attaching RFID (radio frequency identifier) ​​tags or the like to the caregivers and installing a reader at an entrance or exit of the target space. The processing unit 210 of the device 200 executes the process shown in step S405 by acquiring this information from the server system 100.

[0215] In step S406, the processing unit 210 makes a determination based on the scene information. Specifically, the processing unit 210 determines whether the number of caregivers is equal to or less than a predetermined threshold. If the number of caregivers is greater than the predetermined threshold (step S406: No), there are enough people available to deal with the risk of falling, so the processing proceeds to step S404, and the processing unit 210 sets the operation mode of the imaging device 410 to mode 0, which corresponds to inactivity.

[0216] If the number of people is equal to or less than the predetermined threshold (step S406: Yes), there are insufficient caregivers, and support from the device 200 is important to reduce the risk of falling. Therefore, in step S407, the processing unit 210 sets the operation mode of the imaging device 410 to mode 1, which corresponds to active. Note that, while steps S405 and S406 have been described in FIG. 19 as being performed by the processing unit 210, the present invention is not limited to this. For example, the server system 100 may perform steps S405 and S406, and the processing unit 210 may receive only the results and determine whether to transition to step S404 or step S407.

[0217] Furthermore, the scene information is not limited to information about the caregiver. For example, the scene information acquisition unit 112 of the server system 100 may obtain information about the person being assisted as the scene information. Specifically, the scene information may be attribute information that indicates the attributes of the person being assisted. The attribute information includes the age, height, weight, sex, medical history, medication history, etc. of the person being assisted. For example, the scene information acquisition unit 112 obtains information that indicates whether the person being assisted has dementia as the scene information.

[0218] To reduce the risk of falling when starting to move, it is important for the caregiver to call out to the person to stop them temporarily, but dementia patients may not respond when the caregiver calls out to them. However, it is known that even dementia patients often remember the voices of close family members and other people, and are more likely to respond when called by a family member.

[0219] Therefore, the imaging device 410 may have an operation mode in which audio data of family members' voices or video data of family members calling attention is stored and the audio data or video data is output using a speaker (not shown). For example, the processing unit 210 of the imaging device 410 determines whether the imaging device 410 is active or inactive based on the ability information and scene information, which is the number of caregivers, as shown in FIG. 19. When the imaging device 410 is set to active, the processing unit 210 then determines whether the person being assisted identified in step S401 is a dementia patient. If the person being assisted is a dementia patient, the imaging device 410 operates in an operation mode in which audio data, etc. of family members are output, and if the person being assisted is not a dementia patient, the imaging device 410 operates in an operation mode in which audio data, etc. of family members are output. In this way, it becomes possible to set an operation mode that matches the attributes of the person being assisted. Note that multiple voice data from family members or the like may be prepared for one person receiving care. For example, if a family member or the like only calls out in one way, the person receiving care may memorize the call and become unresponsive. Therefore, the image capture device 410 may perform a process of outputting one voice data randomly selected from the multiple voice data stored in association with the target person receiving care. This increases the variety of calls from family members or the like, making it possible to effectively stop the person receiving care from starting to move.

[0220] Furthermore, the scene information in this embodiment may be information indicating the type of assistance, such as meal assistance, excretion assistance, or mobility / transfer assistance. For example, the scene information acquisition unit 112 may acquire scene information indicating the type of assistance based on a user input by a caregiver. The scene information acquisition unit 112 may also determine the type of assistance being provided based on the relationship between an assistance schedule at a nursing facility or the like and the current time. Alternatively, the scene information acquisition unit 112 may determine the type of assistance by estimating the location of the person being assisted at the nursing facility or the like. For example, the scene information acquisition unit 112 determines that meal assistance is being provided if the person being assisted is in the dining room, and that excretion assistance or assistance to prevent falls is being provided if the person being assisted is in the toilet. The location determination may be performed using a motion sensor or the like installed at various locations in the facility. Alternatively, access points (APs) may be installed at various locations in the facility, and the location determination may be performed depending on which AP a station device (STA) carried by the caregiver connects to.

[0221] For example, the seat sensor 440 shown in Fig. 10 can determine whether the person is slipping forward or to the side, and whether there is a possibility of falling, as described above. However, when the person is eating while sitting in a wheelchair 630, a table on which food is served is placed in front of the person being assisted, as shown in Fig. 12, and the table provides support, making it less likely that the person will fall. Therefore, by deactivating the seat sensor 440's function for determining whether there is a possibility of falling while the person is eating, it is possible to omit less-essential processing.

[0222] 20 is a flowchart illustrating the process of determining the operation mode in the seat sensor 440. First, in step S501, the processing unit 210 of the seat sensor 440 performs a process of identifying the target person being assisted. For example, the processing unit 210 may identify the person being assisted based on an assistance schedule or the like, or may identify the person being assisted based on user input.

[0223] In step S502, the processing unit 210 acquires the ability information of the person being assisted. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 in Fig. 18 from the server system 100.

[0224] In step S503, the processing unit 210 determines whether the ability of the person being assisted has declined to the point where the person is unable to walk, based on the ability information. If the ability has not declined (step S503: No), there is little need for a determination using the seat sensor 440, and therefore in step S504 the processing unit 210 sets the operation mode of the seat sensor 440 to mode 0, which corresponds to inactive. Also, as in the example described above in step S403, in step S503, processing may be performed that takes into account more detailed information about the ability information. Therefore, the process of step S503 may be executed in the server system 100.

[0225] If the ability of the person being assisted has deteriorated to the point where the person is unable to walk (Yes in step S503), in step S505 the processing unit 210 acquires information indicating the type of assistance as scene information. As described above, the processing in step S505 may be performed based on a user input or on some kind of sensing data.

[0226] In step S506, the processing unit 210 makes a determination based on the scene information. Specifically, the processing unit 210 determines whether the assistance type is meal assistance. If it is meal assistance (step S506: Yes), the forward slippage / lateral slippage determination is useful, but the need for determining the possibility of a fall is low. Therefore, in step S507, the processing unit 210 sets the operation mode of the seat sensor 440 to a mode that performs the forward slippage / lateral slippage determination but does not perform the determination of the possibility of a fall.

[0227] If the assistance type is not meal assistance (step S506: No), there is not necessarily a table in front of the person being assisted, and there is a high risk of the person falling from the wheelchair 630. Therefore, in step S508, the processing unit 210 sets the operation mode of the seat sensor 440 to a mode that performs both the forward / lateral slippage determination and the determination of the possibility of a fall. Also, in step S508, the processing unit 210 may set the operation mode of the seat sensor 440 to a mode that does not perform the forward / lateral slippage determination, but performs the determination of the possibility of a fall.

[0228] The above describes an example in which processing based on scene information is performed on the imaging device 410 and the seat sensor 440. However, it goes without saying that the operation mode of the other devices 200 may also be determined based on scene information.

[0229] 3.3 Example of processing based on device type information The server system 100 (the device type information acquisition unit 113) may also obtain device type information that identifies the type of a concurrent device used to assist the same person being assisted as the device 200. The server system 100 transmits the device type information to the device 200. The device 200 determines in which of a plurality of operation modes the device 200 will operate based on the capability information and the device type information. For example, the storage unit 220 of the device 200 stores information that associates the capability information, device type information, and operation modes. The processing unit 210 determines the operation mode based on the stored information and the capability information and device type information acquired from the server system 100. The information that associates the capability information, device type information, and operation modes may be, for example, table data that associates the value of the capability information, the value of the device type information, and the active / inactive status of each application. Alternatively, the information that associates the capability information, device type information, and operation modes may be an algorithm that determines the active / inactive status of each application based on the capability information and device type information.

[0230] Fig. 21 is a diagram illustrating an example of the combined device and the choking hazard detection device 460 described above with reference to Fig. 12. The combined device here may specifically be the device 200 according to this embodiment. The choking hazard detection device 460 is used during meals, but meals may be taken using a wheelchair 630 or a bed 610.

[0231] For example, the device used in combination with the choking detection device 460 may be the seat sensor 440 shown in Fig. 10. The device type information acquiring unit 113 may determine that the choking detection device 460 and the seat sensor 440 are used in combination when the choking detection device 460 is active and a log of the detection results and sensing data from the seat sensor 440 has been acquired. Alternatively, the device type information acquiring unit 113 may determine that the choking detection device 460 and the seat sensor 440 are used in combination when it is determined that the person being assisted is sitting in a wheelchair 630 based on the detection results and sensing data log from the seat sensor 440. In this case, it is considered that the person being assisted is eating using the wheelchair 630.

[0232] Similarly, the device used in combination with the choking stool detection device 460 may be the detection device 430 shown in Fig. 9. The device type information acquiring unit 113 may determine that the choking stool detection device 460 and the detection device 430 are used in combination when the choking stool detection device 460 is active and a log of the detection results and sensing data from the detection device 430 has been acquired. Alternatively, the device type information acquiring unit 113 may determine that the choking detection device 460 and the detection device 430 are being used in combination when it is determined that the person being assisted is in bed based on the detection result from the detection device 430 or the log of sensing data. In this case, it is considered that the person being assisted is eating in bed 610.

[0233] 22 is a flowchart illustrating the process of determining the operation mode in the choking stool detection device 460. First, in step S601, the processing unit 210 of the choking stool detection device 460 performs a process of identifying the target person being assisted. For example, the processing unit 210 may identify the person being assisted based on face recognition processing of an image captured by the terminal device 462.

[0234] In step S602, the choking hazard detection device 460 acquires ability information of the person being assisted. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 in FIG. 18 from the server system 100.

[0235] In step S603, the choking hazard detection device 460 determines, based on the ability information, whether the ability of the person being assisted has declined to the point where the person is unable to eat properly. If the ability has not declined (step S603: No), there is little need for determination using the choking hazard detection device 460, and therefore in step S604 the processing unit 210 sets the operation mode of the choking hazard detection device 460 to mode 0, which corresponds to inactive.

[0236] If the ability of the person being assisted has deteriorated to the point where they are unable to eat properly (Yes in step S603), in step S605 the choking hazard detection device 460 acquires information identifying the combined device from the device type information acquisition unit 113. As can be seen from the above example, it is sufficient to determine whether the person is eating in bed 610 or in wheelchair 630, so while the type of combined device is important, there is little need to identify the vendor or model number. Therefore, in step S605 the choking hazard detection device 460 acquires the device type ID of the combined device.

[0237] In step S606, the choking hazard detection device 460 makes a determination based on the device type information. Specifically, the choking hazard detection device 460 determines whether the meal is being eaten using the wheelchair 630. Specifically, as described above, the choking hazard detection device 460 may determine whether the combined device represented by the device type information is the seat sensor 440 or the detection device 430. If the meal is being eaten using the wheelchair 630 (step S606: Yes), the condition of the person being assisted is relatively good because it means that the person can move to a restaurant or the like. Therefore, in step S607, the processing unit 210 sets the operation mode of the choking hazard detection device 460 to a mode that makes a normal determination using swallowing time, etc.

[0238] On the other hand, if the meal is being eaten using the bed 610 (step S606: No), it is estimated that the person being assisted has difficulty moving around or would have difficulty eating without a device that allows for flexible adjustment of the back angle, such as the bed 610. Therefore, in step S608, the processing unit 210 sets the operation mode of the choking detection device 460 to a mode that can further reduce the risk of aspiration. For example, the choking detection device 460 may not only detect the presence or absence of a choking, but also perform processing to determine the nature of the choking. For example, the choking detection device 460 may determine whether the choking is a dangerous choking that is likely to lead to aspiration. The choking detection device 460 may also determine whether the person being assisted looks sleepy. The determination of whether the person looks sleepy may be made based on, for example, the opening and closing of the eyes, the frequency of mouth and hand movements, etc., based on images captured by the terminal device 462. The determination of whether the person looks sleepy may also be performed using the detection device 430.

[0239] Although the above examples show two types of combined devices, the wheelchair 630 and the bed 610, the present invention is not limited to this. For example, a normal wheelchair and a reclining wheelchair may be used as the wheelchair 630. In this case, the flexibility of back angle adjustment increases in the order of the normal wheelchair, the reclining wheelchair, and the bed 610. Therefore, the operation mode of the choking stool detection device 460 may be controlled so that more thorough care can be provided in the order of the normal wheelchair, the reclining wheelchair, and the bed 610. For example, the choking stool detection device 460 may increase the number of applications set to active in the above order.

[0240] In the above, an example has been described in which the choking detection device 460 is the device 200 for which the operation mode is to be set, and the seat sensor 440 and the detection device 430 are the combined devices. However, the seat sensor 440 or the detection device 430 may be the device 200 for which the operation mode is to be set, and the choking detection device 460 may be the combined device. In other words, in the method of this embodiment, when multiple devices 200 are used in combination, the operation mode may be changed by the multiple devices 200 working together. For example, using either the seat sensor 440 shown in FIG. 10 or the terminal device 450 used to adjust the wheelchair position shown in FIG. 11 is useful for reducing the risk of falling. However, both the seat sensor 440 and the terminal device 450 may be used in combination. In this case, the control of one device 200 may be linked to the control of the other device 200.

[0241] For example, when it is determined based on the ability information that the person being assisted has transitioned to a state where he or she is unable to walk, the seat sensor 440 may first transition to an operation mode corresponding to active, while the terminal device 450 may maintain an operation mode corresponding to inactive. In this way, first, a determination is made as to whether the person is shifting forward or sideways, and the ability to maintain a sitting position is calculated using the log data.

[0242] If it is determined that the ability to maintain a seated position has dropped below a predetermined level, the terminal device 450 transitions to an active operation mode. This causes the device type information acquisition unit 113 to transmit data indicating that the terminal device 450 is a combined device to the seat sensor 440. The seat sensor 440 may be set to a different operation mode when the terminal device 450 is combined with the seat sensor 440 than when it is not combined with the seat sensor 440. For example, the processing unit 210 of the seat sensor 440 may change the threshold value used for the determination. Alternatively, the seat sensor 440 may switch from an operation mode in which a criterion (initial value) for determining forward or lateral shift, etc., is internally determined to an operation mode in which the criterion is determined based on, for example, the timing of a position adjustment using the terminal device 450. Similarly, in other cases, the control of the seat sensor 440 or the terminal device 450 may be changed based on information from the other device.

[0243] Alternatively, the operation mode of the seat sensor 440 may be controlled based on information acquired by the terminal device 450. For example, the terminal device 450 can input points that a caregiver should pay attention to when adjusting the posture of the person being assisted. For example, points such as paying attention to the position of the right shoulder or placing a cushion under the right arm may be input, and the terminal device 450 can estimate the attributes of the person being assisted based on the input points. In the above example, the attribute that the person being assisted has a tendency toward right shoulder paralysis is determined. The seat sensor 440 may determine the operation mode based on this information. For example, the seat sensor 440 may store multiple applications that perform forward / lateral slippage detection, which differ depending on the presence or absence of paralysis and the location of the paralysis. The seat sensor 440 then controls the activation / deactivation of each application based on the attributes acquired from the terminal device 450. In the above example, the seat sensor 440 activates an application that performs forward / lateral slippage detection suitable for a person being assisted with right shoulder paralysis, and deactivates other applications that perform forward / lateral slippage detection. In this way, in the seat sensor 440 that can use a plurality of implicit knowledge for determining whether the seat is shifting forward or sideways, it becomes possible to switch the implicit knowledge according to the attributes of the person being assisted.

[0244] Furthermore, the server system 100 of this embodiment may acquire mode information that specifies the operation mode of the concurrent device, and transmit the device type information and the mode information to the device 200. The device 200 determines in which of a plurality of operating modes to operate based on the capability information, device type information, and mode information.

[0245] For example, when the seat sensor 440 transitions to an operation mode in which it determines the possibility of falling in addition to determining whether the seat sensor 440 is shifting forward or sideways, the terminal device 450 may transition to an operation mode in which it performs an additional process of recommending a cushion or the like in addition to the normal process of determining whether the wheelchair position is appropriate. Alternatively, when the operation mode of the seat sensor 440 changes, the terminal device 450 may execute a process of switching the teacher data that is determined to be correct. Various other variations are possible regarding the specific method of linking operation modes.

[0246] In this way, it becomes possible to link multiple devices 200 using more detailed operation modes, not limited to simple active / inactive. In the above example, the device 200 related to detecting the wheelchair position and the device 200 related to assisting in adjusting the wheelchair position can be appropriately linked, which makes it possible to further reduce the risk of the person being assisted falling. Note that although the linkage between the seat sensor 440 and the terminal device 450 has been described here, this does not prevent other devices 200 from operating in linkage.

[0247] Furthermore, although the combination of capability information and scene information, and the combination of capability information and device type information have been described above, a combination of capability information, scene information, and device type information may also be used.

[0248] Furthermore, in the method of this embodiment, the type of assistance may be associated with the type of device 200. For example, the choking spurt detection device 460 shown in Fig. 12 is a device 200 specific to meal assistance, and when the choking spurt detection device 460 is operating, there is a high probability that the person being assisted is receiving meal assistance. In this way, among the devices 200, there are devices for which device type information can be associated with the type of assistance. For example, when the choking spurt detection device 460 is active, it communicates with the server system 100 via the gateway 300. Therefore, the scene information acquisition unit 112 can determine the operating state of the choking spurt detection device 460 based on whether or not communication with the choking spurt detection device 460 is occurring.

[0249] For example, the scene information acquisition unit 112 of the server system 100 may obtain scene information indicating the type of assistance based on the device type information acquired by the device type information acquisition unit 113, and transmit the scene information to the device 200. For example, the scene information acquisition unit 112 may determine that a person is eating when the choking spurt detection device 460 is active, and that a person is not eating when the choking spurt detection device 460 is inactive. Alternatively, the server system 100 may transmit the device type information acquired by the device type information acquisition unit 113 to the device 200, and the device 200 may perform processing to determine an operation mode according to the type of assistance associated with the device 200. For example, when the choking spurt detection device 460 is a combined device, the device 200 may perform processing to select an operation mode suitable for meal assistance. As described above, setting the operation mode according to the type of assistance may be performed as processing related to the scene information or as processing related to the device type information, and various modifications are possible in specific embodiments.

[0250] 4. Other examples of devices Furthermore, the device 200 in this embodiment is not limited to the one described above. For example, the device 200 in this embodiment may be an MCI assessment device that assesses mild cognitive impairment (MCI). For example, the MCI assessment device performs processing to ask questions to the person being assisted using voice or images and receive responses to the questions. The questions here may be those using the Mini-Mental State Examination (MMSE) or other methods. The MCI assessment device makes an MCI assessment based on the responses of the person being assisted. The MCI assessment device may also make an MCI assessment based on information about the sleep of the person being assisted.

[0251] Furthermore, the device 200 in this embodiment may be an assistance recording device that automatically records the history of assistance. For example, the assistance recording device may be device 200 that detects the position information of at least one of the caregiver and the person being assisted. The assistance recording device determines, for example, the time the person in question spends in bed 610, the toilet, the bath, the dining room, etc., and stores, as an assistance record, the type of assistance provided, how often, and for how long, based on the determination results. The assistance recording device is useful, for example, in home care where assistance schedules are not strictly set.

[0252] Furthermore, device 200 of this embodiment may include a reclining wheelchair 510 with an adjustable backrest angle shown in Fig. 23, or a nursing bed 520 with an adjustable bottom angle shown in Fig. 24. Reclining wheelchair 510 and nursing bed 520 are, for example, the controllable devices shown in Fig. 7 and Fig. 18. For example, when choking is detected by choking detection device 460, the angles of the backrest and bottom are controlled to angles suitable for the person being assisted to swallow.

[0253] Furthermore, the reclining wheelchair 510 may be controlled based on the processing results of the seat sensor 440 and the terminal device 450. For example, the wheelchair 630 shown in Figures 10 and 11 may be the reclining wheelchair 510, which is a device to be controlled. Similarly, the nursing bed 520 may be controlled based on the processing results of the bedside sensor 420, the detection device 430, the bed position detection device 470, the glasses-type device 480, etc. For example, the bed 610 shown in Figures 9, 13, and 14 may be the nursing bed 520, which is a device to be controlled.

[0254] Additionally, the device 200 used in this embodiment can be modified in various ways with respect to the shape, the number and type of sensors, the processing contents, and the like.

[0255] 5. Device-to-device communication In the above description, for example, as shown in step S303 of Fig. 18, the server system 100 receives a processing result from the device 200. Also, as shown in step S304 of Fig. 18, the server system 100 transmits a control signal to the device to be controlled to cause the device to perform control based on the processing result. Also, as shown in step S306 of Fig. 18, when information that determines the operation mode is updated, the server system 100 also notifies the device 200 of the information.

[0256] However, in order to prevent an excessive increase in the processing load on the server system 100, in this embodiment, at least one of the control of the control target device based on the processing result in the device 200 and the notification of scene information, etc. may be performed without a server. Hereinafter, a specific example of the information processing system 10 will be described.

[0257] Fig. 25 is a diagram showing a configuration example of the information processing system 10 when serverless control is executed. The information processing system 10 of this embodiment may include an entry / exit control device 700 that detects the entry and exit of devices 200 into a given space, and a gateway 300 that relays communication between the devices 200 located in the space and the server system 100. As in Fig. 1, the information processing system 10 also includes the server system 100 and the devices 200. In the example of Fig. 25, devices 200-1, 200-2, and 200-3 are illustrated as the devices 200.

[0258] As shown in FIG. 25, gateway 300 is placed in a room in a care facility or the like, and is a device capable of transmitting and receiving radio waves with sufficient strength at least to and from device 200 located in the room.

[0259] 25 is a device that is placed near the entrance to a room, for example, and detects the entry or exit of device 200. The entry and exit control device 700 is a device that performs short-range wireless communication using, for example, Bluetooth Low Energy (BLE), and communicates with Bluetooth-compatible devices located within a predetermined distance. For example, the entry and exit control device 700 may be placed in a room and determine that a device 200 that can communicate with the entry and exit control device 700 using BLE is located in the room.

[0260] Alternatively, the entry / exit control device 700 may make a more detailed determination. For example, multiple entry / exit control devices 700 may be placed at different positions in a room. Each of the multiple entry / exit control devices 700 communicates with the device 200 and estimates the distance to the device 200 based on the radio wave strength (e.g., RSSI: Received Signal Strength Indicator) in the communication. In this way, the distance from each of multiple points whose positions are known can be determined, and the position of the device 200 can be estimated. The entry / exit control device 700 may determine whether the device 200 is located within the room based on the estimated position and known information such as the size and shape of the room. Various other variations are possible for the specific determination of entry and exit.

[0261] The storage unit 220 of the device 200 stores information about the device 200. The information about the device 200 includes at least a device ID that identifies the device 200. The information about device 200 may include part or all of device information 122 described above with reference to Fig. 3. In BLE communication with device 200, entry and exit control device 700 acquires information about device 200, which is the communication target. In this way, entry and exit control device 700 can appropriately identify device 200 that has entered the target space.

[0262] The storage unit 220 of the device 200 may also store address information that the target device 200 uses for communication via the gateway 300. The address information here may be, for example, an IP address, or other information that can identify the IP address (for example, a MAC address). In BLE communication with the device 200, the entry and exit control device 700 acquires the address information of the device 200 that is the communication target.

[0263] Furthermore, the entry and exit control device 700 is not limited to a device that uses BLE, and other communication methods such as NFC (Near Field Communication) can be widely applied.

[0264] The entry / exit control device 700 has a memory (not shown), and the memory may store existing device information that identifies existing devices located in a room. This makes it possible to appropriately manage what devices 200 are located in a target room. In the example of FIG. 25, devices 200-1 and 200-2 have already entered the room. For example, when device 200-1 enters the room, the entry / exit control device 700 executes a process of adding the device ID of device 200-1 to the existing device information based on BLE communication with device 200-1. Similarly, when device 200-2 enters the room, the entry / exit control device 700 executes a process of adding the device ID of device 200-2 to the existing device information. As a result, the state that devices 200-1 and 200-2 are present in the room is maintained as existing device information.

[0265] The entry and exit control device 700 may perform a process of transmitting the existing device information it holds to the device 200 that has entered the room. For example, the entry and exit control device 700 transmits existing device information including the device IDs of device 200-1 and device 200-2 to device 200-1. In this way, device 200-1 can recognize that device 200-2 exists in the room in addition to device 200-1 itself. Similarly, by the entry and exit control device 700 transmitting the existing device information to device 200-2, device 200-2 can recognize that device 200-1 exists in the room.

[0266] The existing device information may also be information in which a device ID is associated with address information. For example, the entry / exit control device 700 transmits the device ID and address information of device 200-1 and the device ID and address information of device 200-2 to each device 200. In this way, each device 200 can obtain not only information indicating the existence of other devices 200, but also address information for communicating with the other devices 200 via the gateway 300.

[0267] The same applies when a new device 200-3 enters the room, as in the example of FIG. When the entry and exit control device 700 detects that a new device has entered the space, it may communicate with the new device to acquire address information of the new device to be used in communication via the gateway 300, and notify the acquired address information to the device 200 located in the space. For example, the entry and exit control device 700 performs a process of adding the device ID and address information of the device 200-3 to existing device information by performing BLE communication with the device 200-3. Note that the device ID and address information do not have to be acquired simultaneously. For example, if the connection between the device 200-3 and the gateway 300 has not been completed immediately after the device 200-3 enters the space and a dynamic IP address has not been assigned, the device 200-3 may wait for the connection with the gateway 300 to be established before transmitting its address information to the entry and exit control device 700.

[0268] After updating the existing device information in response to the entry of device 200-3, entry and exit management apparatus 700 transmits the existing device information to devices 200-1 to 200-3. This allows each device 200 located in the room to identify the other devices 200 in the room and their address information.

[0269] The entry / exit control device 700 may switch the information to be sent between existing devices (device 200-1 and device 200-2) and a new device (device 200-3) that has just entered the room. For example, the entry / exit control device 700 may send only the device ID and address information of the new device to the existing devices, and omit sending other records of the existing device information. The entry / exit control device 700 may also send the existing device information before it is updated to the new device. In this way, the transmission of less necessary information can be omitted, thereby reducing the communication load.

[0270] The same applies when any of the devices 200 leaves the room. For example, the entry / exit control device 700 determines that a device 200 that can no longer perform BLE communication or a device 200 whose estimated location is determined to be outside the room has left the room. In this case, the entry / exit control device 700 deletes the device ID and address information of the device 200 that has left the room from the existing device information. The entry / exit control device 700 also notifies the existing devices of information indicating that the device 200 has left the room. For example, the entry / exit control device 700 may transmit updated existing device information to each device 200. The entry / exit control device 700 may also transmit the device ID, etc. of the device 200 that has left the room to each device 200, and instruct each device 200 to delete the corresponding record from the existing device information held by each device 200.

[0271] Furthermore, the entry / exit management device 700 is not limited to storing information about existing devices located in the target space, but may store entry / exit log data. The log data is information in which, for example, the entry time, the exit time, the device ID, etc. are associated with each other.

[0272] Next, serverless communication will be described. For example, assume that device 200-1 is operating in an active operation mode, and as a result of the processing, it is determined that control of device 200-2 is necessary. For example, device 200-1 is choking detection device 460, and since dangerous choking has been detected, it determines that the bottom angle of device 200-2, which is nursing bed 520, needs to be changed.

[0273] In the examples described above with reference to FIGS. 7 and 18, processing was performed via the server system 100. However, in the example of FIG. 25, as described above, device 200-1 knows that device 200-2 is in the same room and also knows its IP address. Therefore, device 200-1 may transmit a control signal to device 200-2 without going through the server system 100. Specifically, device 200-1 transmits a control signal to gateway 300 using a packet specifying the IP address of device 200-2. Based on the IP address, gateway 300 determines that the packet destination is device 200-2, which is a device in the same network. Therefore, the packet containing the control signal is transmitted to device 200-2 without going through the server system 100, as shown as "Packet 1" in FIG. 25. This allows necessary control of device 200 to be performed without going through the server system 100, thereby reducing the load on the server system 100 and shortening the time required for control.

[0274] Furthermore, the scene information and device type information are not limited to those calculated by the processing unit 110 of the server system 100. For example, when the number of caregivers is used as scene information, if the device 200-1 is an image capture device 410 or the like, it is possible to calculate the number of caregivers in the room based on image processing of the captured image. Furthermore, the device type information is information that indicates the type of other devices 200 that are used together with a certain device 200. Therefore, the device type ID of the device 200-1 may serve as device type information for the devices 200-2 and 200-3 that operate in the same room.

[0275] Furthermore, the device 200 of this embodiment may also calculate ability information such as an ADL index. For example, as described above, since the log of sensing data can be used to calculate the ability information, the device 200 that acquires the sensing data may calculate the ability information.

[0276] In the examples described above using FIGS. 7 and 18, the capability information, scene information, and device type information were obtained in the server system 100, and data including these was transmitted to the device 200. However, in the example of FIG. 25, the device 200-1 knows that the device 200-2 is in the same room, and the IP address of the device 200-2 is also known. Therefore, the device 200-1 transmits the capability information, scene information, device type information, and the like to the device 200-2 without going through the server system 100. Specifically, similar to the control signal described above, the device 200-1 can transmit the capability information, and the like to the device 200-2 without going through the server system 100 by using a packet specifying the IP address of the device 200-2. This allows the operating mode to be set according to the capability information, and the like, without going through the server system 100, thereby reducing the load on the server system 100 and shortening the time required for control. For example, a plurality of devices 200 located in a room may execute a process of identifying a concurrent device by each notifying the other devices 200 of its own device type information.

[0277] Furthermore, the device 200-1 may transmit log data relating to the control of a control target device (e.g., the device 200-2) to the server system 100. In the above example, the log data may include data indicating when and to what extent the device 200-1, which is the choking choke detection device 460, changed the back angle of the device 200-2, which is the nursing bed 520. Similarly, when capability information and the like is notified to device 200-2, the timing of the notification and details of the notification content, such as the capability information, become log data. By transmitting this log data to server system 100, it becomes possible to share the content with server system 100 even when communication is performed without a server.

[0278] For example, as shown in FIG. 25, device 200-1 may use packet 1 to transmit control signals, capability information, scene information, device type information, etc., and may use packet 2 to transmit log data. Here, log data is data that represents the history of communication using packet 1, as described above. Packet 1 is a packet that is transmitted and received within the network to which device 200-1 belongs. Packet 2 is a packet that is transmitted beyond gateway 300 that forms the network to which device 200-1 belongs. In a narrow sense, packet 2 may be a packet that specifies the IP address of server system 100. In this way, by switching packets according to the content of the data to be transmitted, it is possible to reduce the communication load on server system 100.

[0279] Although the above describes an example in which data is transmitted from device 200-1 to other devices 200 without a server, other devices 200 such as device 200-2 may also be the source of data transmission.

[0280] As described above, the device 200 according to this embodiment may select whether to transmit the operation result to the server system 100 (step S203 in FIG. 7, etc.), or whether to transmit information for controlling a control target device other than the device 200 based on the operation result (packet 1 in FIG. 25) without going through the server system 100. In this way, it is possible to flexibly change whether processing is led by the server system 100 or performed serverlessly.

[0281] Specifically, when the address information of the control target device is acquired based on information from the entry / exit management apparatus 700, the device 200 may control the control target device without going through the server system 100. In this way, it becomes possible to appropriately determine whether to perform serverless processing, taking into consideration the specific communication situation.

[0282] Although the above describes an example of serverless communication using the entry / exit control device 700, the method of this embodiment is not limited to this. For example, the server system 100 may refer to a packet transmitted from a device 200 to determine through which gateway 300 the data in the packet was transmitted. When there are multiple devices 200 connected to the same gateway 300, the server system 100 may determine that the multiple devices 200 are close to each other.

[0283] The server system 100 then transmits to each device 200 the address information of other devices 200 determined to be close to the device 200. For example, the server system 100 obtains information that associates identification information and address information of multiple devices 200 connected to the same gateway 300, and periodically transmits the information to the multiple devices 200. Even with this method, each device 200 can identify the address information of other devices 200 that are nearby, making it possible to achieve serverless communication.

[0284] 6. Detailed example of communication 6.1 Overview In the information processing system 10 shown in FIG. 1, communication is performed between at least one device 200 and the gateway 300 using a wireless communication method. The wireless communication method here may be, for example, a communication method conforming to IEEE802.11. The server system 100 may perform communication using a wired communication method or a wireless communication method. The wired communication method here may be, for example, a communication method conforming to IEEE802.3. For the sake of simplicity, an example will be described below in which the server system 100, the device 200, and the gateway 300 each use a communication method conforming to IEEE802.11.

[0285] Fig. 26 is a diagram showing an example of the configuration of the communication processing unit 114 and the communication unit 130 of the server system 100. As shown in Fig. 26, the communication processing unit 114 includes an upper layer processing unit 1141, a MAC layer processing unit 1142, and a physical layer processing unit 1143. The communication unit 130 includes a transmission / reception circuit 131 and an antenna array 132. However, the configurations of the communication processing unit 114 and the communication unit 130 are not limited to those shown in Fig. 26, and various modifications are possible, such as omitting some components or adding other components.

[0286] The upper layer processing unit 1141 performs processing at a layer higher than the MAC layer. The upper layer here may be TCP / IP (Transmission Control Protocol / Internet Protocol), UDP / IP (User Datagram Protocol / Internet Protocol), or the application layer. For example, the processing for obtaining the capability information, scene information, and device type information described above may be executed by software running on the server system 100, and the application layer here may correspond to that software. The upper layer processing unit 1141 is connected to the MAC layer processing unit 1142.

[0287] The MAC layer processing unit 1142 performs transmission and reception processing in the MAC layer. The MAC layer processing unit 1142 is connected to the physical layer processing unit 1143.

[0288] The physical layer processing unit 1143 performs processing in the physical layer. The physical layer processing unit 1143 is connected to the antenna array 132 via the transmission / reception circuit 131.

[0289] The transmission / reception circuit 131 includes circuits such as a digital / analog conversion circuit (hereinafter referred to as a D / A conversion circuit) and an RF circuit, converts a digital signal from the physical layer processing unit 1143 into an analog signal, and outputs the analog signal to the antenna array 132. The transmission / reception circuit 131 also includes an analog / digital conversion circuit (hereinafter referred to as an A / D conversion circuit), converts a signal received by the antenna array 132 into a digital signal, and outputs the converted digital signal to the physical layer processing unit 1143. It should be noted that the physical layer processing unit 1143 may also have an A / D conversion circuit or a D / A conversion circuit.

[0290] The antenna array 132 includes a plurality of antennas that transmit radio waves based on analog signals output from the transmission / reception circuit, and output analog signals based on received radio waves.

[0291] 26 operates as follows when receiving data, for example. First, the antenna array 132 receives radio waves in a predetermined frequency band and outputs an analog signal corresponding to the radio waves to the transmission / reception circuit 131. The transmission / reception circuit 131 converts the analog signal received by the antenna array 132 into a baseband signal. The transmission / reception circuit 131 also performs A / D conversion of the baseband signal and outputs the converted digital signal to the physical layer processing unit 1143.

[0292] The physical layer processing unit 1143 performs processes such as demodulation and error correction of the received signal. The physical layer processing unit 1143 also removes the physical header, which is a header corresponding to the physical layer, and outputs the payload portion to the MAC layer processing unit 1142.

[0293] The MAC layer processing unit 1142 processes the received payload portion as a MAC frame. The MAC frame may be an MPDU (MAC Protocol Data Unit) in IEEE 802.11. The MAC layer processing unit 1142 passes the MAC header, which is a header in the MAC layer, and the frame body, which is the portion of the MAC frame excluding the trailer, to the upper layer processing unit 1141.

[0294] The upper layer processing unit 1141 outputs the data corresponding to the frame body received from the MAC layer processing unit 1142 to software or the like.

[0295] When transmitting data, the reverse process is performed: First, the software running on the server system 100 generates data to be transmitted, and outputs the data to the upper layer processing unit 1141.

[0296] The upper layer processing unit 1141 creates data equivalent to a frame body by adding a header in the upper layer to the data, and outputs the data to the MAC layer processing unit 1142 .

[0297] The MAC layer processing unit 1142 generates a MAC frame by adding a MAC header and a trailer to the received data, and outputs the generated MAC frame to the physical layer processing unit 1143.

[0298] The physical layer processing unit 1143 creates a physical packet by adding a physical header and the like to the received data. The physical layer processing unit 1143 outputs the created physical packet to the transmission / reception circuit 131.

[0299] The transmission / reception circuit 131 generates an analog signal by performing A / D conversion, modulation, etc. on the data received from the physical layer processing unit 1143. The transmission / reception circuit outputs the analog signal to the antenna array 132. The antenna array 132 transmits a radio wave corresponding to the received analog signal.

[0300] The MAC layer processing unit 1142 may use a data frame, a control frame, or a management frame as the MAC frame. A data frame is a frame used when transmitting and receiving data between terminals when a communication link between the terminals is established. For example, as described above, a data frame is used when software running on the server system 100 creates data including capability information and the like and transmits the data to the device 200.

[0301] Furthermore, a management frame is a frame used to manage communication links between terminals. IEEE 802.11 defines various types of information to be transmitted and received using management frames, and these can be widely applied to this embodiment. A control frame is a frame used to control the transmission and reception of management frames and data frames. Control frames include various types of frames such as an RTS (Request to Send) frame, a CTS (Clear to Send) frame, and an ACK (Acknowledgement) frame. These frames are defined within the IEEE 802.11 standard, so a detailed description of them will be omitted.

[0302] Although the above describes an example of a configuration related to communication of the server system 100, a similar configuration may be used for the device 200 and the gateway 300. As described above, the server system 100 may perform communication using a wired communication method, in which case some of the above content may be replaced with content that complies with IEEE802.3.

[0303] 6.2 Frame Structure In the method of this embodiment, the configuration of the MAC frame (data frame) may be standardized regardless of the device 200 with which communication is to be performed. As described above with reference to FIG. 7 and other figures, the device 200 in this embodiment and the application running on the device 200 may be created by a device vendor. Therefore, devices 200 created by various vendors may be registered in the information processing system 10. In this regard, standardizing the configuration of the data frame makes it possible to unify communication control regardless of the vendor. In a narrow sense, the configuration of the data frame refers to the bit allocation in the frame body.

[0304] Figure 27 shows an example of a MAC frame format. Note that the format example shown in Figure 27 can also be applied to management frames and control frames, but the following explanation will be given using a data frame as an example. As shown in Figure 27, a MAC frame includes a MAC header, a frame body, and a trailer.

[0305] The MAC header includes the following fields: Frame Control, Duration ID, Address 1, Address 2, Address 3, Sequence Control, Address 4, QoS Control, and HT Control. Some of these fields may be omitted.

[0306] Frame Control contains a type field to determine whether the target MAC frame is a data frame, management frame, or control frame. Frame Control may also contain a subtype field to specify a more specific type. Duration / ID stores information indicating the planned period of radio wave use. The planned period of radio wave use can also be rephrased as the time required to transmit the frame. Duration / ID is used for RTS / CTS, etc.

[0307] Address 1 to Address 4 store information indicating the addresses of the destination and source devices. For example, Address 1 corresponds to the destination address, and Address 2 corresponds to the source address. Address 3 and Address 4 store data according to the frame purpose.

[0308] Sequence Control corresponds to the sequence number of the data to be transmitted. QoS Control stores information used for QoS control. QoS control refers to control that takes into account the priority of frames when transmitting. HT Control is a field used in management frames, for example.

[0309] The structure of the frame body will be described later with reference to Figures 28A and 28B. The trailer is, for example, a Frame Check Sequence (FCS). The FCS is information used to detect errors in the frame, such as a checksum code. The FSC is, for example, a Cyclic Redundancy Code (CRC).

[0310] The information processing system 10 according to this embodiment includes a server system 100 and a device 200, as described above with reference to FIG. 1. The device 200 includes an application that executes processing corresponding to the tacit knowledge of an expert. As shown in FIG. 27, the server system 100 transmits a data frame including a Media Access Control (MAC) header, a frame body, and a trailer at the data link layer of communication with the device 200. Here, the server system 100 transmits a data frame in which a fixed-length field including a first field for storing ability information indicating the activity ability of the person being assisted, a second field for storing scene information specifying an assistance scene for the person being assisted, and a third field for storing device type information specifying the type of a concurrent device used with the device 200 is included in the frame body. Then, the device 200 determines whether to activate or deactivate an application based on at least one of the ability information, the scene information, and the device type information, as described above.

[0311] This makes it possible to standardize the data format when the server system 100 transmits capability information and the like to the device 200. For example, even when devices 200 from various vendors are registered in the information processing system 10 of this embodiment, the configuration of the data frame can be standardized regardless of the vendor. In particular, by making the first to third areas a fixed length, it becomes easier for the device 200 that receives the data frame to interpret the data frame, thereby reducing the processing load related to communication. For example, the first to third areas may be arranged at the beginning of the frame body, excluding the header and the like in the upper layer.

[0312] As shown in step S203 of FIG. 7, when an application is set to active, the device 200 may transmit the operation result of the application to the server system 100. The operation result here refers to the output of the application and represents the execution result of a process corresponding to tacit knowledge. For example, the operation result may include a determination result regarding an event that may occur during assistance, the person being assisted, or the caregiver. Specifically, the operation result may be a determination result regarding whether an event occurred during assistance, whether the condition of the person being assisted is normal, or whether the assistance provided by the caregiver was correct. In this way, information representing the operation result can be aggregated in the server system 100. However, processing via the server system 100 is not required, and serverless processing may be performed as described above with reference to FIG. 25.

[0313] 28A and 28B are diagrams illustrating an example of bit allocation in the frame body of a data frame transmitted by the server system 100. As shown in Fig. 28A, the frame body may include the following fields: User ADL, scene flag, device type ID, data type ID, Instruction length, and contents.

[0314] The User ADL field stores the ability information of the person being assisted and corresponds to the first area described above. For example, the ability information is numerical data indicating which level the person being assisted belongs to when the degree of ability is classified into a predetermined number of levels. For example, if there are eight or fewer levels, the User ADL field is a three-bit field. If there are nine or more levels, the User ADL field may be a four-bit or larger field. Since the number of bits required is known depending on the definition of the ability information, the User ADL field may be a fixed-length field. The User ADL field may also include an ID that identifies the person being assisted. As described above in step S403 of FIG. 19, the ability information in this embodiment is not limited to ADL index values, but may include more detailed information such as information indicating how to stand up, ability to maintain a sitting position, ability to swallow, and ability to walk. Therefore, the User ADL field may have a number of bits sufficient to express each of these abilities.

[0315] The Scene flag is a field for storing scene information and corresponds to the second field described above. For example, the scene information may include a bit indicating whether the number of caregivers is greater than or equal to a predetermined number. When the bit has a first value (e.g., 0), the number of caregivers is greater than or equal to the predetermined number, and when the bit has a second value (e.g., 1), the number is less than the predetermined number. The scene information may also include a bit specifying the type of assistance. For example, if four types of assistance are identified: meal assistance, toilet assistance, moving / transfer assistance, and other, the scene information includes two bits specifying the type of assistance. For example, 00 of the two bits indicates meal assistance, 01 of the two bits indicates toilet assistance, 10 of the two bits indicates moving / transfer assistance, and 11 of the two bits indicates other. The scene information is not limited to these, and other information may also be used. Therefore, the specific number of bits and meaning of the Scene flag can be modified in various ways. However, since the type of scene information to be used and the number of bits required to represent the scene information are known, the Scene flag may be a fixed-length field.

[0316] The device type ID is a field for storing device type information and corresponds to the third field described above. The device type information may be different for each device 200 described above with reference to FIGS. 8 to 14. Alternatively, the same device type ID may be assigned to the image capture device 410 in FIG. 8 and the bedside sensor 420 in FIG. 9, provided that the device 200 is adapted to a fall risk. Alternatively, the device type ID may be assigned based on the type of sensor that the device 200 has. Since the number of target device type IDs is known, the device type ID may be a fixed-length field.

[0317] The server system 100 may also determine the control type and control content of the control for the controlled device based on the operation result of the device 200. The server system 100 may then transmit to the controlled device a data frame including a fixed-length fourth field for storing the control type and a fifth field for storing the control content, the fifth field having a length that varies depending on the control type, in the frame body after the first, second, and third fields. The control type indicates the type of control to be executed, and the control content indicates information that specifically identifies the content of the control to be executed. This makes it possible to use a data frame that is common to both notifying capability information and controlling the controlled device. In this case, the first to third fields, which are fixed-length fields, are positioned relatively forward, and the fifth field, which is a variable-length field, is positioned relatively backward, thereby fixing the data structure of the forward portion, including the number of bits. As a result, the device 200 can easily interpret the data frame.

[0318] For example, the data type ID, instruction length, and contents shown in FIG. 28A are fields used to send a control signal to a controlled device. The data type ID is a field that stores information indicating the type of instruction to be output to the controlled device, and corresponds to the fourth area described above. The instructions here may include four types: "notification (alarm)," "movement / transport," "control," and "recommendation, etc." In this case, the data type ID is a fixed-length field of 2 bits that can identify the four instructions.

[0319] "Notification" refers to information used when the control target device notifies the processing result of the device 200. For example, "notification" may be an instruction used when a fall risk is detected by a device that assesses the risk of falling and the control target device notifies the risk. "Move / Transport" refers to an instruction to move a movable control target device, such as the reclining wheelchair 510 or a walker. For example, a move / transport instruction may be used to control the reclining wheelchair or a walker to move closer so that the person being assisted can grab hold of it when a fall risk is detected. "Control" broadly includes control other than "move / transport" that operates the control target device, such as changing the angle of the back of the reclining wheelchair 510 or the bottom angle of the nursing bed 520. "Recommendations, etc." include, for example, recommendations for purchasing products used to improve the quality of care. For example, a recommendation may represent an instruction to output a recommendation to the control target device, such as the caregiver's terminal device, when the bed position detection device 470 determines that a cushion should be used. Furthermore, "recommendations, etc." may include news distribution, etc. For example, "recommendations, etc." may be used to introduce popular devices 200.

[0320] "contents" is a field that stores information specifying the specific content of the instruction, and corresponds to the fifth area. FIG. 28B is a diagram explaining specific examples of "contents" according to the value of data type ID. For example, data type ID = 0 represents the above-mentioned "alarm," and in this case, "contents" includes an "alarm contents" field. "alarm contents" is a variable-length field that stores information indicating the specific alarm content. "alarm contents" may include information specifying the alarm mode (display, light emission, vibration, sound output), or may include information specifying the specific alarm content (text, light emission color, vibration pattern, sound).

[0321] Data type ID = 1 represents the above-mentioned "movement / transportation," and in this case, the contents include the fields of current location and destination. The current location is information that specifies the current location of the target device 200. The destination is information that specifies the target location of the target device 200. How the current location and target location are expressed is arbitrary, but they are, for example, fixed-length fields that represent two-dimensional or three-dimensional coordinate values.

[0322] Data type ID = 2 represents the above-mentioned "control," and in this case, the contents includes the fields "controlled part" and "how to control." The controlled part stores information identifying the controlled part of the target device 200. For example, the nursing bed 520 shown in FIG. 24 includes multiple bottoms, each of which is movable. The controlled part may store information identifying one of the multiple bottoms. The "how to control" stores information identifying a specific control method for the controlled part. In the example of the nursing bed 520, the "how to control" may be information indicating in which direction and how many times the bottom identified by the controlled part is to be driven. The controlled part and the "how to control" may each be fixed-length fields. Note that "control" here may include instructions for multiple controlled parts. Therefore, the contents when data type ID = 2 may be configured so that the set of the controlled part and the "how to control" is repeated the same number of times as the number of controlled parts.

[0323] Data type ID = 3 represents the "recommendations, etc." mentioned above, and in this case, the contents includes the recommend contents field. The recommend contents is a variable-length field that represents the contents of the recommendation or news. The recommend contents here may include information that identifies the recommended product, or may include link information to the product sales site, the manufacturer's product introduction web page, etc. The recommend contents may also include text that represents the contents of the news, or link information to the web page where the news is displayed, etc.

[0324] As mentioned above, the length of the contents field in the frame body varies depending on the data type ID and the specific instruction content. Therefore, as shown in Figure 28A, the frame body may include an instruction length field before the contents field, which stores the length of the contents field. Since the maximum length of the contents field is considered to be known, the instruction length may be, for example, a fixed-length field.

[0325] 25, in this embodiment, data including control signals, capability information, and the like may be transmitted from a device 200 to another device 200 without going through the server system 100. However, if the formats of the data transmitted from the server system 100 (such as step S304 in FIG. 7) and the data transmitted from the device 200 (packet 1 in FIG. 25) are significantly different, the receiving device 200 may need to interpret the data according to the transmission source. Therefore, even when the device 200 is the transmission source, a frame configuration similar to the frame configurations shown in FIGS. 28A and 28B may be used.

[0326] For example, when the device 200 controls a controlled device without going through the server system 100, the device 200 may transmit to the controlled device a data frame whose frame body includes a fixed-length field including the second and third fields at the data link layer of communication with the controlled device. As described above, the second field corresponds to scene information, and the third field corresponds to device type information. The data frame may also include a first field corresponding to capability information. This makes it possible to standardize at least the portions of the frame body of the data frame that correspond to capability information, scene information, and device type information. For example, the first to third fields may be located at the beginning of the frame body, excluding headers and the like in higher layers.

[0327] Furthermore, the device 200 may determine the control type and control content of the control for the control target device based on the operation result of the application on the device 200. Then, a data frame including a fourth field and a fifth field in the portion of the frame body after the field including the second and third fields may be transmitted to the control target device without going through the server system 100. As described above, the fourth field corresponds to the control type, and the fifth field corresponds to the control content. In this way, it is possible to standardize the configuration of the data frame received by the device 200 even when control is performed without a server.

[0328] 29A and 29B are diagrams illustrating an example of the configuration of a frame body when a device 200 transmits data including a control signal, capability information, and the like to another device 200. As can be seen by comparing FIG. 29A with FIG. 28A, the configuration of the frame body is the same as the configuration of a frame body when transmitted by the server system 100, except that the order of the device type ID, scene flag, and User ADL is different. The device type ID field here may store, for example, device type information of the transmission source device 200. FIG. 29B is a diagram illustrating a specific example of contents according to the value of data type ID. Since the device 200 is the transmission source, the contents are also the same as those in FIG. 28B, except that log data is transmitted instead of recommendations, etc., when data type ID = 3. The log data is as described above and includes information such as the history of serverless control of the control target device and notification history of capability information, etc.

[0329] In this way, it is possible to make the data frame configuration similar whether the server system 100 is the sender or the device 200 is the sender as shown in Fig. 25. As a result, the device 200 that receives the data frame can determine the operation mode and operate in accordance with the control signal based on the same processing, regardless of the sender. Therefore, even when both processing initiated by the server system 100 and serverless processing are performed, an increase in the processing load can be suppressed.

[0330] 28A and 29A, particularly the order of the User ADL, device type ID, and scene flag can be modified in various ways. Therefore, the configuration of the data frame transmitted by the server system 100 and the configuration of the data frame transmitted by the device 200 can be the same. Furthermore, some of the User ADL, device type ID, and scene flag may be omitted from at least one of the configuration of the data frame transmitted by the server system 100 and the data frame transmitted by the device 200. For example, the above describes an example in which the device 200 outputs sensing data and the capability information acquisition unit 111 of the server system 100 calculates capability information based on the sensing data. In this case, the device 200 does not calculate capability information, and therefore the User ADL may be omitted from the data frame transmitted by the device 200. Alternatively, a User ADL field may be provided in the data frame transmitted by the device 200, and the values ​​of this field may be set to a fixed value (for example, all 0s). The device 200 may also calculate capability information, and in this case, information such as an index value of the ADL is stored in the User ADL of the data frame transmitted by the device 200.

[0331] 29A and 29B may be used for transmitting data from the device 200 to the server system 100. For example, when the device 200 determines that an alarm is required as a processing result, the device 200 transmits to the server system 100 a data frame in which the value of the data type ID is set to 1 and information indicating the alarm content is stored in the alarm contents of the contents. In addition, the device 200 can appropriately transmit the processing result to the server system 100 using the data frames shown in FIGS. 29A and 29B by changing the value of the data type ID or the value of the contents according to the processing result. Furthermore, the log data stored in the contents when the data type ID is 3 is not limited to a log of serverless communication and may include a log of sensing data. That is, the data frames shown in FIGS. 29A and 29B may be used when transmitting a log of sensing data to the server system 100.

[0332] The technique of this embodiment may also be applied to an information processing device. The information processing device corresponds, for example, to the server system 100. The information processing device includes, in a data link layer for communication with a device 200 including an application that executes processing corresponding to the tacit knowledge of an expert, a communication unit (corresponding to the communication unit 130 in FIG. 3 ) that transmits a data frame including a Media Access Control (MAC) header, a frame body, and a trailer, and a communication processing unit (corresponding to the communication processing unit 114 in FIG. 3 ) that controls the communication unit. The communication unit transmits to the device 200 a data frame including a fixed-length area in the frame body, the fixed-length area including a first area that stores ability information indicating the activity ability of the person being assisted, a second area that stores scene information that identifies the scene in which the person being assisted is being assisted, and a third area that stores device type information that identifies the type of concurrent device used with the device, as information for determining whether an application is active or inactive. This allows the server system 100 to standardize the data format when transmitting ability information and the like to the device 200, regardless of the vendor of the device 200.

[0333] The technique of this embodiment can also be applied to an information processing method in an information processing system 10 including a device 200 including an application that executes processing corresponding to the tacit knowledge of an expert, and a server system 100 that transmits a data frame including a Media Access Control (MAC) header, a frame body, and a trailer in a data link layer for communication with the device 200. The information processing method includes the steps of: transmitting, from the server system 100 to the device 200, a data frame having a frame body containing fixed-length areas including a first area for storing ability information that indicates the activity ability of the person being assisted, a second area for storing scene information that identifies a scene in which the person being assisted is being assisted, and a third area for storing device type information that identifies a type of concurrent device used with the device 200; and determining whether to activate or deactivate the application based on at least one of the ability information, the scene information, and the device type information.

[0334] 6.3 Priority In addition, methods for setting communication priorities are known in various communication systems. For example, IEEE802.11 has standardized a method for realizing QoS in IEEE802.11e. Specifically, IEEE802.11e uses a method called EDCA (Enhanced Distributed Channel Access), which prioritizes the transmission of high-priority frames.

[0335] For example, an EDCA parameter set element used in EDCA may be included in the frame body of a management frame among MAC frames. Fig. 30 is a diagram showing an example of a default EDCA parameter set element. In EDCA, packets are classified into four access categories (hereinafter referred to as AC). The default access categories are AC_VO, AC_VI, AC_BE, and AC_BK. AC_VO corresponds to voice, AC_VI corresponds to video, AC_BE corresponds to best effort, and AC_BK corresponds to background. AC_VO has the highest priority, followed by AC_VI, AC_BE, and AC_BK in decreasing order of priority.

[0336] Each AC is also associated with parameters such as CWmin, CWmax, and AIFSN. CWmin and CWmax are parameters that determine the transmission waiting time, and represent the minimum and maximum values ​​of the contention window (hereinafter referred to as CW). The transmission waiting time is set to a value between CWmin and CWmax. Since the shorter the transmission waiting time, the easier it is to transmit, so the higher the priority, the smaller the CWmin and CWmax values ​​are set.

[0337] For example, if CWmin is set to aCWmin and CWmax is set to aCWmax for AC_BE and AC_BK, which have low priority, then for AC_VI, which has high priority, CWmin is set to (aCWmin+1) / 2 - 1 and CWmax is set to aCWmin, resulting in smaller CWmin and CWmax values ​​than for the above two ACs. Also, for AC_VO, which has the highest priority, CWmin is set to (aCWmin+1) / 4 - 1, which is even shorter than for AC_VI, and CWmax is set to (aCWmin+1) / 2 - 1, which is even shorter than for AC_VI.

[0338] The AIFSN (Arbitration Inter Frame Space Number) is a parameter that indicates the frame transmission interval, and the smaller this value, the higher the queue priority. In the example of Figure 30, the AIFSN of AC_VO and AC_VI is 2, the AIFSN of AC_BE is 3, and the AIFSN of AC_BK is 7.

[0339] Furthermore, the parameters associated with an AC are not limited to those described above. For example, although not shown in Fig. 30, a TXOP (Transmission Opportunity) limit may be associated with each AC. TXOP represents the channel occupancy time. For example, by setting a non-zero value as the TXOP limit for high-priority AC_VI or AC_VO, the channel can be occupied by packets sent from these ACs.

[0340] The information processing system 10 of this embodiment is assumed to include a large number of devices 200. Each device 200 performs an operation according to an operation mode and transmits the processing result to the server system 100 via the gateway 300. The processing result here can be various contents such as "notification," "movement / transportation," "control," and "log" described above with reference to FIG. 29B.

[0341] The devices 200 of this embodiment may include products from various vendors, and the priority settings may vary depending on the vendor. For example, there is a possibility that only communications with devices 200 from a specific vendor may be prioritized over communications with devices 200 from other vendors. Therefore, in this embodiment, priority settings may be performed to improve communication efficiency. For example, the method of this embodiment updates the allocation of ACs.

[0342] The server system 100 of this embodiment may determine a priority based on device type information and transmit priority information specifying the priority to the device. The priority information is, for example, information associating device type information with a priority, and more specifically, information in which an AC is assigned to each device type. The device 200 then transmits a data frame according to the priority specified based on the device type and the priority information. In this way, priority can be set according to the device type, thereby improving the efficiency of communication in the information processing system 10 including various devices 200.

[0343] Furthermore, the server system 100 may determine a priority based on the device type information and the control type, and transmit priority information specifying the priority to the device 200. The priority information is, for example, information in which an AC is assigned to each pair of device type and control type. The device 200 then transmits the data frame according to the device type, the control type included in the data frame to be transmitted, and the priority specified based on the priority information. In this way, the control type can be reflected in the priority in addition to the device type, thereby making it possible to improve the efficiency of communication in the information processing system 10 even when various devices 200 transmit various information. Below, a specific example will be described using both the device type information and the control type. However, in the following description, either the device type information or the control type may be omitted.

[0344] Fig. 31 is an example of a table showing the allocation of ACs. In the method of this embodiment, ACs may be determined based on the device type and the data type. The device type here corresponds to the device type information described above, and is, for example, a device type ID. The data type here corresponds to the data type ID described above with reference to Figs. 28A to 29B, for example. In other words, the data type may be information indicating which of "notification," "movement / transport," "control," or "log" will be transmitted based on the processing result of the device 200.

[0345] For example, the server system 100 performs a process of determining the device type, data type, and AC relationship for each gateway 300 that communicates with the server system 100, and transmitting the determined information to each gateway 300. For example, based on information from the gateway 300, the server system 100 can identify the number and type of devices 200 connected to the gateway 300, the type of data that each device 200 can transmit, and the like.

[0346] For example, the data type "log" refers to the case where a log of sensing data or a log of serverless communication is transmitted, as described above, and therefore has a low communication priority, and transmission at night when communication volume is low is unlikely to cause any problems. On the other hand, "alert," "movement / transport," and "control" are used to notify caregivers of the occurrence of a risk or to mitigate the impact of the risk. Therefore, it is desirable that the time between the detection of a risk in the device 200 and data transmission is short. Therefore, the server system 100 sets a higher priority for the data type "alert," "movement / transport," or "control" than for the data type "log." The server system 100 may also assign different priorities to "alert," "movement / transport," and "control."

[0347] Furthermore, since the output from the choking stool detection device 460 corresponds to the occurrence of aspiration risk, the severity may increase if a prompt response is not made. On the other hand, the risk of bedsore detected by the bed position detection device 470 is of high importance when assistance is assumed over a relatively long period of time, so even if a notification of a risk of bedsore is issued late, the severity is relatively small. Therefore, even if the data type is the same "notification," the server system 100 may prioritize a device type corresponding to the choking stool detection device 460 over a device type corresponding to the bed position detection device 470. In addition, the server system 100 can set priorities according to the device type for the various devices described above using Figures 8 to 14, etc.

[0348] For example, the server system 100 determines priorities according to device type and data type and assigns ACs according to the priorities. It is known that if the proportion of packets assigned to high-priority ACs becomes excessively high, collisions are more likely to occur, thereby reducing the effectiveness of using EDCA. Therefore, the server system 100 may change the AC assignment according to device type and data type for each gateway 300. For example, even for the same device type and data type, a case may occur in which a relatively low-priority AC_VI is assigned in the first gateway and a relatively high-priority AC_VO is assigned in the second gateway. For example, if a network formed by the first gateway has many devices 200 that detect a high risk of seriousness, such as choking detection devices 460, the priority of packets assigned to AC_VO in a normal network may be lowered to AC_VI.

[0349] The gateway 300 and the device 200 connected to the gateway 300 communicate based on the AC allocation transmitted from the server system 100. Note that the communication here is not limited to data transmission from the device 200 to the gateway 300, but also includes data transmission from the gateway 300 to the device 200.

[0350] In addition to determining the allocation of ACs based on the device type and data type, the server system 100 may also adjust parameters corresponding to each AC. The parameters include at least one of the above-mentioned CWmin, CWmax, AIFSN, and TXOP limit. This allows for more detailed adjustments in the priority settings.

[0351] The priority setting process in this embodiment is not limited to the one using EDCA described above. For example, RTS / CTS may be used as a method for avoiding collisions in wireless communication. In this system, a STA that wants to send data sends an RTS frame to the AP, and the AP then sends a CTS frame to any one of the STAs that sent the RTS frame and is authorized to send data. Since only the STA that received the CTS frame can send data, data collisions can be reduced.

[0352] In the method of the present embodiment, the server system 100 may transmit information indicating the priority order for transmitting CTS frames to the gateway 300 as priority information. For example, the server system 100 transmits to the gateway 300 information for determining the transmission order of CTS frames, such as in descending order of the ADL of the target person being assisted, in descending order of the urgency of data based on the data type, or a combination of these. When performing collision avoidance based on RTS / CTS, the gateway 300 determines the priority order based on a list of devices 200 that have transmitted RTS frames and information from the server system 100, and transmits CTS frames in descending order of priority. This method also enables appropriate priority setting, making it possible to prevent, for example, only a specific vendor from being given a higher priority.

[0353] Furthermore, when the gateway 300 receives an RTS frame, it may perform processing to notify the device 200 that it is in a busy state and the estimated time for completion of the current communication. If the device 200 then transmits data indicating that it cannot tolerate waiting for transmission, it may accept an interrupt using a suspend command and allow the target device 200 to transmit data. When the data transmission is completed, the original data transmission is resumed using a resume command. In this way, it becomes possible to prioritize communication of particularly urgent data by suspending other data transmissions. However, because this processing interrupts data transmission that has already started, it is not desirable to perform this processing frequently or for the interruption time to be excessively long. Therefore, the data for which an interrupt using the suspend command is permitted may be limited to certain data. For example, an interrupt may be permitted when the conditions that the amount of data to be transmitted is small and the time required for transmission is short are met.

[0354] 7. Application to home care The method of this embodiment may also be applied to home care provided at the home of a person receiving care.

[0355] 7.1 Examples of systems used in the home Fig. 32 is a diagram showing an example of the configuration of an information processing system 10 at home. The information processing system 10 in Fig. 32 includes a server system 100, a third terminal device 810, a fourth terminal device 820, and motion sensors 831 to 833. For example, of the information processing system 10, the third terminal device 810, the fourth terminal device 820, and the motion sensors 831 to 833 are arranged in a home where home care is provided. In addition, a bed 610, a wheelchair 630 (not shown), etc. may be used for home care.

[0356] The third terminal device 810 is a mobile terminal device such as a smartphone or tablet terminal device used by a family member assisting the person being assisted. The fourth terminal device 820 is a mobile terminal device such as a smartphone or tablet terminal device used by the person being assisted.

[0357] The motion sensors 831-833 are sensors placed at predetermined locations in the home, and are sensors that detect human movement using, for example, infrared rays. The motion sensors 831-833 may be sound sensors that react to sound or ultrasonic sensors that detect objects using ultrasonic waves, and various modifications are possible in specific embodiments. The motion sensors 831-833 are connected to the third terminal device 810 using, for example, BLE.

[0358] The bed 610 is a bed used by a person receiving care, and may be a nursing bed 520 whose bottom angle, height, etc. are adjustable.

[0359] For example, an assistance recording device, which is an example of the device 200 according to this embodiment, may include the third terminal device 810 and the human sensors 831 to 833 in Fig. 32. The assistance recording device identifies the location where assistance was performed based on the detection results of the human sensors 831 to 833, and creates an assistance record based on the identified location.

[0360] For example, the motion sensors 831 to 833 may each be placed in a location where a specific assistance is to be provided. For example, the motion sensor 831 is placed near the bed 610 used by the person being assisted. The motion sensor 832 is placed near the toilet. The motion sensor 833 is placed near the dining table where meals are eaten. The motion sensors may also be placed in other locations in the home, such as the bathroom where bathing assistance is provided.

[0361] For example, if human movement is detected by the motion sensor 831, it is assumed that the caregiver is assisting the person being assisted, such as adjusting the bed position or changing a diaper. If human movement is detected by the motion sensor 832, it is assumed that the caregiver is assisting the person being assisted with excretion in the toilet. If human movement is detected by the motion sensor 833, it is assumed that the caregiver is assisting the person being assisted with eating. Therefore, it is possible to calculate the assistance time by determining the time at which each motion is detected. For example, by calculating the total time at which motion is detected by the motion sensor 831 in a day, the assistance time in bed 610 for that day can be calculated. Similarly, the time spent assisting with excretion in the toilet and the time spent assisting with eating can be calculated by the motion sensors 832 and 833. More detailed information, such as the time from and to which time and where assistance was provided, may also be calculated.

[0362] Even if the motion sensors 831-833 detect human movement, if the movement is that of a single caregiver or a single person being assisted, assistance may not be provided. In this embodiment, for the sake of simplified determination, a determination may be made simply based on the presence or absence of motion detection without distinguishing whether the detected movement is that of a single person or multiple people. Alternatively, the presence or absence of assistance may be determined in more detail using a different method. For example, instead of the motion sensors 831-833, RFID readers may be installed at each location. The caregiver and the person being assisted may carry IC tags, or IC tags may be embedded in the third terminal device 810 and the fourth terminal device 820, thereby making it possible to determine whether the caregiver and the person being assisted are near each RFID reader. The assistance recording device may determine that assistance is being provided if it is determined that both the caregiver and the person being assisted are present in a predetermined location. Furthermore, cameras may be installed at each location in the home instead of the motion sensors 831-833. For example, a facial recognition process may be performed based on images captured by each camera to determine whether assistance is being provided at each location.

[0363] Furthermore, when meals are eaten in bed, meal assistance is provided, but it may not be easy to distinguish between meal assistance and other assistance such as adjusting the bed position based solely on the presence or absence of human movement. In this case, the assistance recording device may be linked to a food intake measurement app, which will be described later. For example, since the food intake measurement app takes photos, assistance during the time including the time the photo was taken is determined to be meal assistance, and assistance during other times is determined to be bed assistance other than meal assistance.

[0364] By using the care recording device, the time spent by the caregiver on care can be easily measured, so that if the caregiver's burden becomes excessive, it is possible to notify the caregiver or the care manager.

[0365] Other applications may also be installed in the third terminal device 810. For example, there is known an application that uses an acceleration sensor or a microphone mounted on a smartphone to determine the sleep state of a user. For example, the third terminal device 810 is placed at the pillow of the bed where the user sleeps, and determines the sleep state based on vibrations during sleep. The third terminal device 810 may also use a microphone to determine snoring, etc. In this way, it becomes possible to easily determine sleep-related conditions without using dedicated equipment such as the detection device 430.

[0366] An MCI assessment application for performing the above-described MCI assessment may be installed in the third terminal device 810. In other words, the third terminal device 810 may function as an MCI assessment device. For example, the caregiver carries the third terminal device 810, which is his or her own smartphone, close to the person being assisted and performs MAC assessment by having the person being assisted respond.

[0367] A food intake measurement app may also be installed on the third terminal device 810. The food intake measurement app is a method for estimating food intake and calorie intake by, for example, comparing a photo of food before a meal with a photo of the food after a meal. Known methods such as those disclosed in JP 2021-086313 A and the like can be widely applied as methods for estimating food intake based on photos.

[0368] 33 to 35 are examples of screens displayed on the display unit of the third terminal device 810. These screens may be user screens that are displayed when, for example, an operation to log in to the information processing system 10 of this embodiment using the account of a caregiver is performed on the third terminal device 810. The user screen may display the results of the above-mentioned processes executed on the third terminal device 810. For example, the third terminal device 810 executes the above-mentioned processes and transmits the execution results to the server system 100. The server system 100 generates a display screen based on the execution results and executes control to display the display screen on the display unit of the third terminal device 810.

[0369] Fig. 33 is an example of a home screen that is displayed, for example, when logging in. As shown in Fig. 33, the home screen may display information indicating the status of the caregiver, notifications, a contact button, and a photo button.

[0370] The information indicating the caregiver's condition may include, for example, an icon indicating the caregiver's sleep state and an icon indicating the caregiver's care burden. In the example of Fig. 33, the sleep state and the degree of care burden are presented in an easy-to-understand manner using facial expressions. However, each piece of information may also be expressed using numerical values, and various modifications of the specific display form are possible.

[0371] For example, the third terminal device 810 may determine the assistance time, which is the time for which assistance was provided, based on processing using the above-mentioned human sensors 831 to 833, and compare the assistance time with a threshold value to determine which of a plurality of stages the care burden belongs to. The third terminal device 810 may also evaluate the care burden based on a chronological change in the assistance time.

[0372] The third terminal device 810 may also determine the sleep state using the acceleration sensor or microphone described above. For example, the third terminal device 810 may determine the sleep state by calculating the sleep time per day. The third terminal device 810 may also determine to which of a plurality of stages the sleep state belongs by comparing the sleep time with a threshold. The third terminal device 810 may also evaluate the sleep state based on a time-series change in the sleep time.

[0373] Additionally, for example, recommendations for caregiving tools may be displayed in the notification section of the screen in Fig. 33. In the example of Fig. 33, text is displayed suggesting the use of a positioning pillow used to adjust the bed position.

[0374] The contact button is, for example, a button for making a call to a care manager. For example, when the contact button is selected, the third terminal device 810 launches a telephone application and executes a process for making a call to the pre-registered phone number of the care manager. By making it easy to contact the care manager in this way, it is possible to encourage appropriate measures when the caregiver's burden increases. For example, as shown in the upper part of Figure 33, if the caregiver's sleep state or care burden worsens, a warning message such as "Please call your care manager or a relative" may be displayed. This can encourage the caregiver to seek support from others before they become seriously ill due to caregiver fatigue.

[0375] The photograph button is a button that triggers the launch of the food amount measurement app described above. For example, when the photograph button is selected, the third terminal device 810 performs a process to launch the food amount measurement app. In the food amount measurement app, a camera app is launched, and photos of food before and after eating are taken.

[0376] FIG. 34 is an example of a history screen that is displayed when the history button displayed in association with the information indicating the caregiver's condition in FIG. 33 is selected. The history screen displays the time series changes in the caregiver's sleep state and the time series changes in the caregiver's care burden. For example, the processing unit 110 of the server system 100 calculates a seven-day moving average of the daily sleeping hours based on information from the third terminal device 810, and performs processing to display the time series changes in the moving average on the history screen. Note that the calculation unit for the moving average is not limited to seven days, and other periods may be used. Furthermore, the time series changes in the daily sleeping hours themselves may be displayed.

[0377] The time series change in care burden is, for example, information showing the change in the average amount of care time per day. As with the example of sleep state, the average amount of care time may be a moving average over seven days, or may be information calculated using another period. The time series change in the amount of care time per day itself may also be displayed.

[0378] In the example of FIG. 34 , the sleep time decreases over time, and the assistance time increases over time. Displaying the screen shown in FIG. 34 makes it easy to understand that the caregiver's condition is deteriorating. Also, as shown in FIG. 34 , the history screen may include a share button for sharing information about the caregiver's condition. When the share button is selected, information including the time series changes in the average sleep time and the time series changes in the average assistance time is sent to a pre-registered contact. The contact may be a care manager or the caregiver's family, etc. Furthermore, the sharing destination of the information about the caregiver's condition is not limited to the pre-registered contact. For example, when the share button is selected, link information to information about the caregiver's condition may be displayed. The link information may be, for example, address information of a web page that displays information about the caregiver's condition. The address information may be a uniform resource locator (URL) or a two-dimensional barcode representing the URL. For example, when a caregiver visits a hospital for a medical examination due to poor health, sharing information about the caregiver's condition with a doctor via the link information can provide the doctor with information on the cause of the poor health.

[0379] FIG. 35 is an example of a list screen that is displayed when the list button displayed in association with "Notification" in FIG. 33 is selected. The list screen displays information such as recommendations sent to the target caregiver in chronological order. In the example of FIG. 35, a recommendation for a positioning pillow and a recommendation for a reclining wheelchair are displayed. Each recommendation includes the reason for the recommendation, a product image, and detailed product information. The detailed product information may include the product name, manufacturer, price, rating, product description, etc.

[0380] In addition, recommendations may present vendor apps and the device 200 according to this embodiment that can be used along with the product. In the example of FIG. 35, the introduction of a positioning app is proposed along with a positioning pillow. The positioning app is an application that performs the same processing as that of the first terminal device 471 of the bed position detection device 470 described above with reference to FIG. 13, for example. By introducing the positioning app, it becomes possible to assist the caregiver in adjusting the bed position. Similarly, a diaper change support app may be installed in the third terminal device 810. For example, the third terminal device 810 may perform the same processing as that of the second terminal device 472 of the bed position detection device 470 described above with reference to FIG. 13, for example.

[0381] 35, the introduction of a seat sensor 440 is proposed along with a reclining wheelchair 510. By introducing the seat sensor 440, the results of the forward and lateral slippage judgment and the judgment result of the possibility of falling can be presented to the caregiver, making it possible to assist with position adjustment in the reclining wheelchair 510.

[0382] The processing executed by the third terminal device 810 is not limited to the above. For example, the timing of meals can be estimated based on a food intake measurement app, and the timing of excretion assistance can be estimated based on processing by an assistance recording device. Therefore, the third terminal device 810 may perform processing to predict fecal leakage based on the rhythm of meals and excretion. For example, the third terminal device 810 may use the type and amount of laxative, type of diaper, amount of food eaten, amount of water consumed, and timing of food and water intake as input data, and predict fecal leakage based on a trained model that determines whether fecal leakage will occur after a predetermined time. Furthermore, the third terminal device 810 may recommend a type of diaper, etc.

[0383] Various processes are also executed in the fourth terminal device 820 used by the caregiver. For example, the fourth terminal device 820 may have a GPS sensor, and a monitoring application using the GPS sensor may be installed. Conventionally, methods for tracking the locations of elderly people and dementia patients for the purpose of preventing wandering, etc., are known, and these methods can be widely applied in this embodiment.

[0384] The fourth terminal device 820 may have installed thereon an application for determining the sleeping state, similar to the third terminal device 810. The fourth terminal device 820 determines the sleeping state of the person being assisted. The fourth terminal device 820 may have installed thereon an MCI determination application for performing the above-described MCI determination, similar to the third terminal device 810.

[0385] 7.2 Cooperation with nursing care facilities, etc. Although the information processing system 10 used in home care has been exemplified above, the system may also be linked to systems outside the home. For example, information acquired during home care may be sent to a terminal device such as a PC used by a care manager. For example, the processing results of the third terminal device 810 and the fourth terminal device 820 are sent to the server system 100, and the server system 100 sends information based on the acquired processing results to the care manager's terminal device.

[0386] 36A to 36C are example screens displayed on the terminal device of a care manager. FIG. 36A is an example of a home screen, with two tabs at the top: In-facility and Out-of-facility. The In-facility tab is a tab for displaying information about those receiving care who are residents of a nursing home or the like, among those who are under the care of the care manager. For example, when the In-facility tab is selected, a process may be performed to launch separate nursing care software installed by the facility. The Out-of-facility tab is a tab for displaying information about those receiving care who are receiving care at home and the caregivers who are assisting those people. FIG. 36A shows an example of a screen with the Out-of-facility tab selected.

[0387] The screen shown in FIG. 36A includes an area for displaying a list of users and an area for displaying detailed information about a user selected in that area. The users here represent family members of a person receiving care who are providing home care. In FIG. 36A, four users, User A to User D, are displayed as a list, with User A being selected. The multiple users displayed here may be sorted in descending order of importance to the care manager. For example, the users may be sorted in descending order of the degree of deterioration in the sleep state or the care burden of the caregiver. In this way, even when a care manager is responsible for a large number of users, it is possible to clearly indicate which users the care manager should pay attention to. The sorting order is not limited to this; users who have recently been contacted may be ranked higher, or users who have installed a new device 200 may be ranked higher. Various modifications to the specific processing are possible.

[0388] The area displaying detailed information displays the sleep state of the caregiver, the care burden, and the sleep state of the person being assisted. This information may include, for example, a graph showing the time series change in average values, icons indicating stages, etc., as in Figure 34.

[0389] In addition, buttons for contacting the user may be displayed in association with the sleep state and care burden of user A. In FIG. 36A, a phone button for calling the user and an email button for sending an email to the user are displayed. When the phone button is selected, a call app is launched and a call is made to the user's pre-registered phone number. When the email button is selected, an email app is launched and an email composition screen with the user's email address already filled in is displayed. In addition, the method by which the care manager contacts the user is not limited to phone or email, and a chat app or the like may also be used. For example, the screen in FIG. 36A may display a chat history between the care manager and the user.

[0390] 36A, the area displaying the detailed information may display a data output button. When the data output button is selected, data related to the user's sleep state, care burden, etc. is output. The data may be a CSV file or data in another format.

[0391] A conversion button may also be displayed in association with the sleep state of the person being assisted. FIG. 36B is an example of information displayed when the conversion button is selected. For example, when the conversion button is selected, the information shown in FIG. 36B may be displayed instead of the graph of changes in average sleep time shown in FIG. 36A. FIG. 36B is a screen that displays the changes in the wakefulness, sleep state, and bed exit state of the person being assisted per day for a specified number of days. By displaying the screen shown in FIG. 36B, not only the average sleep time but also more detailed information about sleep can be presented. For example, it is possible to present information about the quality of sleep, such as whether or not the patient has woken up during the night and the number of times, to the care manager.

[0392] 36B may be the same as the screen used to display sensing data from detection device 430. In this way, even if detection device 430 is not installed, it is possible to display the same data as when detection device 430 is installed.

[0393] Furthermore, as shown in Fig. 36A, the area displaying the detailed information may also include information regarding the ability information of the person being assisted. In the example of Fig. 36A, the estimated results of changes in ADL are displayed based on changes in the amount of food eaten, the time spent assisting with toileting, the time spent assisting with bathing, and the time spent in bed. For example, as shown in Fig. 36A, a display may be made indicating that the ability to maintain a sitting position is declining based on a decrease in the amount of food eaten or an increase in the amount of time spent providing assistance in each location. In this way, even when providing care at home, it is possible to estimate changes in the ability information and notify the care manager or caregiver of these changes.

[0394] In nursing care facilities and the like, it is relatively easy to introduce the various devices 200 described above using Figures 8 to 14, etc., and various data can be acquired as sensing data. For example, data for determining the details of the movement to start moving, the ability to maintain a sitting position, the ability to swallow, the amount of activity in bed, etc. can be acquired as sensing data, which makes it possible to improve the accuracy of estimating ability information. On the other hand, in home care, it is not easy to introduce devices 200 equivalent to those in nursing care facilities. Therefore, the information that can be acquired is limited to information such as the amount of food eaten and the amount of assistance time, as described above.

[0395] Therefore, in this embodiment, the accuracy of estimating ability information in home care may be improved by associating information acquired in a care facility with information acquired in home care. For example, consider a care recipient who normally receives home care but regularly attends day care. The ability information estimated using device 200 at the care facility during day care is highly reliable as the ability information at that time. Ability information is unlikely to change rapidly in a short period of time unless there is a cause such as the onset of illness or an accident. Therefore, ability information during a predetermined period before and after day care is considered equivalent to the ability information at the time of day care. For example, machine learning may be performed using the amount of food eaten and the amount of care time acquired during home care during a predetermined period before and after day care as input data, and the ability information estimated during day care as accurate data for the input data. This allows for accurate estimation of ability information based on information obtainable during home care.

[0396] Furthermore, the ability information may be estimated based on the overlapping content between home care and day care. 8 to 14 are installed in a nursing care facility, and the capability information acquiring unit 111 of the server system 100 obtains capability information based on sensing data from each device 200. For example, the server system 100 may store table data in which a group of sensing data acquired from each device 200 is associated with capability information as one record.

[0397] In addition, while it may not be easy to introduce all of the devices 200 shown in Figures 8 to 14 into home care, some of the devices 200 may be introduced. For example, a detection device 430 may be introduced into the home to obtain sensing data corresponding to the sleep and activity level of the care recipient. In this case, the server system 100 may compare the sensing data acquired during home care with the sensing data group included in the table data to extract records with high similarity from the table data. For example, since the sensing data group includes sensing data from the detection device 430, the similarity is calculated based on a comparison between the sensing data and the sensing data acquired during home care. The server system 100 then outputs the ability information included in the records with high similarity as the ability information of the care recipient receiving home care. This also makes it possible to accurately estimate ability information based on information obtainable during home care.

[0398] Returning to Fig. 36A, the home screen shown in Fig. 36A may display information about the user's reaction to a "notice" presented to the user. The notice here is, for example, the information shown in Fig. 35.

[0399] For example, the screen shown in Fig. 36A displays whether the user has purchased the device 200 recommended to the user. In the example of Fig. 36A, the user has an intention to purchase a positioning pillow and the corresponding recommended positioning app. For example, by selecting the manufacturer order button shown in Fig. 36A, a care manager may order the positioning pillow on behalf of the user.

[0400] As described above, various applications (vendor apps) can be installed on one device 200. However, since the user is not an expert in caregiving, it may not be easy for the user to determine which application is suitable for providing care to a care recipient. Therefore, as shown in FIG. 36A , a care manager with knowledge of caregiving may determine the application to install. Determining the application corresponds to determining the tacit knowledge to be used, as described above. In the example of FIG. 36A , tacit knowledge suitable for reducing the risk of bedsores for a care recipient with paralysis in the right arm is selected from several candidates. Furthermore, tacit knowledge may be created for each nursing care facility, and the care manager may be able to select the facility that created the tacit knowledge on the screen shown in FIG. 36A . For example, when the selected application is to be installed on the device 200 to be purchased, the product may be shipped with the target application pre-installed based on the selection operation of the manufacturer order button.

[0401] It is also conceivable that the device 200 to be purchased (e.g., a positioning pillow) and the device 200 on which the application is to be installed (e.g., the third terminal device 810) may be different. In this case, information identifying the user and the application may be transmitted to the server system 100, triggered by, for example, selecting a manufacturer order button. The server system 100 then executes a process of transmitting the designated application to the device 200 used by the target user. In this way, an appropriate application can be installed on the device 200 without the user having to perform any procedures. Note that, although the example here illustrates application selection at the time of device purchase, the screen of FIG. 36A may also be used for changing applications during use. For example, the screen viewed by the care manager displays the device 200 and applications currently being used by the target user, and the care manager changes the application to be used according to the situation of the person being assisted, etc. For example, as in the example of FIG. 36A, the application to be used (tacit knowledge) may be selected from a pull-down menu. In this case, the application is similarly uninstalled or installed via, for example, the server system 100.

[0402] FIG. 36C is an example of information additionally displayed on the screen of FIG. 36A. As shown in FIG. 36C, the screen displayed on the care manager's terminal device may display the device 200 and applications used by the target user. In the example of FIG. 36C, the user has installed the choking detection device 460 and is using the normal function as a choking detection app. The normal function is a function for detecting swallowing and choking, and measuring swallowing time. In contrast, the care manager may be able to install additional functions. For example, as shown in FIG. 36C, a function for detecting the presence or absence of highly dangerous choking may be added. This allows the care manager to add optional functions.

[0403] As described above, the method of this embodiment can switch the operation mode of each device 200 based on capability information, etc. Specifically, it can change the active / inactive status of each application. Therefore, even in home care, the determination of active / inactive status may be performed automatically. For example, the server system 100 may estimate capability information from sensing data acquired during home care and determine the operation mode of the device 200 based on the capability information. However, this is not limited to this, and an application activation operation may be possible on a screen such as FIG. 36C. In this way, the care manager can manually change the operation mode according to the situation. As a result, even in home care where there is a limited variety of sensing data, it is possible to appropriately change the operation mode according to the situation.

[0404] Although the present embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel features and advantages of the present embodiment. Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader or equivalent meaning may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of the present embodiment and modifications are also included within the scope of the present disclosure. Furthermore, the configurations and operations of server systems, devices, information processing systems, etc. are not limited to those described in the present embodiment, and various modifications are possible.

[0405] [Additional Notes] One aspect of the present disclosure includes a device including an application that performs processing corresponding to the tacit knowledge of an expert, and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer at a data link layer for communication with the device, wherein the server system transmits the data frame, the frame body containing a fixed-length area including a first area that stores ability information representing the activity ability of a person being assisted, a second area that stores scene information that identifies an assistance scene for the person being assisted, and a third area that stores device type information that identifies a type of concurrent device used with the device, and the device is related to an information processing system that determines whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

[0406] Another aspect of the present disclosure relates to an information processing device that includes, in a data link layer for communication with a device including an application that performs processing corresponding to the tacit knowledge of an expert, a communication unit that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer, and a communication processing unit that controls the communication unit, wherein the communication unit relates to an information processing device that transmits to the device the data frame, which includes a fixed-length area including a first area that stores ability information representing the activity ability of a person being assisted, a second area that stores scene information that identifies a scene of assistance for the person being assisted, and a third area that stores device type information that identifies the type of concurrent device used together with the device, as information for determining whether the application is active or inactive.

[0407] Yet another aspect of the present disclosure relates to an information processing method in an information processing system including a device including an application that performs processing corresponding to the tacit knowledge of an expert, and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer for communication with the device, wherein the data frame includes a fixed-length area in the frame body, the fixed-length area including a first area that stores ability information representing the activity ability of a person being assisted, a second area that stores scene information that identifies a scene of assistance for the person being assisted, and a third area that stores device type information that identifies a type of concurrent device used with the device, and the information processing method determines whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information. [Explanation of symbols]

[0408] 10...information processing system, 100...server system, 110...processing unit, 111...capability information acquisition unit, 112...scene information acquisition unit, 113...device type information acquisition unit, 114...communication processing unit, 120...storage unit, 121...user information, 122...device information, 123...application information, 130...communication unit, 131...transmission / reception circuit, 132...antenna array, 200, 200-1 to 200-3...device, 210...processing unit, 220...storage unit, 230...communication unit, 240...display unit, 250...operation unit, 300...gateway, 410...imaging device, 420...bedside sensor, 430...detection device, 440...seat sensor, 441...cushion, 442...control box , 450...terminal device, 460...choking detection device, 461...throat microphone, 462...terminal device, 470...bed position detection device, 471...first terminal device, 472...second terminal device, 473...display, 480...glasses-type device, 510...reclining wheelchair, 520...nursing bed, 610...bed, 620...mattress, 630...wheelchair, 700...entrance / exit control device, 810...third terminal device, 820...fourth terminal device, 831...human presence sensor, 832...human presence sensor, 833...human presence sensor, 1141...upper layer processing unit, 1142...MAC layer processing unit, 1143...physical layer processing unit, IM1...output image, ReD...diaper area, Se1 to Se4...pressure sensors

Claims

1. a device including an application that executes processing corresponding to the tacit knowledge of an expert; a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with the device; Including, The server system includes: transmitting the data frame in which a fixed-length field including a first field for storing ability information representing the activity ability of the person being assisted is included in the frame body; The device comprises: An information processing system that determines whether to activate or deactivate the application based on the capability information.

2. In claim 1, The device comprises: When the application is set to active, the information processing system transmits the operation result of the application to the server system.

Images