Information processing system, information processing apparatus, and information processing method

Abstract

To provide an information processing system, an information processing apparatus, and an information processing method for appropriately supporting a caregiver's assistance for a person to be cared.SOLUTION: An information processing system includes: a device including an application that executes processing corresponding to tacit knowledge of a skilled person; and a server system that transmits a data frame including a media access control (MAC) header, a frame body, and a trailer in a data link layer of communication with the device. The server system transmits the data frame in which an area with a fixed length including a first area storing ability information, a second area storing scene information, and a third area storing device type information, is included in the frame body. The device determines active or non-active of the application based on at least one of the ability information, scene information, and device type information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and the like.

Background Art

[0002] Conventionally, a system used in a scene where a caregiver assists a care recipient is known. Patent Document 1 discloses a method of arranging sensors in a living space and generating provision information regarding the state of a resident living in the living space based on the temporal change of detection information acquired by the sensors.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided are an information processing system, an information processing apparatus, an information processing method, and the like that appropriately support a caregiver in assisting a care recipient.

Means for Solving the Problems

[0005] One aspect of the present disclosure relates to an information processing system including a device that executes a process corresponding to the tacit knowledge of a skilled person, and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with the device. The server system includes a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information specifying a scene of assistance for the assisted person, and a third area for storing device type information specifying the type of a combined device used together with the device. A fixed-length area including these areas is included in the frame body and transmits the data frame. The device is related to an information processing system that determines whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

[0006] Another aspect of the present disclosure relates to an information processing apparatus including a communication unit that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with a device including an application that executes a process corresponding to the tacit knowledge of a skilled person, and a communication processing unit that controls the communication unit. The communication unit includes a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information specifying a scene of assistance for the assisted person, and a third area for storing device type information specifying the type of a combined device used together with the device. A fixed-length area including these areas is included in the frame body and transmits the data frame to the device as information for determining whether the application is active or inactive.

[0007] Still another aspect of the present disclosure relates to an information processing method in an information processing system including a device that executes a process corresponding to the tacit knowledge of a skilled person, and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with the device, the method including: a first area that stores ability information representing the activity ability of an assisted person; a second area that stores scene information identifying a scene of assistance to the assisted person; and a third area that stores device type information identifying the type of a combined device used together with the device, the fixed-length area being included in the frame body; transmitting the data frame included in the frame body from the server system to the device; and determining whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15A

Figure 15B

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28A

Figure 28B

Figure 29A

Figure 29B

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36A

Figure 36B

Figure 36C

Mode for Carrying Out the Invention

[0009] Hereinafter, this embodiment will be described with reference to the drawings. For the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted. Note that the embodiment described below does not unduly limit the content described in the claims. Also, not all of the configurations described in this embodiment are essential constituent elements of the present disclosure.

[0010] 1. Example of System Configuration FIG. 1 is a configuration example of an information processing system 10 according to the present embodiment. The information processing system 10 according to the present embodiment digitizes the "intuition" and "tacit knowledge" involved in the work performed by an assistant based on their "intuition" and "tacit knowledge" in, for example, a medical facility or a nursing facility, and gives instructions to the assistant so that appropriate assistance can be provided regardless of the level of proficiency.

[0011] Here, the assistant may be a nursing staff member in a nursing facility, or a nurse or practical nurse in a medical facility such as a hospital. That is, the assistance in the present embodiment includes various actions for supporting the person to be assisted, and may include nursing care or actions related to medical treatment such as injections. Also, the person to be assisted here is a person who receives assistance from the assistant, and may be a resident of a nursing facility or a patient who is hospitalized or visits a hospital.

[0012] Also, the assistance in the present embodiment may be performed at home. For example, the person to be assisted in the present embodiment may be a person requiring home care who receives home care, or a patient who receives home medical treatment. Also, the assistant may be a family member of the person requiring care or the patient, or a visiting helper or the like.

[0013] The information processing system 10 shown in FIG. 1 includes a server system 100, a device 200, and a gateway 300. However, the configuration of the information processing system 10 is not limited to that shown in FIG. 1, and various modifications such as omitting a part, adding other configurations, etc. are possible. For example, in FIG. 1, as the device 200, a tablet-type terminal device such as a smartphone, a seat surface sensor 440 (described later with reference to FIG. 10) arranged on a wheelchair 630, and a detection device 430 (described later with reference to FIG. 9) placed on a bed 610 are exemplified, but the number and types of the device 200 are not limited thereto. For example, the information processing system 10 may include various devices 200 described later with reference to FIGS. 8 to 14. Also, the information processing system 10 may include devices other than the devices 200 shown in FIGS. 8 to 14. Hereinafter, when there is no need to distinguish between a plurality of devices 200 from each other, they are simply referred to as the device 200. Also, the point that modifications such as omitting or adding configurations are possible is the same in FIGS. 3, 4, etc. described later.

[0014] The information processing device of the present embodiment corresponds to, for example, the server system 100. However, the method of the present embodiment is not limited thereto, and the processing of the information processing device may be executed by distributed processing using the server system 100 and other devices. For example, the information processing device of the present embodiment may include the server system 100 and the device 200. Hereinafter, an example in which the information processing device is the server system 100 will be described.

[0015] The server system 100 is connected to the device 200 via, for example, a network. For example, the server system 100 is connected to the gateway 300 via a public communication network such as the Internet, and the gateway 300 is connected to the device 200 using a LAN (Local Area Network) or the like. For example, the gateway 300 may be an access point (AP) that performs communication according to the IEEE802.11 standard, and the device 200 may be a station (STA) that performs communication according to the IEEE802.11 standard. However, various modifications are possible for the communication method between each device.

[0016] The server system 100 may be one server or may include a plurality of servers. For example, the server system 100 may include a database server and an application server. The database server stores various data to be described later with reference to FIG. 3. The application server performs the processes to be described later with reference to FIGS. 6 to 7 and the like. Here, the plurality of servers may be physical servers or virtual servers. Further, when virtual servers are used, the virtual servers may be provided on one physical server or may be distributed and arranged on a plurality of physical servers. As described above, the specific configuration of the server system 100 in the present embodiment can be variously modified and implemented.

[0017] The device 200 has, for example, various sensors and performs processing based on data sensed by the sensors (hereinafter referred to as sensing data). The digitization of the tacit knowledge of the above-described experts may be executed, for example, by the vendor of the device 200. For example, assume that an expert has tacit knowledge of performing a forward and lateral deviation determination based on the posture of a care recipient in a wheelchair. In this case, it is possible to digitize the tacit knowledge by collecting sensing data corresponding to the posture of the care recipient detected by the seat surface sensor 440 and creating an application that performs a forward and lateral deviation determination based on the sensing data. For example, the vendor provides the seat surface sensor 440 and the above application. As a result, a person who is not an expert (e.g., a new employee) can utilize a forward and lateral deviation determination equivalent to that of an expert.

[0018] Also, the tacit knowledge digitized in one device 200 is not limited to one. For example, the tacit knowledge digitized using the seat surface sensor 440 is not limited to the determination of forward and lateral displacement, and may include those for determining the possibility of falling or those that combine both the determination of forward and lateral displacement and the possibility of falling. The determination of forward and lateral displacement corresponds to the determination of the quality of the posture of the care recipient, and the determination of the possibility of falling corresponds to the determination of slipping off the seat surface. Also, in the determination of forward and lateral displacement, there can be a plurality of tacit knowledges with different criteria for determining how much displacement is determined as forward or lateral displacement, the content of the determination process, etc. Therefore, each device 200 is capable of executing processing corresponding to one or more tacit knowledges, and may switch whether to execute the processing corresponding to each tacit knowledge. For example, the vendor may implement the tacit knowledge as application software (hereinafter, also simply referred to as an application) and register it in the server system 100. Each device 200 downloads and installs the executable applications among the registered applications to execute the processing corresponding to the tacit knowledge.

[0019] FIG. 2 is a diagram showing an example of the relationship between the device 200 and the tacit knowledge (application). In FIG. 2, five devices 200a to 200e are exemplified as the devices 200 connected to the server system 100. In the example of FIG. 2, two tacit knowledges, tacit knowledge 1 and tacit knowledge 2, are associated with the device 200a. For example, the applications of tacit knowledge 1 and tacit knowledge 2 are installed in the device 200a. The device 200a may be, for example, a device related to the fall risk described later. Specifically, the device 200a may be the seat surface sensor 440, tacit knowledge 1 may be the tacit knowledge related to the determination of forward and lateral displacement, and tacit knowledge 2 may be the tacit knowledge related to the determination of the possibility of falling. The processing results corresponding to tacit knowledge 1 and tacit knowledge 2 are transmitted to, for example, the server system 100.

[0020] Devices 200b and 200c are devices related to, for example, the risk of aspiration described later, and each is associated with a plurality of tacit knowledges for dealing with the risk of aspiration and the like. Devices 200d and 200e are devices related to, for example, the risk of pressure ulcers described later, and each is associated with a plurality of tacit knowledges for dealing with the risk of pressure ulcers and the like. In this way, it is possible to digitize various tacit knowledges using various devices. Here, two tacit knowledges are exemplified for one device 200, but the number of tacit knowledges associated with one device 200 is not limited to this.

[0021] In the method of this embodiment, it is possible to switch the tacit knowledge used according to the situation. For example, it is not necessary to use all of the tacit knowledges 1 to 10 shown in FIG. 2, and the use / non-use can be switched as needed. The switching of the tacit knowledge may be realized by switching the device 200 to be used. For example, when targeting a care recipient with a high risk of falling but low risks of aspiration and pressure ulcers, device 200a is used and devices 200b - 200e are not used. In this way, at least one of tacit knowledge 1 and tacit knowledge 2 is used and the other tacit knowledges are not used, so that it is possible to appropriately use the tacit knowledges that are highly necessary for the care recipient. Also, when the risk of aspiration of the care recipient increases, it is possible to switch the tacit knowledge to be used by using device 200b or device 200c. Also, when dealing with the risk of aspiration, cases such as using only device 200b, using only device 200c, and using both devices 200b and 200c may be switched. In this way, for example, when the risk of aspiration is high, the necessary tacit knowledges among tacit knowledges 3 to 6 in FIG. 2 can be used. The same applies when the risk of pressure ulcers increases, and it is possible to switch the tacit knowledge to be used by using device 200d or device 200e. Also, it is possible to switch, such as stopping the use of device 200a when the risk of falling decreases.

[0022] The switching of tacit knowledge may also be realized by switching the tacit knowledge used within the device 200 while maintaining the device 200 to be used. For example, in the device 200a, cases such as using only tacit knowledge 1, using only tacit knowledge 2, and using both tacit knowledge 1 and tacit knowledge 2 may be switched. This process can be realized, for example, by controlling the activation / inactivation of the applications corresponding to each tacit knowledge.

[0023] And the switching of tacit knowledge may be executed using the ability information of the assisted person. The ability information here is information representing the activity ability of the assisted person, and is, for example, information obtained as a result of the device 200 performing processing using some tacit knowledge. The ability information may be, for example, information related to the levels of fall risk, aspiration risk, and pressure ulcer risk. Details of the ability information will be described later.

[0024] For example, when the processing result of tacit knowledge is obtained in a certain device 200, a process of switching the tacit knowledge used within the same device 200 may be performed. For example, based on the processing result of tacit knowledge 1 in the device 200a of FIG. 2, the activation / inactivation of the application corresponding to tacit knowledge 1 and the application corresponding to tacit knowledge 2 may be switched.

[0025] Also, when the processing result of tacit knowledge is obtained in a certain device 200, the processing result may affect other devices 200. For example, based on the processing result of tacit knowledge 1 in the device 200a, the tacit knowledge used in the device 200b is switched. For example, a switch may be made from a state where the device 200b is not used to a state where the device 200b is used. Alternatively, for tacit knowledge 3 and tacit knowledge 4 associated with the device 200b, the activation / inactivation may be switched individually.

[0026] Furthermore, the switching of tacit knowledge is not limited to being based on ability information, and may be executed based on the assistance scene of the assisted person, the combined use situation of a plurality of devices 200, etc. Information used for switching tacit knowledge and details of the process for switching tacit knowledge will be described later.

[0027] Note that the correspondence between the device 200 and tacit knowledge can be flexibly changed. For example, based on sensing data acquired using a plurality of devices 200, a process corresponding to one piece of tacit knowledge may be executed. For example, based on the sensing data acquired by the device 200a and the sensing data acquired by the device 200b, a process corresponding to tacit knowledge 1 may be executed. In this way, since the types of sensing data used for the process can be increased, it is possible to improve the processing accuracy and the like. In the above example, the process corresponding to tacit knowledge 1 may be performed by the device 200a, may be performed by the device 200b, or may be realized by distributed processing of the device 200a and the device 200b. Furthermore, the process corresponding to tacit knowledge is not limited to being executed by the device 200 that acquires the sensing data. For example, based on the sensing data acquired by the device 200a and the sensing data acquired by the device 200b, in a device 200 that is different from both the device 200a and the device 200b, a process corresponding to tacit knowledge 1 may be executed. For example, in FIG. 1, based on the sensing data acquired by the seat surface sensor 440 or the sensing data acquired by the detection device 430, the device 200 which is a smartphone may execute a process corresponding to tacit knowledge.

[0028] Note that although an example in which tacit knowledge is digitized by the vendor of the device 200 has been described above, it is not limited to this. For example, an assistant who uses the device 200 is not prevented from digitizing his or her own tacit knowledge. For example, the said assistant may create an application corresponding to tacit knowledge and register the said application in the server system 100. In this way, it becomes possible to promote the digitization and utilization of tacit knowledge.

[0029] FIG. 3 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit 130.

[0030] The processing unit 110 of the present embodiment is configured by the following hardware. The hardware can include at least one of a circuit that processes digital signals and a circuit that processes analog signals. For example, the hardware can be configured by one or more circuit devices mounted on a circuit board or one or more circuit elements. The one or more circuit devices are, for example, IC (Integrated Circuit), FPGA (field-programmable gate array), etc. The one or more circuit elements are, for example, resistors, capacitors, etc.

[0031] The processing unit 110 may also be implemented by the following processors. The server system 100 of the present embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various types of data. The memory may be the storage unit 120 or another memory. The processor includes hardware. The processor can use various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor). The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, a register, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores instructions readable by a computer, and the functions of the processing unit 110 are realized as processing when the processor executes the instructions. The instructions here may be instructions in the instruction set that constitutes a program or instructions that instruct operations to the hardware circuit of the processor.

[0032] The processing unit 110 includes, for example, a capability information acquisition unit 111, a scene information acquisition unit 112, a device type information acquisition unit 113, and a communication processing unit 114.

[0033] The ability information acquisition unit 111 performs a process of acquiring ability information representing the activity ability of the assisted person. For example, the ability information acquisition unit 111 acquires sensing data from the device 200. The sensing data here may be, for example, log data for a predetermined period. The ability information acquisition unit 111 obtains a time-series change in the state of the assisted person based on the log data, and estimates the ability information based on the change. The ability information here may be index information regarding daily activities (ADL: Activities of Daily Living). Various methods for evaluating ADL, such as the Barthel index, are known, and in this embodiment, they can be widely applied. Also, as the ability information, the nine-level index disclosed as the Clinical Frailty Scale in “Frailty and the potential kidney transplant recipient: time for a more holistic assessment?”, Henry H.L. Wu, et al. may be used. For example, the ability information acquisition unit 111 of this embodiment determines which of the nine levels the assisted person belongs to based on the sensing data.

[0034] Note that the ability information may be obtained in the device 200 or the like. The ability information acquisition unit 111 may execute a process of acquiring the ability information from the device 200 or the like via the communication unit 130.

[0035] The scene information acquisition unit 112 determines the scene in which the assistance for the assisted person is performed. The scene information here may be information specifying the type of assistance to be executed, such as meal assistance, excretion assistance, transfer assistance, etc. Also, the scene information may be information regarding the assistant, such as the number and proficiency of the assistants who execute the assistance for the assisted person. Also, the scene information may be information regarding the assisted person, such as the attributes of the assisted person. For example, the scene information acquisition unit 112 performs a process of obtaining the scene information based on user input, the schedule of the assistant, or the like.

[0036] The device type information acquisition unit 113 is information for identifying the type of the device 200 that operates together with the target device 200. The device type here represents a broad classification such as a wheelchair or a bed, and may be information that does not distinguish between vendors. For example, the device types of a wheelchair from a first vendor and a wheelchair from a second vendor different from the first vendor may be the same. For example, the device type information acquisition unit 113 may identify other devices 200 used by the care recipient (or the caregiver who assists the care recipient) who uses the target device 200, and perform a process of acquiring information representing the type of the other device 200 as device type information.

[0037] The communication processing unit 114 controls communication using the communication unit 130. For example, the communication processing unit 114 executes a process of creating data to be transmitted, such as a MAC frame in the data link layer. Also, the communication processing unit 114 may perform a process of interpreting the frame structure and the like on the data received by the communication unit 130, extracting necessary data, and outputting it to an upper layer such as an application.

[0038] The storage unit 120 is a work area of the processing unit 110 and stores various information. The storage unit 120 can be realized by various memories, and the memory may be a semiconductor memory such as SRAM, DRAM, ROM, or flash memory, or may be a register, a magnetic storage device, or an optical storage device.

[0039] The storage unit 120 may store user information 121, device information 122, and application information 123.

[0040] In the method of this embodiment, for example, tacit knowledge is registered in the server system 100 as an application. And each user who uses tacit knowledge may register the device 200 in the system and then download and use the necessary application on the device 200.

[0041] User information 121 includes information such as a user ID and a user name that uniquely identify a user of the information processing system 10, and a device ID and the like that are information that uniquely identifies the device 200 used by the user.

[0042] Device information 122 is information related to the device 200, and includes a device ID, a device type ID representing the type of the device 200, a vendor, an application ID of the installed application, and the like. The application ID is information that uniquely identifies an application.

[0043] Application information 123 is information related to an application, and includes an application ID, an application name, a creator, and the like. The application information may also include information that specifies the specific processing content of the application. The information that specifies the processing content may be the source code of the program or the executable file. When the application corresponds to a learned model, the application information may also include information on the structure of the learned model. For example, when the learned model is a neural network (hereinafter referred to as NN), the structure of the learned model includes the number of layers of the NN, the number of nodes included in each layer, the connection relationship between the nodes, weights, activation functions, and the like.

[0044] By using the user information 121, it becomes possible to appropriately manage the users who use the information processing system 10. Also, by referring to the device information 122, it becomes possible to check the details of the device 200 used by each user. Furthermore, by referring to the application information 123, it becomes possible to check the details of each application registered in the server system 100.

[0045] The communication unit 130 is an interface for performing communication via a network. When the server system 100 performs wireless communication, it includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. However, the server system 100 may perform wired communication. In this case, the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 operates according to the control by the communication processing unit 114. However, it is not precluded that the communication unit 130 includes a processor for communication control different from the communication processing unit 114. The communication unit 130 may perform communication according to a method defined in, for example, IEEE802.11 or IEEE802.3. However, various modifications of the specific communication method are possible.

[0046] FIG. 4 is a block diagram showing a detailed configuration example of the device 200. The device 200 includes, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250. As will be described later with reference to FIGS. 8 to 14, in the method of this embodiment, devices 200 of various aspects can be used. The configuration of each device 200 is not limited to that shown in FIG. 4, and modifications such as omitting some configurations and adding other configurations are possible. For example, the device 200 may have various sensors according to the device 200, such as a motion sensor such as an acceleration sensor or a gyro sensor, an imaging sensor, a pressure sensor, and a GPS (Global Positioning System) sensor.

[0047] The processing unit 210 is constituted by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. Further, the processing unit 210 may be realized by a processor. As the processor, various processors such as a CPU, a GPU, and a DSP can be used. The functions of the processing unit 210 are realized as processing by the processor executing instructions stored in the memory of the device 200.

[0048] The memory unit 220 is a work area of the processing unit 210 and is realized by various memories such as SRAM, DRAM, and ROM.

[0049] The communication unit 230 is an interface for performing communication via a network and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100 via a network, for example. The communication unit 230 may perform wireless communication compliant with the IEEE802.11 standard with the gateway 300 and communicate with the server system 100 via the gateway 300.

[0050] The display unit 240 is an interface for displaying various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface for receiving user operations. The operation unit 250 may be buttons or the like provided on the device 200. Further, the display unit 240 and the operation unit 250 may be a touch panel configured integrally.

[0051] The device 200 may also include components not shown in FIG. 4, such as a light emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light emitting unit is, for example, an LED (light emitting diode) and performs notification by light emission. The vibration unit is, for example, a motor and performs notification by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and performs notification by sound.

[0052] 2. Device Control Based on Ability Information As shown in FIG. 1, the information processing system 10 of the present embodiment includes a server system 100 and a device 200. The device 200 may operate in any of a plurality of operation modes. The operation mode in the present embodiment may be determined by a combination of tacit knowledge (applications) to be used. For example, as described above, the device 200 can install a plurality of applications corresponding to different tacit knowledge, and for each of the plurality of applications, switch between active / inactive. In this way, it becomes possible to appropriately switch the tacit knowledge to be used according to the situation.

[0053] In particular, the server system 100 (ability information acquisition unit 111) may obtain ability information representing the activity ability of the care recipient based on the sensing data transmitted from the device. Then, the server system 100 transmits the obtained ability information to the device 200. The device 200 determines which of the plurality of operation modes to operate in based on the received ability information. Since the assistance to be performed changes according to the change in the ability information of the care recipient, the desired operation of the device 200 may change. According to the method of the present embodiment, it becomes possible to appropriately switch the operation mode according to the ability information. For example, the device 200 can appropriately switch the active / inactive (use / non-use of tacit knowledge) of the application according to the ability information. Hereinafter, the processing based on the ability information will be described.

[0054] 2.1 Overview FIG. 5 is a diagram showing an example of the relationship between the ability information of a care recipient and the assumed risks. For example, when the ability of the care recipient is sufficiently high, since daily activities can be performed without the assistance of others, the risks in daily life are not high. However, when the ability begins to decline, for example, it becomes difficult for the care recipient to perform daily living activities such as getting up and sitting down. Daily living activities refer to actions such as getting up and sitting down. In this case, it becomes difficult to maintain balance during the initial movement including daily living activities, so the risk of the care recipient falling increases. The initial movement refers to the movement starting from a state of small movement (narrowly speaking, a stationary state). On the other hand, it is assumed that there are few problems with daily activities other than the initial movement at this stage. For example, the care recipient can hold the sitting position for a long time, walk using a walker after getting up, eat to a certain extent freely, etc.

[0055] As the ability further declines, for example, it becomes difficult for the care recipient to walk, and assistance with movement using a wheelchair or the like is required. In this case, the high risk of falling during the initial movement is the same as in the above example, but since the ability to maintain the sitting position also declines, it is necessary to consider the risk of falling. For example, a care recipient at this stage may lose balance and fall from the mattress of the bed or the seat surface of the wheelchair even when sitting on the bed or in the wheelchair.

[0056] As the ability further declines, for example, the care recipient can no longer eat well. For example, since the swallowing ability declines, the risk of aspiration increases. The risk of aspiration means, for example, that the possibility of developing aspiration pneumonia becomes high. At this stage, since it is considered that the care recipient can move the body, the above-mentioned high risks of falling and toppling are maintained, and it becomes necessary to consider the risk of aspiration pneumonia.

[0057] As the ability further declines, for example, the care recipient transitions to a bedridden state where assistance is required for most daily activities. Even in this case, since there is a possibility of changing diapers in bed or moving in a wheelchair, etc., the risk of falling is high. Also, unless there are special circumstances such as a gastrostomy, oral feeding continues, so the risk of aspiration pneumonia is high. Furthermore, since the time spent in bed is very long and spontaneous turning over becomes difficult, the risk of pressure ulcers also increases. On the other hand, in the case of being bedridden, since it is not assumed that the care recipient will start moving on their own, the risk of falling when starting to move is low.

[0058] As shown in FIG. 5, since the assumed risks differ according to the ability of the care recipient, the assistance to be provided by the caregiver also changes. Therefore, when using the tacit knowledge digitized using the device 200, the tacit knowledge to be used also changes according to the ability. In this regard, in the present embodiment, since the operation mode of the device 200 can be set according to the ability information, it is possible to execute processing according to changes in ability. For example, when providing assistance to about 10 care recipients as a unit in a nursing facility or the like, it is conceivable that care recipients with different abilities are mixed within the unit. However, according to the method of the present embodiment, it is not necessary for the caregiver to manually set the necessary assistance for each care recipient. That is, even when taking care of a plurality of care recipients, it is possible to set an appropriate operation mode without increasing the burden on the caregiver.

[0059] In FIG. 5, the case where the ability decreases from top to bottom was taken as an example for explanation, but the direction of the ability change is not limited to this. For example, the ability may be restored by the cure or remission of a disease, the implementation of rehabilitation, etc. Since the method of this embodiment sets the operation mode according to the ability, it can flexibly respond even when the ability is restored. Also in FIG. 5, four stages of "unable to get up", "unable to walk", "unable to eat well", and "bedridden" were exemplified, but the stages of the ability represented by the ability information are not limited to this, and some stages may be omitted, or other stages may be added. Also, although the risk of falling is low because walking is possible, a state where the risk of aspiration is high due to a decrease in swallowing ability may be considered. That is, the above four stages are not limited to changing in the above order, and more complex combinations may be considered.

[0060] FIG. 6 is a sequence diagram for explaining the operations of the server system 100 and the device 200, and is a diagram for explaining the preprocessing executed before the device 200 executes the process corresponding to tacit knowledge.

[0061] First, in step S101, the server system 100 performs a process of accepting the registration of applications in advance. For example, as described above, each application corresponds to tacit knowledge and is created by the vendor of the device 200 or the like. The creator of the application, for example, after logging in to the information processing system 10 of this embodiment using an arbitrary terminal device (such as a PC or a smartphone), uses a vendor screen (not shown) displayed on the display unit of the terminal device to register the application with the server system 100. The processing unit 110 of the server system 100 stores the information regarding the registered application in the storage unit 120 as application information 123. Here, an example where vendor apps 1 to 3, which are applications created by the vendor, are registered is shown.

[0062] In step S102, based on the operation of the user who uses the device 200, a registration request for the device 200 is sent to the server system 100. Here, the user may be an assistant who utilizes tacit knowledge, or may be an administrator of a nursing facility or the like. For example, when the user introduces a new device 200 into their environment, the process of step S102 is executed. For example, after the user logs in to the information processing system 10 using the operation unit of the device 200 or the operation unit of the terminal device connected to the device 200, the user uses an unillustrated user screen to perform the process of registering the device 200 with the server system 100. The registration request includes, for example, a user ID that identifies the user and information such as the vendor and model number of the device 200.

[0063] In step S103, the processing unit 110 of the server system 100 executes a process based on the registration request. For example, the processing unit 110 assigns a device ID that uniquely identifies the device 200 to the target device 200 and sends the device ID to the device 200. Further, the processing unit 110 may execute a process of associating the logged-in user with the device 200 for which the registration request was made. For example, a process of adding the device ID of the device 200 for which the registration request was made to the user information 121 of the logged-in user may be performed. Further, the processing unit 110 may store, in the device information 122, the device ID of the device 200 for which the registration request was made in association with the device type ID and the like of the device 200. The device type ID can be specified, for example, based on information such as the vendor and model number included in the registration request. Through the above processes, the newly introduced device 200 is registered in the information processing system 10.

[0064] Next, in step S104, based on the operation of the user who uses the device 200, an application used by the device 200 is selected. For example, when the registered device 200 accesses the server system 100, the server system 100 may return an application list screen available on the device 200. The user performs a user operation to select an application to use from the applications listed. Here, consider an example where applications including vendor application 1 to vendor application 3 registered in the process shown in step S101 are listed, and the user performs a selection operation on vendor application 1 to vendor application 3.

[0065] In step S105, the server system 100 permits the download of the selected application, and the device 200 executes the download of the selected application. Also, the server system 100 may perform a process of associating the device 200 with the application downloaded to the device 200. For example, the processing unit 110 executes a process of adding the application IDs corresponding to vendor application 1 to vendor application 3 to the device information 122 regarding the device 200.

[0066] In step S106, the device 200 executes a process of installing the downloaded vendor application 1 to vendor application 3. As a result, the device 200 becomes capable of operating in any of a plurality of operation modes. For example, the device 200 may switch between active / inactive for each of vendor application 1 to vendor application 3. In this case, the device 200 is 2 3Eight operation modes can be selected. In this embodiment, a state where all of Vendor Apps 1 to 3 are inactive is also considered as one operation mode. Also, the relationship between the applications and the operation modes is not limited to this. For example, a plurality of applications may operate exclusively. In the above example, the device 200 may be able to set four operation modes: a mode where all vendor apps are inactive, a mode where only Vendor App 1 is active, a mode where only Vendor App 2 is active, and a mode where only Vendor App 3 is active.

[0067] FIG. 7 is a sequence diagram for explaining the operations of the server system 100 and the device 200, and is a diagram for explaining an example in which the operation mode of the device 200 changes based on the ability information of the assisted person.

[0068] First, in step S201, the server system 100 performs a process of transmitting data including ability information to the device 200. FIG. 7 shows an example in which data with an ADL index value of 2 is transmitted.

[0069] In step S202, the device 200 controls the activation / inactivation of the installed vendor apps based on the acquired capability information. For example, the storage unit 220 of the device 200 may store information associating the capability information with the operation mode. The processing unit 210 of the device 200 performs a process of obtaining the operation mode based on the said information and the capability information acquired from the server system 100. For example, the storage unit 220 may store table data associating the index value of the ADL with the activation / inactivation of each application. The processing unit 210 determines the activation / inactivation of each application by extracting the record that matches the received ADL index value from the said table data. Here, among vendor app 1 to vendor app 3, an operation mode is set in which vendor app 1 and 2 are active and vendor app 3 is inactive. However, the process of determining the operation mode based on the capability information is not limited to the above example, and various modifications can be made.

[0070] After the process of step S202, the device 200 executes the process according to vendor app 1 and the process according to vendor app 2. Specifically, the processing unit 210 of the device 200 acquires sensing data using a sensor, and obtains a processing result by executing the process defined in the application with the said sensing data as input. In step S203, the device 200 transmits the processing result to the server system 100. The processing result here corresponds to the result of a judgment executed using the tacit knowledge of an expert. Also, the information transmitted to the server system 100 here is not limited to the processing result, and may include information such as the log of the sensing data.

[0071] In step S204, the server system 100 executes control based on the processing result received from the device 200. For example, the processing unit 110 may identify the device to be controlled and perform a process of transmitting a control signal for operating the device to be controlled. The device to be controlled here may be the reclining wheelchair 510 described later with reference to FIG. 23 or the nursing bed 520 described later with reference to FIG. 24. The control signal in this case may be a signal for instructing a change in the angle of the backrest of the reclining wheelchair 510 or the bottom angle of the nursing bed 520. The control signal may also be a signal for instructing the execution of notification to the device to be controlled. For example, the device to be controlled is a device including a notification unit such as a display unit or a light emitting unit, and the control signal is a signal for instructing the device to be controlled to execute notification using display of an image, light emission, or the like.

[0072] Also, the process of determining the device to be controlled and the control content based on the processing result in the device 200 may be performed by the processing unit 110 of the server system 100 or the processing unit 210 of the device 200. In the latter case, in step S203, in addition to the processing result, information specifying the device to be controlled and the control content may be transmitted. Also, the device 200 or the server system 100 may only identify the device to be controlled, and the determination of the specific control content may be executed in the device to be controlled. In this case, in steps S203 and S204, processes of transmitting the processing result are respectively executed. In addition, various modifications are possible for the device to be controlled, the control signal, and the like. For example, the device to be controlled may be the device 200. Also, in step S204, the server system 100 may perform a process of storing the log of the sensing data transmitted from the device 200 in the storage unit 120.

[0073] Also in step S205, the server system 100 executes a process of updating the assisted person's ability information. For example, the ability information acquisition unit 111 may obtain the ability information based on the log of the sensing data. For example, the storage unit 120 of the server system 100 may store information associating the sensing data with the ability information. The processing unit 110 executes a process of obtaining the ability information based on the information and the sensing data transmitted from the device 200. Here, the information associating the sensing data with the ability information may be a learned model. The training data for generating the learned model here is, for example, data in which the ability information of the assisted person judged by a person with specialized knowledge (such as a doctor or a skilled caregiver) is given as correct data for the sensing data related to the assisted person. The correct data may be an index value representing the ability as described above, or a set of information representing the presence or absence and level of individual abilities (such as the seat holding ability and swallowing ability described later). The processing unit 110 obtains the ability information by inputting the sensing data into the learned model. Alternatively, the information stored in the storage unit 120 may be reference data that is sensing data with a known correspondence relationship with the ability information. The processing unit 110 determines the similarity between the acquired sensing data and the reference data, and may obtain the ability information based on the similarity. The reference data here may be, for example, sensing data acquired for a highly capable assisted person. In this case, if the similarity to the reference data is high, it is determined that the ability value is high, and if the similarity is low, it is determined that the ability value is low. The reference data may also be other information such as sensing data acquired for a less capable assisted person. Also, the processing unit 110 may use the current ability information in the processing. For example, the processing unit 110 may obtain the amount of change in the ability information based on the sensing data, and obtain the updated ability information based on the amount of change and the current ability information. A specific example of the process of obtaining the ability information will be described later. Also, in the process of step S205, information other than the sensing data, such as a report input by the caregiver or a diagnosis result by a doctor, may be used.

[0074] In step S206, the server system 100 performs a process of transmitting data including the updated ability information to the device 200. FIG. 7 shows an example in which data with an ADL index value of 3 is transmitted.

[0075] In step S207, the device 200 controls the activation / inactivation of the installed vendor apps based on the acquired ability information. For example, as described above, the device 200 determines the activation / inactivation of each vendor app based on the table data. In the example of FIG. 7, vendor apps 1 and 2 maintain their active states, and vendor app 3 is changed from inactive to active. As a result, after step S207, the device 200 transitions to a state of operating in an operation mode in which all of vendor apps 1 to 3 are active. The operations after step S207 are, for example, the same as those in steps S203 - S206.

[0076] Note that the method of this embodiment is not limited to being applied to the information processing system 10 including the server system 100 and the device 200, and may be applied to an information processing apparatus. Here, the information processing apparatus is, in a narrow sense, the server system 100. The information processing apparatus operates in any of a plurality of operation modes, and includes a communication unit (corresponding to the communication unit 130 in FIG. 3) that communicates with the device 200 used for assisting the assisted person, and a processing unit (corresponding to the processing unit 110 in FIG. 3, and in a narrow sense, the ability information acquisition unit 111) that performs a process of obtaining ability information representing the activity ability of the assisted person based on the sensing data transmitted from the device 200. Then, the processing unit of the information processing apparatus performs a process of transmitting the ability information to the device 200 via the communication unit as information for determining in which of the plurality of operation modes the device 200 operates. In this way, it becomes possible to estimate the transition of the ability information of the assisted person based on the information collected from the device 200 and to operate the device 200 according to the ability information.

[0077] In addition, part or all of the processing performed by the information processing system of the present embodiment may be realized by a program. The processing performed by the information processing system includes, for example, at least one of the processing performed by the processing unit 110 of the server system 100 and the processing performed by the processing unit 210 of the device 200. Similarly, part or all of the processing performed by the information processing apparatus of the present embodiment may be realized by a program.

[0078] The program according to the present embodiment can be stored in a non-temporary information storage device (information storage medium), which is a computer-readable medium, for example. The information storage device can be realized by, for example, an optical disk, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 and the like perform various processes of the present embodiment based on the program stored in the information storage device. That is, the information storage device stores a program for causing a computer to function as the processing unit 110 and the like. A computer is a device including an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to the present embodiment is a program for causing a computer to execute each step described above with reference to FIGS. 6, 7, and the like.

[0079] Further, the method of the present embodiment can be applied to an information processing method in an information processing system 10 that operates in any of a plurality of operation modes and includes a device 200 used for assisting a person to be assisted and a server system 100 connected to the device 200 via a network. The information processing method includes a step of obtaining ability information representing the activity ability of the person to be assisted based on sensing data acquired by the device 200, and a step of determining in which of a plurality of operation modes the device 200 operates based on the obtained ability information.

[0080] Hereinafter, specific devices 200 and the processing executed in the devices 200 will be described by taking each stage shown in FIG. 5 as an example.

[0081] 2.2 Unable to get up <Examples of Devices and Operations: Fall Risk Judgment> First, a device 200 for coping with the risk of falling at the start of movement in a state where daily living activities have become difficult will be described. FIGS. 8 to 9 are examples of the device 200 used for determining the risk of falling at the start of movement.

[0082] FIG. 8 is a diagram showing an example of an imaging device 410 that images a care recipient and an example of an output image IM1 of the imaging device 410. In addition to each configuration shown in FIG. 4, the imaging device 410 has an image sensor that outputs an imaging image as sensing data. The imaging device 410 may be disposed in a place where a large number of people gather and move, such as a living area or a hall in a nursing facility. In the example of FIG. 8, the imaging device 410 is disposed on the upper part of a television device.

[0083] The processing unit 210 of the imaging device 410 may perform processing for detecting the start of movement of a person based on the imaging image. For example, by operating according to an application installed in the imaging device 410, the processing unit 210 acquires the imaging image as input data, performs processing for detecting a person from the imaging image, and performs processing for determining the presence or absence of the start of movement of the detected person.

[0084] For example, the imaging device 410 performs face recognition processing for recognizing a person's face based on the captured image. For example, the storage unit 220 of the imaging device 410 stores a face image of a person to be detected, and the processing unit 210 may perform face recognition processing based on matching processing using the face image as a template. Also, various face recognition processing methods are known, and in this embodiment, they can be widely applied. For example, when the movement of the detected face area is below a given threshold for a certain period of time, the imaging device 410 sets the position of the face area in that state as a reference position. Then, the imaging device 410 sets a detection area at a position a predetermined distance away from the reference position, and when the face area reaches the detection area, it may be determined that there has been a movement start. For example, when a standing-up operation is performed, since it is assumed that the position of the face moves relatively upward, the detection area may be an area set at a position a predetermined distance above the reference position. In this case, when the position of the face area on the image moves upward by a predetermined distance or more with respect to the reference position, a movement start is detected. Note that the detection area here is, for example, a linear area, but other shaped areas may be set.

[0085] Also, the imaging device 410 can identify the target care recipient through face recognition processing. Therefore, the imaging device 410 may perform movement start detection for a care recipient whose ability represented by the ability information is below a predetermined threshold (corresponding to being unable to perform a getting-up movement), and may omit movement start detection for a care recipient whose ability is higher than the threshold.

[0086] Also, the movement start detection processing is not limited to the above method. For example, the imaging device 410 may perform skeleton tracking processing based on the captured image. As a method of skeleton tracking based on an image, various methods are known, such as "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" (https: / / arxiv.org / pdf / 1611.08050.pdf), OpenPose disclosed by Zhe Cao et al., and in this embodiment, they can be widely applied.

[0087] In addition, OpenPose discloses a method of performing skeleton tracking for each of a plurality of persons imaged in an image and displaying the results. In the example of FIG. 8, the image sensor outputs a captured image including three assisted persons, and the imaging device 410 determines whether or not each of the three assisted persons has started to move.

[0088] For example, an assisted person whose ability has declined and who has difficulty getting up and down may fall even when taking a standing-up posture. Therefore, the imaging device 410 may determine whether the person is taking a standing-up posture by skeleton tracking. For example, when the imaging device 410 determines that the person has bent forward with their hands on their knees or the seat surface of the chair from a sitting state, it determines that the person is in a standing-up posture and notifies the caregiver of the fall risk. For example, when the imaging device 410 detects that the distance between the position of the hand and the position of the knee is equal to or less than a predetermined value, or that the position of the shoulder has moved downward by a predetermined value or more, etc. from the skeleton tracking result, it may determine that the assisted person is taking a standing-up posture.

[0089] Alternatively, the imaging device 410 may divide the processing target data into windows in units of several seconds, and determine that a posture change such as a standing-up movement has occurred when a specific position such as the head or neck has moved by a predetermined threshold or more within each window. Note that the part to be detected for movement may be other than the head and neck. Also, the movement direction may be vertical, horizontal, or diagonal. Also, the threshold used for detection may be changed according to the part to be detected. Alternatively, the imaging device 410 may obtain a region including the feature points detected by skeleton tracking for a stationary assisted person, and determine that there has been a starting movement such as a standing-up movement when a predetermined number or more of feature points have moved out of the region. In addition, various modifications of the method of detecting the starting movement using the imaging device 410 are possible.

[0090] IM1 in FIG. 8 is an example of the output image of the imaging device 410. The imaging device 410 may superimpose and display some display object on the captured image. In the example of FIG. 8, an object including an "!" mark is displayed in association with the assisted person whose movement has been detected. In this way, it becomes possible to easily notify the caregiver of the assisted person whose movement has been detected. For example, in step S203 of FIG. 7, the imaging device 410 transmits the output image IM1 to the server system 100. In step S204, the server system 100 outputs the output image IM1 to a smartphone or the like used by the caregiver. However, the output of the imaging device 410 may be information for identifying the assisted person whose movement has been detected (for example, the ID of the assisted person), and various modifications can be made to the specific mode. For example, here an example of notifying the assisted person whose movement has been detected is shown, but information for stopping the movement of the assisted person may be output. For example, the imaging device 410 may identify the moving assisted person and output voice data, video data, etc. of the family members of the assisted person. Especially in the case of patients with dementia, the response to calls becomes dull, but in many cases, they remember the faces and voices of family members, etc., which is effective for stopping the movement. By stopping the movement of the assisted person in this way, it is possible to gain time until the caregiver intervenes. Note that an example of outputting voice data and video data of family members will be described later in relation to device control based on scene information.

[0091] FIG. 9 is a diagram for explaining an example of the bedside sensor 420 and the detection device 430 arranged at the bottom of the bed 610. The bedside sensor 420 and the detection device 430 are, for example, as shown in FIG. 9, a sheet-like or plate-like device 200 provided between the bottom of the bed 610 and the mattress 620. Although both the bedside sensor 420 and the detection device 430 are shown in FIG. 9, only one of them may be used. Also, as will be described below, the bedside sensor 420 and the detection device 430 have a pressure sensor in common, so the bedside sensor 420 may also serve as the detection device 430, or the detection device 430 may also serve as the bedside sensor 420. In addition, various modifications can be made to the specific mode.

[0092] The bedside sensor 420 includes a pressure sensor that outputs a pressure value as sensing data, and is disposed on the bottom side where the caregiver uses it for getting on and off the bed. In the example of FIG. 9, the caregiver gets on and off the bed using the front side of the bed 610. At this time, as shown in FIG. 9, a fall prevention fence is disposed on the front side of the bed 610, and the bedside sensor 420 may be disposed at a position where the fence is not provided. In this way, the user who gets on and off the bed 610 once performs an operation of sitting on the bedside sensor 420.

[0093] The processing unit 210 of the bedside sensor 420 operates according to, for example, an application installed in the bedside sensor 420, acquires the pressure value as input data, and executes a process of determining the movement of the care recipient on the bed 610 from the pressure value.

[0094] For example, when the care recipient gets up from the bed 610, the care recipient shifts from the lying position on the bed to the sitting position at the bedside (hereinafter referred to as the end sitting position), and further applies force by putting hands on the knees and the bottom surface to perform the getting up operation. It is assumed. The pressure values detected by the bedside sensor 420 increase in the order of the lying position, the end sitting position, and the getting up operation. For example, the bedside sensor 420 may determine that a movement start has been detected when a change from the end sitting position to the getting up operation is detected based on a comparison process between the pressure value and a given threshold value. Alternatively, from the viewpoint of detecting the getting up operation at a faster stage, the bedside sensor 420 may determine that a movement start has been detected when a change from the lying position to the end sitting position is detected based on a comparison process between the pressure value and a given threshold value.

[0095] Alternatively, if the rising motion continues, the hips of the care recipient will lift off the bottom surface, causing the pressure value output from the pressure sensor to decrease significantly. Therefore, based on the time-series change of the pressure value, the processing unit 210 may determine that the rising motion has been performed when the pressure value increases above the first threshold and then decreases below the second threshold, which is smaller than the first threshold. In addition, various modifications can be made to the specific processing content of the start-of-motion determination.

[0096] When the start of movement of the care recipient is detected, the bedside sensor 420 transmits information indicating this to the server system 100. The server system 100 transmits the information to, for example, a smartphone used by the caregiver, and notification processing is executed on the smartphone or the like. In this way, it becomes possible to easily notify the caregiver of the care recipient whose movement has been detected.

[0097] Further, the detection device 430 shown in FIG. 9 is a device 200 that senses information related to the sleep of the care recipient. The detection device 430 includes a pressure sensor that outputs a pressure value.

[0098] When the user lies down on the bed, the detection device 430 detects the body vibration (body movement, vibration) of the user via the mattress 620. Based on the body vibration detected by the detection device 430, information regarding the respiratory rate, heart rate, activity level, posture, wakefulness / sleep, getting out of bed / staying in bed is obtained. In addition, the detection device 430 may determine non-REM sleep and REM sleep, and the depth of sleep. For example, the periodicity of body movement may be analyzed, and the respiratory rate and heart rate may be calculated from the peak frequency. The periodicity analysis is, for example, Fourier transform or the like. The respiratory rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time is, for example, 1 minute. Also, body vibration may be detected per sampling unit time, and the number of detected body vibrations may be calculated as the activity level. Also, when the user gets out of bed, since the pressure value detected is lower than when in bed, it is possible to determine getting out of bed / staying in bed based on the pressure value and its time-series change.

[0099] For example, based on the determination result of getting out of bed / staying in bed, when the cared-for person transitions from staying in bed to getting out of bed, the processing unit 210 of the detection device 430 may determine that the start of movement has been detected.

[0100] When the start of movement of the cared-for person is detected, the detection device 430 transmits information indicating that fact to the server system 100. The server system 100 transmits the information to, for example, a smartphone used by the caregiver, and notification processing is executed on the smartphone or the like. In this way, it becomes possible to easily notify the caregiver of the cared-for person whose start of movement has been detected.

[0101] For example, each device 200 shown in FIGS. 8 to 9 is set to the inactive operation mode 0 when the ability of the cared-for person represented by the ability information is equal to or greater than a predetermined level, and is set to the active operation mode 1 when the ability is less than the predetermined level. Here, the ability being less than the predetermined level means that the cared-for person cannot perform daily living actions. In this way, the above-described determination operation of the start of movement can be executed in an appropriate situation. As a result, when there is a cared-for person with a high risk of falling, the risk of falling of the cared-for person can be appropriately reduced.

[0102] <Update of ability information> Also, as shown in step S205 of FIG. 7, the ability information acquisition unit 111 of the server system 100 may perform processing to update the ability information based on the sensing data. For example, the ability information acquisition unit 111 determines a change in the state related to daily living based on the sensing data.

[0103] For example, the ability information acquisition unit 111 may determine the way of rising based on the sensing data. When rising, as described above, the hands are placed on the bottom surface or the like from the sitting position, and further weight is applied to the feet, and the back is extended while extending the knees. At this time, if the weight transfer to the feet is not sufficient, the center of gravity will be relatively biased backward, and the person will fall onto the bottom surface. Also, if the weight transfer to the feet is excessive, the center of gravity will be biased forward, so there is a risk of falling forward. Also, if the sitting posture in the sitting position is too shallow, the buttocks may fall off the bottom surface.

[0104] Therefore, the ability information acquisition unit 111 may determine whether the movement of the body of the care recipient at the time of rising is appropriate based on the log of the skeleton tracking by the imaging device 410 and the log of the pressure values from the bedside sensor 420 and the detection device 430. For example, it is determined that the higher the ability, the closer the movement at the time of rising is to the normal state, and the lower the ability, the more the deviation of the center of gravity and the position of the buttocks in the sitting position deviate from the normal state.

[0105] Also, the ability information acquisition unit 111 may obtain ability information based on the number of rising movements executed within a predetermined period. For example, it is determined that the higher the ability, the more the number of rising times, and the lower the ability, the fewer the number of times.

[0106] Also, the ability information acquisition unit 111 may obtain ability information based on the elapsed time from when the person is in the sitting position until they rise. The lying position, sitting position, and standing position can be determined by the positional relationship of the feature points of the skeleton tracking and the time-series change of the pressure value. For example, it is determined that the higher the ability, the shorter the elapsed time from when the person is in the sitting position until they rise, and the lower the ability, the longer the elapsed time.

[0107] Also, the ability information acquisition unit 111 may obtain ability information based on the activity level on the bed 610. The activity level may be detected by the detection device 430 as described above, for example. Also, the activity level may be obtained based on the result of skeleton tracking or the output of the bedside sensor 420. For example, it is determined that the higher the activity level, the higher the ability, and the lower the activity level, the lower the ability.

[0108] In the method of this embodiment, the determination of the ability information may be executed using any one of the above, or two or more may be combined. Also, as described above, the ability information may be obtained based on a combination of the sensing data and data other than the sensing data. According to the method of this embodiment, it is possible to reduce the risk of falling of a care recipient who cannot perform daily living movements and appropriately determine changes in the ability information of the care recipient.

[0109] 2.3 Unable to walk <Examples of Devices and Operations: Fall Risk> Next, a device 200 that operates to address the risk of falling from a wheelchair 630 or the like in a state where walking has become difficult will be described. FIGS. 10 to 11 are examples of the device 200 used for fall risk determination.

[0110] FIG. 10 is a diagram showing a seat surface sensor 440 which is a device 200 arranged on the seat surface of a wheelchair 630, for example. The seat surface sensor 440 includes a pressure sensor that outputs a pressure value, and based on the pressure value, determines which of a plurality of postures including a normal posture, a forward shift, a lateral shift, etc. is the posture of the care recipient when sitting on the wheelchair 630 (hereinafter also referred to as the sitting posture). A forward shift represents a state where the user's center of gravity is shifted forward more than normal, and a lateral shift represents a state where the user's center of gravity is shifted to either the left or right more than normal. Both a forward shift and a lateral shift correspond to a state where the risk of slipping off is relatively high. Note that the seat surface sensor 440 may be a sensor device arranged on a normal chair, or may be a sensor device that determines the posture of a user sitting on a bed or the like. Further, the seat surface sensor 440 may perform a determination of the possibility of falling to determine whether there is a possibility that the care recipient will fall from the seat surface.

[0111] In the example of FIG. 10, four pressure sensors Se1 to Se4 are arranged on the back side of a cushion 441 arranged on the seat surface of the wheelchair 630. The pressure sensor Se1 is a sensor arranged in the front, the pressure sensor Se2 is a sensor arranged in the rear, the pressure sensor Se3 is a sensor arranged on the right side, and the pressure sensor Se4 is a sensor arranged on the left side. Here, the front, rear, left, and right represent the directions as seen from the care recipient when the care recipient is sitting on the wheelchair 630.

[0112] As shown in FIG. 10, the pressure sensors Se1 to Se4 are connected to a control box 442. The control box 442 includes a processor that controls the pressure sensors Se1 to Se4 inside and a memory that serves as a work area for the processor. For example, the processor corresponds to the processing unit 210, and the memory corresponds to the storage unit 220. The processor detects a pressure value by operating the pressure sensors Se1 to Se4.

[0113] The care recipient sitting in the wheelchair 630 may feel pain in the buttocks and may shift the position of the buttocks. For example, the state where the buttocks are shifted forward more than usual is called forward shift, and the state where they are shifted to the left or right is called lateral shift. Also, forward shift and lateral shift may occur simultaneously, and the center of gravity may shift obliquely. By using the pressure sensors arranged in the cushion 441 as shown in FIG. 10, the change in the position of the buttocks can be appropriately detected, so it becomes possible to accurately detect forward shift and lateral shift.

[0114] For example, first, the timing when the care recipient transfers to the wheelchair 630 and assumes a normal posture is set as the initial state. In the initial state, since the care recipient sits deeply on the seat surface of the wheelchair 630, it is assumed that the value of the rear pressure sensor Se2 is relatively large. On the other hand, when forward shift occurs, since the position of the buttocks moves forward, the value of the front pressure sensor Se1 becomes large. For example, the processor of the control box 442 may determine that forward shift has occurred when the value of the pressure sensor Se1 increases by a predetermined amount or more compared to the initial state. When the value of the pressure sensor Se1 exceeds a certain threshold, it is determined that the care recipient is sitting on the wheelchair 630, and it may be determined that forward shift has occurred only based on the change in the value of the pressure sensor Se2 without comparing it with the pressure sensor Se1. Also, instead of using the value of the pressure sensor Se1 alone, processing may be performed using the relationship between the values of the pressure sensor Se2 and the pressure sensor Se1. For example, the difference in the voltage values, which are the outputs of the pressure sensor Se2 and the pressure sensor Se1, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used.

[0115] Similarly, when lateral displacement occurs, the position of the buttocks moves in either the left or right direction. Thus, if there is a left displacement, the value of the pressure sensor Se4 increases, and if there is a right displacement, the value of the pressure sensor Se3 increases. Therefore, the processor may determine that a left displacement has occurred when the value of the pressure sensor Se4 has increased by a predetermined amount or more compared to the initial state, and may determine that a right displacement has occurred when the value of the pressure sensor Se3 has increased by a predetermined amount or more compared to the initial state. Alternatively, the processor may determine right and left displacements using the relationship between the values of the pressure sensor Se4 and the pressure sensor Se3. Similar to the example of forward displacement, the difference in voltage values, which are the outputs of the pressure sensor Se4 and the pressure sensor Se3, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used.

[0116] When forward displacement, lateral displacement, etc. are detected, the seat surface sensor 440 transmits information indicating that to the server system 100. The server system 100 transmits the information to, for example, a smartphone used by a caregiver, and notification processing is executed on the smartphone or the like. Further, the control box 442 may include a light emitting unit or the like, and notification to the caregiver may be performed using the light emitting unit. In this way, since changes in the sitting posture in a wheelchair 630 or the like can be easily notified to the caregiver, it becomes possible to suppress the fall of the care recipient.

[0117] FIG. 11 is a diagram showing a terminal device 450 which is a device 200 used for assisting in adjusting a wheelchair position. The wheelchair position is information regarding the position and posture of the care recipient in the wheelchair 630. The wheelchair position may represent the sitting posture described above, or may represent information including the arrangement of cushions and the like. As shown in FIG. 11, in adjusting the wheelchair position, a terminal device 450 having a camera and fixed at a height at which at least the upper body of the care recipient sitting in the wheelchair 630 can be imaged by the camera may be used. Note that the terminal device 450 may be able to image a wider range of the care recipient, for example, may image up to the knees or may image the whole body. The terminal device 450 is arranged at a predetermined position in a nursing facility, for example. After the caregiver transfers the care recipient to the wheelchair 630 and moves the care recipient to the front of the terminal device 450, the wheelchair position is adjusted.

[0118] The terminal device 450 includes a display unit and displays a comparison result between an image captured by the camera and teacher data. The teacher data here is, for example, image data obtained by imaging the care recipient in a state where a highly skilled caregiver has seated the care recipient in the wheelchair 630 in an appropriate posture. For example, the teacher data is registered in advance using the terminal device 450 or the like. Also, the teacher data may be registered in the storage unit 120 of the server system 100. Further, the teacher data is not limited to the image data itself representing an appropriate posture, and may be data to which additional information is added. The additional information here may be information indicating points considered important by a highly skilled caregiver. Also, the teacher data may be the result of skeleton tracking.

[0119] The terminal device 450 (processing unit 210) may output an image in which teacher data subjected to transparency processing is superimposed and displayed on the actual captured image. Also, the terminal device 450 may determine whether the position of the care recipient in the wheelchair 630 is appropriate based on a comparison process between the teacher data and the actual captured image, and output the determination result. Also, when the position of the care recipient is not appropriate, points to be corrected may be presented. The points to be corrected are, for example, parts where the difference between the teacher data and the actual captured image is a predetermined value or more.

[0120] For example, each device 200 shown in FIGS. 10 to 11 is set to an inactive operation mode 0 when the ability of the assisted person represented by the ability information is equal to or greater than a predetermined level, and is set to an active operation mode 1 when the ability is less than the predetermined level. Here, the ability being less than the predetermined level means that the person cannot walk. In this way, the position determination in the wheelchair 630 and the like described above can be executed in an appropriate situation. As a result, when there is an assisted person with a high risk of falling from the wheelchair 630 or the bed 610, the risk of falling of the assisted person can be appropriately reduced.

[0121] <Operation of the device related to the risk of falling> Also, as shown in FIG. 5, even an assisted person who cannot walk may not be bedridden and may perform a starting movement such as getting up, so the risk of falling is high. Even when targeting an assisted person who cannot walk, it is desirable to continue the starting movement detection by each device 200 shown in FIGS. 8 to 9.

[0122] At that time, in a state where the person cannot get up but can walk and a state where the person cannot walk, the latter has lower ability and may have a higher risk of falling. Therefore, each device 200 shown in FIGS. 8 to 9 may operate in operation mode 1 in a state where the person cannot get up but can walk, and may operate in an operation mode 2 different from operation mode 1 in a state where the person cannot walk. For example, when the detection device 430 is used as the device 200, the processing unit 210 of the detection device 430 may detect a starting movement based on the determination result of getting out of bed / being in bed in operation mode 1, and may detect a starting movement based on the determination result of waking / sleeping in operation mode 2. For example, in operation mode 2, the processing unit 210 may determine that there is a possibility of a starting movement when transitioning from the sleep state to the waking state. In this way, in operation mode 2, a starting movement can be detected at an earlier stage than in operation mode 1, so that the risk of falling can be further reduced.

[0123] <Update of ability information> Also, as shown in step S205 of FIG. 7, the ability information acquisition unit 111 of the server system 100 may perform a process of updating the ability information based on the sensing data. For example, the ability information acquisition unit 111 determines a change in the seat holding ability, which is the ability to maintain the sitting state, based on the sensing data.

[0124] For example, the ability information acquisition unit 111 may determine the number of forward slips and lateral slips based on the sensing data. For example, it is determined that the higher the seat holding ability, the fewer the number of forward slips and lateral slips, and the lower the seat holding ability, the fewer the number of forward slips and lateral slips.

[0125] Also, the ability information acquisition unit 111 may obtain the seat holding ability based on the time that the posture can be maintained (hereinafter referred to as the posture holding time). The posture holding time is, for example, the length of a period starting from the timing when a given reference posture is achieved and ending at the timing when the posture of the assisted person has changed by a predetermined amount or more with respect to the reference posture. The posture holding time may be, for example, the time from when the assistant corrects the posture of the assisted person using the terminal device 450 until it is determined as a forward slip or a lateral slip by the seat surface sensor 440. For example, it is determined that the higher the seat holding ability, the longer the posture holding time, and the lower the seat holding ability, the shorter the posture holding time. Also, if the seat holding ability is high, the same posture can be maintained, so a state where the same degree of pressure is applied to each pressure sensor continues. On the other hand, when the seat holding ability decreases, even if it is not determined as a forward slip or a lateral slip, the body frequently tilts or corrects the posture. As a result, a timing when the pressure value decreases (pressure loss) is likely to occur. Therefore, the ability information acquisition unit 111 may estimate the seat holding ability based on the number, frequency, degree of value decrease, and direction of such pressure loss (which of the pressure sensors Se1 to Se4 has a decreased value).

[0126] In the method of this embodiment, the determination of the ability information may be executed using any one of the above, or two or more may be combined. Also, as described above, the ability information may be obtained based on a combination of the sensing data and data other than the sensing data. According to the method of this embodiment, it is possible to reduce the risk of falling for a care recipient who cannot walk and appropriately determine changes in the ability information of the care recipient.

[0127] 2.4 Unable to eat well <Examples of Devices and Operations: Risk of Aspiration> Next, in a state where the ability to eat well has deteriorated, a device 200 that enters an operating state to address the risk of aspiration (narrowly, the risk of aspiration pneumonia) will be described. FIG. 12 is an example of a device 200 used for determining the risk of aspiration during a meal.

[0128] FIG. 12 is a diagram illustrating a swallowing cough detection device 460, which is a device 200 used in a meal scene. As shown in FIG. 12, for the swallowing cough detection device 460, a throat microphone 461 worn around the neck of the care recipient and a terminal device 462 having a camera are used. Note that instead of the terminal device 462, another device having a camera may be used. The throat microphone 461 outputs voice data due to the care recipient's swallowing, coughing, etc. The camera of the terminal device 462 outputs a captured image of the care recipient's meal situation. The terminal device 462 is, for example, a smartphone placed on the table where the care recipient eats. The throat microphone 461 is connected to the terminal device 462 using Bluetooth (registered trademark), etc., and the terminal device 462 is connected to the server system 100 via the gateway 300. However, both the throat microphone 461 and the terminal device 462 may be connectable to the gateway 300, and various modifications of the specific connection mode are possible.

[0129] For example, the processor included in the swallowing cough detection device 460 acquires voice data from the throat microphone 461 and captured images captured using a camera. The processor here corresponds to the processing unit 210 and may be, for example, a processor included in the terminal device 462.

[0130] Based on the voice data of the throat microphone 461, the processor determines the cough and swallowing of the care recipient. A device that detects swallowing using a microphone worn around the neck is described, for example, in US Patent Application No. 16 / 276768 filed on February 15, 2019, titled "Swallowing action measurement device and swallowing action support system". This patent application is hereby incorporated by reference in its entirety into this specification. The processor can detect the number of coughs, the time of coughs (occurrence time, duration, etc.), and whether swallowing has occurred based on the voice data.

[0131] Also, the camera of the terminal device 462 can detect the mouth, eyes of the care recipient, and utensils such as chopsticks and spoons used by the care recipient by imaging the care recipient from the front direction, as shown in FIG. 12 for example. Various methods for detecting these facial parts and objects based on image processing are known, and in this embodiment, known methods can be widely applied.

[0132] For example, based on the captured image of the camera, the processor can determine whether the care recipient's mouth is open, whether food is coming out of the mouth, and whether the food is being chewed. Also, based on the captured image of the camera, the processor can determine whether the care recipient's eyes are open. Further, based on the captured image of the camera, the processor can determine whether chopsticks, spoons, etc. are near the tableware, whether the care recipient is holding them, and whether food is being spilled.

[0133] In the method of this embodiment, based on this information, the situation regarding the swallowing and coughing of the care recipient is estimated. For example, the processor may perform processing based on the detection results of coughing and swallowing and the determination result of the opening and closing of the care recipient's mouth.

[0134] For example, the processor may determine whether choking occurs frequently based on the number of chokes or the time, and output the determination result. For example, the processor may determine that choking occurs frequently when the number of chokes per unit time exceeds a threshold. In this way, the situation regarding choking can be automatically determined.

[0135] Also, the processor may obtain the swallowing time from when the assisted person opens their mouth until swallowing based on the detection result of swallowing and the determination result of the opening and closing of the assisted person's mouth. In this way, for example, even if it is found that the number of swallows has decreased, it is possible to determine specific situations such as whether the action of putting food into the mouth itself has not been performed, or whether swallowing has not occurred after putting food into the mouth. For example, the processor may start counting up the timer when the mouth transitions from a closed state to an open state based on the captured image of the terminal device 462, and stop measuring the timer when swallowing is detected by the throat microphone 461. The time at the stop represents the swallowing time. In this way, it is possible to accurately determine whether the risk of aspiration during meals is high and whether it is a situation where the caregiver should take some action, so that the tacit knowledge of experts can be appropriately utilized.

[0136] For example, the choking and swallowing detection device 460 may output the swallowing time as a processing result to the server system 100. Also, the processor may determine the pace of the meal based on the swallowing time. Further, the processor may determine whether the swallowing time is long based on the change in the swallowing time during one meal (for example, the increase amount or ratio with respect to the swallowing time at the beginning). Alternatively, the processor may obtain the average swallowing time for each of multiple meals for the same assisted person, and determine whether the swallowing time has become longer based on the change in the average swallowing time.

[0137] Moreover, by using the determination result of the opening and closing of the mouth based on the captured image of the terminal device 462, it is possible to determine whether the situation is such that the assisted person does not open their mouth even when the caregiver approaches with a spoon or the like. In this way, in a situation where the assisted person has difficulty opening their mouth, if the swallowing time becomes long, it can be presumed that there is a situation where food remains in the mouth and choking has occurred. Also, by using the captured image to determine whether food is coming out of the mouth, or by using the recognition result of whether the assisted person is chewing food or not, it is possible to determine whether the assisted person is in a situation where they cannot chew the food properly. For example, if the number of chewing times is normal but the swallowing time is long, it is presumed that the situation is such that the person cannot chew properly. Also, if it is determined using the captured image that the eyes are closed, it is possible to determine whether the assisted person seems sleepy.

[0138] Moreover, by performing recognition processing of chopsticks, spoons, etc. using the captured image, it may be determined whether the situation is such that the person is playing with food, cannot hold utensils, is spilling food, etc.

[0139] As described above, by using the swallowing cough detection device 460, various situations during meals can be determined. In the swallowing cough detection device 460 for input data of the present embodiment, each of these determinations may be realized as an application corresponding to tacit knowledge. For example, the swallowing cough detection device 460 may include an application for detecting coughs and swallowing, calculating the swallowing time, an application for detecting frequent coughs, an application for detecting dangerous coughs, an application for determining whether the person is getting sleepy, an application for determining whether the person is playing with food, and so on. And in the present embodiment, based on ability information and the like, the activation / inactivation of each application is controlled. In other words, the above-described various processes may be flexibly set to be executed / not executed. And when the swallowing cough detection device 460 determines that it is a predetermined situation, it transmits information indicating that fact to the server system 100. The server system 100 may, for example, based on the said information, cause the terminal device used by the caregiver to execute a notification process. Further, the server system 100 may output information indicating an appropriate action according to the meal situation detected by the swallowing cough detection device 460 to the device used by the caregiver, and the device may present an action based on the said information. The presentation to the caregiver may be, for example, an audio output to a headset, a display on a display unit such as a smartphone, or a presentation using other methods.

[0140] <Operation of the device regarding the risk of falling> Also, as shown in FIG. 5, even for a care recipient who has difficulty eating, there is a possibility of performing a starting movement such as standing up, so the risk of falling is high. Therefore, even when targeting a care recipient who has difficulty eating, it is desirable to continue the starting movement detection by each device 200 shown in FIGS. 8 to 9. Each device 200 may operate, for example, in operation mode 2 described above. Further, each device 200 may operate in operation mode 3 in which starting movement detection is possible at an earlier stage than operation mode 2.

[0141] <Operation of the device regarding the risk of falling> Even for care recipients who have difficulty eating, it is conceivable to use a wheelchair 630 or the like, so the risk of falling is high. Therefore, even when targeting care recipients who have difficulty eating, it is desirable to continue the processing related to the wheelchair position by each device 200 shown in FIGS. 10 to 11. For example, each device 200 operates in operation mode 1 in the same manner as when targeting care recipients who cannot walk. Also, each device 200 may operate in a mode different from when targeting care recipients who cannot walk. For example, when targeting care recipients who cannot walk, it operates in an operation mode that only performs forward shift and lateral shift determination, and when targeting care recipients who have difficulty eating, it may operate in an operation mode that can execute determination of the possibility of falling in addition to forward shift and lateral shift determination.

[0142] <Update of ability information> Also, as shown in step S205 of FIG. 7, the ability information acquisition unit 111 of the server system 100 may perform processing to update the ability information based on the sensing data. For example, the ability information acquisition unit 111 determines a change in swallowing ability based on the sensing data.

[0143] For example, the swallowing ability acquisition unit 111 may determine that the shorter the swallowing time from when the care recipient opens their mouth until swallowing, the higher the swallowing ability, and the longer the swallowing time, the lower the swallowing ability. For example, when the average swallowing time of the most recent care recipient becomes longer than the average swallowing time of past care recipients by a threshold value or more, the swallowing ability acquisition unit 111 may determine that the swallowing ability has declined. When the average swallowing time does not become longer than the threshold value, the swallowing ability acquisition unit 111 may determine that the swallowing ability has not declined (been maintained). When the average swallowing time becomes shorter, the swallowing ability acquisition unit 111 may determine that the swallowing ability has recovered. Also, if the change in the average swallowing time before and after changing the eating form is less than or equal to the threshold value, the swallowing ability acquisition unit 111 may determine that the swallowing ability has recovered. Further, the swallowing ability acquisition unit 111 may obtain ability information based on the combination of the swallowing ability and the sitting posture holding ability. For example, the swallowing ability acquisition unit 111 may obtain ability information based on the usage time and usage frequency of the wheelchair 630. For example, since a care recipient with a decreased sitting posture holding ability may shift to eating in bed, the ability to use the wheelchair 630 indicates a high sitting posture holding ability. For example, it is determined that the higher the usage frequency of the wheelchair, the higher the ability, and the lower the usage frequency, the lower the ability. Also, as described above, the swallowing cough detection device 460 may be capable of executing various applications that perform different processes. For example, the swallowing cough detection device 460 first activates an application that detects coughs and swallowing and calculates the swallowing time, and the swallowing ability acquisition unit 111 may obtain ability information based on the output of the application. When the ability information changes, another application such as an application that detects frequent coughing is changed from inactive to active. In this way, based on the ability information, the processing content executed by the swallowing cough detection device 460 can be appropriately controlled. Also, when the number of active applications increases, the swallowing ability acquisition unit 111 may obtain the swallowing ability based on more sensing data by combining the outputs of the respective applications.

[0144] 2.5 Bedridden <Examples of Devices and Operations: Pressure Ulcer Risk> Next, in a state where the ability further deteriorates and the care recipient becomes bedridden, a device 200 that operates to address the risk of pressure ulcers will be described. FIGS. 13 to 14 are examples of the device 200 used for pressure ulcer risk determination.

[0145] FIG. 13 is a diagram illustrating a bed position detection device 470, which is the device 200 arranged around the bed 610. As shown in FIG. 13, the bed position detection device 470 includes a first terminal device 471 fixed to the footboard side of the bed 610, a second terminal device 472 fixed to the side rail of the bed 610, and a display 473 fixed on the opposite side of the second terminal device 472. Note that the display 473 is not limited to being fixed to the bed 610 and may be arranged at other positions where it can be naturally viewed by a caregiver who adjusts the bed position. For example, the display 473 may be fixed to a wall surface or fixed to a stand that stands on the floor surface. Note that the display 473 can be omitted. Also, either the first terminal device 471 or the second terminal device 472 may be omitted. For example, in the following, an example will be described in which the first terminal device 471 is used for adjusting the bed position and the second terminal device 472 is used for diaper changing, but this is not limiting.

[0146] The first terminal device 471 and the second terminal device 472 are devices such as smartphones having cameras, for example. The first terminal device 471 directly transmits a captured image to the server system 100. The second terminal device 472 transmits a captured image of the camera to the server system 100 directly or via the first terminal device 471. The display 473 receives the image transmitted from the server system 100 directly or via another device such as the first terminal device 471 and displays the received image. Note that the first terminal device 471 and the second terminal device 472 may have a depth sensor instead of or in addition to the camera. That is, these devices may output a depth image.

[0147] For example, in bed position adjustment, a teacher data registration process and a position adjustment process using the teacher data may be executed. The teacher data is information registered by, for example, a skilled caregiver. When an unskilled caregiver adjusts the bed position, the caregiver selects the teacher data and adjusts the bed position so that the state of the actual care recipient is close to that of the teacher data. For example, the first terminal device 471 acquires a captured image of the care recipient to be adjusted lying on the bed (including the state of cushions, etc.), and the display 473 displays an image representing the comparison result between the captured image and the teacher data. In this way, it becomes possible to perform the same position adjustment as a skilled person regardless of the caregiver's skill level. For example, a skilled person lays the care recipient on the bed 610, positions the care recipient in a position suitable for preventing bedsores, etc., and then uses the first terminal device 471 to capture an image of the target care recipient. After confirming that the bed position is appropriate, the skilled person selects the registration button. As a result, the still image displayed when the registration button is operated is transmitted to the server system 100 as teacher data. In this way, it becomes possible to register the position considered preferable by the skilled person as teacher data. At this time, additional information may be added, which is the same as in the above example of the wheelchair position.

[0148] Also, when the caregiver actually adjusts the bed position, first, the first terminal device 471 is activated to start image capture. For example, the caregiver activates the first terminal device 471 by voice, and the display 473 displays the moving image captured by the first terminal device 471. Also, the bed position detection device 470 may receive a process of selecting teacher data by the caregiver. The processing unit 110 determines the teacher data based on the user operation and performs control to display the teacher data on the display 473.

[0149] Alternatively, the processing unit 110 may perform a process of automatically selecting teacher data based on the similarity determination between the attributes of the care recipient whose bed position is to be adjusted and the attributes of the care recipient imaged in the teacher data. The attributes here include information such as the age, gender, height, weight, medical history, and medication history of the care recipient.

[0150] Alternatively, the bed position detection device 470 may perform a process of automatically selecting teacher data based on a comparison process between the attributes of the care recipient whose bed position is to be adjusted and the additional information included in the teacher data. For example, assume that the additional information of the teacher data includes text such as "For care recipients showing a tendency of XX, it is advisable to adjust the left shoulder to YY". In this case, when the care recipient to be adjusted corresponds to XX, the teacher data is more likely to be selected. For example, an assistant who performs bed position adjustment may input information identifying the care recipient into the first terminal device 471, and the bed position detection device 470 may identify the attributes of the care recipient based on the information.

[0151] The bed position detection device 470 may output an image that superimposes and displays teacher data subjected to a transparency process on a real-time captured image captured by, for example, the first terminal device 471. At this time, the additional information of the teacher data may be displayed in a recognizable manner. For example, when it is detected that the assistant has spoken "Tell me the points" using a microphone of a headset or the like, the bed position detection device 470 may output the text as voice from the headset via, for example, the server system 100.

[0152] The bed position detection device 470 may determine whether it is OK or NG based on, for example, the similarity between the image captured during position adjustment and the teaching data, and output the determination result. The determination result is displayed on the display 473 directly or via the server system 100. Further, the bed position detection device 470 may perform a process of displaying the specific points determined as NG. For example, the server system 100 or the bed position detection device 470 may compare the image captured by the first terminal device 471 with the teaching data, and perform a process of highlighting the locations determined to have a large difference.

[0153] In this way, it becomes possible to compare and present the bed position of the care recipient with the ideal bed position, and to present information for realizing the ideal bed position.

[0154] Further, the bed position detection device 470 may be used to assist with diaper changing. As tacit knowledge in diaper changing, it has been found that skilled persons attach importance to the following points. A. Whether in the lateral lying position B. Whether the position of the diaper is appropriate C. Whether the pad has come out of the diaper D. Whether the diaper is properly worn

[0155] Therefore, the processing unit 210 of the bed position detection device 470 determines whether the above points A to D are satisfied, and transmits the determination result to the server system 100. Here, the processing unit 210 corresponds to, for example, the processor of the second terminal device 472. This makes it possible to appropriately perform diaper changing regardless of the skill level of the caregiver.

[0156] For example, the second terminal device 472 performs skeleton tracking processing on each image constituting a moving image obtained by imaging the assisted person using a camera, and outputs, as a processing result, an image in which the skeleton tracking result is superimposed on the original image. The processing result may be displayed on, for example, the display 473. In this way, the caregiver can check the display 473 in a natural posture while changing the diaper of the assisted person.

[0157] In consideration of the case where diaper changing is performed at night, the second terminal device 472 may include an illumination unit. Also, in consideration of the privacy of the assisted person, a depth sensor or the like may be used instead of a camera. The depth sensor may be a sensor using the ToF (Time of Flight) method, a sensor using structured illumination, or a sensor of another method.

[0158] FIGS. 15A and 15B are examples of images displayed on the display 473 when changing a diaper. As described above, each image includes the assisted person and the skeleton tracking result of the assisted person.

[0159] In the state of FIG. 15A, the assisted person is stable in the lateral recumbent position, and the camera of the second terminal device 472 is in a state of imaging the assisted person straight from the back side. For example, in FIG. 15A, the difference between the front-rear direction of the assisted person's body and the optical axis direction of the camera is small. As a result, as shown in FIG. 15A, a large number of points to be detected by skeleton tracking are detected.

[0160] On the other hand, FIG. 15B is less stable in posture than FIG. 15A and is in a state of being about to fall backward. Since the camera of the second terminal device 472 is in a state of imaging the assisted person obliquely from the rear, the number of points detected by skeleton tracking decreases. For example, the point corresponding to the waist is not detected because it is hidden by the diaper or the like.

[0161] Therefore, the second terminal device 472 may determine whether or not the user is in the side-lying position shown in A above based on the result of skeleton tracking. For example, when a point corresponding to a specific part such as the waist is detected by skeleton tracking, the second terminal device 472 may determine that the user is in the side-lying position. However, the presence or absence of detection of points other than the waist and the relationship between a plurality of points may be used for the determination of the side-lying position, and the specific method is not limited to this.

[0162] Further, the bed position detection device 470 continuously detects the diaper area in the image by performing object tracking processing based on the moving image captured by the camera of the second terminal device 472. Since object tracking is well-known, detailed description thereof is omitted. For example, in FIGS. 15A and 15B, the diaper area ReD is detected.

[0163] The bed position detection device 470 may determine whether or not the position of the diaper shown in B above is appropriate based on, for example, the relationship between the result of skeleton tracking and the diaper area ReD detected by object tracking. For example, considering the position where the diaper is worn, it is determined whether or not there is a predetermined positional relationship between the position of the waist detected by skeleton tracking and the diaper area ReD. For example, the processing unit 210 may determine that the position of the diaper is appropriate when a straight line including two points corresponding to the pelvis passes through the diaper area ReD. Alternatively, machine learning may be performed in which the result of skeleton tracking and the detection result of the diaper area ReD are extracted as feature amounts from teacher data by a skilled person, and the feature amounts are used as input data. The learned model is, for example, a model that outputs the probability that the position of the diaper is appropriate when receiving the result of skeleton tracking and the detection result of the diaper area ReD.

[0164] Further, the bed position detection device 470 may determine whether a pad has come out of the diaper as shown in C above, based on the horizontal length of the diaper area ReD. Normally, since the pad is contained inside the diaper, the length of the diaper area ReD in the image corresponds to the length of the diaper body. Note that the assumed size of the diaper area ReD can be estimated based on the type and size of the diaper and the optical characteristics of the camera of the second terminal device 472, etc. On the other hand, when the pad protrudes, the length of the diaper area ReD in the image increases by that amount. Therefore, when the length of the diaper area ReD detected from the image is greater than or equal to a predetermined threshold value compared to the assumed length, the bed position detection device 470 determines that the diaper has come out of the pad and is inappropriate.

[0165] Further, the bed position detection device 470 may determine whether the diaper shown in D above is properly worn by detecting a tape that fixes the diaper in a worn state. Normally, a member of a color different from that of the diaper body is used for the tape. As an example, the diaper body is white and the tape is blue. Also, how the tape should be fixed in order to properly wear the diaper is known from the structure of the diaper. Therefore, the processing unit 210 detects the tape area in the image based on the color, and determines whether the diaper is properly worn based on the relationship between the tape area and the diaper area ReD, or the relationship between the tape area and the position of the waist etc. detected by skeleton tracking. Note that when a plurality of diapers of different manufacturers and types are used, the bed position detection device 470 may acquire information for identifying the diaper and determine whether the diaper is properly worn based on the type etc. of the identified diaper.

[0166] For example, the bed position detection device 470 determines OK or NG for each of A to D above and transmits the determination result to the server system 100. The server system 100 transmits the determination result to the display 473 etc. Also, when it is NG, the bed position detection device 470 may highlight a portion with a large deviation from the correct data.

[0167] By doing so, not only can the caregiver be made to assume a position that reduces the risk of bedsores, but also the tacit knowledge in diaper changing can be appropriately utilized to enable the caregiver to appropriately perform diaper changing. In the work of diaper changing, it is necessary to move the body of the care recipient. For example, the caregiver performs operations such as temporarily placing the care recipient in a lateral position so that the diaper can be easily placed, and raising the feet to measure the diaper. Therefore, when the diaper changing is completed, there is a possibility that the care recipient is not in a suitable position when lying on the bed horizontally. Thus, when it is detected that the diaper changing has been completed, the bed position detection device 470 may automatically execute a process for adjusting the above-described bed position. For example, the first terminal device 471 starts imaging the care recipient, and the display 473 superimposes and displays teacher data subjected to a transparency process on the real-time captured image captured by the first terminal device 471.

[0168] In the above, an example of using the first terminal device 471 such as a smartphone or the second terminal device 472 for bed position adjustment support and diaper changing support has been described, but the specific device 200 is not limited to this.

[0169] FIG. 14 is a diagram illustrating an example of a glasses-type device 480 such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses, which is another example of the device 200 used for bed position adjustment support and diaper changing support. The glasses-type device 480 has, for example, a camera that images a region corresponding to the user's field of view. In the glasses-type device 480, a part or all of the lens portion serves as a display, and by transmitting light from the outside world or by displaying an image corresponding to the user's field of view captured by the camera, the user can visually recognize the situation of the outside world. Furthermore, the glasses-type device 480 adds and displays some information on the user's field of view by using the display.

[0170] Even when using the glasses-type device 480, it is the same in that an image of the user on the bed 610 can be acquired. Therefore, the glasses-type device 480 may perform processing to assist in bed position adjustment by superimposing and displaying the captured image and the teacher data as described above. Further, the glasses-type device 480 may perform processing to assist in diaper changing by determining the diaper area ReD. Also, the point that bed position adjustment may be started when diaper changing is completed is the same when using the glasses-type device 480. For example, when completion of diaper adjustment is detected, the correct data related to bed position adjustment may be transparently displayed on the display unit of the glasses-type device 480.

[0171] Further, the glasses-type device 480 may perform processing to automatically detect the presence or absence and the range of pressure ulcers based on the captured image of the skin of the care recipient. By doing so, it becomes possible to determine whether a pressure ulcer has actually occurred and, if so, its state. For example, a learned model is generated by machine learning based on training data in which an image of the care recipient is associated with correct data for specifying the pressure ulcer area given by an expert such as a doctor. The glasses-type device 480 determines the pressure ulcer based on the processing based on the learned model.

[0172] Mattresses and pillows that can detect pressure and automatically change positions are also known. Regarding pressure detection, it is performed using pressure sensors in the same manner as the bedside sensor 420 and the detection device 430. Also, the mattresses and pillows here may be those that encourage the cared-for person to turn over by adjusting the height (thickness) of each part using air or the like. Further, the device 200 for detecting pressure and the device 200 for promoting automatic position change may be different. For example, when the detection device 430 determines that the same posture is continuing based on the pressure value, it transmits information indicating that fact to the server system 100. The server system 100 may prompt the cared-for person to change positions by controlling the mattress or pillow based on the said information. Even in this way, it is possible to reduce the risk of bedsores for the bedridden cared-for person. Also, the control methods for these mattresses and pillows are not limited to one type. For example, even among the same bedridden people, detailed ability information (such as ADL) is determined, and the way of prompting a position change may vary according to the level of the said ability information. For example, in the case of a cared-for person whose ability has extremely declined, it is difficult to maintain a posture, etc., so there may be cases where the posture changes excessively when prompted to change positions. Therefore, the mattress and pillow have a plurality of operation modes in which the frequency of prompting a position change and the amount of air when prompting a position change are different, and the operation mode may be controlled according to the level of ability information. Also, the mattress, etc. may operate in a mode that improves the sleep situation (for example, suppressing snoring by encouraging the side-lying position) while the ability is high, and operate in a mode that reduces the risk of bedsores when the cared-for person becomes bedridden, and operation mode control according to ability information may be executed including states other than being bedridden.

[0173] When targeting bedridden care recipients, tacit knowledge regarding terminal care may be used. The device 200 on which the application corresponding to the tacit knowledge operates may be a terminal device such as a smartphone. For example, the processing unit 210 of the device 200 acquires, as input data, five types of information: the intake amount or intake ratio for each type of food in each meal (which may be a main dish or a side dish, or may be for each ingredient such as meat or fish), the intake amount of water, the timing of intake, information regarding diseases, and weight (or BMI). These input data may be input by the caregiver. Alternatively, some of the input data may be automatically acquired by the device 200 by using an automatic meal recording device.

[0174] Based on the input data, the processing unit 210 outputs output data indicating whether terminal care should be started after a predetermined period and whether it is the timing to change the content of care after the start of terminal care. For example, machine learning may be performed based on training data to which correct data by an expert is assigned for the input data. In this case, the processing unit 210 obtains output data by inputting the input data into the learned model. Other machine learning methods such as SVM may be used, or methods other than machine learning may be used.

[0175] Terminal care here refers to the assistance provided to care recipients who are considered likely to die in the near future. Terminal care differs from normal assistance in that it emphasizes the alleviation of physical and mental pain and the support of a dignified life for the care recipient in question. Also, during the provision of terminal care, as the condition of the care recipient changes over time, the assistance suitable for the patient in question may also change. That is, by presenting the start timing of terminal care and the timing of changing the assistance content during terminal care, it becomes possible to provide appropriate assistance to the care recipient until the end. For example, skilled caregivers have tacit knowledge to estimate the timing and care content for which terminal care is necessary from various perspectives such as the amount of food intake, and by digitizing this tacit knowledge, other caregivers can also provide appropriate terminal care.

[0176] When the ability represented by the ability information has declined to a state representing bedridden, the processing unit 210 of the device 200 starts the determination based on the input data and outputs an analysis result screen representing the determination result to the server system 100. Note that the analysis result screen may be displayed on the display unit 240 of the device 200.

[0177] FIG. 16 is a diagram showing an example of an analysis result screen. The analysis result screen may include the time-series change of the feature amount obtained based on the input data and the determination result as to whether or not the bedside care should be started after a predetermined period. The feature amount here may be information determined to be important among the input information, such as the moving average of the food intake amount, or may be information calculated based on the above five input information. For example, when an NN is used, the feature amount may be the output of a given intermediate layer or output layer. For example, the input data includes the measured values of the main dish intake amount, the water amount, and the BMI up to February 13, 2020. The processing unit 210 may estimate the transition of the main dish intake amount, the water amount, and the BMI after February 14, 2020 based on the learned model. The analysis screen may include a graph representing the time-series change of the measured values and the estimated values for these three items. Note that in FIG. 16, a graph of the moving average of these values for seven days is illustrated. In this way, it becomes possible for the caregiver to easily grasp the transition of the items important in the bedside care. Note that, as described above, the input data may include other items, and the information displayed on the analysis result screen is not limited to the example of FIG. 16.

[0178] Also, the analysis result screen may display the period during which bedside care may be performed. In the example of FIG. 16, the text "There is a possibility of bedside care from March 14, 2020" is displayed, and the corresponding period in the graph is displayed in a distinguishable manner using a background color different from that of the other periods. In this way, since the timing and period when bedside care is estimated to be necessary are clearly shown, it becomes possible to appropriately present information regarding bedside care to the user.

[0179] <Operation of the device regarding the risk of falling> Also, as shown in FIG. 5, since the caregiver who is bedridden has a low probability of spontaneously starting movements such as getting up, the risk of falling is assumed to be low. Therefore, when targeting bedridden care recipients, each device 200 shown in FIGS. 8 to 9 may be set to the inactive operation mode 0.

[0180] <Operation of the device regarding the risk of falling> On the other hand, even for bedridden care recipients, since it is conceivable to use a wheelchair 630 or the like, the risk of falling is high. Therefore, even when targeting bedridden care recipients, it is desirable to continue the processing regarding the risk of falling by each device 200 shown in FIGS. 10 to 11. At this time, since it is difficult for the care recipient to change their posture spontaneously, the risk of bedsores increases. Therefore, the seat surface sensor 440 shown in FIG. 10 may operate in operation mode 3 to determine the level of the risk of bedsores in the wheelchair 630 in addition to determining forward or lateral displacement and the risk of falling. For example, the seat surface sensor 440 may determine that there is a risk of bedsores when a state where the change in the pressure value is small has continued for a predetermined time or more. Similarly, the terminal device 450 shown in FIG. 11 may operate in an operation mode that presents a position for suppressing bedsores to the caregiver.

[0181] <Operation of the device regarding the risk of aspiration> Even bedridden care recipients may continue to eat orally. Therefore, the swallowing mucus detection device 460 shown in FIG. 12 operates even when targeting bedridden care recipients. Note that the swallowing mucus detection device 460 may operate in the same operation mode whether targeting care recipients who are not bedridden but have difficulty eating or bedridden care recipients. Also, the swallowing mucus detection device 460 may operate in an operation mode that can reduce the risk of aspiration more when targeting bedridden care recipients compared to when targeting care recipients who are not bedridden but have difficulty eating. For example, the swallowing mucus detection device 460 may make it easier to detect the risk of aspiration by lowering the threshold value used for comparison with the swallowing time. Alternatively, the swallowing mucus detection device 460 may perform an operation to reduce the risk of aspiration by adding a determination regarding the eating form when targeting bedridden care recipients. Alternatively, when targeting bedridden care recipients, the swallowing mucus detection device 460 may perform an operation to reduce the risk of aspiration by adding a determination regarding the presence or absence of dangerous mucus in addition to processes such as normal mucus detection and calculation of the swallowing time. Also, the swallowing mucus detection device 460 may add a process to determine whether mucus occurs frequently or a process to determine whether the care recipient seems sleepy.

[0182] Also, as described above in the above-mentioned end-of-life care, when targeting bedridden care recipients, automatic recording of the amount of food intake may be performed. For example, the terminal device 462 of the swallowing mucus detection device 460 may capture images of the food before and after the meal and automatically record the amount of food intake based on the difference. That is, the swallowing mucus detection device 460 may operate in an operation mode that includes automatic recording of the amount of food intake when targeting bedridden care recipients. Also, a device different from the swallowing mucus detection device 460 may be used as the device 200 for automatically recording the amount of food intake.

[0183] <Update of ability information> Also, as shown in step S205 of FIG. 7, the ability information acquisition unit 111 of the server system 100 may perform a process of updating the ability information based on the sensing data. For example, the ability information acquisition unit 111 performs an evaluation regarding bedsores based on the sensing data, and determines a change in ability based on the evaluation result.

[0184] For example, the ability information acquisition unit 111 determines the degree of dispersion of the pressure value using the detection device 430 or the like, and if pressure is applied for a predetermined time or more at a fixed location, it determines that the ability to realize a posture change for suppressing bedsores is not available and the ability is low.

[0185] Also, the ability information acquisition unit 111 may determine the number of times determined as NG during positioning adjustment and the number of times the position of the diaper is inappropriate based on the information from the bed position detection device 470, and obtain the ability information based on these numbers of times. For example, the fewer the number of times determined as NG, or the fewer the number of times the position of the diaper is inappropriate, the higher the ability is determined, and the more the number of times, the lower the ability is determined.

[0186] Also, the ability information acquisition unit 111 may update the ability information based on the presence or absence of bedsores output from the glasses-type device 480 such as MR glasses. For example, when there are no bedsores, the ability is determined to be higher than when there are bedsores.

[0187] 2.6 Summary As described above, specific examples and operation examples of the device 200 have been described with reference to FIGS. 8 to 14 and the like. As described above, each device 200 switches the operation mode between at least the operation mode 0 which is inactive and the operation mode 1 which is active according to the ability information. Also, as described above, the operation mode in the active case is not limited to one, and a plurality of operation modes with different processing contents may be set.

[0188] FIG. 17 shows an example of the operation modes of each device 200. As shown in FIG. 17, the imaging device 410 that detects the risk of falling operates in operation mode 1 for the care recipient who cannot get up, operates in operation mode 2 for the care recipient who cannot walk and the care recipient who cannot eat well, and operates in operation mode 0 which is inactive for the bedridden care recipient.

[0189] For example, operation mode 1 may be a mode in which it is determined that there is a movement when a sitting position is detected, and operation mode 2 may be a mode in which it is determined that there is a movement when awakening is detected. Since a movement is detected at an earlier stage in operation mode 2 than in operation mode 1, the risk of falling can be reduced more.

[0190] Also, the seat surface sensor 440 that detects the risk of falling operates in operation mode 0 which is inactive for the care recipient who cannot get up, operates in operation mode 1 for the care recipient who cannot walk, operates in operation mode 2 for the care recipient who cannot eat well, and operates in operation mode 3 for the bedridden care recipient.

[0191] For example, operation mode 1 may be a mode that only performs forward and lateral displacements, operation mode 2 may be a mode in which a determination of the possibility of falling is added, and operation mode 3 may be a mode in which a pressure ulcer determination for a wheelchair is added. In this example, since the determination target is added as the ability decreases, it is possible to appropriately respond to the increase in risk associated with the change in ability.

[0192] Also, the swallowing cough detection device 460 that detects the risk of aspiration operates in operation mode 0 which is inactive for the care recipient who cannot get up and the care recipient who cannot walk, operates in operation mode 1 for the care recipient who cannot eat well, and operates in operation mode 2 for the bedridden care recipient.

[0193] For example, operation mode 1 is a mode that makes a determination based on the swallowing time, and operation mode 2 may be a mode in which options such as the eating form are added in addition to operation mode 1. In this example, since the determination target is added as the ability decreases, it is possible to appropriately respond to the increase in risk associated with the ability change.

[0194] Also, the bed position detection device 470 that detects the risk of pressure ulcers operates in operation mode 0 that is inactive for care recipients who cannot get up, care recipients who cannot walk, and care recipients who cannot eat well, and operates in operation mode 1 for bedridden care recipients.

[0195] As described above, according to the method of the present embodiment, it is possible to appropriately determine the presence or absence of a risk and perform processing for reducing the risk in a scene with high necessity in consideration of the relationship between the ability and various risks. Note that FIG. 17 shows an example of the relationship among the device 200, the ability information, and the operation mode, and the method of the present embodiment is not limited thereto.

[0196] When setting the operation mode of the device 200 based on the ability information, it is assumed that as the ability of the care recipient decreases, the functions of the device 200 are added to compensate for it. However, the first device among the devices 200 is set to an operation mode in which the functions used increase as the ability value decreases in the range where the ability value represented by the ability information is equal to or higher than a predetermined threshold, and in the range where the ability value is less than the predetermined threshold, it may be set to an operation mode in which the functions used are fewer than in the range where the ability value is equal to or higher than the predetermined threshold. That is, a part of the device of the present embodiment operates in a direction of increasing functions to compensate for a certain degree of ability decrease, but may shift in a direction of decreasing functions when a certain level or more of ability decrease is observed. In this way, it is possible to appropriately consider the necessity of each tacit knowledge according to the ability, and reduce the processing load by making the applications corresponding to the tacit knowledge with low necessity inactive.

[0197] The first device here may be the device 200 used to determine the risk of falling when the assisted person starts to move. For example, the first device may be at least one of the imaging device 410 shown in FIG. 8, the bedside sensor 420 shown in FIG. 9, and the detection device 430 shown in FIG. 9. In the example of FIG. 17, within the range of "unable to get up", "unable to walk", and "unable to eat properly", the operation modes of these devices 200 change in the direction of adding processing content, while in the case of the more incapacitated "bedridden", an operation mode corresponding to non-activation is set. In this way, it is possible to suppress the continuation of the operation of the device whose necessity has decreased. Note that the detection device 430 may be used for the watch-care determination in the "bedridden" state, and it is not necessary for all of the devices 200 used to determine the risk of falling when starting to move to reduce their functions in a state of low ability values.

[0198] 3. Device Control Based on Scene Information and Device Type Information Also, in the method of this embodiment, information other than the ability information may be used to set the operation mode of the device 200. Hereinafter, the scene information and the device type information will be described.

[0199] 3.1 Overview FIG. 18 is a sequence diagram for explaining the operations of the server system 100 and the device 200, and is a diagram for explaining an example in which the operation mode of the device 200 changes based on the ability information, scene information, and device type information of the assisted person.

[0200] First, in step S301, the server system 100 performs a process of transmitting data including the ability information, scene information, and device type information to the device 200. Note that one of the scene information and the device type information may be omitted. FIG. 18 shows an example in which data with an ADL index value of 2, a scene ID for specifying the scene of 0, and a device type ID for specifying the device type of xx is transmitted.

[0201] In step S302, the device 200 controls the activation / inactivation of the installed vendor apps based on the acquired capability information, scene information, and device type information. For example, the storage unit 220 of the device 200 may store table data associating the ADL index value, scene ID, device type ID, and the activation / inactivation of each application. The processing unit 210 of the device 200 determines the activation / inactivation of each application by extracting a record that matches the received data from the table data. Also, as will be described later with reference to FIGS. 19, 20, and 22, each device 200 may store an algorithm for determining an operation mode based on the capability information, scene information, and device type information. FIG. 18 shows an example in which, among vendor apps 1 to 3, vendor apps 1 and 2 are activated and vendor app 3 is inactivated.

[0202] After the processing of step S302, the device 200 executes the processing according to vendor app 1 and the processing according to vendor app 2. In step S303, the device 200 transmits the processing result to the server system 100. The processing result here corresponds to the result of a judgment executed using the tacit knowledge of an expert. Also, the processing result here may include a log of sensing data detected by the device 200.

[0203] In step S304, the server system 100 executes control based on the processing result received from the device 200. For example, the processing unit 110 may identify the control target device and perform processing to transmit a control signal for operating the control target device.

[0204] Also in step S305, the server system 100 executes a process of updating the ability information, scene information, and device type information of the assisted person. The update process of the ability information is as described above. Also, the scene information acquisition unit 112 obtains scene information based on at least one of the log of the sensing data acquired from the device 200, information about the assisted person such as attributes, and information about the assistant such as a schedule. Also, the device type information acquisition unit 113 acquires, as device type information, information representing the types of other devices 200 used together with the target device 200. Specific examples of the process will be described later.

[0205] In step S306, the server system 100 performs a process of transmitting data including the updated ability information, scene information, and device type information to the device 200. FIG. 18 shows an example in which the scene ID is changed from 0 to 1.

[0206] In step S307, the device 200 controls the activation / inactivation of the installed vendor apps based on the acquired data. For example, as described above, the device 200 determines the activation / inactivation of each vendor app based on the table data. In the example of FIG. 18, vendor apps 1 and 2 maintain their active states, and vendor app 3 is changed from inactive to active. As a result, after step S307, the device 200 transitions to a state in which it operates in an operation mode in which all of vendor apps 1 to 3 are active. The operations after step S307 are, for example, the same as steps S303 - S306.

[0207] Hereinafter, an example of setting the operation mode based on the scene information and an example of setting the operation mode using the device type information will be described respectively.

[0208] 3.2 Specific Example of Processing Based on Scene Information The server system 100 may request scene information for identifying the scene of assisting the person in need of care, and transmit the requested scene information to the device. Then, based on the ability information and the scene information, the device 200 determines which of the plurality of operation modes to operate in. For example, the storage unit 220 of the device 200 stores information associating the ability information, the scene information, and the operation mode. The processing unit 210 obtains the operation mode based on the said information, the ability information, and the scene information acquired from the server system 100. The information associating the ability information, the scene information, and the operation mode may be, for example, table data associating the value of the ability information, the value of the scene information, and the active / inactive state of each application. Alternatively, the information associating the ability information, the scene information, and the operation mode may be an algorithm for obtaining the active / inactive state of each application based on the ability information and the scene information. In this way, it becomes possible to operate the appropriate device 200 according to the assistance scene, in other words, to execute assistance using appropriate tacit knowledge.

[0209] For example, the scene information acquisition unit 112 may request information about the caregiver who executes the care for the person in need of care as the scene information. Specifically, the scene information may be information about the number of caregivers capable of executing the care for the person in need of care, or information about the years of service, proficiency, qualifications held, etc. Hereinafter, an example using the number of caregivers will be described. In this case, the device 200 determines the operation mode according to the number of caregivers.

[0210] The device 200 here may be, for example, the imaging device 410 described above with reference to FIG. 8. For example, when there are a certain number or more of caregivers in a living room or the like where the care recipient conducts activities together, the caregivers can follow each other. Therefore, when the caregiver visually determines the start of movement of the care recipient, or when the caregiver determines that there is a risk of falling, appropriate intervention can be carried out. The intervention here means, for example, moving close to the care recipient who is about to start moving and supporting the movement such as supporting the body as needed. In this case, the necessity for the imaging device 410 to become active is relatively low.

[0211] On the other hand, when the number of caregivers located in the target space is small, the tasks to be executed by a single caregiver increase. Therefore, it is difficult to observe the movement of the care recipient in detail, and there is a risk that appropriate intervention at the right timing is difficult. In this case, the necessity for the imaging device 410 to become active becomes relatively high.

[0212] In view of the above, by determining the operation mode of the imaging device 410 using the scene information in addition to the ability information, it becomes possible to operate the imaging device 410 appropriately.

[0213] FIG. 19 is a flowchart for explaining the determination process of the operation mode in the imaging device 410. First, in step S401, the processing unit 210 of the imaging device 410 performs a process of identifying the target care recipient. For example, the processing unit 210 identifies the care recipient by performing face recognition processing based on the captured image.

[0214] In step S402, the processing unit 210 acquires the ability information of the care recipient. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 of FIG. 18 from the server system 100.

[0215] In step S403, the processing unit 210 determines whether the ability of the care recipient has declined to a state where they are unable to get up based on the ability information. If the ability has not declined (step S403: No), since the risk of falling is low, in step S404, the processing unit 210 sets the operation mode of the imaging device 410 to mode 0 corresponding to non-active. Note that the ability information in this embodiment is not limited to the index value of ADL, and may be more detailed information, for example, information representing the way of getting up described above, information representing seat holding ability, swallowing ability, walking ability, etc. Furthermore, it is possible to use various sensing data when obtaining the seat holding ability, etc., and various modifications can also be made to the algorithm for obtaining the seat holding ability, etc. from the sensing data. In the determination of whether the ability of the care recipient has declined to a state where they are unable to get up shown in step S403, as described above, processing using detailed ability information with a larger data volume compared to the index value of ADL may be executed. In this case, since the processing load of step S403 increases, the processing may be executed in the server system 100.

[0216] If the ability of the care recipient has declined to a state where they are unable to get up (step S403: Yes), in step S405, the processing unit 210 identifies the number of caregivers based on the scene information. For example, the scene information acquisition unit 112 of the server system 100 may determine the number of caregivers based on the captured image, or may determine the number of caregivers based on the care schedule (for example, the work schedule) in a care facility or the like. Alternatively, by having the caregivers wear RFID (radio frequency identifier) tags or the like and providing a reading device at the entrance and exit of the target space, information representing the number of caregivers in the target space may be obtained. The processing unit 210 of the device 200 executes the processing shown in step S405 by obtaining this information from the server system 100.

[0217] In step S406, the processing unit 210 makes a determination based on the scene information. Specifically, the processing unit 210 determines whether the number of assistants is equal to or less than a predetermined threshold. If the number is more than the predetermined threshold (step S406: No), since personnel capable of handling the risk of falling are ensured, the process proceeds to step S404, and the processing unit 210 sets the operation mode of the imaging device 410 to mode 0 corresponding to non - active.

[0218] If the number is equal to or less than the predetermined threshold (step S406: Yes), the assistants are insufficient, and the support of the device 200 is important for suppressing the risk of falling. Therefore, in step S407, the processing unit 210 sets the operation mode of the imaging device 410 to mode 1 corresponding to active. Although FIGS. 19 illustrate the form in which the processing unit 210 performs steps S405 and S406, the present invention is not limited thereto. For example, the server system 100 may execute steps S405 and S406, and the processing unit 210 may receive only the results thereof and determine whether to transition to step S404 or step S407.

[0219] Also, the scene information is not limited to information about the assistants. For example, the scene information acquisition unit 112 of the server system 100 may obtain information about the care recipient as scene information. Specifically, the scene information may be attribute information representing the attributes of the care recipient. The attribute information includes the age, height, weight, gender, medical history, medication history, etc. of the care recipient. For example, the scene information acquisition unit 112 obtains information indicating whether the care recipient has dementia as scene information.

[0220] In order to reduce the risk of falling at the start of movement, it is important for the assistant to call out and temporarily stop the movement. However, dementia patients may not respond even when called out by an assistant. However, even for dementia patients, there is a finding that they often remember the voices of people in close relationships such as family members, and there is a high probability that they will respond to calls from family members.

[0221] Therefore, the imaging device 410 may have an operation mode in which it stores voice data obtained by recording the voices of family members or video data in which family members issue warnings, and outputs the voice data or video data using a speaker (not shown). For example, as shown in FIG. 19, the processing unit 210 of the imaging device 410 determines active / inactive based on the ability information and the scene information which is the number of caregivers. Then, when the imaging device 410 is set to active, the processing unit 210 determines whether the care recipient specified in step S401 is a dementia patient. When the care recipient is a dementia patient, the imaging device 410 operates in an operation mode in which it outputs voice data of family members or the like, and when the care recipient is not a dementia patient, it operates in an operation mode in which it does not output voice data or the like. In this way, it becomes possible to set an operation mode according to the attributes of the care recipient. Note that a plurality of voice data of family members or the like may be prepared for one care recipient. For example, if there is only one call from family members or the like, the care recipient may remember the call and the reaction may become dull. Therefore, the imaging device 410 may perform a process of outputting one randomly selected from among a plurality of voice data stored in association with the target care recipient. In this way, since the variations of calls from family members or the like can be increased, it becomes possible to effectively stop the movement of the care recipient.

[0222] In addition, the scene information in this embodiment may be information representing the type of assistance, such as meal assistance, excretion assistance, movement / transfer assistance, etc. For example, the scene information acquisition unit 112 may acquire scene information representing the type of assistance based on user input by the caregiver. Also, the scene information acquisition unit 112 may determine the type of assistance being executed based on the relationship between the assistance schedule in a nursing facility or the like and the current time. Alternatively, the scene information acquisition unit 112 may determine the type of assistance by estimating the position of the care recipient in a nursing facility or the like. For example, if the care recipient is in the cafeteria, the scene information acquisition unit 112 determines that meal assistance is being provided, and if the care recipient is in the toilet, excretion assistance and assistance for preventing falls are being provided. The position determination may be performed using a human presence sensor or the like arranged in various places in the facility. Alternatively, access points (APs) may be arranged in various places in the facility, and the position determination may be performed according to which AP the station device (STA) carried by the caregiver is connected to.

[0223] For example, as described above, the seat surface sensor 440 shown in FIG. 10 is capable of performing forward and lateral displacement determination and fall possibility determination. However, when eating while sitting in the wheelchair 630, as shown in FIG. 12, since a table or the like for arranging meals is placed in front of the care recipient, the table serves as a support and the possibility of a fall occurring is low. Therefore, the seat surface sensor 440 can omit unnecessary processing by making the fall possibility determination inactive during meals.

[0224] FIG. 20 is a flowchart for explaining the determination process of the operation mode in the seat surface sensor 440. First, in step S501, the processing unit 210 of the seat surface sensor 440 performs a process of identifying the target care recipient. For example, the processing unit 210 may identify the care recipient based on an assistance schedule or the like, or may identify the care recipient based on user input.

[0225] In step S502, the processing unit 210 acquires the ability information of the care recipient. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 of FIG. 18 from the server system 100.

[0226] In step S503, the processing unit 210 determines whether the ability of the assisted person has decreased to a state where they cannot walk based on the ability information. If the ability has not decreased (step S503: No), since the necessity of determination using the seat surface sensor 440 is low, in step S504, the processing unit 210 sets the operation mode of the seat surface sensor 440 to mode 0 corresponding to non - active. Also, similar to the example described above in step S403, in step S503, for the ability information, processing considering more detailed information may be executed. Therefore, the processing of step S503 may be executed in the server system 100.

[0227] If the ability of the assisted person has decreased to a state where they cannot walk (Yes in step S503), in step S505, the processing unit 210 acquires information representing the type of assistance as scene information. As described above, the processing of step S505 may be performed based on user input or based on some sensing data.

[0228] In step S506, the processing unit 210 makes a determination based on the scene information. Specifically, the processing unit 210 determines whether the type of assistance is meal assistance. If it is meal assistance (step S506: Yes), the forward and lateral displacement determination is useful, but the determination of the possibility of falling is less necessary. Therefore, in step S507, the processing unit 210 sets the operation mode of the seat surface sensor 440 to a mode that performs forward and lateral displacement determination and does not perform the determination of the possibility of falling.

[0229] When the type of assistance is not meal assistance (step S506: No), there is not always a table in front of the person receiving assistance, and the risk of falling from the wheelchair 630 is high. Therefore, in step S508, the processing unit 210 sets the operation mode of the seat surface sensor 440 to a mode that performs both forward and lateral displacement determination and fall possibility determination. Also in step S508, the processing unit 210 may set the operation mode of the seat surface sensor 440 to a mode that does not perform forward and lateral displacement determination and performs fall possibility determination.

[0230] Note that above, an example of performing processing based on scene information for the imaging device 410 and the seat surface sensor 440 has been described. However, it goes without saying that the operation mode of other devices 200 may be determined based on scene information.

[0231] 3.3 Specific Examples of Processing Based on Device Type Information Also, the server system 100 (device type information acquisition unit 113) may obtain device type information that identifies the types of auxiliary devices used for assisting the same person receiving assistance as the device 200. The server system 100 transmits the device type information to the device 200. The device 200 determines which of the plurality of operation modes to operate in based on the ability information and the device type information. For example, the storage unit 220 of the device 200 stores information associating the ability information and the device type information with the operation mode. The processing unit 210 obtains the operation mode based on the said information, the ability information, and the device type information acquired from the server system 100. The information associating the ability information and the device type information with the operation mode may be, for example, table data associating the values of the ability information, the values of the device type information, and the active / inactive states of each application. Alternatively, the information associating the ability information and the device type information with the operation mode may be an algorithm for obtaining the active / inactive states of each application based on the ability information and the device type information.

[0232] FIG. 21 is a diagram for explaining an example of an auxiliary device used in combination with the swallowing muscle detection device 460 described above with reference to FIG. 12. The auxiliary device here may specifically be the device 200 according to the present embodiment. The swallowing muscle detection device 460 is used during a meal, and the meal may be taken using the wheelchair 630 or the bed 610.

[0233] For example, the auxiliary device for the swallowing muscle detection device 460 may be the seat surface sensor 440 shown in FIG. 10. When the swallowing muscle detection device 460 is active and the detection results and sensing data logs from the seat surface sensor 440 are acquired, the device type information acquisition unit 113 may determine that the swallowing muscle detection device 460 and the seat surface sensor 440 are used in combination. Alternatively, when the device type information acquisition unit 113 determines based on the detection results and sensing data logs from the seat surface sensor 440 that the care recipient is sitting on the wheelchair 630, it may determine that the swallowing muscle detection device 460 and the seat surface sensor 440 are used in combination. In this case, it is considered that the care recipient is having a meal using the wheelchair 630.

[0234] Similarly, the auxiliary device for the swallowing muscle detection device 460 may be the detection device 430 shown in FIG. 9. When the swallowing muscle detection device 460 is active and the detection results and sensing data logs from the detection device 430 are acquired, the device type information acquisition unit 113 may determine that the swallowing muscle detection device 460 and the detection device 430 are used in combination. Alternatively, when the device type information acquisition unit 113 determines based on the detection results and sensing data logs from the detection device 430 that the care recipient is in a bedridden state, it may determine that the swallowing muscle detection device 460 and the detection device 430 are used in combination. In this case, it is considered that the care recipient is having a meal using the bed 610.

[0235] FIG. 22 is a flowchart for explaining the determination process of the operation mode in the swallowing mucus detection device 460. First, in step S601, the processing unit 210 of the swallowing mucus detection device 460 performs a process of identifying a target care recipient. For example, the processing unit 210 may identify the care recipient based on face recognition processing on the captured image captured by the terminal device 462.

[0236] In step S602, the swallowing mucus detection device 460 acquires the ability information of the care recipient. For example, the processing unit 210 identifies the ability information by receiving the data shown in steps S301 and S306 of FIG. 18 from the server system 100.

[0237] In step S603, the swallowing mucus detection device 460 determines whether the ability of the care recipient has declined to a state where they cannot eat well based on the ability information. If the ability has not declined (step S603: No), since the necessity of determination using the swallowing mucus detection device 460 is low, in step S604, the processing unit 210 sets the operation mode of the swallowing mucus detection device 460 to mode 0 corresponding to inactive.

[0238] If the ability of the care recipient has declined to a state where they cannot eat well (Yes in step S603), in step S605, the swallowing mucus detection device 460 acquires information for identifying the combined device from the device type information acquisition unit 113. As can be understood from the above example, here, it is sufficient to determine whether it is a meal in the bed 610 or a meal in the wheelchair 630. Therefore, while the type of the combined device is important, there is little need to identify the vendor and model number. Thus, in step S605, the swallowing mucus detection device 460 acquires the device type ID of the combined device.

[0239] In step S606, the swallowing mucus detection device 460 makes a determination based on the device type information. Specifically, the swallowing mucus detection device 460 determines whether the meal is being taken using the wheelchair 630. Specifically, as described above, the swallowing mucus detection device 460 may determine whether the combined device represented by the device type information is the seat surface sensor 440 or the detection device 430. If the meal is being taken using the wheelchair 630 (step S606: Yes), since the movement to a cafeteria or the like is possible, the state of the care recipient is relatively good. Therefore, in step S607, the processing unit 210 sets the operation mode of the swallowing mucus detection device 460 to a mode for performing normal determination using the swallowing time or the like.

[0240] On the other hand, if the meal is being taken using the bed 610 (step S606: No), it is presumed that the care recipient has difficulty moving or has difficulty eating without a device such as the bed 610 that allows flexible back angle setting. Therefore, in step S608, the processing unit 210 sets the operation mode of the swallowing mucus detection device 460 to a mode that can reduce the risk of aspiration more. For example, the swallowing mucus detection device 460 may not only detect the presence or absence of mucus but also perform a process of determining the content of the mucus. For example, the swallowing mucus detection device 460 may determine whether the mucus is a dangerous mucus that is likely to lead to aspiration. In addition, the swallowing mucus detection device 460 may determine whether the care recipient seems sleepy. The determination of whether the care recipient seems sleepy may be made based on, for example, the captured image captured by the terminal device 462, based on the opening and closing of the eyes, the frequency of movement of the mouth and hands, etc. Also, the determination of whether the care recipient seems sleepy may be executed using the detection device 430.

[0241] In the above description, two combined devices of the wheelchair 630 and the bed 610 have been exemplified, but the present invention is not limited thereto. For example, as the wheelchair 630, a normal wheelchair and a reclining wheelchair may be used. In this case, the flexibility of the back angle adjustment increases in the order of a normal wheelchair, a reclining wheelchair, and the bed 610. Therefore, the swallowing cough detection device 460 may have its operation mode controlled so that more attentive care can be performed in the order of a normal wheelchair, a reclining wheelchair, and the bed 610. For example, the swallowing cough detection device 460 may increase the number of applications set to active in the above order.

[0242] In the above description, an example has been described in which the swallowing cough detection device 460 is the device 200 for setting the operation mode, and the seat surface sensor 440 and the detection device 430 are combined devices. However, the seat surface sensor 440 and the detection device 430 may be the devices 200 for setting the operation mode, and the swallowing cough detection device 460 may be a combined device. In other words, in the method of the present embodiment, when a plurality of devices 200 are used in combination, the operation mode may be changed by the plurality of devices 200 interlocking with each other. For example, the seat surface sensor 440 shown in FIG. 10 and the terminal device 450 used for adjusting the wheelchair position shown in FIG. 11 are each useful for reducing the risk of falling even when used alone. However, both the seat surface sensor 440 and the terminal device 450 may be used in combination. And in this case, the control of the other device 200 may be performed in conjunction with the control of one device 200.

[0243] For example, when it is determined based on the ability information that the care recipient has shifted to a state where they cannot walk, first, the seat surface sensor 440 may shift to an operation mode corresponding to active, and the terminal device 450 may maintain an operation mode corresponding to non-active. In this way, first, the forward deviation and lateral deviation determination is performed, and the seat holding ability is obtained using the log data.

[0244] When it is determined that the seat holding ability has dropped below a predetermined level, the terminal device 450 shifts to an operation mode corresponding to active. As a result, the device type information acquisition unit 113 transmits data indicating that the terminal device 450 is a combined device to the seat surface sensor 440. And when the terminal device 450 is used in combination, the seat surface sensor 440 may be set to an operation mode different from when it is not used in combination. For example, the processing unit 210 of the seat surface sensor 440 may change the threshold value in the determination. Alternatively, the seat surface sensor 440 may switch from an operation mode in which the reference (initial value) in the front shift and lateral shift determination etc. is determined internally to an operation mode in which the reference is determined based on the timing etc. when position adjustment using the terminal device 450 is performed. Similarly, in other cases as well, the control of one of the seat surface sensor 440 and the terminal device 450 may be changed based on the information of the other.

[0245] Alternatively, the operation mode of the seat surface sensor 440 may be controlled based on the information acquired by the terminal device 450. For example, in the terminal device 450, points to be noted when the caregiver adjusts the posture of the care recipient can be input. For example, points such as paying attention to the position of the right shoulder and putting a cushion under the right arm may be input, and in the terminal device 450, the attributes of the care recipient can be estimated based on the points. In the above example, an attribute that the care recipient is somewhat right shoulder paralyzed is required. The seat surface sensor 440 may determine the operation mode based on this information. For example, the seat surface sensor 440 may store a plurality of applications different according to the presence or absence of paralysis and the part where paralysis has occurred as an application for performing the front shift and lateral shift determination. And the seat surface sensor 440 controls the activation / inactivation of each application based on the attributes acquired from the terminal device 450. In the above example, the seat surface sensor 440 activates an application for performing a front shift and lateral shift determination suitable for a care recipient with right shoulder paralysis and sets other applications for performing a front shift and lateral shift determination to be inactive. In this way, in the seat surface sensor 440 that can use a plurality of tacit knowledges of the front shift and lateral shift determination, it becomes possible to switch the tacit knowledge according to the attributes of the care recipient.

[0246] In addition, the server system 100 of the present embodiment may acquire mode information for specifying the operation mode of the combined device and transmit the device type information and the mode information to the device 200. The device 200 determines which of the plurality of operation modes to operate in based on the ability information, the device type information, and the mode information.

[0247] For example, in the case of the seat surface sensor 440, when shifting to an operation mode for determining the possibility of falling in addition to the forward shift and lateral shift determination, the terminal device 450 may shift to an operation mode for executing an additional process of recommending a cushion or the like in addition to the normal process of determining the appropriateness of the wheelchair position. Alternatively, when the operation mode of the seat surface sensor 440 changes, a process of switching the teacher data determined as correct in the terminal device 450 may be executed. In addition, various modifications are possible for the specific operation mode cooperation method.

[0248] In this way, it is possible to interlock a plurality of devices 200 using more detailed operation modes without being limited to simple active / inactive. In the above example, since the device 200 related to the detection of the wheelchair position and the device 200 related to the adjustment support of the wheelchair position can be appropriately interlocked, it is possible to further reduce the risk of falling of the care recipient. Although the cooperation between the seat surface sensor 440 and the terminal device 450 has been described here, the operation of other devices 200 in cooperation is not precluded.

[0249] In addition, although the combination of the ability information and the scene information, and the combination of the ability information and the device type information have been described above, the three of the ability information, the scene information, and the device type information may be combined.

[0250] Also, in the method of this embodiment, the type of assistance and the type of device 200 may be associated. For example, the swallowing muscle detection device 460 shown in FIG. 12 is a device 200 specific to meal assistance. When the swallowing muscle detection device 460 is operating, the probability that the meal assistance for the assisted person is being performed is high. Thus, among the devices 200, there are devices that can associate device type information with the type of assistance. For example, since the swallowing muscle detection device 460 communicates with the server system 100 via the gateway 300 when it is in an active state, the scene information acquisition unit 112 can determine the operating state of the swallowing muscle detection device 460 based on the presence or absence of communication with the swallowing muscle detection device 460.

[0251] For example, the scene information acquisition unit 112 of the server system 100 may obtain scene information representing the type of assistance based on the device type information acquired by the device type information acquisition unit 113 and transmit the scene information to the device 200. For example, the scene information acquisition unit 112 determines that it is during a meal when the swallowing muscle detection device 460 is active and determines that it is not during a meal when the swallowing muscle detection device 460 is inactive. Alternatively, the server system 100 may transmit the device type information acquired by the device type information acquisition unit 113 to the device 200, and the device 200 may perform a process of determining an operation mode according to the type of assistance associated with the device 200. For example, the device 200 may perform a process of selecting an operation mode suitable for meal assistance when the swallowing muscle detection device 460 is an auxiliary device. As described above, the operation mode setting according to the type of assistance may be executed as a process related to scene information or as a process related to device type information, and various modified embodiments are possible for specific implementation modes.

[0252] 4. Other Examples of Devices Also, the device 200 in this embodiment is not limited to the above-described one. For example, the device 200 in this embodiment may be an MCI determination device that determines mild cognitive impairment (MCI). For example, the MCI determination device asks questions to the care recipient using voice or images and performs a process of receiving responses to the questions. The questions here may use the Mini-Mental State Examination (MMSE) or the like, or may be questions of other methods. The MCI determination device makes an MCI determination based on the responses of the care recipient. Also, the MCI determination device may make an MCI determination based on information regarding the sleep of the care recipient or the like.

[0253] Also, the device 200 in this embodiment may be an assistance recording device that automatically records the history of assistance. For example, the assistance recording device may be a device 200 that detects the position information of at least one of the caregiver and the care recipient. The assistance recording device determines, for example, the time when the target person is in the bed 610, the toilet, the bath, the cafeteria, etc., and performs a process of storing, as an assistance record, what types of assistance were performed at what frequencies and times based on the determination result. The assistance recording device is useful, for example, for home care where the assistance schedule or the like is not strictly set.

[0254] Also, the device 200 of this embodiment may include a reclining wheelchair 510 whose backrest angle can be adjusted as shown in FIG. 23, or a nursing bed 520 whose bottom surface angle can be adjusted as shown in FIG. 24. The reclining wheelchair 510 and the nursing bed 520 are control target devices shown in FIGS. 7 and 18, for example. For example, when the swallowing muscle is detected by the swallowing muscle detection device 460, the angle of the backrest or the bottom is controlled to an angle suitable for the swallowing of the care recipient.

[0255] Further, the reclining wheelchair 510 may be controlled based on the processing results of the seat surface sensor 440 and the terminal device 450. For example, the wheelchair 630 shown in FIGS. 10 and 11 may be the reclining wheelchair 510 which is a device to be controlled. Similarly, the nursing bed 520 may be controlled based on the processing results of the bedside sensor 420, the detection device 430, the bed position detection device 470, the glasses-type device 480, etc. For example, the bed 610 shown in FIGS. 9, 13, and 14 may be the nursing bed 520 which is a device to be controlled.

[0256] In addition, the device 200 used in this embodiment can be variously modified in terms of shape, the number and type of sensors, processing content, etc.

[0257] 5. Communication between Devices In the above description, for example, as shown in step S303 of FIG. 18, the server system 100 receives the processing result in the device 200. Also, as shown in step S304 of FIG. 18, the server system 100 transmits a control signal for causing the control target device to perform control based on the processing result. Further, as shown in step S306 of FIG. 18, when the information for determining the operation mode is updated, the server system 100 also executes the notification of the information to the device 200.

[0258] However, from the viewpoint of suppressing an excessive increase in the processing load in the server system 100, in this embodiment, at least one of the control of the control target device based on the processing result in the device 200 and the notification of scene information, etc. may be executed in a serverless manner. Hereinafter, a specific information processing system 10 will be described while exemplifying it.

[0259] FIG. 25 is a diagram showing a configuration example of the information processing system 10 when serverless control is executed. The information processing system 10 of the present embodiment may include an entrance / exit management device 700 that detects the entry and exit of the device 200 into and out of a given space, and a gateway 300 that relays communication between the device 200 located in the space and the server system 100. Also, similar to FIG. 1, the information processing system 10 includes the server system 100 and the device 200. In the example of FIG. 25, the devices 200-1, 200-2, and 200-3 are illustrated as the device 200.

[0260] As shown in FIG. 25, the gateway 300 is, for example, arranged in a living room in a nursing facility or the like, and is a device capable of transmitting and receiving radio waves with sufficient strength at least with the device 200 located in the living room.

[0261] Also, the entrance / exit management device 700 shown in FIG. 25 is, for example, arranged near the entrance / exit of the living room, and is a device that detects the entry or exit of the device 200. The entrance / exit management device 700 is, for example, a device that performs short-range wireless communication such as BLE (Bluetooth Low Energy), and communicates with a Bluetooth-compatible device located at a position within a predetermined distance or less. For example, the entrance / exit management device 700 may be arranged in the living room, and determine that a device 200 capable of BLE communication with the entrance / exit management device 700 is located in the living room.

[0262] Alternatively, the entry / exit management device 700 may perform a more detailed determination. For example, a plurality of entry / exit management devices 700 may be arranged at different positions in the living room. Then, the plurality of entry / exit management devices 700 communicate with the device 200 respectively, and estimate the distance from the device 200 based on the radio wave intensity (e.g., RSSI: Received Signal Strength Indicator) in the communication. In this way, since the distances from each of the plurality of points with known positions can be obtained, the position of the device 200 can be estimated. The entry / exit management device 700 may determine whether the device 200 is located in the living room based on the estimated position and the known information such as the size and shape of the living room. In addition, various modifications can be made to the specific determination of entry / exit.

[0263] The storage unit 220 of the device 200 stores information about the device 200. The information about the device 200 includes at least the device ID that identifies the device 200. Also, the information about the device 200 may include some or all of the device information 122 described above with reference to FIG. 3. The entry / exit management device 700 acquires information about the device 200 that is the communication target in the BLE communication with the device 200. In this way, the entry / exit management device 700 can appropriately identify the device 200 that has entered the target space.

[0264] Also, the storage unit 220 of the device 200 may store address information used for communication via the gateway 300 by the target device 200. The address information here may be, for example, an IP address, or other information (e.g., MAC address) that can identify the IP address. The entry / exit management device 700 acquires the address information of the device 200 that is the communication target in the BLE communication with the device 200.

[0265] Also, the entry / exit management device 700 is not limited to a device using BLE, and other communication methods such as NFC (Near Field Communication) can be widely applied.

[0266] The entrance / exit management device 700 may have a memory (not shown), and the memory may store existing device information, which is information for identifying existing devices located in the living room. In this way, it becomes possible to appropriately manage what devices 200 are located in the target living room. In the example of FIG. 25, devices 200-1 and 200-2 have already entered the living room. For example, when device 200-1 enters the room, the entrance / exit management device 700 executes a process of adding the device ID of the device 200-1 to the existing device information based on BLE communication with the device 200-1. Similarly, when device 200-2 enters the room, the entrance / exit management device 700 executes a process of adding the device ID of the device 200-2 to the existing device information. As a result, the state where devices 200-1 and 200-2 exist in the living room is retained as the existing device information.

[0267] The entrance / exit management device 700 may perform a process of transmitting the stored existing device information to the devices 200 that have already entered the room. For example, the entrance / exit management device 700 transmits the existing device information including the device IDs of devices 200-1 and 200-2 to the device 200-1. In this way, the device 200-1 can recognize that, in addition to itself, the device 200-2 exists in the living room. Similarly, when the entrance / exit management device 700 transmits the existing device information to the device 200-2, the device 200-2 can recognize that the device 200-1 exists in the living room.

[0268] Also, the existing device information may be information in which the device ID and the address information are associated. For example, the entrance / exit management device 700 transmits the device ID and address information of the device 200-1 and the device ID and address information of the device 200-2 to each device 200. In this way, each device 200 can obtain, in addition to the information indicating the existence of other devices 200, the address information for communicating with the other devices 200 via the gateway 300.

[0269] Similarly, when a new device 200-3 enters the living room as in the example of FIG. 25, the same applies. When the entry / exit management device 700 detects that a new device has entered the space, it communicates with the new device to obtain the address information of the new device used in the communication via the gateway 300, and may notify the device 200 located in the space of the obtained address information. For example, the entry / exit management device 700 may perform a process of adding the device ID and address information of the device 200-3 to the existing device information by performing BLE communication with the device 200-3. Note that the device ID and address information are not limited to being acquired simultaneously. For example, immediately after entering the room, if the connection between the device 200-3 and the gateway 300 is not completed and a dynamic IP address has not been assigned, the device 200-3 may wait for the connection to the gateway 300 to be established and then send the address information to the entry / exit management device 700.

[0270] After updating the existing device information due to the entry of the device 200-3 into the room, the entry / exit management device 700 sends the existing device information to the devices 200-1 to 200-3. As a result, each device 200 located in the living room can identify the other devices 200 in the living room and the address information.

[0271] Note that the entry / exit management device 700 may switch the information to be sent between the existing devices (devices 200-1 and 200-2) and the newly entered new device (device 200-3). For example, the entry / exit management device 700 may send only the device ID and address information of the new device to the existing devices and omit the transmission of other records of the existing device information. Also, the entry / exit management device 700 may send the existing device information before the update to the new device. By doing so, the transmission of unnecessary information can be omitted, and thus the communication load can be reduced.

[0272] The same applies when any of the devices 200 leaves the room. For example, when the device 200 for which BLE communication is no longer possible or the device 200 whose estimated position is determined to be outside the room, the entrance / exit management device 700 determines that the device has left the target room. In this case, the entrance / exit management device 700 performs a process of deleting the device ID and address information of the device 200 that has left the room from the existing device information. Further, the entrance / exit management device 700 notifies the existing devices of information indicating that the device 200 has left the room. For example, the entrance / exit management device 700 may transmit the updated existing device information to each device 200. Further, the entrance / exit management device 700 may transmit the device ID, etc. of the device 200 that has left the room to each device 200 and instruct each device 200 to delete the corresponding record from the existing device information held by each device 200.

[0273] Further, the entrance / exit management device 700 is not limited to storing the existing device information located in the target space, and may store the entrance / exit log data. The log data is information in which, for example, the entrance time, exit time, device ID, etc. are associated.

[0274] Next, communication in a serverless manner will be described. For example, assume that the device 200-1 is operating in an active operation mode, and as a result of its processing, it is determined that the device 200-2 needs to be controlled. For example, the device 200-1 is a swallowing mucus detection device 460, and since dangerous mucus has been detected, it is determined that it is necessary to change the bottom angle of the device 200-2, which is a care bed 520.

[0275] In the example described above with reference to FIGS. 7 and 18, the processing via the server system 100 was performed. However, in the example of FIG. 25, as described above, the device 200-1 knows that the device 200-2 is in the same living room, and its IP address is also known. Therefore, the device 200-1 may send a control signal to the device 200-2 without going through the server system 100. Specifically, the device 200-1 sends a control signal to the gateway 300 using a packet specifying the IP address of the device 200-2. The gateway 300 determines, based on the determination according to the IP address, that the destination of the packet is the device 200-2 which is a device within the same network. Therefore, the packet containing the control signal is sent to the device 200-2 without going through the server system 100, as shown as "Packet 1" in FIG. 25. Thereby, since the control of the necessary device 200 can be executed without going through the server system 100, it becomes possible to reduce the load on the server system 100 and shorten the time required for control.

[0276] Also, the scene information and the device type information are not limited to those required in the processing unit 110 of the server system 100. For example, when using the number of assistants as the scene information, if the device 200-1 is an imaging device 410 or the like, it is also possible to obtain the number of assistants in the living room based on image processing of the captured image. Also, the device type information is information representing the types of other devices 200 used together with a certain device 200. Therefore, the device type ID of the device 200-1 may be device type information for the devices 200-2 and 200-3 operating in the same living room.

[0277] Also, in the device 200 of the present embodiment, it is not precluded that ability information such as an ADL index is required. For example, as described above, since it is possible to use the log of the sensing data for calculating the ability information, the device 200 that acquires the sensing data may calculate the ability information.

[0278] In the example described above using FIGS. 7 and 18, in the server system 100, ability information, scene information, and device type information were requested, and data including these were transmitted to the device 200. However, in the example of FIG. 25, the device 200-1 knows that the device 200-2 is in the same living room, and its IP address is also known. Therefore, the device 200-1 transmits ability information, scene information, device type information, etc. to the device 200-2 without going through the server system 100. Specifically, similar to the control signal described above, the device 200-1 can transmit ability information, etc. to the device 200-2 without going through the server system 100 by using a packet specifying the IP address of the device 200-2. As a result, since the operation mode can be set according to the ability information, etc. without going through the server system 100, it is possible to reduce the load on the server system 100 and shorten the time required for control. For example, a plurality of devices 200 located in the living room may execute a process of identifying a combined-use device by each notifying other devices 200 of the device type information of its own device 200.

[0279] Also, the device 200-1 may transmit log data related to the control of the controlled device (for example, the device 200-2) to the server system 100. In the above example, data indicating when and to what extent the back angle of the device 200-2, which is the swallowing mucus detection device 460, was changed with respect to the device 200-1, which is the care bed 520, becomes log data. Similarly, when notifying the device 200-2 of ability information, etc., the timing of the notification and the details of the ability information, etc., which is the content of the notification, become log data. By transmitting these log data to the server system 100, it becomes possible to share the content with the server system 100 even when serverless communication is performed.

[0280] For example, as shown in FIG. 25, the device 200-1 may transmit control signals, capability information, scene information, device type information, etc. using Packet 1, and may transmit log data using Packet 2. The log data here is data representing the communication history using Packet 1 as described above. Also, Packet 1 is a packet transmitted and received within the network to which the device 200-1 belongs. Packet 2 is a packet transmitted beyond the gateway 300 that forms the network to which the device 200-1 belongs. Packet 2 may, in a narrow sense, be a packet specifying the IP address of the server system 100. By switching the packet according to the content of the data to be transmitted in this way, it becomes possible to reduce the communication load on the server system 100.

[0281] Note that, in the above, an example of serverless data transmission from the device 200-1 to another device 200 has been shown, but other devices 200 such as the device 200-2 may be the data transmission source.

[0282] As described above, the device 200 according to the present embodiment may select whether to transmit the operation result to the server system 100 (steps S203 etc. in FIG. 7) or, based on the operation result, to transmit information for controlling a control target device different from the device 200 without going through the server system 100 (Packet 1 in FIG. 25). In this way, it is possible to flexibly change whether to perform processing under the leadership of the server system 100 or to perform serverless processing.

[0283] Specifically, when the address information of the control target device has been acquired based on information from the entry / exit management device 700, the device 200 may control the control target device without going through the server system 100. In this way, it becomes possible to appropriately determine whether to perform serverless processing in consideration of the specific communication situation.

[0284] In the above, an example of performing serverless communication using the entry / exit management device 700 has been described. However, the method of this embodiment is not limited to this. For example, the server system 100 may determine, by referring to the packets transmitted from the device 200, through which gateway 300 the packets are transmitted data. And when there are a plurality of devices 200 connected to the same gateway 300, the server system 100 may determine that the plurality of devices 200 are in a close distance.

[0285] Then, the server system 100 transmits the address information of other devices 200 determined to be close to the distance from each device 200 to the corresponding device 200. For example, the server system 100 obtains information associating the identification information and address information of a plurality of devices 200 connected to the same gateway 300, and periodically transmits the information to the plurality of devices 200. Even with such a method, each device 200 can specify the address information of other devices 200 in the vicinity, so serverless communication can be realized.

[0286] 6. Detailed Example of Communication 6.1 Overview Among the information processing systems 10 shown in FIG. 1, communication using a wireless communication method is performed between at least one device 200 and the gateway 300. The wireless communication method here may be, for example, a communication method compliant with IEEE802.11. Also, the server system 100 may perform communication using a wired communication method or may perform communication using a wireless communication method. The wired communication method here may be, for example, a communication method compliant with IEEE802.3. Hereinafter, for the sake of simplifying the explanation, an example in which each of the server system 100, the device 200, and the gateway 300 uses a communication method compliant with IEEE802.11 will be described.

[0287] FIG. 26 is a diagram showing a configuration example of the communication processing unit 114 and the communication unit 130 of the server system 100. As shown in FIG. 26, the communication processing unit 114 includes an upper layer processing unit 1141, a MAC layer processing unit 1142, and a physical layer processing unit 1143. The communication unit 130 includes a transmission / reception circuit 131 and an antenna array 132. However, the configurations of the communication processing unit 114 and the communication unit 130 are not limited to those shown in FIG. 26, and various modifications such as omitting some configurations and adding other configurations are possible.

[0288] The upper layer processing unit 1141 performs processing at a layer higher than the MAC layer. The upper layer here may be TCP / IP (Transmission Control Protocol / Internet Protocol), UDP / IP (User Datagram Protocol / Internet Protocol), or the application layer. For example, the processing for obtaining the above-described capability information, scene information, and device type information is executed by software operating on the server system 100, and the application layer here may correspond to the software. The upper layer processing unit 1141 is connected to the MAC layer processing unit 1142.

[0289] The MAC layer processing unit 1142 performs transmission processing and reception processing in the MAC layer. The MAC layer processing unit 1142 is connected to the physical layer processing unit 1143.

[0290] The physical layer processing unit 1143 performs processing in the physical layer. The physical layer processing unit 1143 is connected to the antenna array 132 via the transmission / reception circuit 131.

[0291] The transmission / reception circuit 131 includes circuits such as a digital / analog conversion circuit (hereinafter referred to as a D / A conversion circuit), an RF circuit, etc., converts the digital signal from the physical layer processing unit 1143 into an analog signal, and outputs it to the antenna array 132. Also, the transmission / reception circuit 131 includes an analog / digital conversion circuit (hereinafter referred to as an A / D conversion circuit), converts the signal received by the antenna array 132 into a digital signal, and outputs the converted digital signal to the physical layer processing unit 1143. Note that it is not precluded that the physical layer processing unit 1143 has an A / D conversion circuit or a D / A conversion circuit.

[0292] The antenna array 132 includes a plurality of antennas that transmit radio waves based on the analog signal output from the transmission / reception circuit and output an analog signal based on the received radio wave.

[0293] For example, at the time of data reception, each part shown in FIG. 26 operates as follows. First, the antenna array 132 receives radio waves in a predetermined frequency band and outputs an analog signal corresponding to the radio waves to the transmission / reception circuit 131. The transmission / reception circuit 131 converts the analog signal received by the antenna array 132 into a baseband signal. Also, the transmission / reception circuit 131 performs A / D conversion of the baseband signal and outputs the converted digital signal to the physical layer processing unit 1143.

[0294] The physical layer processing unit 1143 performs processing such as demodulation and error correction of the received signal. Also, the physical layer processing unit 1143 removes the physical header which is the header corresponding to the physical layer, and outputs the payload part to the MAC layer processing unit 1142.

[0295] The MAC layer processing unit 1142 processes the received payload part as a MAC frame. The MAC frame may be an MPDU (MAC Protocol data unit) in IEEE802.11. The MAC layer processing unit 1142 delivers the frame body, which is the part excluding the MAC header and the trailer which are the headers in the MAC layer, of the MAC frame to the upper layer processing unit 1141.

[0296] The upper layer processing unit 1141 outputs data corresponding to the frame body received from the MAC layer processing unit 1142 to software or the like.

[0297] Also, at the time of data transmission, the reverse corresponding process is executed. First, software operating on the server system 100 generates data to be transmitted, and the data is output to the upper layer processing unit 1141.

[0298] The upper layer processing unit 1141 creates data corresponding to the frame body by attaching headers and the like at the upper layer to the data, and outputs it to the MAC layer processing unit 1142.

[0299] The MAC layer processing unit 1142 creates a MAC frame by attaching a MAC header and trailer to the received data. The MAC layer processing unit 1142 outputs the created MAC frame to the physical layer processing unit 1143.

[0300] The physical layer processing unit 1143 creates a physical packet by attaching a physical header and the like to the received data. The physical layer processing unit 1143 outputs the created physical packet to the transmission / reception circuit 131.

[0301] The transmission / reception circuit 131 generates an analog signal by performing A / D conversion and modulation on the data received from the physical layer processing unit 1143. The transmission / reception circuit outputs the analog signal to the antenna array 132. The antenna array 132 transmits radio waves corresponding to the received analog signal.

[0302] The MAC layer processing unit 1142 may use a data frame, a control frame, or a management frame as the above-described MAC frame. A data frame is a frame used for transmitting and receiving data between terminals when a communication link between the terminals is established. For example, as described above, when software operating on the server system 100 creates data including capability information or the like and transmits the data to the device 200, a data frame is used.

[0303] A management frame is a frame used for managing a communication link between terminals. Note that various types of information transmitted and received using a management frame are defined in IEEE802.11, and these can also be widely applied in this embodiment. A control frame is a frame used for controlling the transmission and reception of management frames and data frames. The control frame includes various frames such as an RTS (Request to Send) frame, a CTS (Clear to Send) frame, and an ACK (Acknowledgement) frame. Since these frames are defined within the IEEE802.11 standard, a detailed description thereof is omitted.

[0304] In addition, the configuration related to the communication of the server system 100 has been exemplified above, but a similar configuration may also be used for the device 200 and the gateway 300. Also, as described above, the server system 100 may perform communication using a wired communication method. In that case, a part of the above content can be replaced with content compliant with IEEE802.3.

[0305] 6.2 Frame Configuration In the method of this embodiment, the configuration of the MAC frame (data frame) may be made common regardless of the device 200 that is the communication target. As described above with reference to FIG. 7 and the like, the device 200 and the application operating on the device 200 in this embodiment may be created by the device vendor. Therefore, there may be cases where devices 200 created by various vendors are registered in the information processing system 10. In that regard, by making the configuration of the data frame common, it becomes possible to unify communication control regardless of the vendor. Note that the configuration of the data frame, in a narrow sense, is the bit assignment in the frame body.

[0306] FIG. 27 is an example of the format of the MAC frame. Note that the format example shown in FIG. 27 is also applicable to management frames and control frames, but hereinafter, the data frame will be used as an example for explanation. As shown in FIG. 27, the MAC frame includes a MAC header, a frame body, and a trailer.

[0307] The MAC header includes fields such as Frame Control, Duration ID, Address 1, Address 2, Address 3, Sequence Control, Address 4, QoS Control, and HT Control. Also, some of these may be omitted.

[0308] Frame Control includes a type field for discriminating whether the target MAC frame is a data frame, a management frame, or a control frame. Frame Control may also include a subtype field for specifying a more detailed type. Information representing the scheduled period for using the radio wave is stored in Duration / ID. The scheduled time for using the radio wave may be rephrased as the time required for frame transmission. Duration / ID is used for RTS / CTS and the like.

[0309] Address 1 to Address 4 store information representing the addresses of the destination or source devices. For example, Address 1 corresponds to the destination address, and Address 2 corresponds to the source address. Data corresponding to the frame usage is stored in Address 3 and Address 4.

[0310] Sequence Control corresponds to the sequence number of the data to be transmitted. QoS Control stores information used for QoS control. QoS control represents control for performing transmission considering the priority of the frame. HT Control is a field used, for example, in management frames.

[0311] The configuration of the frame body will be described later with reference to FIGS. 28A and 28B. Also, the trailer is, for example, FCS (Frame Check Sequence). FCS is information used for error detection of the frame and is, for example, a checksum code. FSC is, for example, CRC (Cyclic Redundancy Code).

[0312] As described above with reference to FIG. 1, the information processing system 10 according to the present embodiment includes the server system 100 and the device 200. The device 200 includes an application that executes processing corresponding to the tacit knowledge of an expert. As shown in FIG. 27, the server system 100 transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in the data link layer of communication with the device 200. Here, the server system 100 transmits a data frame included in the frame body, where the data frame includes a fixed-length area including a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information for specifying the scene of assistance to the assisted person, and a third area for storing device type information for specifying the type of combined device used together with the device 200. Then, as described above, the device 200 determines whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

[0313] In this way, it becomes possible to unify the data format when the server system 100 transmits ability information and the like to the device 200. For example, even when devices 200 of various vendors are registered in the information processing system 10 of the present embodiment, the configuration of the data frame can be unified regardless of the vendor. In particular, by setting the first to third regions to a fixed length, it becomes easier for the device 200 that has received the data frame to interpret the data frame, so that the processing load related to communication can be reduced. For example, the first to third regions may be arranged in the leading portion of the frame body excluding headers and the like in the upper layer.

[0314] As shown in step S203 of FIG. 7, the device 200 may transmit the operation result of the application to the server system 100 when the application is set to active. The operation result here is the output of the application and represents the execution result of the process corresponding to tacit knowledge. For example, the operation result includes a determination result regarding any of the events that may occur in assistance, the assisted person, and the assistant. Specifically, the operation result may be a determination result as to whether or not any event has occurred in assistance, a determination result as to whether the state of the assisted person is normal, or a determination result as to whether the assistance by the assistant is correct. In this way, it becomes possible to aggregate the information representing the operation result in the server system 100. However, the process via the server system 100 is not essential, and a serverless process may be executed as described above with reference to FIG. 25.

[0315] FIGS. 28A and 28B are diagrams for explaining an example of bit allocation of the frame body of the data frame transmitted by the server system 100. As shown in FIG. 28A, the frame body may include fields such as User ADL, scene flag, device type ID, data type ID, Instruction length, and contents.

[0316] User ADL is a field for storing the ability information of the person to be assisted, corresponding to the first area described above. For example, the ability information is numerical data indicating which stage the ability of the person to be assisted belongs to when the degree of ability is divided into a predetermined number of stages. For example, if the number of stages is 8 or less, User ADL is a 3-bit field. When the number of stages is 9 or more, User ADL may be a 4-bit or larger field. Since the number of bits required according to the definition of the ability information is known, User ADL may be a fixed-length field. User ADL may include an ID for identifying the person to be assisted. Also, as described above in step S403 of FIG. 19 and the like, the ability information of the present embodiment is not limited to the index value of ADL, and may be more detailed information such as information representing the way of getting up, sitting holding ability, swallowing ability, walking ability, etc. Therefore, User ADL may be a field having the number of bits capable of expressing these respective abilities.

[0317] Scene flag is a field for storing scene information, corresponding to the second area described above. For example, the scene information may include a bit indicating whether the number of assistants is a predetermined number or more. When the bit is the first value (for example, 0), it means that the number of assistants is a predetermined number or more, and when it is the second value (for example, 1), it means that it is less than the predetermined number. The scene information may also include bits for specifying the type of assistance. For example, when identifying four types of assistance such as meal assistance, excretion assistance, movement / transfer assistance, and others as the type of assistance, the scene information includes 2 bits as bits for specifying the type of assistance. For example, when the 2 bits are 00, it represents meal assistance, when it is 01, it represents excretion assistance, when it is 10, it represents movement / transfer assistance, and when it is 11, it represents others. The scene information is not limited to these, and other information may be used. Therefore, various modifications can be made to the specific number of bits and meaning of Scene flag. However, since it is known what kind of scene information is used and the number of bits required for expressing the scene information is also known, Scene flag may be a fixed-length field.

[0318] The device type ID is a field that stores device type information and corresponds to the third region described above. The device type information may be different for each device 200 described above using FIGS. 8 to 14. Alternatively, for a device 200 corresponding to a fall risk, the same device type ID may be assigned to the imaging device 410 in FIG. 8 and the bedside sensor 420 in FIG. 9. Alternatively, the device type ID may be assigned based on the type of sensor or the like that the device 200 has. Since the number of target device type IDs is known, the device type ID may be a fixed-length field.

[0319] Further, the server system 100 may obtain the control type and the control content for the control target device based on the operation result of the device 200. Then, the server system 100 stores a fixed-length fourth region for storing the control type and a fifth region for storing the control content and having different lengths according to the control type in a portion of the frame body that is behind the region including the first region, the second region, and the third region, and may transmit a data frame including the fifth region to the control target device. The control type represents the type of control to be executed, and the control content represents information that specifically identifies the content of the control to be executed. In this way, it becomes possible to use a common data frame for both the notification of ability information and the like and the control of the control target device. At this time, the first to third regions, which are fixed-length fields, are arranged relatively forward, and the fifth region, which is a variable-length field, is arranged relatively backward, so that the data structure on the front side can be fixed including the number of bits. As a result, the interpretation process of the data frame by the device 200 becomes easy.

[0320] For example, the data type ID, Instruction length, and contents shown in FIG. 28A are fields used for transmitting control signals to the device to be controlled. The data type ID is a field that stores information representing the type of instruction output to the device to be controlled, and corresponds to the fourth region. The instructions here may include four types: "notification (alarm)", "movement / conveyance", "control", and "recommendation, etc.". In this case, the data type ID is a 2-bit fixed-length field that can identify the four instructions.

[0321] "Notification" is information used when notifying the device to be controlled of the processing result in device 200. For example, "notification" may be an instruction used to notify the device to be controlled of the fact that a fall risk has been detected in a device that determines the fall risk. "Movement / conveyance" is an instruction to move a movable device to be controlled, such as a reclining wheelchair 510 or a walker. For example, the instruction for movement / conveyance may be used for control to move a walker or the like closer so that the target care recipient can be caught when a fall risk is detected. "Control" broadly includes controls other than "movement / conveyance" that operate the device to be controlled, and includes changing the angle of the back part of the reclining wheelchair 510, changing the bottom angle of the nursing bed 520, etc. "Recommendation, etc." includes, for example, recommendations for purchasing products used to improve the quality of care. For example, the recommendation may represent an instruction to output the fact that a determination has been made to recommend the use of a cushion in the bed position detection device 470 to the device to be controlled, which is a terminal device of the caregiver or the like. Also, "recommendation, etc." may include news distribution, etc. For example, "recommendation, etc." may be used to introduce popular devices 200.

[0322] contents is a field that stores information identifying the content of a specific instruction and corresponds to the fifth region described above. FIG. 28B is a diagram for explaining a specific example of contents according to the value of the data type ID. For example, data type ID = 0 represents the "notification" described above, and in this case, contents includes an alarm contents field. Alarm contents is a variable-length field that stores information representing specific notification content. Alarm contents may include information identifying the notification mode (display, light emission, vibration, sound output), or may include information identifying specific notification content (text, light emission color, vibration pattern, voice).

[0323] Data type ID = 1 represents the "movement / conveyance" described above, and in this case, contents includes fields for current location and destination. Current location is information identifying the current position of the target device 200. Destination is information identifying the target position of the target device 200. How the current position and target position are represented is arbitrary, but for example, they are fixed-length fields representing two-dimensional or three-dimensional coordinate values.

[0324] Data type ID = 2 represents the "control" described above. In this case, the contents include fields for the controlled part and how to control. The controlled part stores information identifying the part of the target device 200 to be controlled. For example, in the case of the care bed 520 shown in FIG. 24, it includes a plurality of bottoms that can each move. The controlled part may store information identifying any one of the plurality of bottoms. How to control stores information identifying the specific control method for the part to be controlled. In the example of the care bed 520, how to control may be information indicating in which direction and how many times the bottom identified by the controlled part is to be driven. The controlled part and how to control may each be a fixed-length field. Note that the "control" here may include instructions for a plurality of parts to be controlled. Therefore, the contents in the case of data type ID = 2 may be configured to repeat the pair of the controlled part and how to control for the number of parts to be controlled.

[0325] Data type ID = 3 represents the "recommendation, etc." described above. In this case, the contents include a recommend contents field. Recommend contents is a variable-length field representing the content of recommendations or news. The recommend contents here may include information identifying the product to be recommended, or link information to the product sales site, the manufacturer's product introduction web page, etc. Also, recommend contents may include text representing the content of the news, or link information to the web page where the news is displayed, etc.

[0326] As described above, the length of the contents field in the frame body varies depending on the data type ID and the specific instruction content. Therefore, as shown in FIG. 28A, the frame body may include an instruction length field for storing the length of the contents field before the contents field. Since the maximum length of the contents field is considered to be known, the instruction length may be, for example, a fixed-length field.

[0327] Also, as described above with reference to FIG. 25, in this embodiment, data including a control signal, capability information, etc. may be transmitted from the device 200 to another device 200 without going through the server system 100. However, if the formats of the data transmitted from the server system 100 (steps S304 in FIG. 7, etc.) and the data transmitted from the device 200 (packet 1 in FIG. 25) are significantly different, the receiving device 200 may need to perform data interpretation processing according to the transmission source. Therefore, even when the device 200 is the transmission source, a frame configuration conforming to the frame configurations shown in FIGS. 28A and 28B may be used.

[0328] For example, when the device 200 controls a device to be controlled without going through the server system 100, in the data link layer of the communication with the device to be controlled, a data frame including a fixed-length area including a second area and a third area may be transmitted to the device to be controlled. As described above, the second area corresponds to the scene information, and the third area corresponds to the device type information. Also, the data frame may include a first area corresponding to the capability information. In this way, it becomes possible to share at least the parts corresponding to the capability information, the scene information, and the device type information in the frame body of the data frame. For example, the first to third areas may be arranged at the head part of the frame body excluding the header, etc. in the upper layer.

[0329] Also, the device 200 may request the control type and control content for the device to be controlled based on the operation result of the application on the device 200. Then, a data frame including a fourth region and a fifth region may be transmitted to the device to be controlled without passing through the server system 100 to a portion of the frame body that is behind the region including the second region and the third region. As described above, the fourth region corresponds to the control type, and the fifth region corresponds to the control content. In this way, even when serverless control is performed, it is possible to standardize the configuration of the data frame received by the device 200.

[0330] FIGS. 29A and 29B are diagrams showing a configuration example of a frame body when the device 200 transmits data including a control signal, capability information, etc. to another device 200. As can be seen by comparing FIG. 29A with FIG. 28A, the configuration of the frame body is the same as the configuration of the frame body transmitted by the server system 100 except that the order of the device type ID, scene flag, and User ADL is different. Here, the device type ID field may store, for example, the device type information of the transmitting device 200. FIG. 29B is a diagram for explaining a specific example of the contents according to the value of the data type ID. Since the device 200 is the transmission source, the contents are the same as those in FIG. 28B except that log data is transmitted instead of a recommendation or the like when the data type ID = 3. The log data is as described above and includes information such as the history when the serverless control of the device to be controlled is performed and the notification history of the capability information, etc.

[0331] In this way, regardless of whether the server system 100 is the source or, as shown in FIG. 25, the device 200 is the source, it is possible to make the data frame configurations similar. As a result, in the device 200 that receives the data frame, it becomes possible to determine the operation mode and perform operations according to the control signal based on the same processing regardless of the source. Therefore, even when both the processing starting from the server system 100 and the serverless processing are performed, an increase in the processing load can be suppressed.

[0332] Note that the order of each field in FIGS. 28A and 29A, particularly the order of User ADL, device type ID, and scene flag, can be variously modified. Therefore, it is not precluded that the configuration of the data frame transmitted by the server system 100 is the same as the configuration of the data frame transmitted by the device 200. Also, in the configuration of the data frame transmitted by the server system 100 and at least one of the data frames transmitted by the device 200, a part of User ADL, device type ID, and scene flag may be omitted. For example, in the above, an example has been described in which the device 200 outputs sensing data and the capability information acquisition unit 111 of the server system 100 calculates capability information based on the sensing data. In this case, since the capability information is not calculated in the device 200, User ADL may be omitted from the data frame transmitted by the device 200. Alternatively, after providing a User ADL field in the data frame transmitted by the device 200, the value of the field may be set to a fixed value (for example, all 0). Also, the device 200 is not precluded from calculating the capability information. In this case, information such as the index value of the ADL is stored in the User ADL of the data frame transmitted by the device 200.

[0333] In addition, the data frame described above with reference to FIGS. 29A and 29B may be used for data transmission from the device 200 to the server system 100. For example, when the device 200 determines that notification is required as a processing result, it sets the value of the data type ID to 1 and transmits to the server system 100 a data frame in which information representing the notification content is stored in the alarm contents among the contents. In addition to this, the device 200 can appropriately transmit the processing result to the server system 100 using the data frame shown in FIGS. 29A and 29B by changing the value of the data type ID and the value of the contents according to the processing result. Also, the log data stored in the contents when the data type ID = 3 is not limited to the log of the serverless communication and may include the log of the sensing data. That is, when transmitting the log of the sensing data to the server system 100, the data frame shown in FIGS. 29A and 29B may be used.

[0334] In addition, the method of this embodiment may be applied to an information processing apparatus. The information processing apparatus corresponds to, for example, the server system 100. The information processing apparatus includes a communication unit (corresponding to the communication unit 130 in FIG. 3) that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in the data link layer of communication with the device 200 including an application that executes processing corresponding to the tacit knowledge of a skilled person, and a communication processing unit (corresponding to the communication processing unit 114 in FIG. 3) that controls the communication unit. The communication unit transmits to the device 200, as information for determining whether the application is active or inactive, a data frame in which a fixed-length area including a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information for specifying the scene of assistance to the assisted person, and a third area for storing device type information for specifying the type of the combined device used with the device is included in the frame body. In this way, it becomes possible to unify the data format when the server system 100 transmits ability information and the like to the device 200 regardless of the vendor of the device 200 and the like.

[0335] Further, the method of this embodiment can be applied to an information processing method in an information processing system 10 including a device 200 including an application that executes processing corresponding to the tacit knowledge of an expert, and a server system 100 that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with the device 200. The information processing method includes a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information for specifying the scene of assistance to the assisted person, and a third area for storing device type information for specifying the type of the combined device used together with the device 200. A fixed-length area including: a step of transmitting a data frame included in the frame body from the server system 100 to the device 200; and a step of determining whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

[0336] 6.3 Priority Also, in various communication systems, methods for setting the priority of communication are known. For example, in IEEE802.11, a method for realizing QoS in IEEE802.11e is standardized. Specifically, in IEEE802.11e, a method called EDCA (Enhanced Distributed Channel Access) that preferentially transmits frames with higher priority is used.

[0337] For example, the frame body of a management frame among MAC frames may include an EDCA parameter set element used in EDCA. FIG. 30 is a diagram showing an example of a default EDCA parameter set element. In EDCA, packets are classified into four access categories (hereinafter referred to as ACs). The default access categories are four: AC_VO, AC_VI, AC_BE, and AC_BK. AC_VO corresponds to voice, AC_VI corresponds to video, AC_BE corresponds to best effort, and AC_BK corresponds to background. AC_VO has the highest priority, and the priorities decrease in the order of AC_VI, AC_BE, and AC_BK.

[0338] Also, parameters such as CWmin, CWmax, and AIFSN are associated with each AC. CWmin and CWmax are parameters that determine the transmission waiting time and represent the minimum and maximum values of the contention window (hereinafter referred to as CW). The transmission waiting time is set to a value between CWmin and CWmax. Since it is easier to transmit when the transmission waiting time is shorter, the values of CWmin and CWmax are set smaller as the priority is higher.

[0339] For example, for AC_BE and AC_BK with low priority, when CWmin is set to aCWmin and CWmax is set to aCWmax, in AC_VI with high priority, CWmin is set to (aCWmin + 1) / 2 - 1 and CWmax is set to aCWmin, so the values of CWmin and CWmax are smaller compared to the above two ACs. Also, in AC_VO with the highest priority, CWmin is set to (aCWmin + 1) / 4 - 1, which is shorter than that of AC_VI, and CWmax is set to (aCWmin + 1) / 2 - 1, which is shorter than that of AC_VI.

[0340] Also, AIFSN (Arbitration Inter Frame Space Number) is a parameter representing the transmission interval of frames. The smaller this value is, the higher the priority of the queue. In the example of FIG. 30, the AIFSN of AC_VO and AC_VI is 2, the AIFSN of AC_BE is 3, and the AIFSN of AC_BK is 7.

[0341] Also, the parameters associated with AC are not limited to the above. For example, although not shown in FIG. 30, each AC may be associated with a TXOP (Transmission Opportunity) limit. TXOP represents the channel occupancy time. For example, in the case of AC_VI and AC_VO with high priority, by setting a non-zero value as the TXOP limit, it becomes possible to occupy the channel for transmitting the packets set for these ACs.

[0342] The information processing system 10 of this embodiment is assumed to include a large number of devices 200. And each device 200 executes an operation according to the operation mode and transmits the processing result to the server system 100 via the gateway 300. The processing results here can include various contents such as "notification", "movement / conveyance", "control", "log", etc. described above with reference to FIG. 29B.

[0343] The devices 200 of this embodiment may include products of various vendors, and the priority settings may vary depending on the vendor. For example, there is a possibility that only the communication of the devices 200 of a specific vendor is prioritized over the communication of the devices 200 of other vendors. Therefore, in this embodiment, communication efficiency may be improved by setting priorities. For example, in the method of this embodiment, the allocation of AC is updated.

[0344] The server system 100 of the present embodiment may obtain a priority based on device type information and transmit priority information specifying the priority to the device. The priority information is, for example, information associating device type information and a priority, and more specifically, information assigning an AC for each device type. Then, the device 200 transmits a data frame according to the priority specified based on the device type and the priority information. In this way, since the priority according to the device type can be set, communication in the information processing system 10 including various devices 200 can be made efficient.

[0345] Furthermore, the server system 100 may obtain a priority based on device type information and a control type, and transmit priority information specifying the priority to the device 200. The priority information is, for example, information assigning an AC for each combination of a device type and a control type. Then, the device 200 transmits a data frame according to the priority specified based on the device type, the control type included in the data frame to be transmitted, and the priority information. In this way, since the control type can be reflected in the priority in addition to the device type, communication in the information processing system 10 can be made efficient even when various devices 200 transmit various information. Hereinafter, an example using both device type information and control type will be specifically described. However, in the following description, either one of the device type information and the control type may be omitted.

[0346] FIG. 31 is an example of a table showing the assignment of ACs. In the method of the present embodiment, the AC may be determined based on the device type and the data type. The device type here corresponds to the device type information described above and is, for example, a device type ID. Also, the data type here corresponds to, for example, the data type ID described above with reference to FIGS. 28A to 29B. That is, the data type may be information indicating which of "notification", "movement / conveyance", "control", and "log" is to be transmitted based on the processing result in the device 200.

[0347] For example, for each gateway 300 that communicates with the server system 100, the server system 100 obtains the relationship between the device type, data type, and AC, and performs a process of transmitting the obtained information to each gateway 300. For example, based on the information from the gateway 300, the server system 100 can identify the number and type of devices 200 connected to the gateway 300, the data types that each device 200 can transmit, and the like.

[0348] For example, the "log" of the data type represents a case where, as described above, the log of the sensing data or the log of communication in the case of serverless communication is transmitted. Therefore, the communication priority is low, and it is not likely to be a problem even if it is transmitted late at night when the communication volume decreases. On the other hand, "notification", "movement / conveyance", and "control" are used to inform the caregiver of the occurrence of a risk or to suppress the impact of the occurred risk. Therefore, it is desirable that the time from when a risk is detected in the device 200 to when the data is transmitted is short. Thus, when the data type is any of "notification", "movement / conveyance", and "control", the server system 100 sets a higher priority than when it is "log". Also, the server system 100 may vary the priorities among "notification", "movement / conveyance", and "control".

[0349] In addition, since the output by the swallowing mucus detection device 460 corresponds to the occurrence of the aspiration risk, the severity may increase if prompt response is not made. On the other hand, since the pressure ulcer risk detected by the bed position detection device 470 is highly important when assuming assistance over a relatively long period, the severity is relatively small even if the notification of the presence of a pressure ulcer risk is delayed in execution. Thus, even if the data type is the same "notification", the server system 100 may set a higher priority when the device type corresponds to the swallowing mucus detection device 460 than when it corresponds to the bed position detection device 470. In addition, the server system 100 can set priorities according to the device type for various devices described above using FIGS. 8 to 14 and the like.

[0350] For example, the server system 100 determines a priority according to the device type and data type, and assigns an AC according to the priority. Note that when the ratio of packets assigned to the AC with a high priority becomes excessively high, collisions are likely to occur, and it is known that the effect of using EDCA is impaired. Therefore, the server system 100 may change the AC assignment according to the device type and data type for each gateway 300. For example, even for the same device type and the same data type, a case may occur where AC_VI with a relatively low priority is assigned to the first gateway, and AC_VO with a relatively high priority is assigned to the second gateway. For example, in a network configured by the first gateway, when there are many devices 200 that detect a high-risk such as a swallowing mucus detection device 460, it is conceivable that the priority of the packets assigned to AC_VO in a normal network is lowered to AC_VI, etc.

[0351] The gateway 300 and the device 200 connected to the gateway 300 communicate based on the AC assignment transmitted from the server system 100. Note that the communication here is not limited to data transmission from the device 200 to the gateway 300, and also includes data transmission from the gateway 300 to the device 200.

[0352] In addition to the process of determining the AC assignment based on the device type and data type, the server system 100 may perform a process of adjusting the parameters corresponding to each AC. The parameters here include at least one of CWmin, CWmax, AIFSN, and TXOP limit described above. In this way, it becomes possible to perform more detailed adjustment in the priority setting.

[0353] Also, the priority setting process in this embodiment is not limited to using the above-described EDCA. For example, RTS / CTS is used as a collision avoidance method in wireless communication. This is a method in which a STA that wants to transmit data sends an RTS frame to the AP, and the AP sends a CTS frame to any one of the STAs that permits transmission among the STAs that have sent the RTS frame. Since data transmission is possible only for the STA that has received the CTS frame, data collisions can be suppressed.

[0354] In the method of this embodiment, the server system 100 may transmit information indicating the priority order when transmitting the CTS frame to the gateway 300 as priority information. For example, the server system 100 transmits information for determining the transmission order of the CTS frame to the gateway 300, such as in ascending order of the low ADL of the target assisted person, in descending order of the urgency of the data based on the data type, or a combination thereof. When performing collision avoidance based on RTS / CTS, the gateway 300 determines the priority order based on the list of devices 200 that have sent the RTS frame and the information from the server system 100, and transmits the CTS frames in order from the highest priority. By such a method, appropriate priority setting becomes possible, and for example, it is possible to suppress the situation where only the priority of a specific vendor becomes high.

[0355] Also, when the gateway 300 receives an RTS frame, it may perform a process of indicating to the device 200 that it is busy and presenting the expected completion time of the currently ongoing communication. Furthermore, when data indicating that transmission waiting cannot be permitted is sent from the device 200, the gateway 300 may accept an interruption by a suspend command and permit the data transmission of the target device 200. When the data transmission is completed, the original data transmission is resumed by a resume command. In this way, for particularly urgent data, it becomes possible to interrupt other data transmissions and perform communication preferentially. However, since this process interrupts an already started data transmission, it is not preferable to perform it frequently or to have an excessively long interruption time. Therefore, the data that permits an interruption by a suspend command may be limited to a part of the data. For example, an interruption may be permitted when conditions such as a small amount of transmission data and a short time required for transmission are satisfied.

[0356] 7. Application to Home Care Also, the method of this embodiment may be applied to home care and the like performed in the home of the care recipient.

[0357] 7.1 Example of a System Used at Home FIG. 32 is a diagram showing a configuration example of an information processing system 10 at home. The information processing system 10 in FIG. 32 includes a server system 100, a third terminal device 810, a fourth terminal device 820, and human sensors 831 to 833. For example, among the information processing system 10, the third terminal device 810, the fourth terminal device 820, and the human sensors 831 to 833 are arranged in the home where home care is performed. Also, in home care, a bed 610, a wheelchair 630 (not shown), etc. may be used.

[0358] The third terminal device 810 is a portable terminal device such as a smartphone or a tablet terminal device used, for example, by a family member who assists the care recipient. The fourth terminal device 820 is a portable terminal device such as a smartphone or a tablet terminal device used, for example, by the care recipient.

[0359] The human sensors 831 to 833 are sensors arranged at predetermined positions within a home, and are, for example, sensors that detect human movement using infrared rays. Note that the human sensors 831 to 833 may be sound sensors that respond to sound, or ultrasonic sensors that detect objects using ultrasonic waves, and various modifications can be made to the specific embodiments. The human sensors 831 to 833 are connected to the third terminal device 810 using, for example, BLE.

[0360] The bed 610 is a bed used by the care recipient, and may be a nursing bed 520 whose bottom angle, height, etc. can be adjusted.

[0361] For example, the assistance recording device, which is an example of the device 200 according to the present embodiment, may include the third terminal device 810 and the human sensors 831 to 833 in FIG. 32. The assistance recording device identifies the location where the assistance was performed based on the detection results of the human sensors 831 to 833, and creates an assistance record based on the identified location.

[0362] For example, the human sensors 831 to 833 may each be arranged at a location where specific assistance is performed. For example, the human sensor 831 is arranged near the bed 610 used by the care recipient. The human sensor 832 is arranged near the toilet. The human sensor 833 is arranged near the dining table where meals are taken. Also, the human sensor may be arranged at other locations within the home, such as a bathroom where bathing assistance is performed.

[0363] For example, when the human presence sensor 831 detects the movement of a person, it is considered that the caregiver is providing assistance such as adjusting the bed position of the care recipient or changing diapers. Also, when the human presence sensor 832 detects the movement of a person, it is considered that the caregiver is providing assistance such as toileting assistance. Further, when the human presence sensor 833 detects the movement of a person, it is considered that the caregiver is providing meal assistance to the care recipient. Therefore, by obtaining the time when movement is detected in each case, it is possible to obtain the assistance time. For example, by obtaining the total time during which movement is detected by the human presence sensor 831 in a day, the assistance time at the bed 610 performed on that day can be obtained. Similarly, the time for toileting assistance and the time for meal assistance can be obtained by the human presence sensors 832 and 833. Also, more detailed information such as from what time to what time and where the assistance was provided may be required.

[0364] Even if the human presence sensors 831 to 833 detect the movement of a person, if that movement is of the caregiver alone or the care recipient alone, there is a possibility that no assistance is being executed. In this embodiment, from the perspective of performing a simple determination, without distinguishing whether the detected movement is of one person or multiple persons, a determination based simply on the presence or absence of movement detection may be executed. Alternatively, the presence or absence of assistance may be determined in more detail by different methods. For example, instead of the human presence sensors 831 to 833, RFID readers may be provided at each location. And by having the caregiver and the care recipient carry IC tags, or by incorporating IC tags in the third terminal device 810 and the fourth terminal device 820, it is possible to determine whether the caregiver and the care recipient are near each RFID reader. The assistance recording device may determine that assistance is being provided when it is determined that both the caregiver and the care recipient are present at a predetermined location. Also, instead of the human presence sensors 831 to 833, cameras may be arranged at each location in the home. For example, by performing face recognition processing based on the captured images of each camera, it may be determined whether assistance is being executed at each location.

[0365] Also, when meals are taken in bed, meal assistance will be provided. However, it may not be easy to distinguish meal assistance from other assistance such as bed position adjustment based solely on a person's movement and presence. In this case, the assistance recording device may be linked with the meal amount measurement application described later. For example, since a photo is taken in the meal amount measurement application, assistance during the time including the timing when the photo is taken is determined to be meal assistance, and assistance at other times is determined to be bed assistance other than meals.

[0366] By using the assistance recording device, the time required for assistance by the caregiver can be easily measured. Therefore, for example, when the caregiver's assistance burden is excessive, it becomes possible to notify the caregiver himself / herself or the care manager.

[0367] Also, other applications may be installed in the third terminal device 810. For example, an application for determining the user's sleep state using an acceleration sensor or a microphone installed in a smartphone is known. For example, the third terminal device 810 is arranged near the pillow of the bed where the user goes to bed, and determines the sleep state based on vibrations during sleep. Also, the third terminal device 810 may make a determination regarding snoring or the like using a microphone. In this way, it becomes possible to easily make a determination regarding sleep without using a dedicated device such as the detection device 430.

[0368] Also, an MCI determination application for performing the above-described MCI determination may be installed in the third terminal device 810. In other words, the third terminal device 810 may function as an MCI determination device. For example, the caregiver carries the third terminal device 810, which is his / her own smartphone, near the care recipient and makes the MAC determination by having the care recipient respond.

[0369] Further, a meal amount measurement app may be installed in the third terminal device 810. The meal amount measurement app is, for example, a method of estimating the amount of food and the calories consumed by comparing a photo of the meal before eating with a photo of the meal after eating. Regarding the method of estimating the amount of food based on a photo, known methods disclosed in, for example, Japanese Patent Application Laid-Open No. 2021-086313 and the like can be widely applied.

[0370] Figs. 33 to 35 are examples of screens displayed on the display unit of the third terminal device 810. These screens may be user screens displayed, for example, when an operation of logging in to the information processing system 10 of the present embodiment is performed using the account of the caregiver in the third terminal device 810. The results of each of the above processes executed in the third terminal device 810 may be displayed on the user screen. For example, the third terminal device 810 executes each of the above processes and transmits the execution result to the server system 100. The server system 100 generates a display screen based on the execution result and executes control to display the display screen on the display unit of the third terminal device 810.

[0371] Fig. 33 is an example of a home screen displayed, for example, at the time of login. As shown in Fig. 33, information indicating the state of the caregiver, announcements, a contact button, and a shooting button may be displayed on the home screen.

[0372] The information indicating the state of the caregiver may include, for example, an icon indicating the sleep state of the caregiver and an icon indicating the care burden of the caregiver. In the example of Fig. 33, the sleep state and the degree of care burden are presented in an easy-to-understand manner using human expressions. However, each piece of information may be expressed using numerical values or the like, and various modifications of the specific display mode are possible.

[0373] For example, the third terminal device 810 obtains the care time, which is the time when care is provided based on the processes using the human presence sensors 831 to 833 and the like described above, and determines which of a plurality of levels the care burden belongs to by comparing the care time with a threshold value. Further, the third terminal device 810 may evaluate the care burden based on the temporal change of the care time.

[0374] Also, the third terminal device 810 determines the sleep state using the acceleration sensor and microphone described above. For example, the third terminal device 810 may obtain the sleep time per day as the sleep state. The third terminal device 810 may determine which of the multiple stages the sleep state belongs to by comparing the sleep time with a threshold value. Also, the third terminal device 810 may evaluate the sleep state based on the temporal change of the sleep time.

[0375] Also, in the notification column of the screen in FIG. 33, for example, a recommendation for an assistive device may be displayed. In the example of FIG. 33, text proposing the use of a positioning pillow used for adjusting the bed position is displayed.

[0376] Also, the contact button is, for example, a button for making a call to the care manager. For example, when a selection operation of the contact button is performed, the third terminal device 810 starts a phone application and executes a process of making a call to the pre-registered phone number of the care manager. In this way, by making it easy to contact the care manager, it becomes possible to prompt appropriate responses when the burden on the caregiver is increasing. For example, as shown at the top of FIG. 33, warning text such as "Please call the assigned care manager or family member" may be displayed when the sleep state or care burden is deteriorating. In this way, it is possible to encourage the caregiver to receive support from others before suffering serious discomfort due to caregiver fatigue.

[0377] The shooting button is a button that serves as a trigger for starting the meal amount measurement application described above. For example, when a selection operation of the shooting button is performed, the third terminal device 810 performs a process of starting the meal amount measurement application. In the meal amount measurement application, the camera application is started, and the dishes before and after the meal are photographed.

[0378] FIG. 34 is an example of a history screen displayed when a selection operation of a history button displayed in association with information indicating the state of the assistant in FIG. 33 is performed. On the history screen, the time-series change of the sleep state of the assistant and the time-series change of the care burden of the assistant are displayed. For example, the processing unit 110 of the server system 100 performs processing to obtain the moving average of the sleep time per day over 7 days based on the information from the third terminal device 810 and display the time-series change of the moving average on the history screen. Note that the calculation unit of the moving average is not limited to 7 days, and other periods may be used. Also, the time-series change of the sleep time per day itself may be displayed.

[0379] Also, the time-series change of the care burden is information indicating, for example, the change in the average value of the care time per day. Similar to the example of the sleep state, the average value of the care time may be the moving average over 7 days or information calculated using other periods. Also, the time-series change of the care time per day itself may be displayed.

[0380] In the example of FIG. 34, the sleep time decreases with the passage of time, and the assistance time increases with the passage of time. By displaying the screen shown in FIG. 34, it is possible to clearly present that the state of the caregiver is deteriorating. Also, as shown in FIG. 34, the history screen may include a sharing button for sharing information regarding the state of the caregiver. When a selection operation of the sharing button is performed, information including the time-series change of the average sleep time, the time-series change of the average assistance time, etc. is sent to a pre-registered contact. The contact here may be a care manager or a family member of the caregiver, etc. Also, the sharing destination of the information regarding the state of the caregiver is not limited to the pre-registered contact. For example, when an operation of selecting the sharing button is performed, link information to the information regarding the state of the caregiver may be displayed. The link information here may be, for example, the address information of a web page where the information regarding the state of the caregiver is displayed. The address information may be a URL (Uniform Resource Locator) or a two-dimensional barcode representing the URL. For example, when the caregiver undergoes a medical examination at a hospital due to poor physical condition, by sharing the information regarding the state of the caregiver with the doctor via the link information, it is possible to provide the doctor with a basis for judgment in diagnosing the cause of the poor physical condition.

[0381] FIG. 35 is an example of a list screen displayed when a selection operation of a list button displayed in association with the "Notification" in FIG. 33 is performed. On the list screen, information such as recommendations sent to the target caregiver is listed in chronological order. In the example of FIG. 35, a recommendation for a positioning pillow and a recommendation for a reclining wheelchair are displayed. Each recommendation includes the reason why the product is recommended, a product image, and detailed information about the product. The detailed information about the product may include the product name, manufacturer, price, evaluation, product description, etc.

[0382] Also, in the recommendation, a vendor app or the device 200 according to the present embodiment that can be used with the product may be presented. In the example of FIG. 35, together with the positioning pillow, the introduction of a positioning app is proposed. The positioning app is an application that executes the same processing as the first terminal device 471 of the bed position detection device 470 described above with reference to FIG. 13, for example. By introducing the positioning app, it becomes possible to assist the caregiver in adjusting the bed position. Similarly, an incontinence care support app may be installed in the third terminal device 810. For example, the third terminal device 810 may execute the same processing as the second terminal device 472 of the bed position detection device 470 described above with reference to FIG. 13, for example.

[0383] Also, in the example shown in FIG. 35, together with the reclining wheelchair 510, the introduction of the seat surface sensor 440 is proposed. By introducing the seat surface sensor 440, it is possible to present the result of the forward slip and lateral slip determination and the result of the fall possibility determination to the caregiver, so that it becomes possible to assist in adjusting the position of the reclining wheelchair 510.

[0384] Also, the processing executed in the third terminal device 810 is not limited to the above. For example, the timing of meals can be estimated based on a meal amount measurement app, and the timing of incontinence care can be estimated based on the processing in the care record device. Therefore, the third terminal device 810 may perform processing to predict fecal incontinence based on the rhythms of meals and excretion. For example, the third terminal device 810 may input the type and amount of laxatives, the type of diapers, the meal amount, the water intake amount, and the intake timing of meals and water as input data, and predict fecal incontinence based on a learned model or the like that determines the presence or absence of fecal incontinence after a predetermined time. Also, as a recommendation, the type of diapers and the like may be recommended.

[0385] Also, various processes are executed in the fourth terminal device 820 used by the caregiver. For example, the fourth terminal device 820 may have a GPS sensor, and a monitoring application using the GPS sensor may be installed. Conventionally, techniques for tracking the positions of the elderly and dementia patients for the purpose of suppressing wandering and the like are known, and in this embodiment, they can be widely applied.

[0386] Also, in the fourth terminal device 820, an application for determining the sleep state may be installed in the same manner as in the third terminal device 810. The fourth terminal device 820 determines the sleep state of the care recipient. Also, in the fourth terminal device 820, an MCI determination application for performing the above-described MCI determination may be installed in the same manner as in the third terminal device 810.

[0387] 7.2 Cooperation with Nursing Facilities, etc. In the above, the information processing system 10 used in home care has been exemplified, but the system may cooperate with a system outside the home. For example, the information acquired in home care may be transmitted to a terminal device such as a PC used by the care manager. For example, the processing results in the third terminal device 810 and the fourth terminal device 820 are transmitted to the server system 100, and the server system 100 transmits information based on the acquired processing results to the terminal device of the care manager.

[0388] FIGS. 36A to 36C are examples of screens displayed on the terminal device of the care manager. FIG. 36A is an example of a home screen and has two tabs, inside the facility and outside the facility, at the top. The inside the facility tab is a tab for displaying information about the care recipients who live in a nursing facility or the like among the care recipients in charge of the care manager. For example, when the inside the facility tab is selected, a process of starting the care software introduced by the target facility may be performed. The outside the facility tab is a tab for displaying information about the care recipients receiving home care and the caregivers assisting the care recipients. FIG. 36A illustrates a screen in a state where the outside the facility tab is selected.

[0389] The screen shown in FIG. 36A includes an area for displaying a list of users and an area for displaying detailed information of the user selected in that area. Here, the users represent the families of the care recipients who receive home care. In FIG. 36A, four users, User A to User D, are listed, and User A among them is in a selected state. Note that the multiple users displayed here may be sorted in descending order of importance to the care manager. For example, they may be sorted in descending order of the degree of deterioration of the sleep state of the user who is a caregiver and the degree of deterioration of the care burden. By doing so, even when the care manager is in charge of a large number of users, it is possible to clearly present which users are the ones that the care manager should pay attention to. Note that the sorting order is not limited to this, and the users who have newly received contact may be ranked higher, or the users who have introduced a new device 200 may be ranked higher, and various specific processes can be implemented with various modifications.

[0390] In the area for displaying detailed information, the sleep state, care burden, and sleep state of the care recipient of the user who is a caregiver are displayed. These pieces of information may include, for example, a graph representing the time-series change of the average value and icons indicating stages, similar to FIG. 34.

[0391] In addition, a contact button with the user may be displayed in association with the sleep state and care burden of User A. In FIG. 36A, a phone button for calling the user and an email button for sending an email to the user are displayed. When a selection operation of the phone button is performed, the call application is launched, and a call is made to the pre-registered phone number of the user. When a selection operation of the email button is performed, the email application is launched, and an email creation screen with the user's email address already filled in is displayed. Also, the means by which the care manager contacts the user is not limited to phone calls and emails, and a chat application or the like may be used. For example, a chat history between the care manager and the user may be displayed on the screen of FIG. 36A.

[0392] Also, as shown in FIG. 36A, the area for displaying detailed information may display a data output button. When a selection operation of the data output button is performed, data regarding the user's sleep state, care burden, etc. is output. The data here may be a csv file or data in other formats.

[0393] Also, a conversion button may be displayed in association with the sleep state of the care recipient. FIG. 36B is an example of the information displayed when a selection operation of the conversion button is performed. For example, when the conversion button is selected, instead of the change graph of the average sleep time shown in FIG. 36A, the information shown in FIG. 36B may be displayed. FIG. 36B is a screen that collectively displays the changes in the awakening state, sleep state, and out-of-bed state of the care recipient per day for a predetermined number of days. By displaying the screen shown in FIG. 36B, not only the average sleep time but also more detailed information regarding sleep can be presented. For example, it is possible to present information regarding the quality of sleep, such as the presence or absence and frequency of mid-sleep awakenings, to the care manager.

[0394] For example, the screen shown in FIG. 36B may be the same as the screen used for displaying the sensing data of the detection device 430. In this way, even when the detection device 430 is not introduced, it is possible to perform the same display as when the detection device 430 is introduced.

[0395] Also, as shown in FIG. 36A, the area for displaying detailed information may include information regarding the ability information of the care recipient. In the example of FIG. 36A, based on the respective changes in the amount of food intake, toilet assistance time, bath assistance time, and time in bed, the estimated result of the ADL change is displayed. For example, as shown in FIG. 36A, based on the decrease in the amount of food intake and the increase in the assistance time at each location, it may be displayed that the sitting retention ability is decreasing. In this way, even when providing home care, it becomes possible to estimate the change in the ability information and notify the care manager or caregiver of the change.

[0396] In addition, in care facilities and the like, it is relatively easy to introduce the various devices 200 described above with reference to FIGS. 8 to 14, and various data can be acquired as sensing data. For example, as sensing data, data for obtaining the content of the starting movement, the seat holding ability, the swallowing ability, the activity level in bed, etc. can be acquired, so that the estimation accuracy of the ability information can be increased. On the other hand, it is not easy to introduce the same devices 200 as in care facilities in home care. Therefore, the information that can be acquired is limited information such as the amount of food intake and the care time as described above.

[0397] Therefore, in the present embodiment, the estimation accuracy of the ability information in home care may be increased by associating the information acquired in a care facility with the information acquired in home care. For example, assume a care recipient who usually receives home care but regularly uses day care. The value of the ability information estimated using the device 200 in the care facility during day care is highly reliable as the ability information at that timing. Also, since it is unlikely that the ability information will change rapidly in a short period of time without factors such as the onset of a disease or an accident, the ability information during a predetermined period before and after day care is considered to be equivalent to the ability information at the time of day care. For example, machine learning may be performed using the amount of food intake and the care time acquired during home care in a predetermined period before and after day care as input data, and using data in which the ability information estimated during day care is given as accurate data for the input data as training data. In this way, it becomes possible to accurately estimate the ability information based on the information that can be acquired during home care.

[0398] Also, the ability information may be estimated based on the overlapping content between home care and day care. For example, each of the devices 200 described above with reference to FIGS. 8 to 14 is introduced in a care facility, and it is assumed that the ability information acquisition unit 111 of the server system 100 obtains the ability information based on the sensing data from each device 200. For example, the server system 100 may store table data in which one record is data associating a group of sensing data acquired from each device 200 with the ability information.

[0399] In addition, in-home care, it may not be easy to introduce all of the devices 200 shown in FIGS. 8 to 14, but there is a possibility that some of the devices 200 may be introduced. For example, the detection device 430 may also be introduced into the home, and sensing data corresponding to the sleep and activity level of the care recipient is required. In this case, the server system 100 may perform a process of extracting records with a high degree of similarity from the above table data by comparing the sensing data obtained in in-home care with the sensing data group included in the above table data. For example, since the sensing data of the detection device 430 is also included in the sensing data group, the similarity is calculated based on the comparison process of the sensing data and the sensing data obtained in in-home care. Then, the server system 100 outputs the ability information included in the record with a high degree of similarity as the ability information of the care recipient receiving in-home care. Even in this way, it is possible to accurately estimate the ability information based on the information that can be obtained during in-home care.

[0400] Returning to FIG. 36A, the description will be continued. The home screen shown in FIG. 36A may display information regarding the reaction of the target user to the "notification" presented to the user. The notification here is, for example, the information shown in FIG. 35.

[0401] For example, on the screen shown in FIG. 36A, it is displayed whether the user has purchased the device 200 recommended to the user. In the example of FIG. 36A, for the positioning pillow and the accompanying recommended positioning application, the user has the intention to purchase. For example, when the selection operation of the manufacturer order button shown in FIG. 36A is performed, the care manager may perform the ordering process of the positioning pillow on behalf of the user.

[0402] As described above, although various applications (vendor apps) can be installed on one device 200, since the user is not an expert in assistance, it may not be easy to determine which application is suitable for the assistance of the care recipient. Therefore, as shown in FIG. 36A, a care manager having knowledge about assistance may determine the application to be installed. Determining the application corresponds to determining the tacit knowledge to be used as described above. In the example of FIG. 36A, among several candidates, the tacit knowledge suitable for suppressing the risk of pressure ulcers for a care recipient with paralysis in the right arm is selected. Also, the tacit knowledge may be created for each care facility, and the care manager may be able to select the facility that created the tacit knowledge on the screen shown in FIG. 36A. For example, when the selected application is installed on the device 200 to be purchased, the product may be shipped with the target application pre-installed based on the selection operation of the manufacturer order button.

[0403] Also, it is conceivable that the device 200 to be purchased (for example, a positioning pillow) is different from the device 200 on which the application is installed (for example, the third terminal device 810). In this case, information identifying the user and the application may be transmitted to the server system 100 triggered by a selection operation of the manufacturer order button or the like. The server system 100 executes a process of transmitting the specified application to the device 200 used by the target user. In this way, an appropriate application can be installed on the device 200 without the user himself / herself performing the procedure. Although the selection of the application at the time of device purchase has been exemplified here, the screen of FIG. 36A may be used for application change during use. For example, the device 200 and the application used by the target user are displayed on the screen viewed by the care manager, and the application used by the care manager is changed according to the situation of the care recipient or the like. For example, the application (tacit knowledge) to be used may be selected from the pull-down menu as in the example of FIG. 36A. Similarly in this case, for example, uninstallation and installation of the application are executed via the server system 100.

[0404] FIG. 36C is an example of information additionally displayed on the screen of FIG. 36A. As shown in FIG. 36C, the device 200 and the application used by the target user may be displayed on the screen displayed on the terminal device of the care manager. In the example of FIG. 36C, the user has introduced the swallowing mucosa detection device 460 and is using the normal function as the swallowing mucosa application. The normal function is a function of detecting swallowing and mucosa and measuring the swallowing time. On the other hand, the care manager may be able to execute the installation of additional functions. For example, as shown in FIG. 36C, a function of detecting the presence or absence of a highly dangerous mucosa may be added. In this way, the care manager can add optional functions.

[0405] As described above, in the method of the present embodiment, the operation mode of each device 200 can be switched based on ability information or the like. Specifically, the active / inactive state of each application can be changed. Therefore, even in home care, the determination of active / inactive may be automatically performed. For example, the server system 100 may estimate ability information from sensing data obtained in home care and determine the operation mode of the device 200 based on the ability information. However, it is not limited thereto, and an activation operation of an application may be possible on a screen such as FIG. 36C. In this way, the care manager can manually change the operation mode according to the situation. As a result, even in home care where the types of sensing data are few, it becomes possible to appropriately change the operation mode according to the situation.

[0406] Although the present embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications that do not substantially depart from the novel matters and effects of the present embodiment are possible. Therefore, all such modified examples are intended to be included in the scope of the present disclosure. For example, in the specification or drawings, a term that is described at least once together with a broader or synonymous different term can be replaced with the different term at any place in the specification or drawings. Also, all combinations of the present embodiment and the modified examples are included in the scope of the present disclosure. Further, the configurations and operations of the server system, device, information processing system, etc. are not limited to those described in the present embodiment, and various modified implementations are possible.

Explanation of Reference Numerals

[0407] 10… Information processing system, 100… Server system, 110… Processing unit, 111… Ability information acquisition unit, 112… Scene information acquisition unit, 113… Device type information acquisition unit, 114… Communication processing unit, 120… Memory unit, 121… User information, 122… Device information, 123… Application information, 130… Communication unit, 131… Transmission / reception circuit, 132… Antenna array, 200, 200-1~200-3… Devices, 210… Processing unit, 220… Memory unit, 230… Communication unit, 240… Display unit, 250… Operation unit, 300… Gateway, 410… Imaging device, 420… Bedside sensor, 430… Detection device, 440… Seat surface sensor, 441… Cushion, 442… Control box, 450… Terminal device, 460… Swallowing muscle detection device, 461… Throat microphone, 462… Terminal device, 470… Bed position detection device, 471… First terminal device, 472… Second terminal device, 473… Display, 480… Glasses-type device, 510… Reclining wheelchair, 520… Nursing bed, 610… Bed, 620… Mattress, 630… Wheelchair, 700… Entrance / exit room management device, 810… Third terminal device, 820… Fourth terminal device, 831… Human presence sensor, 832… Human presence sensor, 833… Human presence sensor, 1141… Upper layer processing unit, 1142… MAC layer processing unit, 1143… Physical layer processing unit, IM1… Output image, ReD… Diaper area, Se1~Se4… Pressure sensors

Claims

1. A device including an application that executes processing corresponding to the tacit knowledge of an expert, A server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer of communication with the device, Including, The server system, A fixed-length area including a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information identifying the assistance scene for the assisted person, and a third area for storing device type information identifying the type of combined device used with the device is included in the data frame included in the frame body and transmitted, The device, An information processing system that determines whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information.

2. In Claim 1, The device, An information processing system that transmits the operation result of the application to the server system when the application is set to be active.

3. In Claim 2, The server system, Based on the operation result, determines the control type and the control content for the control target device, In the frame body, a fixed-length fourth area for storing the control type and a fifth area for storing the control content and having different lengths according to the control type are included in a part behind the area including the first area, the second area, and the third area, and the data frame is transmitted to the control target device. An information processing system.

4. In Claim 3, The device, Whether to transmit the operation result to the server system, Based on the operation result, without passing through the server system, transmits information for controlling the control target device to the control target device, An information processing system that makes a selection.

5. In Claim 4, The device, When controlling the control target device without passing through the server system, in the data link layer of communication with the control target device, a fixed-length area including the second area and the third area is included in the data frame included in the frame body and transmitted to the control target device. An information processing system.

6. In Claim 5, The device, Based on the operation result, determine the control type and the control content of the control for the device to be controlled. An information processing system that transmits the data frame including the fourth area and the fifth area to the device to be controlled without passing through the server system to a part behind the area including the second area and the third area in the frame body. **Claim 7** In claim 5 or 6, The device is An information processing system that transmits log data related to the control for the device to be controlled to the server system. **Claim 8** In any one of claims 4 to 6, An entrance / exit management device that detects the entry and exit of the device into / from a given space, A gateway that relays communication between the device located in the space and the server system, further includes The entrance / exit management device is When detecting that a new device has entered the space, by communicating with the new device, obtain the address information of the new device used in the communication via the gateway, and notify the device located in the space of the obtained address information. An information processing system. **Claim 9** In claim 8, The device is An information processing system that controls the device to be controlled without passing through the server system when the address information of the device to be controlled is obtained based on the information from the entrance / exit management device. **Claim 10** In any one of claims 4 to 6, The device is Obtain the address information of other devices located within a predetermined distance from the device from the server system, An information processing system that controls the device to be controlled without passing through the server system when the address information of the device to be controlled is obtained. **Claim 11** In any one of claims 1 to 6, The server system is Obtain a priority based on the device type information, and transmit priority information specifying the priority to the device, The device is An information processing system that transmits the data frame according to the priority specified based on the type of the device and the priority information. **Claim 12** In claim 3, The server system is Obtain a priority based on the device type information and the control type, and transmit priority information specifying the priority to the device, The device is An information processing system that transmits the data frame according to the type of the device, the control type included in the data frame to be transmitted, and the priority determined based on the priority information.

13. In the data link layer of communication with a device including an application that executes processing corresponding to the tacit knowledge of an expert, a communication unit that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer, A communication processing unit that controls the communication unit, Including, The communication unit, An information processing device that stores, in the frame body, a fixed-length area including a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information specifying the scene of assistance for the assisted person, and a third area for storing device type information specifying the type of the combined device used with the device, and transmits the data frame to the device as information for determining whether the application is active or inactive.

14. An information processing method in an information processing system including a device including an application that executes processing corresponding to the tacit knowledge of an expert and a server system that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in the data link layer of communication with the device, A fixed-length area including a first area for storing ability information representing the activity ability of the assisted person, a second area for storing scene information specifying the scene of assistance for the assisted person, and a third area for storing device type information specifying the type of the combined device used with the device is included in the frame body, and the data frame is transmitted from the server system to the device, Determining whether the application is active or inactive based on at least one of the ability information, the scene information, and the device type information, Information processing method.

Images