Information processing system and control method

Abstract

To provide an information processing system and a control method and the like suitable for fields such as assistance.SOLUTION: An information processing system includes: a first device used for assistance of a person to be assisted; a second device used in coordination with the first device; and a server system. The server system stores association information in which a plurality of devices used in coordination are associated with each other. The first device, when acquiring trigger information representing start of assistance, transmits a coordination request to the second device that has been determined as a coordination target device on the basis of the association information. The first device and the second device start processing related to assistance after executing preprocessing concerning the association information in accordance with a connection state with the server system.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, a control method, and the like. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there are known systems for managing a user's health condition, etc. For example, Patent Document 1 discloses a system for sharing real-time measurement data of a user (patient). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-503017 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention provides an information processing system and a control method suitable for fields such as caregiving. [Means for solving the problem]

[0005] In one aspect of the present disclosure, a system includes a first device used to assist a person being assisted, a second device used in conjunction with the first device in the assistance of the person being assisted, and a server system connected to the first device and the second device, wherein the server system stores correspondence information that associates a plurality of devices used in conjunction with the assistance of the person being assisted, and when the first device acquires trigger information indicating the start of the assistance, the first device determines a device to be linked based on the correspondence information and sends a link request to the second device determined to be the device to be linked, and the first device and the second device are related to an information processing system that performs pre-processing related to the correspondence information depending on the connection state with the server system and then starts processing related to the assistance.

[0006] Another aspect of the present disclosure relates to a control method for an information processing system including a first device used to assist a person being assisted, a second device used in conjunction with the first device in the assistance of the person being assisted, and a server system connected to the first device and the second device, wherein correspondence information that associates a plurality of devices used in conjunction with the assistance of the person being assisted is stored in the server system, and when trigger information indicating the start of the assistance is acquired in the first device, a device to be linked is determined based on the correspondence information, a link request is sent to the second device determined to be the device to be linked, pre-processing is performed on the correspondence information depending on the connection state between the first device and the second device and the server system, and then processing related to the assistance is started. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a configuration of an information processing system. [Diagram 2] FIG. 1 illustrates an example of the configuration of a server system. [Diagram 3] FIG. 13 is a diagram illustrating an example of the configuration of association information (table data). [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a first device. [Figure 5A] This is an example of training data for a positioning application. [Figure 5B] 1 is a specific example of a usage mode of a positioning application. [Figure 6] This is an example of a seat sensor placed on a wheelchair. [Figure 7] 1 is an example of a choking detection device. [Figure 8] 13 is an example of a screen displaying a processing result of the choking detection device. [Figure 9] FIG. 11 is a sequence diagram illustrating a periodic process of the information processing system. [Figure 10] FIG. 11 is a sequence diagram illustrating a cooperative process of the information processing system. [Figure 11]13 is an example of a screen displayed on a display unit of a first device. [Figure 12] 13 is an example of a screen displayed on a display unit of a first device. [Figure 13] FIG. 11 is a sequence diagram illustrating a periodic process of the information processing system. [Figure 14] FIG. 11 is a sequence diagram illustrating a cooperative process of the information processing system. [Figure 15] FIG. 11 is a sequence diagram illustrating a cooperative process of the information processing system. [Figure 16] 13 is an example of a screen displayed on a display unit of a first device. [Figure 17A] 13 is an example of a display screen of a positioning application using a smartphone. [Figure 17B] 13 is an example of a display screen of a positioning application using a smartphone. [Figure 18] 13 is an example of a display screen in a setting mode of a positioning application. [Figure 19A] 13 is an example of a display screen in a setting mode of a positioning application. [Figure 19B] 13 is an example of a display screen in a setting mode of a positioning application. [Figure 20] 1 is an example of a dashboard screen of a positioning application. [Figure 21A] 1 is an example of a dashboard screen of a positioning application. [Figure 21B] 1 is an example of a dashboard screen of a positioning application. [Figure 22A] 1 is an example of a dashboard screen of a positioning application. [Figure 22B] 1 is an example of a dashboard screen of a positioning application. [Figure 22C] 1 is an example of a dashboard screen of a positioning application. [Diagram 23] FIG. 2 is a diagram illustrating an example of the configuration of a MAC frame. [Figure 24]FIG. 2 is a diagram illustrating an example of a configuration of a frame body. [Diagram 25] FIG. 13 is a diagram illustrating an example of the relationship between a data type ID and contents. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the present embodiment will be described with reference to the drawings. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Note that the present embodiment described below does not unduly limit the contents described in the claims. In addition, all of the configurations described in the present embodiment are not necessarily essential configurations of the present disclosure.

[0009] 1. System configuration example 1 is a configuration example of an information processing system 1 according to this embodiment. The information processing system 1 includes a server system 10, a first device 100, and a second device 200. The server system 10 is connected to the first device 100 and the second device 200 via a network. The first device 100 and the second device 200 may be directly connected without going through the server system 10 (without going through a network used for communication with the server system 10). However, the configuration of the information processing system 1 is not limited to that shown in FIG. 1, and various modifications such as omitting some components or adding other components are possible.

[0010] The information processing system 1 according to this embodiment digitizes the tacit knowledge of an experienced caregiver, for example in a medical facility or nursing home, which is performed based on the "intuition" or "tacit knowledge" (hereinafter simply referred to as tacit knowledge), thereby supporting other caregivers to provide appropriate care regardless of their level of skill. For example, the tacit knowledge of an experienced caregiver is realized as application software. Hereinafter, application software will be referred to simply as application. Furthermore, an application that uses tacit knowledge will be referred to as a tacit knowledge application.

[0011] For example, in the information processing system 1 of this embodiment, the server system 10 may store and manage the tacit knowledge application, and transmit the tacit knowledge application to the first device 100 or the second device 200 as necessary. In this way, it becomes possible to centrally manage the tacit knowledge application using the server system 10, which has a relatively high processing capacity. In addition, by transmitting the tacit knowledge application to the first device 100 and the second device 200, a caregiver using each device can provide care similar to that of an expert.

[0012] However, the application that runs on at least one of the first device 100 and the second device 200 is not limited to digitized tacit knowledge, and may be software that supports the caregiver's assistance without using tacit knowledge. Below, an example in which a tacit knowledge application runs on both the first device 100 and the second device 200 will be described, but the application on at least one of the devices can be replaced with an application that does not use tacit knowledge.

[0013] In addition, the caregiver in this embodiment is a person in charge of providing care to the person being assisted, and includes, for example, a care manager, a caregiver, a visiting helper, etc. In addition, in home care, the family of the person being assisted may provide care to the person being assisted, so the caregiver in this embodiment may include the family of the person being assisted. In addition, the person being assisted in this embodiment may be a patient suffering from some kind of illness, and the caregiver may include medical professionals such as doctors, nurses, and pharmacists.

[0014] The information processing system 1 and each device included in the information processing system 1 will be described in detail below. The server system 10 is connected to the first device 100 and the second device 200 via a network, for example. The network here may be a public communication network such as the Internet, or a network such as a LAN. For example, the server system is connected to the first device 100 and the second device 200 using a communication method conforming to the IEEE802.11 standard. However, various modifications can be made to the communication method between the devices.

[0015] The server system 10 may be one server or may include multiple servers. For example, the server system 10 may include a database server and an application server. The database server stores various data such as association information, which will be described later with reference to FIG. 3. The application server performs processing, which will be described later with reference to FIG. 9, etc. The multiple servers here may be physical servers or virtual servers. In addition, when a virtual server is used, the virtual server may be provided in one physical server, or may be distributed and arranged in multiple physical servers. As described above, the specific configuration of the server system 10 in this embodiment can be modified in various ways.

[0016] The first device 100 and the second device 200 are devices used to assist a person being assisted (a patient) in, for example, a medical facility or a nursing home. In particular, the first device 100 and the second device 200 may be apparatuses used in cooperation with each other in providing specific assistance to a person being assisted.

[0017] 1 illustrates a smartphone as an example of the first device 100 and the second device 200. For example, at least one of the first device 100 and the second device 200 may be a device on which a positioning application, which will be described later with reference to FIGS. 5A and 5B, operates. The positioning application is an example of a tacit knowledge application.

[0018] However, the first device 100 and the second device 200 in this embodiment may be any device used in care and are not limited to smartphones. For example, the first device 100 and the second device 200 may be a seat sensor 440 described later with reference to Fig. 6, or a choking detection device 460 described later with reference to Fig. 7. The first device 100 and the second device 200 may be a detection device that detects information such as sleep, heart rate, and breathing based on the body movement of the person being assisted in bed, or may be other devices.

[0019] 2 is a block diagram showing a detailed configuration example of the server system 10. The server system 10 includes a processing unit 11, a storage unit 12, and a communication unit 13, for example.

[0020] The processing unit 11 of this embodiment is configured by the following hardware. The hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware may be configured by one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices are, for example, an integrated circuit (IC) or a field-programmable gate array (FPGA). The one or more circuit elements are, for example, a resistor, a capacitor, etc.

[0021] The processing unit 11 may be realized by the following processor. The server system 10 of the present embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The memory may be the storage unit 12 or another memory. The processor includes hardware. The processor can use various processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a digital signal processor (DSP). The memory may be a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, or may be a register, or may be a magnetic storage device such as a hard disk drive (HDD), or may be an optical storage device such as an optical disk device. For example, the memory stores computer-readable instructions, and the processor executes the instructions to realize the function of the processing unit 11 as processing. The instructions here may be instructions of an instruction set that constitutes a program, or may be instructions that instruct the hardware circuit of the processor to operate.

[0022] The processing unit 11 includes, for example, a capability information acquisition unit 14, a scene information acquisition unit 15, and a correspondence information processing unit 16.

[0023] The ability information acquiring unit 14 performs a process of acquiring ability information representing the activity ability of the person being assisted. For example, the ability information acquiring unit 14 acquires sensing data from at least one of the first device 100 and the second device 200. The sensing data here may be an output value of a sensor included in each device, or may be a processing result based on the output value. For example, the sensing data may include log data for a predetermined period of time of a tacit knowledge application.

[0024] For example, the ability information acquiring unit 14 obtains a time-series change in the state of the person being assisted based on the sensing data, and estimates the ability information based on the change. The ability information here may be index information related to daily activities (ADL: Activities of Daily Living). For example, various methods for evaluating ADL, such as the Barthel Index, are known, and these can be widely applied in this embodiment. In addition, the ability information may be a nine-level index disclosed as the Clinical Frailty Scale in "Frailty and the potential kidney transplant recipient: time for a more holistic assessment?", Henry HL Wu, etc. For example, the ability information acquiring unit 14 in this embodiment determines which of the nine levels the person being assisted belongs to based on the sensing data.

[0025] The capability information may be obtained in the first device 100 or the like. The capability information acquisition unit 14 may execute a process of acquiring the capability information from the first device 100 or the like via the communication unit 13.

[0026] The scene information acquisition unit 15 determines a scene in which the person being assisted is to be assisted. The scene information here may be information specifying the type of assistance to be performed, such as meal assistance, excretion assistance, and movement / transfer assistance. The scene information may also be information about the assistants who will be assisting the person being assisted, such as the number and skill level of the assistants. The scene information may also be information about the person being assisted, such as the attributes of the person being assisted. For example, the scene information acquisition unit 15 performs a process of obtaining scene information based on a user input, the schedule of the assistant, and the like. The scene information may also be information about the place where assistance is to be performed, as shown in FIG. 3.

[0027] The corresponding information processing unit 16 performs processing related to the creation, update, and transmission of the corresponding information. Fig. 3 shows an example of the corresponding information. The corresponding information in this embodiment may include information for identifying the person being assisted, information on the ability of the person being assisted, information for identifying the assistance scene, information for identifying the type of assistance to be performed, information for identifying the device used for assistance, information for identifying tacit knowledge used for assistance, and information on the update date and time of the corresponding information. The corresponding information may also include information on the expiration date of the corresponding information, which is not shown in Fig. 3. The corresponding information is, for example, table data (or in a narrow sense, a relational database) in which item values ​​are associated with each item.

[0028] However, the configuration of the correspondence information is not limited to the example in Fig. 3, and some items may be omitted, and other items may be added. Furthermore, the correspondence information is not limited to table data, and other formats of data may be used. Furthermore, while Fig. 3 illustrates four rows of data as table data, the item values ​​of each row are merely examples, and various modifications of the specific item values ​​are possible, as described below.

[0029] The information identifying the person being assisted is, for example, an assisted person ID, which is an ID uniquely assigned to the person being assisted. The assisted person ID is associated with information such as the name, address, and facility of the person being assisted. The assisted person ID may also be associated with attribute information of the person being assisted, such as age, sex, height, weight, medical history, and medication history. Hereinafter, the information associated with the assisted person ID is also referred to as user information.

[0030] The ability information of the person being assisted corresponds to the ability information acquired by the ability information acquisition unit 14. The assistance scene corresponds to the scene information acquired by the scene information acquisition unit 15.

[0031] The information specifying the type of assistance to be performed is information specifying specific assistance content, such as, for example, meal assistance, excretion assistance, movement assistance, etc. Note that meal assistance and excretion assistance may be further subdivided.

[0032] The information identifying the device used for assistance may be information (type ID) identifying the type of device, such as a "smartphone," a "choking detection device," or a "seat sensor." Alternatively, the information identifying the device may be information (device ID) identifying one of multiple devices belonging to a specific type. The device ID is information that uniquely identifies a device, such as a serial number or a MAC address.

[0033] The information for identifying the tacit knowledge used for the assistance is, for example, an application ID that is an ID uniquely assigned to each of a plurality of tacit knowledge applications.

[0034] The update date and time is information indicating the timing when the target association information was updated, and is, for example, information specifying the year, month, and day. The update date and time may also include hour and minute information. For example, when the association information is table data with the configuration shown in FIG. 3, the data of each row includes, as an item value of the update date and time, information indicating the timing when the row was newly added or when at least one of the item values ​​of the row was updated. Note that, although an example in which an update date and time is associated with each row of data has been shown here, this is not limited to this. For example, the association information may include one update date and time information for the entire association information. In this case, the update date and time indicates the timing when the last update process was performed on any row of the association information.

[0035] As described above, the association information may include expiration date information not shown in FIG. 3. The expiration date is information indicating a period during which the target data is valid (or, in a narrow sense, the timing at which the data becomes invalid). For example, the expiration date is information used in a device (the first device 100 or the second device 200) that received the association information to determine whether or not the association information may be used. The expiration date here may be, for example, a value of expiration data, which will be described later with reference to FIG. 24. Details of the expiration date will be described later. The expiration date may also be data set for each row of the association information, or may be data set only once for the entire association information received by the first device 100 or the like.

[0036] As described above, the update date and time is information that indicates the update timing, and is used, for example, for version management (described later using steps S203-S207 in FIG. 10). On the other hand, the expiration date is used to determine whether the received association information may be used, as described above (described later using FIGS. 23-25). Therefore, the update date and time and the expiration date may be different pieces of information. However, since it is conceivable that the expiration date may be set based on the update deadline, one piece of information may also serve as the other piece of information, and this does not prevent the other piece of information from being omitted. Furthermore, the expiration date is information that is acquired as part of a packet, as described later using FIG. 24, and may be treated as information different from the association information (metadata of the association information).

[0037] For example, even when providing the same meal assistance, an experienced caregiver is considered to change the tacit knowledge he or she uses, taking into consideration the person being assisted and the situation (ability, scene, etc.) in which the person being assisted is placed. In other words, not only the content of the assistance, but also the condition judgment for performing the assistance is considered to be tacit knowledge. In this regard, the association information processing unit 16 of this embodiment associates a device ID and an application ID with the ID of the person being assisted, the ADL, and the scene, respectively. This makes it possible not only to digitize the assistance itself performed by the expert (create a tacit knowledge application), but also to appropriately digitize the conditions for using the tacit knowledge application.

[0038] The correspondence information may also include information associating multiple devices used in cooperation with each other in assisting a person being assisted. For example, consider table data including a row in which the assisted person ID, ADL, scene, and type of assistance are common, and the device ID is device ID1, and another row in which the device ID is device ID2. In this case, it is represented that in one assistance specified by the assisted person ID, ADL, scene, and type of assistance, two devices, a device corresponding to device ID1 and a device corresponding to device ID2, are used in cooperation with each other. However, one row may include information on multiple device IDs used in cooperation with each other, and various modifications are possible for the data configuration for associating multiple devices used in cooperation with each other.

[0039] The association information processing unit 16 performs a process of updating the association information when, for example, sensing data related to the assisted person is acquired from the first device 100 and the second device 200. For example, when the ADL of the target assisted person is deteriorated, the association information processing unit 16 performs a process of adding data to the association information so that more thorough assistance is provided than before. Specifically, the association information processing unit 16 may add a row including item values ​​of the assistance type, device ID, and application ID that were not associated with the target assisted person to the table data.

[0040] Furthermore, the association information processing unit 16 may perform a process of transmitting association information to the first device 100 when connected to the first device 100 via a network. Specific processing will be described later with reference to FIG. 10 and the like. In this case, by including information on the update date and time in the association information, it is possible to prevent the first device 100 from using excessively old data for processing. Similarly, the association information processing unit 16 may perform a process of transmitting association information to the second device 200 when connected to the second device 200 via a network.

[0041] The storage unit 12 is a work area of ​​the processing unit 11 and stores various information. The storage unit 12 can be realized by various memories, and the memory may be a semiconductor memory such as an SRAM, a DRAM, a ROM, or a flash memory, a register, a magnetic storage device, or an optical storage device.

[0042] The storage unit 12 stores, for example, tacit knowledge applications created in the server system 10 or other devices, and the association information shown in Fig. 3. The storage unit 12 may also store user information, device information, application information, and the like. As described above, the user information is various information related to the person being assisted that is associated with the person being assisted ID. The device information is information such as a type ID, a vendor, and an application ID of an installed application that is associated with the device ID. The application information is information such as an application name, a creator, and the like that is associated with the application ID.

[0043] The communication unit 13 is an interface for performing communication via a network, and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit when the server system 10 performs wireless communication. However, the server system 10 may perform wired communication, in which case the communication unit 13 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 13 may operate under the control of the processing unit 11, or may include a processor for communication control different from the processing unit 11.

[0044] FIG. 4 is a block diagram showing a detailed configuration example of the first device 100. The first device 100 includes, for example, a processing unit 110, a storage unit 120, a communication unit 130, a display unit 140, and an operation unit 150. As will be described later with reference to FIG. 6 and FIG. 7, the method of this embodiment allows devices of various aspects to be used as the first device 100. Therefore, the configuration of the first device 100 is not limited to that shown in FIG. 4, and modifications such as omitting some components or adding other components are possible. For example, the first device 100 may have various sensors according to the aspect of the device, such as a motion sensor such as an acceleration sensor or a gyro sensor, an image sensor, a pressure sensor, a GPS (Global Positioning System) sensor, etc.

[0045] The processing unit 110 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 110 may also be realized by a processor. The processor may be of various types, such as a CPU, a GPU, or a DSP. The processor executes instructions stored in the memory of the first device 100, thereby realizing the function of the processing unit 110 as processing.

[0046] The storage unit 120 is a work area of ​​the processing unit 110, and is realized by various memories such as SRAM, DRAM, ROM, etc. For example, the storage unit 120 stores the association information and the tacit knowledge application acquired from the server system 10.

[0047] The communication unit 130 is an interface for performing communication via a network, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 130 performs communication with the server system 10, for example, via the network. The communication unit 130 may perform wireless communication conforming to, for example, the IEEE802.11 standard with a gateway (relay device) such as a router, and may perform communication with the server system 10 via the gateway.

[0048] The communication unit 130 of the first device 100 may include a first communication unit that communicates with the server system 10 and a second communication unit that communicates with the second device 200. The first communication unit and the second communication unit may have different communication methods. For example, the first communication unit connects to the server system 10 via a network such as a LAN or the Internet by communication according to the IEEE802.11 standard. The second communication unit connects to the second device 200 by using short-range wireless communication such as Bluetooth (registered trademark) or ZigBee.

[0049] For example, the first device 100 controls the operation of the first communication unit and the operation of the second communication unit independently. For example, the first device 100 may turn on the second communication unit when the first communication unit is off or in a standby state. In this case, the first device 100 is not connected to the server system 10 but is connected to the second device 200. By reducing the frequency of communication with the server system 10, the processing load of the server system 10 and the communication load of the network can be reduced. Even if the first device 10 is not connected to the server system 10, the second device 200 can be connected, so that a cooperative process related to assistance can be executed. Note that, although an example in which the communication methods of the first communication unit and the second communication unit are different has been described above, this is not limiting. For example, a configuration in which the communication method of the first communication unit and the second communication unit is common and the connection / disconnection with the server system 10 and the connection / disconnection with the second device 200 can be controlled independently may be used.

[0050] The display unit 140 is an interface that displays various information, and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 150 is an interface that accepts user operations. The operation unit 150 may be a button or the like provided on the first device 100. Furthermore, the display unit 140 and the operation unit 150 may be a touch panel that is integrally configured.

[0051] The first device 100 may also include a configuration not shown in FIG. 4. For example, the first device 100 has an imaging unit (image sensor) that outputs image information by imaging a predetermined imaging range. The image information here may be a still image or a moving image. The image information may also be color or monochrome. The imaging unit may also include a depth sensor that detects the distance to the subject, or a sensor (e.g., an infrared sensor) that detects the heat of the subject. The first device 100 may also include a light-emitting unit, a vibration unit, a sound input unit, a sound output unit, and the like. The light-emitting unit is, for example, an LED (light emitting diode) and notifies by emitting light. The vibration unit is, for example, a motor and notifies by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and notifies by sound.

[0052] The second device 200 is similar and includes, for example, a processing unit, a storage unit, a communication unit, a display unit, and an operation unit. The communication unit of the second device 200 may also be capable of independently controlling communication with the server system 10 and communication with the first device 100. Also, like the first device 100, devices of various aspects described later with reference to Figures 6 and 7 can be used as the second device 200, and the configuration of the second device 200 can be modified in various ways.

[0053] As described above, the information processing system 1 according to this embodiment includes the first device 100 used to assist the person being assisted, the second device 200 used in cooperation with the first device 100 in assisting the person being assisted, and the server system 10 connected to the first device 100 and the second device 200. As described above with reference to Fig. 3, the server system 10 stores association information that associates a plurality of devices used in cooperation in assisting the person being assisted.

[0054] In the present embodiment, when the first device 100 acquires trigger information indicating the start of assistance, the first device 100 may determine a device to be linked based on the association information and transmit a link request to the second device determined to be a device to be linked. The link request corresponds to, for example, step S209 in FIG. 10, which will be described later. Then, the first device 100 and the second device 200 execute pre-processing related to the association information according to the connection state with the server system 10, and then start processing related to the assistance. The pre-processing may be an update process or a determination process of the association information used in the linking process, and the specific processing content differs according to the connection state. For example, in the example shown in steps S204-S217 in FIG. 10 or steps S404-S418 in FIG. 14, which will be described later, a process of updating the association information used in the linking process is performed as a pre-processing. Note that the update process here may include a process of inquiring whether an update is necessary, and if an update is not necessary, using the existing association information as it is. Also, in the example shown in steps S504-S506 in FIG. 15, which will be described later, a process of determining the association information used in the linking process is performed as a pre-processing.

[0055] For example, in a care facility or the like, the first device 100 and the second device 200 (for example, a smartphone used by a caregiver or a device described later with reference to FIG. 6 and FIG. 7) are not always connected to the server system 10 via a network. For example, these devices may be connected to the server system 10 for a limited period, such as when they transmit log data of operation results, and may be disconnected from the server system 10 during other periods. Even in this case, if the first device 100 and the second device 200 acquire association information in advance, they can identify the equipment to be linked and the tacit knowledge to be used based on the association information. Furthermore, if the first device 100 and the second device 200 are directly connected as described above, multiple devices can work in conjunction with each other. By shortening the connection period with the server system 10, it is possible to reduce communication load and power consumption.

[0056] However, since the assistance to be performed and the tacit knowledge used in the assistance change depending on the ability information of the person being assisted, the association information may be updated as described above. Therefore, if the first device 100 is not connected to the server system 10, there is a possibility that the latest association information may not be available depending on the timing. The same applies to the second device 200.

[0057] In this regard, in this embodiment, a pre-processing according to the connection state with the server system 10 is performed, and then the first device 100 and the second device 200 start a cooperative process. Therefore, it is possible to perform a cooperative process appropriate to the situation in each of the cases where both devices are connected to the server system 10 (for example, FIG. 10 described later), where only one of the devices is connected to the server system 10 (for example, FIG. 14 described later), and where neither device is connected to the server system 10 (for example, FIG. 15 described later). Details of the process will be described later.

[0058] In the above, an example has been described in which the information processing system 1 includes the first device 100 and the second device 200, and these two devices perform cooperative processing. However, the information processing system 1 may include three or more devices. And, the cooperative processing related to a specific care may be performed by three or more devices working in cooperation with each other.

[0059] In addition, a part or all of the processes performed by the information processing system 1 of the present embodiment may be realized by a program. In a narrow sense, the processes performed by the information processing system 1 are the processes performed by the processing unit 11 of the server system 10, the processes performed by the processing unit 110 of the first device 100, and the processes performed by the processing unit of the second device 200, but one or two of these processes may be omitted.

[0060] The program according to this embodiment can be stored in a non-transitory information storage medium (information storage device), which is a medium readable by a computer, for example. The information storage medium can be realized by, for example, an optical disk, a memory card, a HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 and the like perform various processes of this embodiment based on the program stored in the information storage medium. That is, the information storage medium stores a program for making a computer function as the processing unit 110 and the like. A computer is a device equipped with an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for making a computer execute each step described later with reference to Figs. 9-10, 13-15, and the like.

[0061] The technique of this embodiment can also be applied to a control method including the following steps. The control method according to this embodiment is a control method for an information processing system including a first device used to assist a person being assisted, a second device used in cooperation with the first device in assisting the person being assisted, and a server system connected to the first and second devices. The control method includes the steps of: storing, in the server system, association information that associates a plurality of devices used in cooperation in assisting the person being assisted; determining a linking target device based on the association information when trigger information indicating the start of assistance is acquired in the first device, and transmitting a linking request to the second device determined to be the linking target device; and executing pre-processing according to the connection state between the first and second devices and the server system, and then starting processing related to assistance.

[0062] 2. Specific Examples of the First Device and the Second Device Next, a specific example of the first device 100 and the second device 200 will be described. In this embodiment, the master device that detects a trigger of the cooperation process is represented as the first device 100, and the slave device that performs the cooperation process based on the cooperation request from the first device 100 is represented as the second device 200. Each device described below may function as the first device 100 or the second device 200. In addition, it is not necessary to fix whether it is the first device 100 or the second device 200. That is, each device described below may function as the first device 100 in one type of assistance and as the second device 200 in another type of assistance. In addition, the first device 100 and the second device 200 are not limited to the devices described below, and may include a device that detects the amount of food intake, a device that detects that the assisted person is trying to touch the skin or the inside of the assisted person's clothing (pants or underwear) by putting his / her hand inside the clothing (trying to touch it), etc. In addition, the devices described below may execute different processes by operating according to different applications. For example, the choking detection device 460 described below may function as a medication management device that manages the administration of medicine by a person being assisted by detecting the swallowing of medicine.

[0063] 2.1 Smartphone or PC (positioning application) For example, at least one of the first device 100 and the second device 200 may be a smartphone or a PC. In the smartphone or the PC, a positioning application may run as the tacit knowledge application.

[0064] A positioning application is an application that performs processing related to the position or posture of at least one of a person and an object during care. The positioning application may be used to adjust the posture of a person being assisted in a bed, or may be used to adjust the posture of a person being assisted in a wheelchair. For example, the positioning application may operate in a setting mode in which settings are made, and a usage mode in which actual position adjustment is supported according to the settings. For example, in the setting mode, the positioning application obtains teacher data in which a person or object is imaged at a desired position based on the operation of an expert. Then, in the operation mode, the positioning application superimposes and displays the teacher data that has been subjected to transparency processing on the image of the person or object to be adjusted.

[0065] FIG. 5A is an example of teacher data acquired in the setting mode. In the example of FIG. 5A, image information showing a desired posture when a person being assisted lies on a bed is acquired as teacher data. In the example shown in FIG. 5A, the teacher data may be an image of a cushion, and a triangular or rectangular figure may be displayed at a position corresponding to the cushion as additional information. The figure is input by, for example, a user (such as an administrator described later using FIG. 18-FIG. 19B) who sets the teacher data. FIG. 5B is a diagram showing a usage status of a positioning application in a usage mode and a specific example of a display screen. For example, a device such as a smartphone on which a positioning application runs superimposes the teacher data shown in FIG. 5A on a captured image of a person being assisted who is the target of position adjustment. For example, the positioning application may perform a transparency process on the teacher data, and then superimpose the teacher data after the transparency process (corresponding to the dashed line in FIG. 5B) on the captured image. The caregiver assists the person being assisted in the captured image so that the person being assisted approaches the teacher data. In this way, it is possible to appropriately support the position adjustment by the caregiver.

[0066] Although an example of superimposing and displaying the teacher data, which is image information, has been described here, the positioning application may output a judgment result (OK / NG) as to whether the posture of the person being assisted is appropriate or not. For example, the positioning application may judge whether it is OK or NG based on the degree of similarity between the image captured during position adjustment and the teacher data, and output the judgment result. Specifically, the positioning application may judge whether the posture of the person being assisted or the caregiver is appropriate or not, or may judge whether the position of a tool such as a cushion is appropriate or not. The judgment of whether it is OK or NG may be based on the skeletal information of the person being assisted contained in the teacher data and the skeletal information of the person being assisted contained in the image captured during position adjustment. For example, since the skeletal information includes information on several feature points as described later with reference to FIG. 19B, it is possible to specify the coordinate values ​​of the feature points. The coordinate values ​​here are values ​​in x and y coordinates that represent the vertical and horizontal positions of the image. Furthermore, when depth information is acquired, the coordinate values ​​may include a z coordinate value that represents the distance from the camera. Furthermore, different coordinate systems may be set as two-dimensional or three-dimensional coordinates. The positioning application determines which feature points contained in the skeletal information being adjusted for position have a degree of difference from the coordinate values ​​in the training data that is greater than or equal to a predetermined value, and determines that the point is NG if the number of points with large differences exceeds a threshold value.

[0067] The positioning application may also perform a process of displaying specific points that are determined to be NG. For example, the positioning application may compare the captured image with the teacher data and perform a process of highlighting the points that are determined to have a large difference. Alternatively, the positioning application may output specific instructions such as how to change the position of the cushion or how to change the posture. The points that are determined to have a large difference may be based on the skeletal information of the person being assisted contained in the teacher data and the skeletal information of the person being assisted contained in the image captured during the position adjustment. For example, the positioning application may determine the coordinate values ​​of the above-mentioned feature points and highlight points with a difference of a predetermined value or more as points with a large difference. For example, the black frame shown in FIG. 5B and the feature points corresponding to the ankles correspond to the highlighting result. Also, the rightward arrow in FIG. 5B is information that indicates the direction in which the posture is changed.

[0068] The positioning application may also accept additional information such as points that an expert considers important in the setting mode, and present the additional information in the use mode. The additional information may include information such as the position and angle of a specific part, the presence or absence, size, and softness of a cushion. The positioning application may also recommend welfare equipment suitable for the person being assisted, if it is determined based on the result of the posture determination of the person being assisted that the person being assisted can easily adjust his / her posture by using welfare equipment such as a cushion. Even if the person being assisted is already using a cushion, if the position adjustment of the cushion is not properly performed, or if the size of the cushion is different from the size of the cushion included in the teacher data, a new cushion or the like may be recommended. The recommendation here may specify the size, hardness, etc. of the cushion, or may specify a specific product (model number, etc.). When a specific product is specified, the positioning application may output link information to an EC site or the like that sells the product.

[0069] The positioning application may also display a questionnaire about the set teacher data when setting the teacher data in the setting mode. This questionnaire about the teacher data is conducted to obtain information necessary for interpreting the teacher data, such as the purpose of setting the teacher data (e.g., bedsore purpose, pressure relief purpose) and the target area for the bedsore purpose, etc.

[0070] The positioning application may accumulate training data by prompting the caregiver to manually input information to a questionnaire at the start of the service. When a predetermined number of pieces of training data or more are accumulated, the positioning application performs a learning process based on the training data. The learning process here is performed based on training data including both teacher data and input results to the questionnaire. As a result, in the trained model generated by the learning process, it is possible to output appropriate teacher data for each purpose of setting the teacher data and each target part. For example, in the case of bed positioning targeting a certain part, the learning model can determine where a cushion of what size should be placed and what posture the person being assisted should be in. In other words, after the trained model is created, it is possible to check the optimal positioning in the position app by simply selecting the target part without setting teacher data. Furthermore, when the service is started, the training data may be accumulated by prompting the caregiver to manually input desired welfare equipment. When a predetermined number of pieces of training data or more are accumulated, the positioning application may create a trained model that recommends welfare equipment by performing a learning process based on the training data.

[0071] Specific examples of the setting mode of the positioning application will be described later with reference to Figs. 17A to 19B. Also, specific examples of the dashboard screen when presenting the processing results of the positioning application to the user will be described later with reference to Figs. 20 to 22C.

[0072] In addition, various modifications are possible regarding how to manage and use application A used to adjust the position of the person being assisted AAA and application B used to adjust the position of the person being assisted BBB. For example, application A and application B may be treated as different applications in the first place. For example, on a smartphone, each application is started by selecting a different icon.

[0073] Alternatively, application A and application B may share a common interface application and be switchable within the interface application. For example, in a smartphone or the like, an icon of a positioning application may be selected first, and then application A and application B may be switched by selecting a tab within the positioning application.

[0074] Note that application A and application B may have a common process (algorithm) and different parameters used in the process, or both the process and the parameters may be different. Furthermore, even if it is the same application A, application A1 that cooperates with other applications (devices) and application A2 that does not cooperate with other applications may be treated as different applications. Furthermore, the above-described management and usage modes of applications are not limited to positioning applications, and the same applies to other applications such as the choking application used in choking detection device 460.

[0075] 2.2 Seat sensor 6 is an example of at least one of the first device 100 and the second device 200, and is a diagram showing a seat sensor 440 arranged on the seat of a wheelchair 630. The seat sensor 440 includes a pressure sensor that outputs a pressure value, and outputs the pressure value.

[0076] In the example of FIG. 6, four pressure sensors Se1 to Se4 are arranged on the back side of a cushion 441 arranged on the seat of a wheelchair 630. Pressure sensor Se1 is a sensor arranged in the front, pressure sensor Se2 is a sensor arranged in the rear, pressure sensor Se3 is a sensor arranged on the right, and pressure sensor Se4 is a sensor arranged on the left. Note that front, back, left, and right here refer to directions seen from the person being assisted when the person is sitting on the wheelchair 630. The pressure sensors Se1 to Se4 are connected to a control box 442. The control box 442 includes a processor therein for controlling the pressure sensors Se1 to Se4, and a memory that serves as a work area for the processor. The processor detects pressure values ​​by operating the pressure sensors Se1 to Se4.

[0077] The seat sensor 440 may determine whether the posture of the person being assisted sitting on the wheelchair 630 (hereinafter also referred to as the sitting posture) is normal, forward shifted, side shifted, or any of a plurality of postures including side shifted, etc., based on the pressure values ​​from the pressure sensors Se1 to Se4. A forward shift indicates a state in which the center of gravity of the user is shifted forward from normal, and a side shift indicates a state in which the center of gravity of the user is shifted to either the left or right from normal. For example, the seat sensor 440 determines a forward shift when the value of the pressure sensor Se1 increases by a predetermined value or more compared to the initial state, and determines a side shift when the value of the pressure sensor Se3 or Se4 increases by a predetermined value or more compared to the initial state. The seat sensor 440 may also perform a fall possibility determination to determine whether or not there is a possibility that the person being assisted will fall off the seat.

[0078] The judgment conditions such as normal, forward slippage, side slippage, etc., or the judgment conditions for the possibility of falling may differ depending on the person being assisted and the scene. For example, the seat sensor 440 may execute the above judgment based on a tacit knowledge application. The seat sensor 440 may also add or update the tacit knowledge application based on communication with the server system 10.

[0079] At least one of the first device 100 and the second device 200 of the present embodiment may be a mattress capable of detecting pressure, or may be a device that communicates with the mattress. These devices determine the posture of the person being assisted on the bed.

[0080] 2.3 Choking Detection Device 7 is a diagram illustrating a choking detection device 460 used in a mealtime as an example of at least one of the first device 100 and the second device 200. As shown in Fig. 7, the choking detection device 460 includes a throat microphone 461 that is attached around the neck of the person being assisted, and a terminal device 462 having a camera.

[0081] The throat microphone 461 outputs voice data generated by swallowing, coughing, etc. of the person being assisted. The camera of the terminal device 462 outputs a captured image of the person being assisted eating. The terminal device 462 is, for example, a smartphone or a tablet PC placed on a table where the person being assisted eats. The throat microphone 461 is connected to the terminal device 462 using Bluetooth or the like. However, the specific connection mode can be modified in various ways.

[0082] The swallowing choking detection device 460 judges whether the person being assisted has choked or swallowed based on the voice data of the throat microphone 461. A device that detects swallowing using a microphone attached around the neck is described, for example, in U.S. Patent Application No. 16 / 276,768, filed February 15, 2019, entitled "Swallowing action measurement device and swallowing action support system." This patent application is incorporated by reference in its entirety into this specification. The swallowing choking detection application can detect the number of choking incidents, the duration of choking (time of occurrence, duration, etc.), and whether or not the person swallowed based on the voice data.

[0083] The camera of the terminal device 462 captures an image of the person being assisted from the front direction, as shown in Fig. 7, for example. Therefore, the choking detection application may detect the mouth and eyes of the person being assisted, and chopsticks and spoons used by the person being assisted, based on the captured image. There are various known methods for detecting these facial parts and objects based on image processing, and known methods can be widely applied in this embodiment.

[0084] For example, the choking detection device 460 may obtain the swallowing time from when the assisted person opens his / her mouth to when he / she swallows, based on the swallowing detection result and the opening / closing determination result of the mouth of the assisted person. In this way, for example, when the number of swallows has decreased, it is possible to determine specific situations, such as whether the action of putting food in the mouth is not performed, or whether food is put in the mouth but not swallowed. As a result, it is possible to accurately determine the risk of aspiration during meals. For example, the choking detection device 460 may perform these determinations based on a tacit knowledge application. The choking detection device 460 may also add or update a tacit knowledge application based on communication with the server system 10.

[0085] Fig. 8 is an example of a display screen that displays a processing result of the choking detection device 460. The screen shown in Fig. 8 may be displayed on the display unit of the terminal device 462, or may be displayed on another device connected to the choking detection device 460.

[0086] The screen displaying the processing results of the choking detection device 460 may include areas RE1 to RE6 as shown in Fig. 8. Area RE1 is an area where an image of the person being assisted captured by an imaging unit of the terminal device 462 is displayed, for example. When the choking detection device 460 and a positioning application work together, area RE1 may display the processing results of the positioning application (for example, an image on which teacher data is superimposed as shown in Figs. 5A and 5B).

[0087] In the region RE2, for example, the swallowing time described above is displayed. Note that the display in the region RE2 may be reset every time the person being assisted opens his / her mouth, or an average time for one meal may be displayed.

[0088] The area RE3 is an area used for inputting parameters used in the judgment process of the choking detection device 460. For example, the sound detected by the throat microphone 461 changes depending on the type of food (solid food, jelly, chopped food, etc.) of the person being assisted, so the judgment accuracy is improved by switching the parameters of the judgment process. The parameters of the judgment process may be, for example, parameters of a trained model used in an implicit knowledge application. For example, the area RE3 is displayed in advance to indicate the type of food, such as solid food, jelly, chopped food, etc., and the display of the check box on the left side changes when one of them is selected. In this way, it is possible to easily switch parameters using the screen of FIG. 8, and to clearly indicate the selected parameter to the user. However, the information displayed in the area RE3 is not limited to this, and the judgment result of the choking detection device 460 may be displayed. For example, the choking detection device 460 may judge the state of the person being assisted as normal, caution, or abnormal based on the occurrence or non-occurrence of choking, the risk level, the state of swallowing, etc., and display the result in the area RE3 using color, text, etc. Furthermore, the result of the determination process may be displayed in a separate pop-up window, rather than in area RE3.

[0089] Area RE4 is an area where the detection result of the throat microphone is displayed. For example, the horizontal axis of area RE4 may represent time, and the vertical axis may represent the strength (loudness) of the audio signal. For example, the choking detection device 460 may display the detection result of the throat microphone in area RE4 in real time, or may display an audio waveform corresponding to one swallow in area RE4 each time a swallow is detected.

[0090] The area RE5 is an area where a camera switching button is displayed. For example, when the terminal device 462 includes multiple imaging units (an in-camera, an out-camera, an external web camera, etc.), the choking detection device 460 determines which imaging unit's image is to be displayed in the area RE1 based on the operation result of the camera switching button.

[0091] An execution button is displayed in the area RE6. When the execution button is operated, the choking detection device 460 starts the various processes described above and the screen update process shown in FIG.

[0092] 3. Processing Details Next, the processing of the information processing system 1 according to the present embodiment will be described for each connection state between the server system 10 and the first device 100 and the second device 200.

[0093] 3.1 When both devices are connected to a server system For example, the process of this embodiment may include two processes: a periodic process that is periodically executed, and a cooperative process in which the first device 100 and the second device 200 cooperate to execute a process related to assistance using the association information. The periodic process here is not necessarily limited to a process executed at a fixed cycle, and the execution timing can be set in various modified forms. The periodic process is assumed to be executed, for example, at a timing when the cooperative process is not being performed, but it is not prevented from being executed in parallel when the cooperative process is being performed. As will be described later with reference to FIG. 9, the periodic process includes a process of transmitting data from a device such as the first device 100 to the server system 10, a process of updating the association information in the server system 10, a process of transmitting association information and tacit knowledge applications from the server system 10 to the first device 100, etc.

[0094] Fig. 9 is a sequence diagram illustrating periodic processing when both the first device 100 and the second device 200 are connected to the server system 10. The information processing system 1, for example, periodically executes the processing shown in Fig. 9. For example, the periodic processing may be performed during a time when the processing load and communication load are relatively low, such as late at night.

[0095] First, in step S101, the first device 100 acquires sensing data using a sensor mounted on the first device 100. Note that the acquisition of sensing data here is not limited to a process of acquiring an output value of a sensor by executing a tacit knowledge application in real time, and may be a process of reading out sensing data accumulated by past execution of a tacit knowledge application from the storage unit 120. In step S102, the communication unit 130 of the first device 100 transmits the acquired sensing data to the server system 10. For example, the first device 100 accumulates sensing data detected after last transmitting sensing data to the server system 10 as log data in the storage unit 120 (step S101), and transmits the log data to the server system 10 (step S102).

[0096] In step S103, the second device 200 acquires sensing data using a sensor mounted on the second device 200. In step S104, the communication unit of the second device 200 transmits the acquired sensing data to the server system 10. Similarly, in the process of step S103, the second device 200 may be executing a tacit knowledge application, or may read from the storage unit 120 a result of a tacit knowledge application executed in the past.

[0097] In step S105, the processing unit 11 (the ability information acquiring unit 14) of the server system 10 calculates the ADL of the person being assisted based on the sensing data acquired from the first device 100 and the second device 200.

[0098] In step S106, the processing unit 11 (association information processing unit 16) of the server system 10 updates the association information as necessary based on the ADL calculated in step S105. If the association information is updated, the processes of steps S107-S117 described below are executed. If the association information is not updated, the processes of steps S107-S117 may be omitted.

[0099] In step S107, the communication unit 13 of the server system 10 transmits the updated association information to the first device 100. The information transmitted here may be the entire association information or a part of it. For example, if the association information is table data, the server system 10 may transmit all the rows of the table data to the first device 100, or may transmit some of the rows of the table data that are related to the first device 100 to the first device 100.

[0100] In step S108, the first device 100 stores the association information transmitted from the server system 10 in the storage unit 120. Note that, hereinafter, the association information stored in the first device 100 is also referred to as first association information.

[0101] In step S109, processing unit 110 of first device 100 determines, based on the newly acquired association information, an application that may be executed by first device 100, and determines whether the application has been installed.

[0102] If there is an application that has not been installed, in step S110, the communication unit 130 of the first device 100 requests the necessary application from the server system 10. In step S111, the server system 10 transmits the requested application to the first device 100. In step S112, the processing unit 110 of the first device 100 installs the obtained application.

[0103] The processes of steps S109-S112 result in the first device 100 having installed therein an application that may be executed when the first device 100 assists the person being assisted. At this time, whether or not there is a possibility of execution is determined based on the correspondence information updated in step S106. That is, the processes of steps S109-S112 enable the first device 100 to execute a process corresponding to the state of the person being assisted at this time point.

[0104] 9, since the second device 200 is also connected to the server system 10, the second device 200 is also subjected to the same processing as that of steps S107-S112 in steps S113-S117. In the following, the association information stored in the second device 200 is also referred to as second association information. The first association information and the second association information may be the same or different. For example, when the server system 10 transmits only a part of the association information, the first association information and the second association information may be different even if the latest association information in the server system (hereinafter, also referred to as a master table) is common. In addition, depending on the timing of updating the association information (S106) and the timing of transmitting the association information (S107, S113), the master table may be different in the first place.

[0105] 9, an example in which the processing unit 11 of the server system 10 updates the association information based on the sensing data from the first device 100 and the second device 200 has been described, but the present invention is not limited to this. For example, a sensing device that only performs sensing without executing processing according to an application may be placed around the person being assisted. The server system 10 may update the association information based on the sensing data from the sensing device. The sensing device here may include devices that include various sensors such as a thermometer, a hygrometer, and a light meter.

[0106] As described with reference to FIG. 9, the server system 10 may transmit the association information to the first device and the second device (steps S107 and S113). Then, the first device 100 may store the association information as the first association information (step S108), and the second device 200 may store the association information as the second association information (step S114). In this way, the association information is distributed to each device used for assistance, so that each device can execute appropriate processing according to the ability, scene, etc. of the person being assisted. In addition, as described above, the association information may include information about the equipment used in cooperation with each other, so that the first device 100 and the second device 200 can be appropriately linked.

[0107] FIG. 10 is a sequence diagram for explaining the cooperation process between the first device 100 and the second device 200. First, in step S201, the first device 100 acquires trigger information indicating a trigger of the cooperation process. The trigger information here is, for example, information indicating that a specific application has been started in the first device 100. For example, when the application (choking application) described above using FIG. 8 is started in the choking detection device 460, it is considered to be the timing when the person being assisted starts eating. Therefore, when the application of the choking detection device 460 is started, a positioning application is started in the second device 200 (smartphone, etc.), so that the person being assisted can take a posture that makes it difficult for choking or aspiration to occur during eating. In this case, the choking detection device 460 is the first device 100, and the smartphone is the second device 200. The device that cooperates based on the trigger of the choking detection device 460 may be the seat sensor 440. As will be described later, the start trigger of the cooperation process is not limited to the start of an application, and may be that the sensing data satisfies some condition.

[0108] In step S202, the processing unit 110 of the first device 100 identifies the person being assisted who is the target of assistance. For example, when the first device 100 is a choking detection device 460, the choking detection device 460 may identify the person being assisted based on image recognition processing using an imaging unit of the terminal device 462. For example, the terminal device 462 may perform face recognition processing to recognize the face of the person being assisted. Alternatively, the person being assisted or the person being assisted may have a marker (such as a barcode or QR code, QR code is a registered trademark) corresponding to the person being assisted, and the terminal device 462 may perform recognition processing of the marker. Alternatively, the application executed by the first device 100 may be an application that is associated one-to-one with the person being assisted, and the processing unit 110 may identify the person being assisted based on the started application. Alternatively, the application may be an application that allows the target user to be switched by tab selection or the like, and the processing unit 110 may identify the person being assisted based on the selected tab.

[0109] When the trigger information is acquired, the first device 100 may determine whether it is connected to the server system 10, and if it is connected, may determine whether the first association information is the latest by communicating with the server system 10. In this way, appropriate pre-processing can be realized when a connection with the server system 10 is established.

[0110] Specifically, when a connection between the first device 100 and the server system 10 has been established, in step S203, the first device 100 performs a process of inquiring of the server system 10 as to whether the association information (first association information) stored in the storage unit 120 is the latest data. For example, the processing unit 110 may perform a process of transmitting the first association information itself to the server system 10, or may transmit information regarding the update timing (corresponding to the update date and time in FIG. 3) and the expiration date and time (not shown in FIG. 3) included in the first association information.

[0111] The server system 10 judges whether the first association information is the latest based on the request in step S203. For example, the processing unit 11 may judge whether the content of the row in the master table corresponding to the first association information matches the first association information. Alternatively, the processing unit 11 may judge whether the update timing of the master table matches the update timing of the first association information. The processing unit 11 may also make the judgment based on the expiration date instead of the update timing.

[0112] If the first association information is the latest, in step S204, the processing unit 11 performs a process of returning information indicating that the first association information is the latest. If the first association information is not the latest, in step S205, the processing unit 11 identifies the latest association information (master table).

[0113] In step S205, the processing unit 11 may determine whether or not there is an application that needs to be newly installed in the first device 100. For example, the processing unit 11 identifies a first list, which is a list of applications required for the first device 100, based on the master table. In addition, the processing unit 11 acquires a second list, which is a list of applications already installed in the first device 100, based on the device information stored in the storage unit 12. The second list may be acquired from the first device 100 in step S203. The processing unit 11 determines that an application that is not included in the second list among the applications included in the first list is an application that needs to be installed in the first device. In step S206, the communication unit 13 transmits the latest association information and the application that needs to be installed to the first device 100. As shown in steps S108-S112 of FIG. 9, the first device 100 may determine whether or not there is an application that needs to be newly installed.

[0114] In step S207, the processing unit 110 of the first device 100 stores the association information transmitted from the server system 10 as the first association information in the storage unit 120. The processing unit 110 also installs the application transmitted from the server system 10. This updates the association information and application of the first device 100 to the latest state. For example, even if the association information is updated between the process of FIG. 9 and the start of the process of FIG. 10, the first association information of the first device 100 is updated to the latest state by the process shown in FIG. 10. That is, by the processes of steps S203-S207, the pre-processing for executing the process according to the latest association information is completed in the first device 100.

[0115] In step S208, the processing unit 110 performs a process of identifying devices that operate in cooperation with each other in the assistance based on the first association information. For example, here, the person being assisted and the application used by the first device 100 are identified (steps S201-S202). Based on this information and the association information, the processing unit 110 may perform a process of searching for rows that have the same person being assisted, the same ADL, scene, and type of assistance, and different device IDs. When a row that satisfies the conditions is found, the processing unit 110 determines the device identified by the device ID of the row as the cooperation destination.

[0116] Here, an example has been described in which the first association information includes information identifying the link destination, but the present invention is not limited to this. For example, the first association information may include only information related to the first device 100, and may not include information on the link destination. In such a case, in step S203, the first device 100 may request information identifying the link destination from the server system 10 in addition to checking the association information. The storage unit 12 of the server system 10 holds information on devices other than the first device 100 as association information (master table), and therefore the link destination of the first device 100 can be appropriately determined.

[0117] 10, the second device 200 is identified as the collaboration destination. Therefore, in step S209, the first device 100 transmits a collaboration request to request the second device 200 to start collaboration processing. For example, the communication unit (second communication unit) of the first device 100 establishes a connection with the second device 200 by performing pairing through Bluetooth communication, and then transmits the collaboration request shown in step S209 using the Bluetooth communication.

[0118] When the second device 200 receives a link request, it determines whether it is connected to the server system 10, and if it is connected, it determines whether the second association information is up to date by communicating with the server system 10. In this way, when a connection with the server system 10 is established, appropriate pre-processing can be realized.

[0119] Specifically, in steps S210-S214, the second device 200 and the server system 10 execute the same processes as those in steps S203-S207 described above. This completes the pre-processing for executing the process according to the latest association information in the second device 200 as well.

[0120] 10 illustrates an example in which the second device 200 directly communicates with the server system 10, but the present invention is not limited to this. For example, the second device 200 may communicate with the server system 10 via the first device 100 that is the sender of the link request. For example, in step S210, the second device 200 may first transmit a confirmation request to the first device 100, and the first device 100 may transmit the confirmation request to the server system 10. Similarly, in steps S211 and S214, the server system 10 may first transmit data to the first device 100, and the first device 100 may transfer the data to the second device 200.

[0121] In step S215, the second device 200 identifies an application to be used in the cooperation process based on the second association information. This puts the second device 200 in a state in which it can start an application that cooperates with the application started in step S201. Therefore, in step S216, the second device 200 transmits an acceptance response to the cooperation request of step S209 to the first device 100.

[0122] In step S217, the first device 100 displays a message indicating that preparation is complete on the screen of the application started in step S201.

[0123] FIG. 11 is an example of an application screen displayed in step S217. For example, when an application executed by the choking detection device 460 is used, a text "Application setting for AAA has been completed" may be superimposed on the display screen described above with reference to FIG. 8. "AAA" is the name of the person being assisted. As shown in FIG. 11, the display unit 140 of the first device 100 (choking detection device 460) may display the second device 200 that performs processing in cooperation with the first device 100. In the example of FIG. 11, it is displayed that the smartphone XXX on which the positioning application runs is the second device 200. Note that "XXX" here may be an ID (device ID) that uniquely identifies the smartphone, or may be information indicating the manufacturer name or model number of the smartphone. When there are multiple second devices 200 that cooperate with the first device 100, information on the multiple second devices 200 may be displayed on the screen shown in FIG. 11. 11 may display information about an application (such as an application ID or an application name) that runs on the second device 200. In this way, it is possible to clearly present to the user that the application of the choking detection device 460 and the application of the second device 200 that cooperates with the application (such as a smartphone positioning application) are in an appropriate setting state. In addition, in the screen shown in FIG. 11, it is assumed that a device that has returned a cooperation acceptance response (step S216) in response to a cooperation request (step S209) is displayed as the second device 200. However, the method of the present embodiment is not limited to this. For example, the first device 100 may display information about a device that is a candidate for the second device 200. For example, the first device 100 may display a list of devices whose Bluetooth communication radio wave intensity is equal to or greater than a threshold on the screen of FIG. 11, and may display a device connected as the second device 200 among them in a manner that makes it possible to distinguish it from a non-connected device. In this way, it is possible to present to the user not only information about the second device 200 that will start the linking process, but also information about a device that may link as the second device 200 in the future.

[0124] 11 may display an OK button used to allow the start of the cooperative process. When the OK button is selected, the processing unit 110 of the first device 100 transitions to a state in which the cooperative process can be started, and when the execution button in area RE6 is selected in this state, the cooperative process is started. Alternatively, when the OK button in FIG. 11 is selected, the cooperative process is started, and the execution button selection operation may be omitted. Alternatively, the OK button and the execution button in area RE6 may have different functions. For example, the OK button in FIG. 11 may be used to start the cooperative process, and the execution button in area RE6 may be used to start a process in which the first device 100 does not cooperate with other devices (a process by the first device 100 alone).

[0125] In this state, since both the first device 100 and the second device 200 are capable of processing based on the latest association information, specific cooperation processing is started in step S218. Note that a detailed description of the cooperation processing will be omitted.

[0126] Although not described above, the first device 100 may not accept operation input of the application started in step S201 until the display in step S217 is performed. For example, the processing unit 110 of the first device 100 does not start processing even if the user presses the execution button in area RE6 during the period from the application start to the display in step S217. In this way, it is possible to prevent processing from being executed before an application is updated. For example, it is possible to prevent inappropriate processing from being started when at least one of the first device 100 and the second device 200 does not correspond to the latest association information.

[0127] At that time, the processing unit 110 of the first device 100 may cause the display unit 140 to display a screen shown in Fig. 12. For example, the processing unit 110 may perform a display process of displaying the screen shown in Fig. 12 during the period from the start of the application until before step S217, and changing the screen to the screen shown in Fig. 11 in step S217. In this way, it is possible to clearly notify the user that necessary settings have not been made during the execution of the preprocessing.

[0128] Also, a details button may be displayed on the screen shown in FIG. 12. When the processing unit 110 receives a selection operation of the details button, the processing unit 110 performs a process of transitioning to a details setting screen. On the details setting screen, for example, a list of tacit knowledge applications required for the first device 100 may be displayed, and a user operation of which application to install may be received. For example, before the processing of steps S206-S207 in FIG. 10, the server system 10 may first transmit a list of tacit knowledge applications required for the first device 100, and the display unit 140 may display the list on the details setting screen. Then, when a user operation is performed to select one or more applications from the list, the first device 100 may request the selected application from the server system 10. In step S206, the server system 10 transmits the tacit knowledge application selected by the user to the first device 100. In this way, the user can manually set the tacit knowledge application to be installed. Also, on the details setting screen, not only the selection of the tacit knowledge application to be installed but also various settings such as setting of parameters used in the tacit knowledge application may be performed.

[0129] The association information may include information on tacit knowledge data that is a digitalized version of the tacit knowledge of an experienced caregiver used for care. Although the tacit knowledge data is a tacit knowledge application in the narrow sense, the tacit knowledge data of this embodiment may include data in a form other than an application. For example, the tacit knowledge data may not be the application itself, but may be setting data (parameters, etc.) used in the application.

[0130] At least one of the first device 100 and the second device 200 acquires tacit knowledge data from the server system based on the association information, and executes a process related to assistance based on the tacit knowledge data. For example, the application transmitted in step S206 or S214 may be a tacit knowledge application that uses tacit knowledge. In this way, when using digitized tacit knowledge in each device, it is possible to appropriately provide data related to the tacit knowledge to each device. As described above, the method of this embodiment does not require the use of tacit knowledge, and the application in one or both of the first device 100 and the second device 200 may not use tacit knowledge.

[0131] In addition, in this embodiment, it is assumed that various devices are arranged around the person being assisted, and that these devices flexibly cooperate with each other according to the situation. Therefore, in the information processing system 1 of this embodiment, various devices of different types and vendors may be mixed. Therefore, in order to deal with the diversity of devices, a data structure used in communication between the server system 10 and each device, or communication between devices, may be made common. Details of the data structure will be described later with reference to Figs. 23 to 25, etc.

[0132] 3.2 If one of the devices is not connected to the server system <If no second device is connected> In a case where the second device 200, which is a slave device, is not connected to the server system 10, but the first device 100, which is a master device, is connected to the server system 10, for example, the second device 200 may communicate with the server system 10 via the first device 100.

[0133] 10 described above, the second device 200 may determine whether it is connected to the server system 10 when it receives a link request, and if it is not connected, determine whether the first device 100 is connected to the server system 10. If the first device 100 is connected to the server system 10, the second device 200 may transmit a confirmation request to the server system 10 to the first device 100.

[0134] Specifically, as described above, in step S210, the second device 200 may first transmit a confirmation request to the first device 100, and the first device 100 may then transmit the confirmation request to the server system 10. Also, in steps S211 and S214, the server system 10 may first transmit data to the first device 100, and the first device 100 may then transfer the data to the second device 200. In this manner, even if the second device 200 is not connected to the server system 10, it is possible to realize a linking process in the same manner as when both devices are connected to the server system 10.

[0135] <If the first device is not connected> Fig. 13 is a sequence diagram illustrating periodic processing when the first device 100 is not connected to the server system 10 and the second device 200 is connected to the server system 10. As in the example of Fig. 9, the information processing system 1 executes the processing shown in Fig. 13 periodically, for example.

[0136] First, in step S301, the first device 100 acquires sensing data using a sensor mounted on the first device 100. However, since the first device 100 is not connected to the server system 10 here, the sensing data is not transmitted to the server system 10 and is accumulated, for example, as log data in the storage unit 120 of the first device 100. For example, the first device 100 transmits the log data to the server system 10 the next time it is connected to the server system 10.

[0137] On the other hand, in step S302, the second device 200 acquires sensing data using a sensor mounted on the second device 200. In step S303, the communication unit of the second device 200 transmits the acquired sensing data to the server system 10. That is, since the second device 200 has already been connected to the server system 10, the same process as that in FIG.

[0138] Steps S304-S305, which are the processing of the server system 10, are similar to steps S105-S106 in Fig. 9, except that the sensing data of the first device 100 acquired in step S301 is not used. In addition, the processing of the following steps S306-S310 is also similar to steps S113-S117, and therefore detailed description thereof will be omitted.

[0139] As described above, the periodic processing is executed for devices connected to the server system 10, and the processing may be omitted for devices not connected to the server system 10. For example, an example in which the first device 100 is not connected is shown in Fig. 13, but if the second device 200 is not connected, the processing related to the second device 200 is similarly omitted.

[0140] 14 is a sequence diagram for explaining the cooperation process between the first device 100 and the second device 200. First, in step S401, the first device 100 acquires trigger information representing a trigger for the cooperation process. In step S402, the processing unit 110 of the first device 100 identifies a person being assisted who is a target for assistance. The processes in steps S401-S402 are similar to those in steps S201-S202 in FIG.

[0141] When the first device 100 is not connected to the server system 10, in step S403, the first device 100 transmits a link request including information on the first association information to the second device 200. Note that in step S403, information on the person being assisted identified in step S402 may also be transmitted. For example, the first device 100 identifies the second device 200 as a link destination based on the first association information stored in the storage unit 120 at that time, the started application, and information on the identified person being assisted. Then, the first device 100 performs pairing with the identified second device 200 for Bluetooth communication, for example.

[0142] If the second device 200 is connected to the server system 10 when it receives a link request, it determines whether the first association information and the second association information are the latest by communicating with the server system 10. In this way, even if the first device 100, which is the master device, is not connected to the server system 10, it is possible to realize appropriate pre-processing using the second device 200, which is the slave device.

[0143] Specifically, in step S404, the second device 200 performs a process of inquiring of the server system 10 as to whether the first association information of the first device 100 and the second association information of the second device 200 are the latest data. As in the example described above with reference to Fig. 10, the second device 200 may perform a process of transmitting the association information itself to the server system 10, or may perform a process of transmitting information such as update timing.

[0144] Based on the request in step S403, the server system 10 determines whether or not each of the first association information and the second association information is the latest.

[0145] If both the first association information and the second association information are the latest, in step S405, the processing unit 11 performs a process of returning information indicating that both the first association information and the second association information are the latest to the second device 200. As a result, the second device 200 recognizes that the second association information is the latest. Also, in step S406, the second device 200 performs a process of transmitting information indicating that the first association information is the latest to the first device 100. As a result, the first device 100 recognizes that the first association information is the latest.

[0146] If the second association information is the latest but the first association information is not the latest, in step S407, the processing unit 11 of the server system 10 identifies the latest association information (master table). In addition, in step S407, the processing unit 11 may determine whether or not there is an application that needs to be newly installed in the first device 100. In addition, in step S407, the processing unit 11 may determine whether or not the second device 200 with which the first device 100 should be linked has been changed. For example, it is assumed that the choking detection device 460 is the first device 100, and the smartphone CP is determined to be the second device 200 based on existing information and the above-mentioned processing is performed. However, as a result of the server system 10 making a determination based on the latest association information, it is possible that the second device 200 with which the choking detection device 460 should be linked has been changed to the seat sensor 440 instead of the smartphone CP. Alternatively, it is possible that the smartphone CP is maintained as the second device, but in addition, the seat sensor 440 is also added to the second device 200 (three or more devices are linked). Therefore, the server system 10 may transmit information indicating these determination results to the first device 100 via the smartphone CP (the second device 200 before the change).

[0147] In step S408, the communication unit 13 transmits the latest association information and the application that needs to be installed in the first device 100 to the second device 200. Also in step S408, the server system 10 may transmit information indicating that the second association information is the latest. In step S409, the second device 200 transmits the latest association information and the application that needs to be installed in the first device 100 to the first device 100.

[0148] In step S410, the processing unit 110 of the first device 100 stores the association information transmitted from the server system 10 via the second device 200 as first association information in the storage unit 120. The processing unit 110 also installs the application transmitted from the server system 10 via the second device 200. As a result, the association information and the application of the first device 100 are updated to the latest state.

[0149] If both the first association information and the second association information are not the latest, in step S411, the processing unit 11 of the server system 10 identifies the latest association information (master table). In addition, in step S411, the processing unit 11 may determine whether or not there is an application (referred to as a first application) that needs to be newly installed in the first device 100, and whether or not there is an application (referred to as a second application) that needs to be newly installed in the second device 200.

[0150] In step S412, the communication unit 13 transmits to the second device 200 the latest association information, the first application, and the second application.

[0151] In step S413, the processing unit of the second device 200 stores the association information transmitted from the server system 10 in the storage unit as second association information. The second device 200 also installs the second application transmitted from the server system 10. As a result, the association information and the application of the second device 200 are updated to the latest state.

[0152] Furthermore, in step S414, the second device 200 transmits the latest association information and the first application to the first device 100. In step S415, the processing unit 110 of the first device 100 stores the association information transmitted from the server system 10 via the second device 200 as first association information in the storage unit 120. Furthermore, the processing unit 110 installs the application transmitted from the server system 10 via the second device 200. As a result, the association information and the application of the first device 100 are updated to the latest state.

[0153] As described above, when it is determined that the first association information is not the latest, the second device 200 may receive the latest first association information and the tacit knowledge data for the first device specified by the latest first association information from the server system (steps S408 and S412). The first device 100 acquires the latest first association information and the tacit knowledge data for the first device from the second device 200 (steps S409 and S414). In this way, even if the first device 100, which is the master device, is not connected to the server system 10, it is possible to execute appropriate pre-processing using the second device 200, which is the slave device.

[0154] 14 omits a case where the first association information is the latest and the second association information is not the latest. However, such a case may occur, and the server system 10 may transmit to the second device 200 a message that the first association information is the latest, as well as the latest association information and the second application. The second device 200 updates the second association information, installs the second application, and transmits to the first device a message that the first association information is the latest.

[0155] In step S416, the first device 100 performs a process of transmitting a request to start a cooperation process to the second device 200. As described above in step S407, the second device 200 with which the first device 100 should cooperate may have been changed in the determination process in the server system 10. Alternatively, although omitted in FIG. 14, before step S416, the processing unit 110 may identify an apparatus that operates in cooperation in the assistance based on the latest first association information. In this way, when the second device 200 needs to be changed, the start request of step S416 is transmitted to the identified device. For example, when the cooperation destination has been changed based on the update process of at least one of the first association information and the second association information, the start request of step S416 may be transmitted to the second device 200 that is different from the transmission destination of the cooperation request of step S403.

[0156] In step S417, the second device 200 identifies an application to be used in the cooperative process, and then returns a start acceptance response (cooperation acceptance response) to the first device 100. In step S418, the first device 100 displays a message indicating that preparation is complete on the screen of the application started in step S401. The screen displayed in step S418 is, for example, the same as the screen described above with reference to Fig. 11. Also, during the period before step S418, the screen described above with reference to Fig. 12 may be displayed.

[0157] In this state, since both the first device 100 and the second device 200 are capable of processing based on the latest association information, specific cooperation processing is started in step S419. Note that a detailed description of the cooperation processing will be omitted.

[0158] In the above, an example has been described in which the connection state is common between the periodic processing and the cooperative processing. For example, in the above description using Fig. 9 and Fig. 10, an example has been described in which both the first device 100 and the second device 200 are connected to the server system 10 in both the periodic processing and the cooperative processing. In the above description using Fig. 13 and Fig. 14, an example has been described in which the first device 100 is not connected and the second device 200 is connected to the server system 10 in both the periodic processing and the cooperative processing.

[0159] However, the process of this embodiment is not limited to a case where the connection state in the periodic process and the connection state in the cooperation process are common. For example, the cooperation process and the periodic process may be executed at different timings, such as the cooperation process being executed during meal assistance in the daytime and the periodic process being executed in the middle of the night. Therefore, the connection state of each device with the server system 100 may change between the cooperation process and the periodic process. For example, if the first device 100 is not connected to the server system 10 at the stage of the periodic process, but is connected when the trigger is detected, the cooperation process shown in FIG. 10 may be executed after the periodic process shown in FIG. 13 is executed. Similarly, the cooperation process shown in FIG. 14 may be executed after the periodic process shown in FIG. 9 is executed. In other words, any combination of the connection state in the periodic process and the connection state in the cooperation process is allowed.

[0160] 3.3 When both devices are not connected to the server system In the periodic processing when neither the first device 100 nor the second device 200 is connected to the server system 10, for example, steps S102 and S104-S117 in Fig. 9 are omitted. That is, the first device 100 and the second device 200 accumulate the sensing data as log data, but do not transmit the data to the server system 10 or receive the association information. Note that the processing described above with reference to Fig. 9 or Fig. 13 may be performed as the periodic processing.

[0161] If both the first device 100 and the second device 200 are not connected to the server system 10, the server system 10 cannot be requested to confirm the first association information and the second association information in the cooperation process. Therefore, if the first device 100 is not connected to the server system 10 when the trigger information is acquired, and the second device 200 is not connected to the server system when the cooperation request is received, at least one of the first device 100 and the second device 200 may select one of the association information based on a comparison process of the first association information and the second association information. Then, the first device 100 and the second device 200 cooperate with each other based on the selected association information to execute a process related to assistance. In this way, even if the latest association information cannot be referred to, the cooperation process can be executed using the acquired association information. At that time, the comparison process of the first association information and the second association information is executed, so that the optimum cooperation process can be realized within the possible range. A specific example of the process will be described below.

[0162] 15 is a sequence diagram for explaining the cooperation process between the first device 100 and the second device 200. First, in step S501, the first device 100 acquires trigger information representing a trigger for the cooperation process. In step S502, the processing unit 110 of the first device 100 identifies a person being assisted who is a target for assistance. The processes in steps S501-S502 are similar to those in steps S201-S202 in FIG.

[0163] Here, the first device 100 is not connected to the server system 10. Therefore, in step S503, the first device 100 transmits a link request including information related to the first association information to the second device 200, similar to step S403 in FIG.

[0164] In step S504, the processing unit of the second device 200 that has received the link request checks the connection state between the second device 200 and the server system 10 and determines that they are not connected. In other words, unlike the process shown in Fig. 14, communication with the server system 10 via the second device 200 cannot be performed.

[0165] Therefore, in step S504, the second device 200 performs a process of comparing the first association information acquired in step S503 with the second association information stored in the storage unit of the second device 200. For example, the second device 200 may select association information with a newer update date and time from among the first association information and the second association information. In step S505, the second device 200 transmits data including the selected association information to the first device 100 as a link acceptance response. Note that the data included in the link acceptance response in step S505 may be the association information itself, or may be information that identifies the association information (such as an ID of the association information). In this way, newer association information is more likely to be selected, so that even if the selected association information is not the latest, data that is close to the latest can be used.

[0166] In step S506 (before executing the process related to assistance), the first device 100 performs a process of presenting the update date and time of the association information to the user. Fig. 16 is an example of an application screen displayed in step S506. For example, when using an application executed by the choking detection device 460, text "Operation will be performed based on data dated MM / DD" may be superimposed on the display screen described above with reference to Fig. 8. "MM / DD" is information indicating the update date of the selected association information.

[0167] According to the method of the present embodiment, it is possible to present to the user in an easily understandable manner what point in time association information will be used as the basis for subsequent link processing.

[0168] Furthermore, as shown in Fig. 16, the screen displayed in step S506 may display information for receiving a user input regarding whether or not to perform the operation. In the example of Fig. 16, in addition to the text "The operation will be performed based on the data dated MM / DD", a confirmation text "Are you sure?" and a selection item "Yes / No" are displayed. Then, when a user input is made to select "Yes", the first device 100 and the second device 200 execute the cooperation process in step S507.

[0169] This makes it possible to suppress the execution of linkage processes that the user considers undesirable. For example, if the ADL of the person being assisted has significantly deteriorated after the displayed date, it is possible that the content of the assistance to be performed for that person has changed. If the user, such as a caregiver, is aware of the timing of the change in the ADL, he or she can consider that the association information before that time of change is old and does not reflect the current state of the person being assisted, and can select "No."

[0170] 15 shows an example in which the second device 200 performs the comparison process and the selection process of the association information, but the present invention is not limited to this. For example, the first device 100 may perform the comparison process and the selection process of the association information after acquiring information related to the second association information from the second device 200.

[0171] In this embodiment, the determination may be made not only from the viewpoint of the update timing of the association information but also from the viewpoint of whether the target association information can be executed. For example, consider an example in which the association information includes information for identifying the first tacit knowledge data used in the first device 100 and the second tacit knowledge data used in the second device 200 as information on tacit knowledge data obtained by digitalizing the tacit knowledge of an experienced caregiver used for care. In this case, at least one of the first device 100 and the second device 200 may select, from the first association information and the second association information, association information in which the first tacit knowledge data has been installed in the first device and the second tacit knowledge data has been installed in the second device. Hereinafter, an example in which the tacit knowledge data is a tacit knowledge application will be described.

[0172] For example, it is assumed that the first association information is information representing that the first device 100 executes a tacit knowledge application 1A, and the second device 200 executes a tacit knowledge application 2A. It is also assumed that the second association information is information representing that the first device 100 executes a tacit knowledge application 1B, and the second device 200 executes a tacit knowledge application 2B. In this example, it is assumed that the first device 100 has the tacit knowledge application 1A installed, but the tacit knowledge application 1B is not installed.

[0173] For example, when simply considering the update timing, the second association information may be newer than the first association information. However, according to the second association information, the first device 100 needs to operate according to the tacit knowledge application 1B, but the tacit knowledge application 1B is not installed in the first device 100. In addition, since both devices are not connected to the server system 10, it is not easy to newly install the tacit knowledge application 1B. In other words, in this example, there is a possibility that the cooperation process based on the second association information cannot be executed.

[0174] In this respect, according to the above process, the selection priority of the second association information requesting the tacit knowledge application 1B is lowered, and the selection priority of the first association information requesting the installed tacit knowledge application 1A is increased. In this way, by taking into consideration the installation status of the tacit knowledge application, it becomes possible to select association information from the viewpoint of whether or not the linking process according to the association information is actually possible.

[0175] 3.4 Variations Specific examples of the processing have been described above with reference to FIGS. 9 to 16, but the processing of this embodiment is not limited to this, and various modifications are possible.

[0176] (Notification processing) 11, 12, and 16 have been described above as specific examples of the notification process (display process) for the user. However, the information notified to the user is not limited to these.

[0177] For example, the first device 100 and the second device 200 may check whether they are connected to the server system 10 (network) at the time of startup, and if they are not connected, may display a message indicating that they are not connected.

[0178] Furthermore, when the first device 100 is not connected to a network, the first device 100 may display a list of devices that are candidates for collaboration and prompt the user to select one. For example, the first device 100 may display devices that are capable of Bluetooth pairing as candidates for collaboration.

[0179] Furthermore, at least one of the first device 100 and the second device 200 may perform a process of displaying a mismatch between the first association information and the second association information. In this case, the first device 100 may notify the server system 10 of a mismatch occurrence notification message, the association information of the device where the mismatch occurs, and the time of the mismatch occurrence via the network. For example, if a mismatch in the association information occurs because one of the first device 100 and the second device 200 is not connected to the server system 10, it is possible to handle the mismatch in the future by notifying the server system 10 of the mismatch while accepting the mismatch and executing a linkage process.

[0180] Furthermore, if a mismatch occurs, a queue for reserving an update of the association information may be automatically set in a device that is not connected to the server system 10. In this case, when the device is connected to the network, updating of the association information is automatically started. For example, if the association information of the second device 200, which is a slave device, has not been updated, the first device 100, which is a master device, may perform a process of setting a queue for updating the second device 200 using Bluetooth communication.

[0181] (Trigger information) Although an example in which the start of an application is the trigger of the cooperation process has been described above, the present invention is not limited to this. For example, in the cooperation process for fall prevention, an application that uses a camera to determine the degree of stability of posture is first operated, and when the application detects standing instability, a foot pressure sensor operates as a slave device to detect posture in detail, or a locking mechanism provided on a fixture such as a desk operates so that the person being assisted can hold it stably. In this way, the trigger in this embodiment may be that the judgment result (application processing result) in the master device satisfies a given condition.

[0182] In this example, since it is assumed that the application is operating for a specific person being assisted, the process of launching the application and identifying the person being assisted shown in steps S201-S202 of Figure 9 is automatically executed when the trigger information is obtained.

[0183] (Identifying partners) In the above, an example has been described in which the first device 100 identifies the second device 200 to be linked based on the latest association information or the association information stored in the storage unit 120. However, depending on the device, the user may be identified. For example, a device placed in a room of a person receiving care is highly likely to be used to assist the person receiving care. In this case, the second device 200 to be linked may be identified based on the result of the process of identifying the person receiving care shown in step S202 of FIG. 9. For example, when a trigger is detected in a master device placed in a room, another device placed in the room may be selected as a slave device.

[0184] Also, the first device 100, which is the master device, may select a nearby device as a device to cooperate with. For example, the first device 100 performs processing related to Bluetooth pairing, and selects a device with a radio wave intensity equal to or higher than a predetermined value as a device to cooperate with. Also, the position of each device may be detected from RFID (Radio Frequency Identification), a camera image, or the like, and a device near the first device 100 may be determined based on the position. In this way, it becomes possible to select a device to cooperate with based on the positional relationship of the devices.

[0185] If there is no device to be linked nearby, the first device 100 may display that fact on the screen of the display unit 140. If the location of the inventory of the device to be linked is managed, the first device 100 may display the location where the device is stored on the display unit 140. The notification of the storage location may be performed on a device other than the first device 100, for example, a terminal device used by a caregiver.

[0186] (ADL, timing of updating association information) In the above, the periodic processing and the cooperative processing have been described as different processes. Specifically, the ADL and the association information based on the communication between each device and the server system 10 are updated by a predetermined time, but these update processes are not executed in the cooperative processing (FIGS. 9-10, 13-15).

[0187] However, this embodiment is not limited to this, and the ADL and the association information may be updated one by one when an application is started in each device. For example, when the first device 100 and the server system 10 are connected, when an application is started (step S201 in FIG. 10), log data may be transmitted from the first device 100 to the server system 10, and the ADL and the association information may be updated in the server system 10. In this case, it may take a long time until preparation for the linking process is completed (until the Ready display shown in FIG. 11 is displayed), but it becomes possible to update the ADL and the association information (master table of the server system 10) to the latest data at the linking process stage.

[0188] For example, by updating the master table during the collaboration process, even if the condition of the person being assisted changes between the processing of Figure 9 and the processing of Figure 10, it is possible to reflect the change in the master table.

[0189] 4. Specific examples of positioning applications Next, a specific example of a screen of the positioning application will be described.

[0190] 4.1 Example of setting screen 17A and 17B are examples of screens displayed on a display unit of a device such as a smartphone when a positioning application is started. FIG. 17A is a screen used by an operator, and FIG. 17B is a screen used by an administrator. The operator here is, for example, a general care worker in a care facility. The administrator is, for example, a functional trainer who has knowledge about positioning suitable for the person being assisted. However, the administrator may also be a care manager or a management staff member of the care facility.

[0191] For example, when the positioning application is started, the screen shown in Fig. 17A for general care staff is displayed first. The general care staff performs a login operation on the screen shown in Fig. 17A, and after completing the login, adjusts the position of the person being assisted by operating the positioning application in the operation mode. The general care staff may be restricted from creating new applications (registering teacher data) in the setting mode.

[0192] For example, when the worker has completed logging in, applications available to the worker may be displayed as shown in Fig. 17A. In the example of Fig. 17A, the person AAA who is assisted is selected, and three applications used to assist the person AAA who is assisted are displayed. By selecting the person BBB or CCC who is assisted, the application corresponding to the selected person may be displayed. By using the text box at the top of the screen, a search process for the person who is assisted may be performed.

[0193] In addition, the positioning application transitions to a screen shown in FIG. 17B when an object OB1 representing a right arrow is selected on the screen shown in FIG. 17A. For example, a manager such as a functional trainer selects the object OB1 on the screen shown in FIG. 17A and executes a login operation on the screen shown in FIG. 17B. After completing the login, the manager creates a new application in the setting mode or adjusts the position in the operation mode. For example, the positioning application operates in the setting mode when an object OB2 corresponding to a plus sign is selected. Note that the positioning application may perform a process of transitioning to the screen shown in FIG. 17A when an object OB3 representing a left arrow is selected on the screen shown in FIG. 17B. Also, in FIG. 17B, applications available to the manager who logged in may be displayed as in FIG. 17A. FIG. 17B shows an example in which the same person being assisted and the same applications as those in FIG. 17A are displayed, but different applications may be displayed depending on the user's authority, division of responsibility, etc.

[0194] In this way, even if the same positioning application is shared by multiple caregivers, it is possible to switch between executable operations according to the authority granted to each caregiver. In the above example, only highly specialized administrators are permitted to operate in the setting mode, and general care staff are permitted to operate only in the operation mode using the set teacher data. In particular, in this embodiment, the screen for the worker is displayed when the application is started, and unless a specific operation (for example, an operation to select object OB1 in FIG. 17A) is performed on that screen, the screen will not transition to the screen for the administrator. Therefore, since the screen for the functional trainer is not displayed at the time of start-up, it is possible to prevent processes such as the creation of teacher data from being performed inadvertently.

[0195] Fig. 18 is an example of a setting screen that is displayed when, for example, the object OB2 is selected in Fig. 17B. In the setting screen shown in Fig. 18, the group, the application name, the name of the person being assisted, and specific registration contents are input. The group is used when dividing the people being assisted in a care facility or the like. The application name is the name of the application to be created. The registration contents include various items including the teacher data shown in Fig. 5A.

[0196] 19A and 19B are examples of screens displayed when adding teacher data to the registered contents, for example, when the camera icon in the registered contents is selected in FIG. 18. In FIG. 19A, a guide message "Please take a picture of the posture you want to register" is displayed, and an image captured by the smartphone camera is displayed. For example, the administrator adjusts the position and angle of the smartphone so that a person in a desired posture is captured in the captured image. When the object OB4, which is a shooting button, is selected, the positioning application acquires the captured image at that time as a candidate for teacher data. Note that the screen shown in FIG. 19A may display an object OB5 corresponding to the in-camera / out-camera switching button and an object OB6 corresponding to the back button.

[0197] FIG. 19B is an example of a display screen when the shooting button is pressed. For example, the positioning application performs a skeleton tracking process of a person and displays the result superimposed on the captured image. In FIG. 19B, as in the examples of FIG. 5A and FIG. 5B, an example is considered in which teacher data is acquired that represents the position and posture of the person being assisted and the position and posture of the cushion that are considered appropriate when lying on a bed. The circles in FIG. 19B represent the detected reference points of the skeleton (shoulders, elbows, wrists, etc.). In the screen shown in FIG. 19B, a text that prompts the selection of one of the reference points, "Please select the first point," may be displayed. For example, the administrator may select a point that is considered important for performing an appropriate position adjustment. When one of the points is selected, the positioning application stores the selected point and then displays a text that prompts further selection, "Please select the second point." From this point onwards, the positioning application may accept the selection of N points in the same manner. N is, for example, 3, but may be a different number.

[0198] For example, the positioning application accepts the selection input of three points, and stores the magnitude of the angle formed by the three points as the correct angle. Among the registration items in FIG. 18, the correct / incorrect judgment is an item for setting ON / OFF of the judgment of whether the position of the person being assisted is an appropriate posture in the operation mode. For example, when the correct / incorrect judgment is ON, the positioning application may perform the skeletal tracking process in the operation mode as well, identify the set three points, and obtain the angle formed by the three points. The positioning application then performs the correct / incorrect judgment by comparing the angle in the operation mode with the correct angle stored in the setting mode. However, the correct / incorrect judgment is not limited to this, and other judgments may be performed based on the positions of the points detected by the skeletal tracking, etc.

[0199] The correct / incorrect judgment in FIG. 18 is a setting item for switching whether to perform an OK / NG judgment based on the teacher data or to perform only the superimposed display of the teacher data. The transparency is an item that indicates the transparency when the teacher data is superimposed. Although not shown in FIG. 18, an item for switching on / off the acquisition process of the skeleton information may be displayed. When the skeleton information is used only for the correct / incorrect judgment, the item for setting on / off of the correct / incorrect judgment may also serve as the item for setting on / off of the acquisition process of the skeleton information. These items are not limited to those displayed on the screen of FIG. 18, and may be displayed on the screens shown in FIG. 19A and FIG. 19B. For example, when an operation for changing the transparency is performed on the screens shown in FIG. 19A and FIG. 19B, the transparency of the captured image may be changed in real time according to the operation. When the acquisition process of the skeleton information is switched on / off, the display of the skeleton information in FIG. 19A and FIG. 19B may be switched. For example, when the skeleton information is off, the display of the skeleton information shown in FIG. 19B and the reception of the selection input of the feature points may be omitted. Adjustment of the transparency in the setting mode is not essential. For example, in the setting mode, the transmittance may be set to a given fixed value, and a display object such as a scroll bar with adjustable transmittance may be displayed on the screen of the use mode (for example, the screen described above with reference to FIG. 5B). Making the transmittance adjustable in the use mode allows for flexible control in situations where care is actually provided. For example, it is possible to make it easier to check the details of the teacher data by setting the transmittance low, and to make it easier to check the captured image in real time by setting the transmittance high. Also, the transmittance may be set in both the setting mode and the use mode.

[0200] Also, points 1 and 2 in Fig. 18 are items in which the manager inputs items that he / she considers important for position adjustment using text, etc. The point display direction is an item that sets the direction in which the text, etc. included in points 1 and 2 are displayed.

[0201] There may be cases where it is desired to have a single person being assisted assume different positions. For example, when adjusting the sleeping position of a person being assisted, if the preferred position is limited to one, maintaining that position for a long period of time may lead to the development of bedsores, etc. Therefore, in the positioning application, multiple sleeping positions (for example, right lateral position and left lateral position) may be registered, and the caregiver may switch between position adjustment using the right lateral position as training data and position adjustment using the left lateral position as training data over time.

[0202] From the above viewpoint, one positioning application may include a plurality of teacher data. For example, in the example of FIG. 18, an example of registering one teacher data in an area marked "Tab1" is shown, but by scrolling down this screen, a screen for registering other teacher data marked "Tab2" may be displayed. Naturally, the number of teacher data is not limited to two, so the number of tabs may be three or more. In the operation mode of the positioning application, for example, tabs are displayed on the screen, and teacher data, transparency, on / off of correct / incorrect judgment, comments, etc. that are superimposed and displayed in association with switching of the tabs are switched.

[0203] Furthermore, the multiple teacher data set using different tabs are not limited to those used at different times. For example, the execution of a specific movement may be supported by using multiple teacher data consecutively. For example, when performing rehabilitation for an assisted person, it is considered that it is possible to have the assisted person execute a desired movement because there is support from experts in a hospital or the like. On the other hand, at home, the assisted person's family may support the rehabilitation, or the assisted person may perform the rehabilitation alone, so it may be difficult to perform the appropriate movement as in a hospital. In this case, for example, the administrator may subdivide the movements to be performed in the rehabilitation and set teacher data representing the intermediate posture and the final posture in different tabs. For example, if the movement in the rehabilitation can be realized by performing three postures, the first posture, the second posture, and the third posture, in this order, the administrator registers the first posture in Tab1, the second posture in Tab2, and the third posture in Tab3. The positioning application then supports the execution of the rehabilitation movement by automatically switching between Tab1 to Tab3. For example, the positioning application first executes processing using the teacher data corresponding to Tab1, and when it is determined that the posture of the person being assisted is sufficiently close to the first posture (OK judgment), it executes processing using the teacher data corresponding to Tab2. Furthermore, when it is determined that the posture of the person being assisted is sufficiently close to the second posture, the positioning application executes processing using the teacher data corresponding to Tab3, and when it is determined that the posture of the person being assisted is sufficiently close to the third posture, it judges that the target rehabilitation movement is completed. In this way, when a certain movement is to be performed by the person being assisted, it is possible to appropriately support the movement. Also, by referring to the history of the positioning application, the caregiver or manager can check whether the rehabilitation is being performed appropriately at home. Also, as will be described later with reference to Figures 20 to 22C, the positioning application can also present the time until it is judged as OK, so it is also possible to present to the caregiver which part of the series of movements is taking time.

[0204] By using the screens shown in Figs. 17A to 19B, it becomes possible to determine the authority of the caregiver and then have the caregiver carry out the setting process using an easy-to-understand interface.

[0205] 4.2 Dashboard screen example In addition, in this embodiment, a dashboard screen may be used that displays data representing the results of using the positioning application. The data representing the results of using the positioning application may be, for example, statistical data obtained when multiple position adjustments have been performed.

[0206] The dashboard screen is expected to be displayed using a device such as a PC having a large display size, since the amount of information displayed is likely to be large. For example, when a PC accesses the server system 10 using a Web browser, the dashboard screen described below is displayed. However, the display of the dashboard screen may be executed using an application other than a Web browser. Also, the dashboard screen may be displayed on a mobile terminal device such as a smartphone.

[0207] FIG. 20 is an example of a screen displayed when using a positioning application in a web browser. In the screen shown in FIG. 20, a menu screen is displayed on the left side, and one of the following items is selected: dashboard, application creation, or device. If dashboard is selected, the dashboard screen described below is displayed. If application creation is selected, a screen for creating a new application is displayed, similar to the example described above using FIG. 18-FIG. 19B. If device is selected, a screen for adding and managing devices on which the positioning application runs is displayed.

[0208] The menu screen may also display the name of the currently logged in user and the number of notifications. For example, in Fig. 20, it is displayed that a user with the name "XXXX" is currently logged in and that three notifications have been received for that user. The menu screen may also display items such as settings and logout.

[0209] Information corresponding to an item selected on the menu screen is displayed in the area on the right side of the screen shown in Fig. 20. Below, an example in which the dashboard is selected on the menu screen will be described.

[0210] As shown in FIG. 20, the dashboard screen displays information on the number of times the positioning application was used in a given period, the accuracy rate of the correct / incorrect judgment, and the time until the correct / incorrect judgment is judged to be OK. Consider a case where a caregiver (worker) adjusts the positions of one or more patients for which he or she is responsible, using a positioning application installed on a smartphone. For example, the caregiver uses positioning application A to perform various position adjustments for patient A according to the care schedule, such as position adjustments for eating (e.g., to prevent swallowing), position adjustments for changing diapers, and position adjustments for preventing bedsores (changing posture). Since one caregiver may be in charge of multiple patients, for example, the same caregiver may use positioning application B to perform various position adjustments for patient B.

[0211] A device such as a smartphone accumulates as log data which positioning application was started and when it was ended. Furthermore, since the positioning application may output the result of the correct / incorrect judgment as described above, the log data may include data on whether the positioning application started at a certain timing was judged as OK and ended or was judged as NG. Furthermore, since the positioning application can also output the timing at which it was judged as OK, the said timing may be included in the log data. Furthermore, as described above, one positioning application may include multiple teaching data (tabs). Therefore, the log data may include data specifying which tab of which positioning application was processed.

[0212] The log data accumulated by the operation of the positioning application may be transmitted, for example, to the server system 10. The transmission timing is arbitrary, but the log data of the positioning application is transmitted, for example, at the timings shown in steps S102 and S104 in FIG.

[0213] Then, the server system 10 displays the number of uses, the accuracy rate, and the time until OK based on the acquired log data. The number of uses is, for example, the total number of times the positioning application was turned on during the display target period. The accuracy rate is the ratio of the number of times that it was judged as OK to the total number of uses during the display target period. The time until OK is a statistical value (average value, median value, etc.) of the time until it is judged as OK after the positioning application is started. In the example of FIG. 20, the tab "Months" is selected to set one month as the display target period, and the results displayed are that the number of uses during that one month was 47 times, the accuracy rate was 14 percent, and the average time until OK was 1 minute and 12 seconds.

[0214] The period to be displayed here may be switchable between one month and one week as shown in Fig. 20. A period different from this may also be set as the display period. For example, the server system 10 displays the number of uses and the like based on data corresponding to the logged-in user among the log data transmitted from various devices and acquired within the display period.

[0215] If the logged-in user is a worker (general staff member), the data corresponding to the logged-in user is data representing the result of the position adjustment performed by the user. Alternatively, the result of the position adjustment of the entire user group to which the user belongs may be displayed. If the logged-in user is an administrator (e.g., a functional trainer), the data corresponding to the logged-in user may be data representing the execution result of a part or all of an application created by the functional trainer. If the application created by the functional trainer is used on the terminals of multiple workers (general staff members), data for each terminal may be displayable. If the user is the person in charge of a care facility, the data corresponding to the logged-in user may be the result of position adjustment of a specific caregiver belonging to the care facility, or the result of position adjustment of the entire care facility.

[0216] Moreover, the log data to be displayed is not limited to being automatically selected, and may be manually determined by the user. For example, the log data may be stored as a file in the server system 10. The file format here is, for example, a CSV (comma separated values) format, but other file formats may be used. For example, when a user operation is performed to select a log data file using the file selection field in FIG. 20, data within the display target period among data included in the log data file is displayed on the dashboard screen. Furthermore, the storage location of the log data file is not limited to the server system 10, and may be a PC. In this case, the PC transmits the selected log data file to the server system 10 via a Web browser, and the server system 10 executes a process of displaying a screen corresponding to the log data file on the Web browser. As described above, the selection process of the data to be displayed can be performed by various methods such as automatic selection and manual selection. For example, the display items for file selection in FIG. 20 may be omitted.

[0217] 21A and 21B are examples of dashboard screens that display detailed information regarding the number of uses, the accuracy rate, and the time until OK. For example, the screens of Fig. 21A and Fig. 21B may be displayed when the scroll bar on the screen of Fig. 20 is scrolled downward.

[0218] The detailed information may be switchable between display in daily units and display in hourly units. The display unit period may be a period of other length. FIG. 21A shows an example in which detailed information on one month's worth of log data is displayed in daily units. As shown in FIG. 21A, the detailed display may display a positioning application (target person being assisted) in a manner that allows identification. In the example of FIG. 21A, there are two people being assisted, AAA and BBB, and position adjustment was performed using the positioning application for these two people being assisted.

[0219] The number of uses per day is displayed as a graph (e.g., a bar graph). The horizontal axis of the graph is time (per day), and the vertical axis is the total number of uses. In this case, the area corresponding to AAA and the area corresponding to BBB are displayed in different ways (e.g., different colors). In this way, the user can easily see the history of how many times position adjustments were made, for which person being assisted, and on which date.

[0220] In addition, when the mouse cursor is placed over any area of ​​the bar graph (mouse over), the accuracy rate corresponding to that area may be displayed separately in a pie chart. In the example of FIG. 21A, the mouse cursor is placed over the bar graph showing the number of uses on June 29, 2023, and the associated information that the number of uses for the assisted person AAA is 4 times and the number of uses for the assisted person BBB is 17 times is displayed. In this case, as the accuracy rate for the assisted person AAA, the percentage of the number of times that were judged to be OK out of the 4 times is displayed as a pie chart (hereinafter also referred to as the first pie chart). In this example, since one time was judged to be OK, a pie chart showing about 25 percent as the accuracy rate is displayed. Similarly, for the accuracy rate for the assisted person BBB, the percentage of the number of times that were judged to be OK out of the 17 times is displayed as a pie chart (hereinafter also referred to as the second pie chart). Although omitted in FIG. 21A, a scale showing the percentage may be displayed together with the pie chart, or a specific value of the accuracy rate may be displayed. In addition, in Fig. 21A, the first pie chart for the assisted person AAA and the second pie chart for the assisted person BBB are arranged separately above and below, but this is not limited to this. For example, on the dashboard screen, one of the first pie chart and the second pie chart may be displayed on the inner circumference side, and the other side may be displayed on the outer circumference side, so that the two pie charts are arranged concentrically. In this way, it is possible to clearly present the difference in the accuracy rate for each assisted person.

[0221] Furthermore, the time to OK is displayed as a graph (e.g., a curved line graph) with the average time per day. The horizontal axis of the graph is time (per day), and the vertical axis is the average time to OK. As with the number of uses, the points corresponding to AAA and the points corresponding to BBB are displayed in different formats. In this way, the user can easily see the history of the time it took to get the person being assisted to assume an appropriate posture for each day and for each person being assisted.

[0222] FIG. 21B is an example of a screen when one hour is selected as the display unit of detailed information. For example, when an item corresponding to Hours is selected in FIG. 21A, a transition to the screen of FIG. 21B is performed, and when an item corresponding to Days is selected in FIG. 21B, a transition to the screen of FIG. 21A is performed. The display contents in FIG. 21B are the same as those in FIG. 21A except that the display unit has been changed to one hour. For example, the number of uses is displayed using a bar graph, the time until OK is displayed using a line graph, and the accuracy rate in a given hour is displayed as a pie chart based on a mouse-over operation. Note that in FIG. 21B, since the display unit time is short, the display is limited to two days' worth of data rather than displaying all information for one month (for example, from midnight on June 30th to midnight on July 2nd). For this reason, data on the assisted person AAA who has no usage history during the two days to be displayed is not displayed in FIG. 21B, but information on multiple assisted people can be displayed in an identifiable manner, as in FIG. 21B.

[0223] As described above with reference to FIG. 18, from the viewpoint of preventing bedsores, the same posture should not be continued for a long time, so it is useful to change the posture every few hours. However, if the unit of displaying the detailed information is set to one day or more, it is difficult to grasp the timing of the position adjustment in a day. For example, even if there is log data in which the position adjustment is performed a fixed number of times in one day, it is not easy to distinguish whether the adjustment is performed at an appropriate interval so as not to continue one posture excessively, or whether the position adjustment is concentrated in a short period (the same posture is continued for a long time). Considering the above, it is useful to display the log data in units of a period shorter than one day, as shown in FIG. 21B. In the example of FIG. 21B, in the data for 6 / 30, which has a sufficient number of usage history for the assisted person BBB, the number of usages peaks every two hours. Therefore, it can be seen that the position change is performed at an appropriate interval so that the same posture is not continued for a long time.

[0224] 22A-22C are examples of screens that display detailed information for each person being assisted (for each positioning application). For example, the screens shown in Fig. 22A-22C may be displayed when the scroll bar is scrolled downward on the screen of Fig. 21A or Fig. 21B.

[0225] As shown in Fig. 22A, the left side of this screen displays the name, number of times used, accuracy rate, and average time until OK for each person being assisted. As in Fig. 20, the display target period may be one month or one week.

[0226] When a selection operation for one of the users is performed in the left area, details about the selected user are displayed in the right area of ​​the screen. The information displayed here may include history, day of the week, and worst NG information. The history is a log of the date and time of use of the positioning application for the target user, the tab, the judgment result, and the time until OK, as shown in FIG. 22A.

[0227] The date and time of use may be, for example, the start or end timing of the application. In other words, when "History" in Fig. 22A is selected, the dashboard screen displays details of each operation history of the positioning application.

[0228] A tab is information indicating which of one or more teacher data registered in a positioning application was used to adjust the position. For example, in the setting mode using FIG. 18, it is assumed that teacher data corresponding to the right lateral position is registered as Tab1, and teacher data corresponding to the left lateral position is registered as Tab2. Then, the text "Right" is registered as the name of Tab1, and the text "Left" is registered as the name of Tab2. In the example shown in FIG. 22A, right and left are recorded as the use history of the teacher data (tab). In other words, the history of the use of Tab1 corresponding to the right lateral position and the history of the use of Tab2 corresponding to the left lateral position remain.

[0229] The judgment result indicates whether the result of the judgment is OK or NG. As described above, whether it is OK or NG is determined based on the angle formed by the selected skeleton (feature points of the skeleton tracking result) and the respective positions of the selected multiple skeletons.

[0230] The time to OK is as described above, and indicates the time elapsed from the start of the application until it is judged as OK. Note that if the judgment result is NG, the OK judgment has not been made, so the time to OK is left blank.

[0231] In this way, in addition to being able to know the timing of use of the target positioning application in detail, the teacher data used, the judgment results at each timing of use, and the time until OK are displayed in detail. For example, in the case of changing the body position for preventing bedsores as described above, it is meaningless to make the user take the same posture every time, and it is important to make the user take a posture different from the previous time. Therefore, for example, in adjusting the position at night, it is possible to have the user judge whether "right" (Tab1) and "left" (Tab2) appear alternately. Alternatively, the server system 10 may automatically judge whether the body position change is appropriate by judging whether the tabs are changed periodically. In this judgment, for example, the text corresponding to each tab may be used to judge whether the postures of each tab are different.

[0232] In addition, since the judgment result and the time until OK are known for each date and time of use, it is possible to judge, for example, that it is easy to adjust the position during the day, but difficult to adjust the position at night. An adjustment that is not successful may be that it is likely to be judged as NG, or that it is judged as OK but the time until OK is greater than a predetermined value. In addition, the content of the assistance may be specified by combining the time and the assistance schedule. In this case, it is possible to judge, for example, that the position adjustment during meals is successful, but the position adjustment for diaper changing is not successful. In addition, since the judgment result and the time until OK are known for each tab, it is possible to judge, for example, that it is easy to adjust the position to posture A, but difficult to adjust the position to posture B.

[0233] Fig. 22B is an example of a screen that is displayed when "By day of the week" is selected in the area on the right side of the screen shown in Fig. 21. As shown in Fig. 22B, when By day of the week is selected, the NG rate for each day of the week may be displayed. The NG rate may be numerical data expressed as 100 - correct answer rate (%).

[0234] In nursing care facilities and the like, caregivers often work in shifts, and it is expected that the same person being assisted will be assigned a different caregiver (or caregiver team) on different days of the week. In this regard, by displaying the display shown in FIG. 22B, the days of the week on which the judgment result is likely to be NG can be clearly shown to the user. This makes it possible to identify caregivers who are likely to be judged NG and provide appropriate guidance, etc. Alternatively, it is also possible to identify caregivers with a low NG rate and share the knowledge of the caregivers (for example, have them create a tacit knowledge application).

[0235] Fig. 22C is an example of a screen that is displayed when "Worst NG" is selected in the area on the right side of the screen shown in Fig. 21. As shown in Fig. 22C, when Worst NG is selected, the reason for the NG determination is displayed.

[0236] For example, when performing a NG judgment using a plurality of feature points selected from the skeleton tracking result as described above, the positioning application can identify the feature point that caused the judgment as NG (hereinafter, NG cause point) among the plurality of feature points. For example, the positioning application compares a plurality of feature points included in the teacher data with a plurality of feature points acquired from an image captured in an operation mode, and judges it as NG if the degree of difference is equal to or greater than a threshold. In this case, the positioning application determines the feature point that is most different from the teacher data as the NG cause point. Alternatively, a priority order of skeletons may be set in advance, and the positioning application may identify the NG cause point based on the priority order when it is judged as NG.

[0237] In the example of FIG. 22C, it is displayed that, of the total number of times judged as NG, the rate at which the left ankle is the NG cause point is 14%, and the rate at which the left wrist is the NG cause point is 34%. In this way, it is possible to allow the user to confirm the reason for the NG judgement as a statistical value. For example, when a functional trainer views this data, it is possible to consider whether the photo used as the training data is bad, or what kind of guidance should be given to the worker (care worker) to improve the accuracy rate.

[0238] In the above, examples of displaying "History", "Day of the week", and "Worst NG" have been described, but the present invention is not limited to this, and other items may be displayed. For example, the history that took the longest time to be judged as OK may be displayed in a ranking format.

[0239] In addition, when using a device such as a PC, there is an advantage that the display size is larger than that of a mobile terminal device such as a smartphone. In addition, since it is easy to use peripheral devices such as a keyboard and a pointing device, the convenience of user operation is also high. In consideration of this point, in the positioning application of this embodiment, the positioning application may be edited using a web browser of a PC.

[0240] For example, a manager such as a functional trainer may use a smartphone to perform the registration process shown in FIG. 18-FIG. 19B, and then use a web browser to perform detailed editing. The editing process here is, for example, editing of photo data, which is teacher data. For example, when a graphic object such as a triangle or a square is superimposed on an image, the shape and size of the graphic object can be easily adjusted by using a PC. In addition, for trimming, brightness and saturation adjustment, filter processing, and the like, using a PC has fewer restrictions on functions and convenience of operation. In addition, in this embodiment, multiple types of terminals with different sizes may be used as mobile terminal devices. The multiple types of terminals here include, for example, tablet terminals that are relatively large in size and easy to view and operate data, and terminals such as smartphones that are relatively small in size and excellent in portability. Therefore, in this embodiment, three types of screens, a setting screen for a PC, a setting screen for a tablet terminal, and a setting screen for a smartphone, may be used in combination. In this way, it is possible to display a screen suitable for the terminal to be used. For example, an embodiment in which the manager uses a PC when editing an application, a tablet terminal when explaining the application to the worker, and a smartphone when actually providing assistance, is conceivable.

[0241] 5. Data Structure Example In the method of this embodiment, the data configuration used in communication may be common regardless of the source device or the destination device. Here, the data used in communication is, for example, a MAC frame, and more specifically, a data frame. By commonizing the data frame configuration, even when various devices are used as the first device 100 and the second device 200, it is possible to unify the communication control regardless of the vendor. In the narrow sense, the data frame configuration is the bit allocation in the frame body.

[0242] 23 shows an example of the format of a MAC frame. The MAC header includes the following fields: Frame Control, Duration ID, Address 1, Address 2, Address 3, Sequence Control, Address 4, QoS Control, and HT Control. Some of these fields may be omitted.

[0243] Frame Control includes a type field for determining whether the target MAC frame is a data frame, a management frame, or a control frame. Duration / ID stores information indicating the planned period for using radio waves. Address 1 to Address 4 store information indicating the addresses of the receiving and sending devices. Sequence Control corresponds to the sequence number of the data to be transmitted. QoS Control stores information used for QoS control. HT Control is a field used, for example, in management frames. The structure of the frame body will be described later with reference to FIG. 24. The trailer is, for example, an FCS (Frame Check Sequence). The FCS is information used to detect errors in the frame, and is, for example, a checksum code.

[0244] As shown in FIG. 23, each of the server system 10, the first device 100, and the second device 200 transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in the data link layer of communication with each other.

[0245] Fig. 24 is a diagram for explaining an example of bit allocation in the frame body of a data frame transmitted by the server system 10 etc. As shown in Fig. 24, the frame body may include the following fields: data type ID1, User ADL, scene flag, device type ID, data type ID2, Instruction length, contents, and expiration date.

[0246] Data type ID1 and data type ID2 are fields that store information indicating the type and use of the data frame. The types here may include five types, for example, ID=1 to ID=5. FIG. 24 shows an example in which the data type ID is divided into two areas. For example, data type ID1 is a fixed-length field of 1 bit, and data type ID2 is a fixed-length field of 2 bits or more. However, these two areas may be combined into one.

[0247] User ADL is a field for storing the ability information of the person being assisted. For example, the ability information is numerical data that indicates which level the ability of the person being assisted belongs to when the degree of ability is classified into a predetermined number of levels. Since the number of bits required is known depending on the definition of the ability information, User ADL may be a fixed-length field.

[0248] The scene flag is a field for storing scene information. For example, the scene information may include a bit indicating whether the number of assistants is a predetermined number or more. The scene information may also include a bit specifying the type of assistance.

[0249] The device type ID is a field for storing device information. The device information may be a type ID that identifies the type of device, or a device ID that uniquely identifies each device.

[0250] The "Instruction length" and "contents" are fields used for transmitting data to a destination device. The "Instruction length" is a field that stores the length of data. The "contents" field stores information that identifies the specific content of the data.

[0251] FIG. 25 is a diagram for explaining specific examples of contents according to the value of data type ID. For example, data type ID=0 indicates an "alert" to the device to which data is sent (alert to the user who uses the device). In this case, the contents includes an alarm contents field. The alarm contents is a variable-length field that stores information indicating the specific alarm contents. The alarm contents may include information specifying the alarm mode (display, light emission, vibration, sound output), or may include information specifying the specific alarm contents (text, light emission color, vibration pattern, sound).

[0252] Data type ID = 1 represents an instruction to "move / transport" the device to which data is being sent. In this case, the contents include the fields current location and destination. Current location is information that specifies the current location of the target device. Destination is information that specifies the target location of the target device. How the current location and target location are expressed is up to you, but they are fixed-length fields that represent, for example, two-dimensional or three-dimensional coordinate values.

[0253] Data type ID = 2 indicates "control" of the data destination device. In this case, the contents includes the fields "controlled part" and "how to control." The controlled part stores information that specifies the part to be controlled of the target device. The how to control stores information that specifies the specific control method of the part to be controlled. In the case of a nursing bed as an example, information on which of multiple bottoms to drive and to what angle may be stored in the contents.

[0254] Data type ID = 3 represents "recommendations, etc." for the device to which the data is being sent (recommendations for the user who uses that device). In this case, the contents includes a recommend contents field. The recommend contents is a variable-length field that represents the contents of the recommendation or news. Alternatively, data type ID = 3 may represent the sending of a "log" to the destination device. In this case, the contents may include a log contents field. The log contents is a variable-length field that represents log data accumulated on the device.

[0255] Data type ID=4 indicates a "table confirmation request" to a data transmission destination device. In this case, the contents include table information and App information fields. This is used, for example, when the confirmation request in steps S203 and S210 in FIG. 10 is transmitted from the first device 100 or the second device 200 to the server system 10. The table information is information about the correspondence information (table data) to be confirmed, and may be the correspondence information itself or information about the update timing of the correspondence information. The App information is information indicating an application already installed in the device. Note that data type ID=4 may be used when making a cooperation request from the first device 100 to the second device 200 shown in step S403 in FIG. 14, step S503 in FIG. 15, etc.

[0256] Data type ID=5 indicates "table delivery" that is executed based on table confirmation. In this case, contents includes table data and application fields. This is used when the server system 10 transmits the latest data to the first device 100 or the second device 200, for example, as in steps S206 and S213 in FIG. 10. Table data is the latest association information. Application is an application that needs to be installed on the device.

[0257] On the other hand, in S107, S111, S113, and S116 in FIG. 9, the server system 10 also transmits the latest data to the first device 100 or the second device 200, but data type ID = 5 is not used in this case. This is because these steps are processes that are not caused by a confirmation request for correspondence information from the first device 100 or the second device 200. In these cases, for example, the correspondence information or the tacit knowledge application is transmitted using recommend contents or log contents of data type ID = 3. For example, when transmitting the correspondence information or the application, the processing unit 11 of the server system 10 may determine whether or not a data packet of data type ID = 4 has been received before (whether or not a confirmation request for correspondence information has been received), and if so, set the data type ID = 5, and if not, set the data type ID = 3.

[0258] As described above, the length of the contents field in the frame body varies depending on the data type ID and the specific instruction content. Therefore, as shown in Fig. 24, the frame body may include an instruction length field that stores the length of the contents field before the contents field. Since the maximum length of the contents field is considered to be known, the instruction length may be, for example, a fixed-length field.

[0259] In addition, in various communication methods, a method for setting communication priority is known. For example, in IEEE802.11, a method for realizing QoS is standardized in IEEE802.11e. Specifically, IEEE802.11e uses a method called EDCA (Enhanced Distributed Channel Access) that prioritizes the transmission of frames with high priority. Since EDCA is a well-known method, a detailed description is omitted. Furthermore, priority may be set using a method other than EDCA.

[0260] In the method of this embodiment, for example, "recommendations, etc." and "logs" are set to have lower priorities than the other functions, "alert," "movement / transport," "control," "table confirmation," and "table distribution." In this way, it is possible to set the priority of the confirmation request for association information from the first device 100 and the second device 200 according to this embodiment higher than the transmission of recommendations, logs, etc.

[0261] Therefore, even if the data to be transmitted is common to the association information and the application, the communication priority is changed between when data type ID = 3 and when data type ID = 5. Specifically, in the case of processing based on a confirmation request for association information from the first device 100 or the like (data type ID = 5), the communication priority is set relatively high. This allows the server system 10 to quickly respond to, for example, the request for confirmation of association information. This makes it possible to shorten the waiting time after the detection of a trigger, for example, the time until S217 is performed after step S201 in FIG. 10.

[0262] On the other hand, if the processing is not based on a request to confirm association information from the first device 100 or the like (data type ID = 3), even if the data content is association information or an application, the communication priority is set relatively low. This makes it possible to reduce the communication load when the link processing is not executed immediately.

[0263] As shown in Fig. 24, the frame body may also include a field corresponding to the expiration date. The expiration date is information indicating the expiration date of each message, which is set in consideration of the timing at which the message (data frame) is issued and the valid duration of each set data (User ADL, etc.). The expiration date may be data specifying the hour and minute, data specifying the year, month, and day, or both.

[0264] For example, if the ADL is updated once a day in the server system 10, the expiration date of the message is set to a maximum of 24 hours. The server system 10, the first device 100, and the second device 200 that receive the message compare the expiration date set in the message with the current time, and discard the message that has passed the expiration date. In this way, it is possible to prevent data that has become outdated after being transferred between multiple terminals and the server system 10 from being referenced. In the above example of the ADL, when a new ADL is calculated in the server system 10, it is possible to invalidate the message whose data of the old ADL is stored in the User ADL.

[0265] Although the present embodiment has been described in detail as above, it will be easily understood by those skilled in the art that many modifications are possible that do not substantially deviate from the novel matters and effects of the present embodiment. Therefore, all such modifications are intended to be included in the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader meaning or synonymy can be replaced with that different term anywhere in the specification or drawings. All combinations of the present embodiment and modifications are also included in the scope of the present disclosure. The configurations and operations of the information processing system, server system, first device, second device, etc. are not limited to those described in the present embodiment, and various modifications are possible. [Explanation of symbols]

[0266] 1...information processing system, 10...server system, 11...processing unit, 12...storage unit, 13...communication unit, 14...capability information acquisition unit, 15...scene information acquisition unit, 16...association information processing unit, 100...first device, 110...processing unit, 120...storage unit, 130...communication unit, 140...display unit, 150...operation unit, 200...second device, 440...seat sensor, 441...cushion, 442...control box, 460...choking detection device, 461...throat microphone, 462...terminal device, 630...wheelchair, OB1-OB6...object, RE1-RE6...area, Se1-Se4...pressure sensor

Claims

1. A first device used to assist a person being assisted; A second device that is used in cooperation with the first device in the assistance of the person being assisted; a server system connected to the first device and the second device; Including, The server system includes: storing correspondence information that associates a plurality of devices used in cooperation with each other in the assistance of the person being assisted; The first device is When trigger information indicating a start of the assistance is acquired, a link target device is determined based on the association information, and a link request is transmitted to the second device determined to be the link target device; The first device and the second device An information processing system that executes pre-processing on the association information according to a connection state with the server system, and then starts processing related to the assistance.

2. In claim 1, The server system includes: Transmitting the association information to the first device and the second device; The first device stores the association information as first association information; The second device is an information processing system that stores the association information as second association information.

3. In claim 2, The first device is When the trigger information is acquired, the information processing system determines whether it is connected to the server system, and if connected, determines whether the first correspondence information is up to date by communicating with the server system.

4. In claim 3, The second device is An information processing system that, when receiving the collaboration request, determines whether it is connected to the server system, and if connected, determines whether the second correspondence information is up to date by communicating with the server system.

5. In claim 2, The first device is If the second device is not connected to the server system, the second device transmits the link request including information regarding first correspondence information to the second device; The second device is An information processing system that, when connected to the server system when the collaboration request is received, determines whether the first association information and the second association information are up to date by communicating with the server system.

6. In claim 2, When the trigger information is acquired, the first device is not connected to the server system, and when the link request is received, the second device is not connected to the server system. At least one of the first device and the second device is selecting one of the pieces of association information based on a comparison process between the first association information and the second association information; The first device and the second device An information processing system that cooperates to execute processing related to the assistance based on the selected association information.

7. In claim 6, At least one of the first device and the second device is selecting the association information having a newer update date and time from among the first association information and the second association information; The first device is The information processing system performs a process of presenting an update date and time of the association information to a user before the execution of the process related to the assistance.

8. In any one of claims 1 to 7, The association information is The information includes information regarding tacit knowledge data obtained by digitalizing the tacit knowledge of the skilled caregiver used in the care, At least one of the first device and the second device is An information processing system that acquires the tacit knowledge data from the server system based on the association information, and executes a process related to the assistance based on the tacit knowledge data.

9. In claim 5, The association information is The information includes information regarding tacit knowledge data obtained by digitalizing the tacit knowledge of the skilled caregiver used in the care, The second device is When it is determined that the first association information is not the latest, the latest first association information and the tacit knowledge data for the first device identified by the latest first association information are received from the server system; The first device is An information processing system that acquires the latest first association information and the tacit knowledge data for the first device from the second device.

10. In claim 7, The association information is The information regarding tacit knowledge data obtained by digitalizing the tacit knowledge of the skilled caregiver used in the care includes information for identifying first tacit knowledge data used in the first device and second tacit knowledge data used in the second device, At least one of the first device and the second device is An information processing system that selects, from the first association information and the second association information, the association information in which the first tacit knowledge data has been installed in the first device and the second tacit knowledge data has been installed in the second device.

11. A control method for an information processing system including a first device used to assist a person being assisted, a second device used in cooperation with the first device in the assistance of the person being assisted, and a server system connected to the first device and the second device, storing, in the server system, correspondence information associating a plurality of devices used in cooperation with each other in the assistance of the person being assisted; When the first device acquires trigger information indicating a start of the assistance, the first device determines a link target device based on the association information, and transmits a link request to the second device determined to be the link target device; execute a pre-processing on the association information according to a connection state between the first device and the second device and the server system, and then start a process on the assistance. Control methods.

Images