Information Processing Apparatus and Information Processing Method

Abstract

To provide an information processing device and information processing method or the like for appropriately supporting a care of a cared person by a carer.SOLUTION: An information processing device includes: a first processing section for generating components each of which expresses a learned model to perform determination in a care; and a second processing section for generating an application which presents information for supporting a carer in the care of a cared person based on the combination of the components. The second processing section executes processing to display objects expressing the components generated by the first processing section in a display area, and processing to receive a user input to determine connection relation of the objects in the display area, so as to generate the application based on the user input.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and the like.

Background Art

[0002] Conventionally, a system used in a scene where a caregiver assists a care recipient is known. Patent Document 1 discloses a method of arranging sensors in a living space and generating provision information regarding the state of a resident living in the living space based on the temporal change of detection information acquired by the sensors.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided are an information processing apparatus, an information processing method, and the like that appropriately support a caregiver in assisting a care recipient.

Means for Solving the Problems

[0005] One aspect of the present disclosure relates to an information processing apparatus including: a first processing unit that creates a component representing a learned model for performing determination in assistance; and a second processing unit that creates an application for presenting information for supporting a caregiver in the assistance of a care recipient based on a combination of the components, wherein the second processing unit performs processing of displaying an object representing the component created by the first processing unit in a display area, and processing of receiving a user input for determining a connection relationship of the object in the display area, and creates the application based on the user input.

[0006] Other aspects of the present disclosure relate to an information processing method for creating a component representing a learned model for making determinations in assistance, performing a process of displaying an object representing the created component in a display area, and performing a process of receiving user input for determining a connection relationship of the object in the display area, and creating an application that presents information for supporting an assistant in the assistance of a person receiving assistance based on the combination of the components determined by the user input.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30A

Figure 30B

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Mode for Carrying Out the Invention

[0008] Hereinafter, this embodiment will be described with reference to the drawings. Regarding the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted. Note that the embodiment described below does not unduly limit the content described in the claims. Also, not all of the configurations described in this embodiment are essential constituent elements of the present disclosure.

[0009] 1. Example of System Configuration FIG. 1 is a configuration example of an information processing system 10 including an information processing apparatus according to this embodiment. The information processing system 10 according to this embodiment digitizes the "intuition" and "tacit knowledge" for the work performed by an assistant based on the "intuition" and "tacit knowledge" in, for example, a medical facility or a nursing facility, and gives instructions to the assistant so that appropriate assistance can be provided regardless of the level of proficiency.

[0010] Here, the assistant may be a nursing staff member in a nursing facility, or a nurse or a practical nurse in a medical facility such as a hospital. That is, the assistance in this embodiment includes various actions for supporting the person to be assisted, and may include nursing care or actions related to medical treatment such as injections. Also, the person to be assisted here is a person who receives assistance from the assistant, and may be a resident of a nursing facility or a patient who is hospitalized or visits a hospital.

[0011] Also, the assistance in this embodiment may be performed at home. For example, the person to be assisted in this embodiment may be a person requiring home care who receives home care, or a patient who receives home medical care. Also, the assistant may be a family member of the person requiring care or the patient, or a visiting helper or the like.

[0012] The information processing system 10 shown in FIG. 1 includes a server system 100 and terminal devices 200. However, the configuration of the information processing system 10 is not limited to that shown in FIG. 1, and various modifications such as omitting a part, adding other configurations, etc. are possible. For example, in FIG. 1, three terminal devices 200-1 to 200-3 are illustrated as the terminal devices 200, but the number of terminal devices 200 is not limited to this. Also, the same applies to FIGS. 2 and 3 described later in that modifications such as omitting or adding configurations are possible. Also, hereinafter, when there is no need to distinguish between a plurality of terminal devices 200 from each other, they are simply denoted as the terminal device 200.

[0013] The information processing device of the present embodiment corresponds to, for example, the server system 100. However, the method of the present embodiment is not limited to this, and the processing of the information processing device may be executed by distributed processing using the server system 100 and other devices. For example, the information processing device of the present embodiment may include the server system 100 and the terminal device 200. Hereinafter, an example in which the information processing device is the server system 100 will be described.

[0014] The server system 100 is connected to the terminal device 200 via, for example, a network. The network here is a public communication network such as the Internet. The server system 100 may be one server or may include a plurality of servers. For example, the server system 100 may include a database server and an application server. The database server stores various data described later with reference to FIG. 2. The application server performs the processing described later with reference to FIGS. 5, 25, etc. Here, the plurality of servers may be physical servers or virtual servers. Also, when virtual servers are used, the virtual servers may be provided on one physical server or may be distributed and arranged on a plurality of physical servers. As described above, the specific configuration of the server system 100 in the present embodiment can be variously modified.

[0015] The terminal device 200 according to this embodiment is connected to the server system 100 via a network using, for example, dedicated software or a general-purpose Internet browser or the like. For example, the terminal device 200 may utilize the functions of the server system 100 using a Web API (Web Application Programming Interface). However, the Web API is not essential, and the processing according to this embodiment may be realized by other means.

[0016] For example, a user using the terminal device 200 may register components, register applications, and use applications by communicating with the server system 100.

[0017] The component in this embodiment is a processing unit for automatically executing determination in assistance, and corresponds to, for example, one learned model. For example, the component may be a binary classification model that classifies whether an object belongs to one of two classes based on input data. The binary classification model may be, for example, a model that determines whether a given event has occurred in assistance, or a model that determines whether some condition related to a given event is satisfied. The component may also be a classification model that performs classification into three or more classes. The component may also be a regression model that obtains an output value based on input data. For example, a skilled caregiver makes appropriate judgments according to the situation in various scenes of daily caregiving work and acts based on the results of such judgments, thereby realizing appropriate caregiving. The component here digitizes the judgments in each scene during caregiving.

[0018] The application in this embodiment is software that performs processing to support the execution of a series of assistance, and is realized by combining a plurality of components. The application may be software represented by, for example, a flowchart or a sequence diagram in which processing branches based on the judgment results in each of a plurality of components.

[0019] In the example of FIG. 1, the terminal device 200-1 creates components and registers them with the server system 100. The terminal device 200-2 creates applications and registers them with the server system 100.

[0020] For example, the terminal devices 200-1 and 200-2 are devices used by developers who create components and applications. Here, the developer is, for example, a skilled person with tacit knowledge regarding assistance, but is not limited thereto. For example, the developer here may be an administrator of a nursing facility, a medical facility, or the like. Also, users who do not perform assistance as a job, such as the family members of care recipients in home care, etc., are not prohibited from creating components and applications based on their own experiences.

[0021] Also, in the example of FIG. 1, the terminal device 200-3 uses an application already registered with the server system 100. For example, the terminal device 200-3 is a device used by caregivers with little work experience, assistant nurses, family members of care recipients who provide home care, etc., i.e., assistants with low proficiency.

[0022] In FIG. 1, a mobile terminal device such as a smartphone is exemplified as the terminal device 200-1, a PC (Personal Computer) is exemplified as the terminal device 200-2, and a wearable device such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses is exemplified as the terminal device 200-3. However, the aspects of the terminal device 200 that creates components, the terminal device 200 that creates applications, and the terminal device 200 that uses applications are not limited to these, and various modifications are possible. For example, an electronic circuit contact lens may be used as the terminal device 200 that uses applications. Also, in FIG. 1, the terminal device 200 that creates components, the terminal device 200 that creates applications, and the terminal device 200 that uses applications are shown as different examples, but one terminal device 200 may serve two or more of these. For example, the terminal device 200-1, which is a smartphone, may create at least one of the applications and the applications in addition to creating the components. The same applies to the terminal device 200-2, which is a PC, and the terminal device 200-3, which is MR glasses.

[0023] In the above, an example has been shown in which the terminal device 200 and the server system 100 communicate using a public communication network such as the Internet, but the present invention is not limited to this. For example, the network may be a LAN (Local Area Network) or the like. Furthermore, the terminal device 200 may be connected to the Internet via another communication device, rather than being directly connected to the Internet. In addition, the connection mode of each device in this embodiment can be modified in various ways.

[0024] 2 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes a processing unit 110, a storage unit 120, and a communication unit 130, for example.

[0025] The processing unit 110 of this embodiment is composed of the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be composed of one or more circuit devices mounted on a circuit board or one or more circuit elements. The one or more circuit devices are, for example, IC (Integrated Circuit), FPGA (field-programmable gate array), etc. The one or more circuit elements are, for example, resistors, capacitors, etc.

[0026] Also, the processing unit 110 may be realized by the following processor. The server system 100 of this embodiment includes a memory for storing information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data, etc. The memory may be the storage unit 120 or another memory. The processor includes hardware. The processor can use various processors such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), etc. The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., or a register, or a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk device. For example, the memory stores instructions readable by a computer, and when the processor executes the instructions, the functions of the processing unit 110 are realized as processing. The instructions here may be instructions in the instruction set constituting the program or instructions for instructing operations to the hardware circuit of the processor.

[0027] The processing unit 110 includes, for example, a first processing unit 111, a second processing unit 112, a management processing unit 113, and an accounting processing unit 114.

[0028] The first processing unit 111 performs first processing related to the creation and registration of components. For example, the first processing unit 111 performs processing to display on the display unit 240 of the terminal device 200 a screen described later using FIGS. 7, 10 to 12, etc. Further, the first processing unit 111 performs processing to acquire, via the network, user input entered by a user who is a developer using the operation unit 250 of the terminal device 200 based on the screen. Further, the first processing unit 111 performs processing to create a component that is a learned model by performing machine learning based on the user input. The first processing unit 111 stores information regarding the created component in the storage unit 120 as component information 122.

[0029] The second processing unit 112 performs second processing related to the creation and registration of applications. For example, the second processing unit 112 performs processing to display on the display unit 240 of the terminal device 200 a screen described later using FIGS. 26 to 29. Further, the second processing unit 112 performs processing to acquire, via the network, user input entered by a user who is a developer using the operation unit 250 of the terminal device 200 based on the screen. The second processing unit 112 performs processing to create an application, for example, by automatically generating source code based on the user input. The second processing unit 112 stores information regarding the created application in the storage unit 120 as application information 123.

[0030] The management processing unit 113 performs management processing and the like for users who use the information processing system 10 of the present embodiment. For example, for each user, the management processing unit 113 stores in the storage unit 120 as user information 121 the user ID, user name, login password, email address, components registered by the user, applications registered by the user, applications downloaded by the user, and the like. For example, a user logs in to the information processing system 10 using at least one of the user ID and user name and the login password. Note that the users in the present embodiment may be divided into developers who create components and applications and users in the narrow sense who use applications. The management processing unit 113 may manage, as user information 121, information for identifying whether each user is a developer or a user in the narrow sense. Further, the user information 121 may include a developer name. The above-described user name is information that uniquely identifies a user in the information processing system 10, and the developer name is information that is displayed as the creator of components and applications registered by the target user. In this way, it is possible to make the user name used for user management and the like different from the name of the creator displayed on an application search screen and the like described later using FIG. 31 and the like.

[0031] The accounting processing unit 114 executes at least one of a process of paying a reward for the provision to a user who has provided a component or an application and a process of requesting payment of a price for the use to a user who has used a component or an application. Note that in the method of the present embodiment, the user who has paid the price and the user who uses the application may be the same or different. For example, as described later with reference to FIGS. 36 to 38, a primary user such as a hospital may pay a price, and a secondary user such as a patient may use the application. Further, the secondary user may be a family member or a friend of the primary user. For example, the accounting processing unit 114 may perform accounting processing related to the purchase of a gift card by a primary user, and a secondary user who has received the gift card may download an application.

[0032] The storage unit 120 is a work area of the processing unit 110 and stores various information. The storage unit 120 can be realized by various memories, and the memory may be a semiconductor memory such as SRAM, DRAM, ROM, or flash memory, or a register, or a magnetic storage device, or an optical storage device.

[0033] The storage unit 120 may store user information 121, component information 122, and application information 123.

[0034] As described above, the user information 121 includes information for identifying a user such as a user ID and a user name, and information for identifying components and applications registered or used by the user.

[0035] The component information 122 is information for identifying a component, and includes a component name, text for explaining the component, data formats of input data, data formats of output data, etc. The component information 122 may also include information for identifying a specific algorithm for obtaining output data based on the input data. For example, when the learned model representing the component is a neural network (hereinafter referred to as NN), the component information 122 may include the number of layers of the NN, the number of nodes included in each layer, the connection relationship and weights between nodes, activation functions, etc. The component information 122 may also include information such as the registration date, creator, and usage fee of the component.

[0036] The application information 123 is information for identifying an application, and includes an application name, text for explaining the application, information for identifying components used, etc. The application information 123 may also include an execution file for executing the application. The application information 123 may also include information such as the registration date, creator, and usage fee of the application.

[0037] The communication unit 130 is an interface for performing communication via a network, and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. The communication unit 130 may operate according to the control by the processing unit 110, or may include a processor for communication control different from the processing unit 110. The communication unit 130 may perform communication according to, for example, a method defined in IEEE802.11. However, various modifications of the specific communication method are possible.

[0038] FIG. 3 is a block diagram showing a detailed configuration example of the terminal device 200. The terminal device 200 includes, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250.

[0039] The processing unit 210 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. Further, the processing unit 210 may be realized by a processor. The processor can use various processors such as a CPU, a GPU, and a DSP. The function of the processing unit 210 is realized as processing by the processor executing instructions stored in the memory of the terminal device 200.

[0040] The storage unit 220 is a work area of the processing unit 210 and is realized by various memories such as SRAM, DRAM, and ROM.

[0041] The communication unit 230 is an interface for performing communication via a network, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100 via, for example, a network.

[0042] The display unit 240 is an interface for displaying various information, which may be a liquid crystal display, an organic EL display, or a display of other types. The operation unit 250 is an interface for receiving user operations. The operation unit 250 may be a button or the like provided in the terminal device 200. Also, the display unit 240 and the operation unit 250 may be a touch panel configured integrally.

[0043] Also, the terminal device 200 may include components not shown in FIG. 3, such as a light emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light emitting unit is, for example, an LED (light emitting diode), and performs notification by light emission. The vibration unit is, for example, a motor, and performs notification by vibration. The sound input unit is, for example, a microphone, and the sound output unit is, for example, a speaker, and performs notification by sound. Also, the terminal device 200 may include various sensors such as a motion sensor such as an acceleration sensor and a gyro sensor, an imaging sensor, and a GPS (Global Positioning System) sensor.

[0044] As described above, the information processing apparatus (for example, the server system 100) according to the present embodiment includes a first processing unit 111 and a second processing unit 112. The first processing unit 111 creates a component representing a learned model for making a determination in assistance. The second processing unit 112 creates an application that presents information for supporting an assistant in the assistance of the person to be assisted based on a combination of components. Here, the information for supporting the assistant may be information representing desirable actions in assistance, or may be information for issuing a warning when the assistant performs inappropriate actions. Also, the information for supporting the assistant may be information for recommending useful instruments or the like for assistance. In addition, the information for supporting the assistant in the present embodiment can widely include various information serving as guidelines in assistance. For example, the application may be software that presents a series of actions recommended in assistance.

[0045] According to the method of this embodiment, it is possible to digitize the tacit knowledge of experts and enable appropriate assistance even for assistants with low proficiency. For example, even assistants with low proficiency can provide assistance equivalent to that of experts, thus improving the reproducibility of assistance. Also, the variation in skills is suppressed, making organizational management easier, and as a result, the occurrence of incidents such as the fall of the assisted person is suppressed. As a result, for example, it is possible to suppress the need for hospitalization and the resulting vacancy in the nursing facility, the occurrence of hospital-acquired conditions (HACs) described later in a hospital, etc., and the overtime work associated with the preparation of accident reports. Also, if incidents are suppressed, it is possible to prevent assistants from becoming overly sensitive to risks, thus reducing stress and, as a result, suppressing the turnover rate of assistants. Also, by enabling the improvement of assistants' skills and the improvement of the working environment, it is possible to improve the satisfaction of patients and their families and the quality of life (QOL).

[0046] At that time, according to the method of this embodiment, the judgment by an expert is digitized as a component, and an application for supporting assistance is created by further combining the components. In this way, it becomes possible to manage the tacit knowledge of experts using multiple layers. For example, in the method of this embodiment, it is also possible to use one component in multiple applications. Since it is also possible to reuse useful components, etc., application creation becomes easier, and as a result, the digitization of tacit knowledge can be promoted. Also, since useful components are expected to be widely used, it is possible to promote the use of digitized tacit knowledge.

[0047] Furthermore, in the method of this embodiment, the second processing unit 112 performs a process of displaying an object representing a component created by the first process in a display area, and a process of receiving a user input for determining a connection relationship of the objects in the display area, and creates an application based on the user input. The display area here may be an area corresponding to a part or all of the display unit 240 of the terminal device 200, for example, the display area RE9 described later with reference to FIGS. 26 to 29. The object is a display object displayed in the display area, and may be an object having a shape corresponding to the processing content, such as a box or a diamond. The connection relationship of the objects is information indicating the relationship between a plurality of components included in the application. For example, in the display area, it is information indicating how the objects are connected to each other using an edge (line) or the like.

[0048] In this way, it becomes possible to create an application using a visually easy-to-understand interface. An expert in this embodiment may have expertise regarding assistance, but may not have expertise regarding programming or system development. In that regard, according to the method of this embodiment, even if lacking knowledge such as programming, an application can be easily created. As a result, it becomes possible to appropriately digitize the tacit knowledge accumulated by experts.

[0049] Also, part or all of the processes performed by the information processing apparatus of this embodiment may be realized by a program. The processes performed by the information processing apparatus are, for example, the processes performed by the processing unit 110 of the server system 100.

[0050] The program according to this embodiment can be stored in a non-transitory information storage device (information storage medium), which is a computer-readable medium, for example. The information storage device can be realized by, for example, an optical disk, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 etc. perform various processes of this embodiment based on the program stored in the information storage device. That is, the information storage device stores a program for causing a computer to function as the processing unit 110 etc. A computer is a device including an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for causing a computer to execute each step described later using FIGS. 5, 25, etc.

[0051] Also, the method of this embodiment can be applied to an information processing method that executes the following steps. The information processing method includes a step of creating a component representing a learned model for making a determination in assistance, a process of displaying an object representing the created component in a display area, and a step of performing a process of receiving a user input for determining a connection relationship of the objects in the display area, and a step of creating an application for presenting information for supporting an assistant in assisting a person to be assisted based on a combination of components determined by the user input.

[0052] 2. Details of the processing The details of the processing in this embodiment will be described.

[0053] 2.1 Overview FIG. 4 is a state transition diagram showing the state transition in the processing unit 110 of the server system 100. Also, FIG. 4 may be considered as a diagram for explaining the transition of the screen displayed on the display unit 240 of the terminal device 200 based on the processing in the processing unit 110.

[0054] For example, when software that utilizes the functions of the server system 100 or a web browser is launched in the terminal device 200, the processing unit 110 first transitions to a state S1 corresponding to the account settings and the login state. For example, the processing unit 110 displays a login screen on the display unit 240 of the terminal device 200 and prompts the user to input a user ID and password. Also, in the case of a new user, a screen for creating a new account may be displayed.

[0055] When the login process is completed, the processing unit 110 selects a user mode or a developer mode. For example, in this embodiment, information indicating whether the user is a user who uses applications in a narrow sense or a developer who creates components and applications may be associated. Alternatively, the user may be both a user in a narrow sense and a developer, and it may be determined based on user input which type of processing to perform.

[0056] When the user who has performed the login process is a developer, the processing unit 110 transitions to a state S2 which is the developer mode, and displays, for example, a developer screen to be described later using FIG. 6 on the display unit 240. In the developer mode, a transition to a state S3 which is the component creation mode and a transition to a state S4 which is the application creation mode are performed. Specific processes executed in the processing unit 110 and examples of display screens will be described later using FIGS. 6 to 31.

[0057] On the one hand, when the user who has performed the login process is a user in the narrow sense, the processing unit 110 transitions to the state S5 which is the user mode, and displays, for example, a user screen described later with reference to FIG. 32, on the display unit 240. In the user mode, the processing unit 110 searches for applications and accepts user selection input for the search results, and can transition to the state S6 of downloading the selected application to the terminal device 200 used by the logged-in user. Further, the processing unit 110 can transition to the state S7 of performing processing to provide the application selected by the logged-in user to a secondary user other than the logged-in user. Further, when an application is provided to the secondary user, the processing unit 110 may transition to the state S8 of performing processing to issue a certificate to the logged-in user and processing to present the certificate. Details of the secondary user and the certificate will be described later with reference to FIGS. 36 to 39.

[0058] Note that in the present embodiment, one user may have the attributes of both a developer and a user in the narrow sense. Therefore, as shown in FIG. 4, state transition between the state S2 corresponding to the developer mode and the state S5 corresponding to the user mode may be possible. Further, the state transition is not limited to the example shown in FIG. 4, and state transition between two states not shown by arrows in FIG. 4 may be possible, or other states not shown in FIG. 4 may be added.

[0059] 2.2 First Process (Component Creation) FIG. 5 is a flowchart for explaining the first process by the first processing unit 111. First, in step S101, the first processing unit 111 performs processing to accept user input for specifying input data and output data of the component. Specifically, as will be described later with reference to FIGS. 7 and 10, the first processing unit 111 accepts input of a sensor device used for acquiring input data and input of a data format of output data, etc., using a visually easy-to-understand interface. Alternatively, the first processing unit 111 may accept user input based on a screen described later with reference to FIGS. 14 to 23.

[0060] In step S102, the first processing unit 111 receives an annotation by a developer. The annotation here is, for example, correct data in machine learning and information representing the judgment of an expert in assistance or the like. Thereby, training data in which input data and correct data are associated is acquired.

[0061] In step S103, the first processing unit 111 generates a learned model by performing machine learning based on the training data. The components in the present embodiment correspond to the learned model as described above. Note that the process of step S103 is not limited to being executed in the server system 100, and an external learning device may be used.

[0062] In step S104, the first processing unit 111 stores information including the generated learned model in the storage unit 120 as component information 122. Hereinafter, the details of each step will be described.

[0063] 2.2.1 Identification of Input and Output FIG. 6 is an example of a developer screen displayed on the display unit 240 in the state S2 of FIG. 4. The developer screen includes an application creation button and a component creation button. When a user input for selecting the component creation button is performed, the processing unit 110 shifts to the state S3 of FIG. 4 and starts the first process. The user input here is a click operation, a tap operation, or the like at a position corresponding to the component creation button. As shown in FIG. 6, the developer screen may include information other than the above two buttons. In the example of FIG. 6, the time, an ID for identifying the logged-in user, the current mode, etc. may be displayed. Also, the mode display may be a pull-down menu as shown in FIG. 6. For example, the pull-down menu includes a plurality of modes corresponding to S1 to S8 in FIG. 4, and the processing unit 110 may perform a process of transitioning to the corresponding mode when any mode is selected from the pull-down menu.

[0064] FIG. 7 is an example of a component creation screen displayed on the display unit 240 when a user input for selecting the component creation button in FIG. 6 is made. As shown in FIG. 7, the component creation screen may include four display areas RE1 to RE4. However, the specific screen configuration is not limited to FIG. 7, and various modifications can be made.

[0065] In the display area RE1, information for determining input data, output data, etc. of the component is displayed. For example, in the initial state, the first processing unit 111 performs a process of displaying a box representing a learning model such as NN, a line displayed on the left side of the box, a line displayed on the right side of the box, etc. in the display area RE1.

[0066] In the display area RE2, parts such as boxes, text boxes, and lines are displayed. When a user input for selecting these parts is made, the first processing unit 111 performs a process of adding the selected parts to the display area RE1.

[0067] In the display area RE3, candidates for sensor devices that are sources of input data are displayed. The sensor device in the present embodiment is a device that outputs information based on a sensor, and may be composed of a single sensor or may include a plurality of sensors. The sensor device may include a processor, a memory, a communication unit, a display unit, an operation unit, etc., or one or more of these may be omitted. Also, the specific shape, installation location, usage mode, etc. of the sensor device can be variously modified according to the sensor device. In the example of FIG. 7, as sensor devices, sleep scan, seat surface sensor, acceleration sensor, etc. are exemplified. Hereinafter, examples of each sensor device will be described. Hereinafter, the data acquired based on the sensor device is also referred to as sensing data. The sensing data is not limited to the output of the sensor itself, and may be data calculated based on the sensor output. Also, the calculation based on the sensor output may be executed in the sensor device or may be executed in another device.

[0068] Sleep scan is a sensor device that senses information related to the sleep of the care recipient. As shown in FIG. 8, for example, the sleep scan is a sheet-like or plate-like detection device 400 provided between the bottom of the bed 410 and the mattress 420. However, various sensor devices for detecting the sleep state are known, and a smartphone or a wristwatch-type device having a microphone and an acceleration sensor may be used.

[0069] When the user goes to bed, the detection device 400 detects the body vibration (body movement, vibration) of the user through the mattress 420. Based on the body vibration detected by the detection device 400, information regarding the respiration rate, heart rate, activity level, posture, wakefulness / sleep, getting out of bed / being in bed is obtained. Further, the detection device 400 may determine non-REM sleep and REM sleep, and may also determine the depth of sleep. For example, the periodicity of body movement may be analyzed, and the respiration rate and heart rate may be calculated from the peak frequency. The analysis of periodicity is, for example, Fourier transform or the like. The respiration rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time is, for example, 1 minute. Further, the body vibration may be detected per sampling unit time, and the number of detected body vibrations may be calculated as the activity level. Note that information such as the respiration rate may be calculated by the detection device 400 or may be calculated by another device connected to the detection device 400.

[0070] Hereinafter, for the sake of simplifying the explanation, an example in which the sensing data of the sleep scan is the determination result of whether the state is wakefulness or sleep will be described.

[0071] The seat surface sensor is, for example, a sensor device that is arranged on the seat surface of a wheelchair and determines the posture of a care recipient using the wheelchair. For example, the sensing data based on the seat surface sensor is a determination result of which of a plurality of postures including a normal posture, a forward shift, a lateral shift, etc. is the posture (hereinafter also referred to as the sitting posture) when the care recipient sits on the wheelchair. The forward shift represents a state where the user's center of gravity has shifted forward more than usual, and the lateral shift represents a state where the user's center of gravity has shifted to either the left or right more than usual. Both the forward shift and the lateral shift correspond to states with a relatively high risk of slipping or falling. Note that the seat surface sensor may be a sensor device arranged on a normal chair or a sensor device that determines the posture of a user sitting on a bed or the like.

[0072] FIG. 9 is an example of a seat surface sensor and is an example of a pressure sensor arranged on a wheelchair 520. In the example of FIG. 9, four pressure sensors Se1 to Se4 are arranged on the back side of a cushion 521 arranged on the seat surface of the wheelchair 520. The pressure sensor Se1 is a sensor arranged in the front, the pressure sensor Se2 is a sensor arranged in the rear, the pressure sensor Se3 is a sensor arranged on the right, and the pressure sensor Se4 is a sensor arranged on the left. Here, the front, rear, left, and right represent the directions as seen from the care recipient when the care recipient is sitting on the wheelchair 520.

[0073] As shown in FIG. 9, the pressure sensors Se1 to Se4 are connected to a control box 523. The control box 523 includes a processor that controls the pressure sensors Se1 to Se4 and a memory that serves as a work area for the processor. The processor detects pressure values by operating the pressure sensors Se1 to Se4.

[0074] The assisted person sitting on the wheelchair 520 may feel pain in the buttocks and may shift the position of the buttocks. For example, the state where the buttocks are shifted forward more than usual is called forward shift, and the state where they are shifted laterally is called lateral shift. Also, forward shift and lateral shift may occur simultaneously, and the center of gravity may shift obliquely. By using the pressure sensors arranged on the cushion 521 as shown in FIG. 9, the change in the position of the buttocks can be appropriately detected, so that forward shift and lateral shift can be accurately detected.

[0075] For example, first, the timing when the person transfers to the wheelchair 520 and assumes a normal posture is set as the initial state. In the initial state, since the assisted person sits deeply on the seat surface of the wheelchair 520, it is assumed that the value of the rear pressure sensor Se2 is relatively large. On the other hand, when forward shift occurs, since the position of the buttocks moves forward, the value of the front pressure sensor Se1 increases. For example, the processor of the control box 523 may determine that forward shift has occurred when the value of the pressure sensor Se1 has increased by a predetermined amount or more compared to the initial state. When the value of the pressure sensor Se1 exceeds a certain threshold, it is determined that the assisted person is sitting on the wheelchair 520, and it may be determined that forward shift has occurred only based on the change in the value of the pressure sensor Se2 without comparing it with the pressure sensor Se1. Also, instead of using the value of the pressure sensor Se1 alone, processing may be performed using the relationship between the values of the pressure sensor Se2 and the pressure sensor Se1. For example, the difference in voltage values, which are the outputs of the pressure sensor Se2 and the pressure sensor Se1, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used.

[0076] Similarly, when lateral displacement occurs, the position of the buttocks moves in either the left or right direction. Therefore, if it is a left displacement, the value of the pressure sensor Se4 increases, and if it is a right displacement, the value of the pressure sensor Se3 increases. Thus, the processor may determine that a left displacement has occurred when the value of the pressure sensor Se4 increases by a predetermined amount or more compared to the initial state, and may determine that a right displacement has occurred when the value of the pressure sensor Se3 increases by a predetermined amount or more compared to the initial state. Alternatively, the processor may determine right and left displacements using the relationship between the values of the pressure sensor Se4 and the pressure sensor Se3. Similar to the example of forward displacement, the difference in voltage values, which are the outputs of the pressure sensor Se4 and the pressure sensor Se3, may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio with respect to the initial state may be used.

[0077] Also, an acceleration sensor, which is an example of the sensor device of this embodiment, is, for example, a three-axis acceleration sensor. Further, instead of or in addition to the acceleration sensor, a gyro sensor that detects the angular velocity around each axis may be used. The acceleration sensor determines, for example, the walking state of the assisted person. For example, the sensing data of the acceleration sensor is the determination result of whether the assisted person is in a stopped or walking state.

[0078] Here, the acceleration sensor may be included in, for example, a seal-shaped device attached to the chest or back of the assisted person. Further, the acceleration sensor may be included in a smartphone carried by the assisted person or a wearable device such as the MR glasses exemplified as the terminal device 200-3 in FIG. 1. Since walking and running are periodic motions, the acceleration data also has periodicity. Therefore, based on the presence or absence of periodicity, its period, amplitude, etc., it is possible to determine whether the assisted person is stopped or walking. Also, various methods for determining walking based on an acceleration sensor are known, and these methods can be widely applied in this embodiment.

[0079] Also, the determination regarding walking or the like may vary depending on the situation of the care recipient, for example, whether the care recipient is near the bed in the nursing facility, in the toilet, in the cafeteria, or the like. In the present embodiment, for example, the developer may not only simply specify the sensor device, but also specify, as input data, the result of extracting the sensing data in a specific situation from the sensing data acquired by the sensor device.

[0080] In the above, the sleep scan, the seat surface sensor, and the acceleration sensor are exemplified as the sensor device, but the sensor device in the present embodiment is not limited to this, and other devices may be added. Also, the candidates for the sensor device displayed in the display area RE3 may be those registered by the developer, or those registered as default in the server system 100.

[0081] The display area RE4 displays information for searching for components registered in the server system 100. For example, the display area RE4 may include a text box for receiving text input by the user. When some text is input, the first processing unit 111 extracts a component that matches the text based on the component information 122 stored in the storage unit 120 and the input text, and displays the extraction result in the display area RE4. For example, when the component information 122 includes text such as the title or description of the component, the search process is executed based on the comparison process between these texts and the text that is the user input. However, the component extracted by the search process is not limited to those that include the text that is the user input in the title or the like, and a component that does not include the text may be extracted. For example, when the component is a learned model that performs classification using the k-means method or the like, a component that does not include the text that is the user input may be displayed as the search result. Also, FIG. 7 exemplifies the search result when "vein" is used as the search word, but other search words may be used.

[0082] The developer may determine whether there is already a component similar to the component to be created by using the display area RE4. Also, as will be described later, in creating a component, other existing components may be used, and the display area RE4 may be used to search for the other components.

[0083] FIG. 10 is an example screen when creating a component for determining the fall of a care recipient. For example, assume that a skilled caregiver has tacit knowledge of estimating the possibility of a fall by combining the sleep of the care recipient, the sitting posture in a wheelchair, etc., and the walking state. In this case, the user who is the developer selects three sensor devices, namely a sleep scan, a seat surface sensor, and an acceleration sensor, as the sensor devices to be used for acquiring input data.

[0084] For example, on the screen shown in FIG. 7, the developer selects a line from the display area RE2 and performs the operation of adding it to the left side of the box twice. As a result, three lines are connected to the left side of the box together with the original line, so that it becomes possible to associate three sensor devices with the input data.

[0085] After that, the developer selects the sleep scan from the display area RE3 and connects it to any of the three lines on the left side of the box. For example, as shown in FIG. 10, the developer may perform an operation of dragging the box corresponding to the sleep scan included in the display area RE3 to the position on the left side of the line. Similarly, the developer performs an operation of adding the seat surface sensor to the display area RE1 and an operation of adding the acceleration sensor to the display area RE1. By the above operations, as shown in FIG. 10, each of the sleep scan, the seat surface sensor, and the acceleration sensor is associated as a source for acquiring input data of the component related to fall determination.

[0086] The developer also inputs an explanation of the output data and the data format. For example, the developer uses a text box corresponding to the output data to input text explaining the output data. Here, since the output data is information indicating the presence or absence of a risk of falling, the developer inputs "Risk of falling" into the text box. Also, the output data here is data indicating whether the risk of falling is "present" or "absent". Therefore, the developer designates binary data as the data format of the output data. Note that the method of this embodiment is not limited to those that specify the data type in advance in this way. For example, as will be described later as a modification, the developer may perform user input to specify the data format of the output data while checking the actual sensing data.

[0087] In this way, the first processing unit 111 may create a component based on a user input associating the sensor device with the input data of the learned model and a user input specifying the output data format of the learned model. By doing so, it is possible to easily realize the association between the input data and the output data required when creating the learned model. As a result, even a user lacking knowledge of programming, data analysis, etc. can easily create a component.

[0088] For example, as shown in FIG. 10, when a sleep scan, a seat surface sensor, and an acceleration sensor are selected and binary data is specified as the data format of the output data, which is the risk of falling, the first processing unit 111 can identify the data formats of the input data and the output data necessary for creating a component that determines the risk of falling. Therefore, the first processing unit 111 displays a data input button for inputting specific training data to be used in machine learning. In the example of FIG. 10, the data input button is displayed in the display area RE1.

[0089] 2.2.2 Annotation FIG. 11 is an example of a component creation screen displayed on the display unit 240 when a user input for selecting a data input button displayed in the display area RE1 of FIG. 10 is performed. Since FIG. 11 is a screen for a user to perform annotation, it is also referred to as an annotation screen. The annotation screen includes display areas RE5 to RE7. The display area RE5 includes a display area RE5a for displaying sensing data, a display area RE5b for inputting the judgment result of an expert for the sensing data, and a display area RE5c for displaying a registration button and a learning button. The display area RE6 is a search screen, which is the same as the display area RE4 of FIG. 7. The display area RE7 is a screen showing the current structure of the component, and performs the same display as the display area RE1 of FIG. 10. Note that an edit button is displayed in the display area RE7, and when a user input for selecting the edit button is performed, the first processing unit 111 may execute a process of returning to FIG. 10. In this way, it is possible to smoothly perform the transition between the annotation screen and the component creation screen.

[0090] For example, the display unit 240 displays the sensing data actually acquired by the sleep scan, seat surface sensor, and acceleration sensor selected in FIG. 10 in the display area RE5a. The sensing data displayed here is, for example, data held by a user who is a developer, and is stored in the terminal device 200 used by the developer or another device connected to the terminal device 200. Alternatively, the sensing data held by the developer may be uploaded to the server system 100, and the first processing unit 111 may perform display processing in the display area RE5a based on the uploaded data. For example, the first processing unit 111 may receive a user input including information (such as name and ID) for identifying the target care recipient and information for identifying the date and time when the sensing data was measured, and based on the user input, perform processing to display the sensing data of a specific care recipient and date and time in the display area 5a. In the example of FIG. 11, an example is considered where the name "XXXX" is input as information for identifying the care recipient and January 1, 2021 is input as information for identifying the date and time. Therefore, in the display area RE5a, the sensing data of the sleep scan, seat surface sensor, and acceleration sensor acquired for the care recipient with the name "XXXX" on January 1, 2021 is displayed.

[0091] Also, in the display area RE5b, the text "Evaluation of fall possibility" and a box for data input are displayed. This box is, for example, a box into which data specified by the developer as the data format of the output data in FIG. 10 can be input. In the example of FIG. 10, since the output data is binary data, for example, either "0" representing no fall possibility or "1" representing a fall possibility is input into the box. Note that instead of the numerical data "0" and "1", inputs such as text data "yes", "no", etc. may be made, and various modifications of specific user inputs are possible.

[0092] The developer determines the presence or absence of the possibility of the assisted person falling based on the data displayed in the display area RE5a. As described above, since the skilled person here has the tacit knowledge to determine the presence or absence of the possibility of falling based on sleep, sitting posture, and walking state, an appropriate determination can be made based on the data displayed in the display area RE5a. For example, when the skilled person determines that there is a possibility of falling, as shown in FIG. 11, "1" is input into the box in the display area RE5b.

[0093] When a user input is made to select the registration button in the display area RE5c in this state, the first processing unit 111 acquires, as training data for machine learning, the data associated with "1" input in the display area RE5b for the sensing data displayed in the display area RE5a. The sensing data displayed in the display area RE5a is input data, and the data input using the display area RE5b is an annotation representing the correct answer data.

[0094] The developer generates training data by annotating a plurality of input data while changing the target assisted person and the date and time when the data was acquired. As a result, the first processing unit 111 can accumulate training data for use in machine learning. Then, when a user input is made to select the learning button in the display area RE5c, the first processing unit 111 generates a learned model for determining the presence or absence of the possibility of falling by performing machine learning based on the training data accumulated so far.

[0095] Note that the first processing unit 111 may acquire training data in which annotations are associated with the sensing data by performing a process of displaying a list of a plurality of sensing data acquired using the sensor device in chronological order and a process of accepting an annotation, which is the user's judgment result, for one or more of the plurality of sensing data. In this way, annotations based on the temporal changes in the sensing data become possible.

[0096] FIG. 12 shows another example of the annotation screen displayed on the display unit 240 when a user input for selecting the data input button in FIG. 10 is made. The annotation screen includes display areas RE6 to RE8. Since the display areas RE6 and RE7 are the same as those in FIG. 11, detailed description thereof is omitted.

[0097] The display area RE8 includes a display area RE8a for listing time-series sensing data, a display area RE8b for inputting an annotation representing the judgment result of an expert for the sensing data, and a display area RE8c for displaying a registration button and a learning button. For example, as shown in FIG. 12, in the display area RE8a, the sensing data of the sleep scan, the seat surface sensor, and the acceleration sensor for a predetermined period is displayed. The predetermined period here is, for example, one month, but other periods may be set. In the example of FIG. 12, since one line represents the sensing data of one day, the number of lines of data corresponding to the number of days included in the predetermined period is displayed in the display area RE8a.

[0098] For example, even if sensing data indicating that the care recipient did not sleep well on a certain day is acquired, the judgment of an expert regarding the possibility of falling may differ depending on whether it is an accidental occurrence on that day or the care recipient is continuously suffering from sleep deprivation. That is, when digitizing the tacit knowledge of an expert, it is useful to consider the time-series changes of the sensing data. In that regard, by performing a list display as shown in FIG. 12, the accuracy of the annotation is improved, so that the determination accuracy using the component can be improved.

[0099] For example, as shown in FIG. 12, the input of the annotation may be in units of one day as in FIG. 11. The developer determines the possibility of falling on each day in view of the time-series changes of the sensing data, and inputs the determination result into the box in the display area RE8b.

[0100] When a user input is made to select the registration button in the display area RE8c, training data is stored. When a user input is made to select the learning button, machine learning based on the training data is executed, which is the same as in FIG. 11.

[0101] 2.2.3 Learning Process In this embodiment, machine learning using a neural network (NN) may be performed. However, machine learning is not limited to NN, and other methods such as SVM (support vector machine) and k-means method may be used, or methods developed from these may be used. Also, although supervised learning is exemplified below, other machine learning such as unsupervised learning may be used.

[0102] FIG. 13 is an example of the basic structure of an NN. One circle in FIG. 13 is called a node or a neuron. In the example of FIG. 13, the NN has an input layer, two or more intermediate layers, and an output layer. The input layer is I, the intermediate layers are H1 and Hn, and the output layer is O. Also, in the example of FIG. 13, the number of nodes in the input layer is 2, the number of nodes in each intermediate layer is 5, and the number of nodes in the output layer is 1. However, the number of intermediate layers and the number of nodes included in each layer can be variously modified. Also, in FIG. 13, an example is shown where each node included in a given layer is connected to all the nodes included in the next layer, but this configuration can also be variously modified.

[0103] The input layer receives the input values and outputs them to the intermediate layer H1. In the example of FIG. 13, the input layer I receives two types of input values. Note that each node in the input layer may perform some processing on the input values and output the values after the processing.

[0104] In an NN, a weight is set between two connected nodes. W1 in FIG. 13 is the weight between the input layer I and the first intermediate layer H1. W1 represents a set of weights between a given node included in the input layer and a given node included in the first intermediate layer. For example, W1 in FIG. 13 is information including 10 weights.

[0105] At each node of the first intermediate layer H1, an operation is performed in which the outputs of the nodes of the input layer I connected to the node are weighted and added using the weights W1, and then a bias is added. Further, at each node, the output of the node is obtained by applying an activation function, which is a non-linear function, to the addition result. The activation function may be a ReLU function, a sigmoid function, or other functions.

[0106] The same applies to the subsequent layers. That is, in a given layer, the output to the next layer is obtained by weighted addition of the output of the previous layer using the weights W, adding a bias, and then applying an activation function. The NN takes the output of the output layer as the output of the NN.

[0107] As can be seen from the above description, in order to obtain desired output data from input data using an NN, it is necessary to set appropriate weights and biases. In learning, training data is prepared by associating given input data with correct answer data representing the correct output data for the input data. The learning process of the NN is a process of obtaining the most probable weights based on the training data. Note that in the learning process of the NN, various learning methods such as the backpropagation method are known. In this embodiment, since those learning methods can be widely applied, detailed description thereof is omitted.

[0108] For example, in step S103 of FIG. 5, the first processing unit 111 inputs the input data among the training data to the NN, and obtains output data by performing a forward operation using the weights at that time. The first processing unit 111 compares the obtained output data with the annotation result, which is the correct answer data, and performs a process of updating the weights using the backpropagation method or the like so that the output data approaches the correct answer data.

[0109] At this time, the first processing unit 111 may perform preprocessing such as normalization on the input sensing data. The preprocessing at this time is determined according to, for example, the data type. The data type includes a character string type, an integer type, a floating point type, a binary type, and the like. Also, if it is a sensor device known to the server system 100, the data type of the sensing data based on the sensor device is also known. Therefore, the first processing unit 111 may determine the preprocessing of the sensing data based on the sensor device. Note that the point that the preprocessing may be determined based on the data type is the same when using components, that is, when creating or executing an application.

[0110] Note that NN is not limited to the configuration shown in FIG. 13. For example, as NN, a network with another configuration such as an RNN (Recurrent Neural Network) may be used. The RNN may be, for example, an LSTM (Long Short Term Memory). Also, a convolutional neural network (CNN) may be used as NN.

[0111] The first processing unit 111 stores the component information 122 including the learned model obtained by machine learning in the storage unit 120. As described above, the component information 122 may include other information such as the creator and the registration date and time in addition to the learned model.

[0112] 2.2.4 Variants Hereinafter, several variants regarding component creation will be described.

[0113] <Use of Existing Components> Note that, in the above, an example of receiving a user input for selecting a sensor device as information for specifying input data of a component has been described. However, the method of this embodiment is not limited to this. For example, a judgment result obtained by certain tacit knowledge may be obtained as input data of other tacit knowledge. For example, in the above example, each piece of information representing sleep, sitting posture, and walking state is used for determining the possibility of falling, and each piece of information is obtained using a sensor device. However, for example, tacit knowledge for determining whether a user is in a sleeping state or an awake state using a sensor device other than sleep scan may be componentized. For example, assuming a nursing facility that does not have a sleep scan, a component for determining sleep / awake using a smartphone camera image, microphone, acceleration sensor, etc. may be created and registered.

[0114] In this case, as one of the inputs of the component for determining the presence or absence of the possibility of falling described above, the output data of the component for determining sleep / awake using a smartphone camera image, microphone, acceleration sensor, etc. may be used. For example, a developer searches for a component that performs sleep determination using the display area RE4, and makes a user input to select a component whose output data is the determination result of sleep / awake. Further, the developer associates the selected component with the input data by connecting the selected component to the line on the left side of the box. In this way, by using existing components in the creation of components, it becomes possible to easily create highly functional components.

[0115] Alternatively, the developer may create a new component by modifying (processing) a part of an existing component. For example, the developer may create a new component by modifying a part of the input data of an existing component or by modifying the data format of the output data. Alternatively, the developer may create a new component by changing the training data used for learning while maintaining the formats of the input data and output data of the existing component. In addition, various modifications can be made to the method of using the existing component.

[0116] <Another example of the component creation screen> The component creation screen (including the annotation screen) in the present disclosure is not limited to the above-described examples, and various modifications can be made. FIGS. 14 to 23 are diagrams showing other examples of the component creation screen.

[0117] FIG. 14 is an example of a basic setting screen among the component creation screens. As shown in FIG. 14, the basic setting screen is a screen for inputting a component name, a component type, and a grouping setting. For example, the basic setting screen includes a text box displayed in association with the text "component name". The developer inputs the name of the component using the text box. In the example of FIG. 14, "death prediction" is input as the component name.

[0118] Also, the type of component is information that specifies the type of processing executed by the component, and as shown in FIG. 14, it may include prediction, anomaly detection, classification, image determination, and the like. Here, prediction means, for example, estimating the temporal change of some information. As shown in FIG. 14, in the basic setting screen, explanations about each type may be displayed. FIG. 14 shows an example in which prediction is selected. The first processing unit 111 may specify a model to be used for learning based on the type of component specified by the user. For example, when prediction is selected, an RNN (such as LSTM) suitable for predicting future information based on time-series data is used. Also, for classification, the k-means method or the like may be used. Note that the association between the type of component and the model is not limited to this, and various modifications can be made.

[0119] In the grouping setting, information assumed to be output data is input. In the example of FIG. 14, two are set: "watching" indicating that the target care recipient needs end-of-life care, and "not watching" indicating that end-of-life care is not necessary. That is, in the example of FIG. 14, the output data of the component is binary data. Here, end-of-life care represents assistance for a care recipient who is considered likely to die in the near future. End-of-life care differs from normal assistance in that it emphasizes alleviating physical and mental pain and supporting a dignified life for the target care recipient. Also, during end-of-life care, as the state of the care recipient changes over time, the assistance suitable for the target patient may change. That is, by presenting the start timing of end-of-life care and the change timing of the assistance content during end-of-life care, it becomes possible to provide appropriate assistance to the care recipient until the end. For example, a skilled caregiver has tacit knowledge to estimate the timing and care content for which end-of-life care is necessary from various viewpoints such as sleep and food intake, and by digitizing this tacit knowledge, other caregivers can also provide appropriate end-of-life care. As described above, here, for the sake of simplicity, a component for determining the necessity of watching is considered.

[0120] When a user input to select the Next button is performed in a state where the input for each item has been made, the first processing unit 111 performs a process of saving the input information and then transitioning to the screen shown in FIG. 16 or FIG. 17. On the other hand, when a user input to select the Back button is performed, the first processing unit 111 performs a process of returning to, for example, the developer screen in FIG. 6 without saving the input information.

[0121] Also, as shown in FIG. 14, the basic setting screen may include history information regarding past component creation. The history information includes the name of the component that was the work target and information on the work date. When a user input to select any component included in the history information is performed, the first processing unit 111 may perform a process of displaying a screen related to the selected component, such as a component creation screen or a detailed information display screen.

[0122] Also, as shown in FIG. 14, the basic setting screen may include a search and processing button. For example, as described above, in the method of this embodiment, a new component may be created using existing components. When a user input for selecting the search and processing button is performed, the first processing unit 111 may display a component search screen and perform a process of presenting the component selected on the search screen to the developer. The developer may create a new component based on the presented component. Alternatively, as will be described later with reference to FIG. 19, in this embodiment, one component may be provided as an application. In this case, when a user input for selecting the search and processing button is performed, the first processing unit 111 may display an application search screen and perform a process of presenting the application selected on the search screen to the developer. The developer may create a new component or application based on the presented application. FIG. 15 is an example of a screen displayed when the search and processing button is selected. In the example of FIG. 15, a screen for searching for and selecting an application is pop-up displayed so as to overlap the screen of FIG. 14. On the screen for searching for and selecting an application, text prompting search and selection, a text box for inputting a search word, recommended applications, etc. may be displayed. The screen may be the same as the user screen displayed in the user mode. Details of the user screen will be described later with reference to FIG. 32.

[0123] Figs. 16 and 17 are examples of a learning data setting screen for setting training data (learning data) used in machine learning. As shown in Figs. 16 and 17, the learning data setting screen may include buttons or the like that allow selection of either "labeled" or "unlabeled". When "labeled" is selected, a screen for registering sensing data with labels, i.e., correct data, already assigned is displayed. Fig. 16 is an example of a screen for registering labeled sensing data. On the other hand, when "unlabeled" is selected, a screen for assigning labels to unlabeled sensing data and then registering it is displayed. Fig. 17 is an example of a screen for registering unlabeled sensing data. Each will be described below.

[0124] As shown in Fig. 16, for example, a developer may register data of a sleep scan with a label as labeled sensing data. For example, the labeled sensing data is stored in a format such as csv (comma-separated values), and registration is performed based on a drag-and-drop operation to a part of the learning data setting screen. Here, four csv files are exemplified, but various modifications are possible for the specific file format, the number of files, etc.

[0125] Here, the labeled sensing data has a label as one of the elements of the csv file. When a label has already been assigned to the sensing data, as shown in FIG. 16, the first processing unit 111 may accept a user input that associates the label with the grouping setting input in FIG. 14. Here, an input is made to the effect that the label "1" represents "dying observation" and the label "0" represents "not a dying observation". When a user input for selecting the save button is made in this state, the first processing unit 111 registers the labeled sensing data as learning data. In this way, it becomes possible to appropriately identify the meaning of the label included in the labeled sensing data and register the labeled sensing data as learning data. Thereafter, the first processing unit 111 performs a process of displaying the learning data setting screen again and executes a process of accepting new learning data. When a user operation of selecting the back button is performed, the first processing unit 111 performs a process of transitioning to the previous screen (for example, the basic setting screen, etc.) without saving the sensing data or the like. When a user operation of selecting the back button is performed, the first processing unit 111 may display a screen for confirming with the user whether to save the sensing data or the like, and when a user operation of saving is performed, perform a process of saving the input data and then transitioning to the previous screen. When a user operation of selecting the next button is performed, the first processing unit 111 generates a learned model by performing machine learning, and then performs a process of transitioning to FIG. 18.

[0126] Also, as shown in FIG. 17, for example, a developer may register sensing data without a label. For example, sensing data without a label is also stored in a format such as csv, and can be registered by a drag-and-drop operation or the like. For example, a developer may pre-classify sensing data corresponding to reading and sensing data that is not reading. FIG. 17 is an example of a screen for registering sensing data that is not reading, for example. For example, a developer performs a drag-and-drop operation on sensing data that is not reading, and selects "not reading" from among "reading" and "not reading" in group selection. The group selection is a drop-down menu including items input in the grouping setting on the basic setting screen, for example. For example, when a user input to select a save button is performed in a state where a drag-and-drop operation of data and group selection are performed, the first processing unit 111 performs a process of registering, as learning data, data in which a label corresponding to the selected group is associated with the sensing data. In the example of FIG. 17, the first processing unit 111 registers, as learning data, data with a label indicating "not reading" for the sensing data without a label. In this way, it becomes possible to register the sensing data without a label as learning data after appropriately attaching a label. Also, when a user operation to select a back button is performed, the first processing unit 111 performs a process of transitioning to the previous screen (for example, the basic setting screen or the like) without saving the sensing data or the like. Note that when a user operation to select a back button is performed, the first processing unit 111 may display a screen for confirming with the user whether to save the sensing data or the like, and perform a process of saving the input data and then transitioning to the previous screen when a user operation to save is performed. Also, when a user operation to select a next button is performed, the first processing unit 111 generates a learned model by performing machine learning, and then performs a process of transitioning to FIG. 18. Note that although sensing data of sleep scan is illustrated in FIGS. 16 and 17, sensing data of other sensor devices may be added.Also, as described above, the setting of the learning data may be repeatedly executed, and both the processes shown in FIGS. 16 and 17 may be performed, or either one of them may be performed.

[0127] In addition, although an example of performing supervised learning has been described above, it is not limited to this. For example, in the above example, it is known that there are two types of output data, "recognition" and "not recognition", or in other words, the component being created is a binary classification model that determines to which of the two classes it belongs. Therefore, the first processing unit 111 may create a binary classification model by unsupervised learning. For example, the first processing unit 111 maps the input data to a feature space and creates a binary classification model by obtaining a hyperplane that divides the feature space. Various specific methods of unsupervised learning are known, and they can be widely applied in this embodiment. For example, in the grouping setting shown in FIG. 14, a value such as "mixing" may be set. Here, "mixing" represents information indicating that machine learning is performed without labeling the two types of output, "recognition" and "not recognition". In this case, for example, "mixing" may be selectable in the group selection in FIG. 17. When "mixing" is selected, the first processing unit 111 performs unsupervised learning based on the sensing data added by the drag-and-drop operation. In addition to the screens shown in FIGS. 16 and 17, a component creation screen for performing unsupervised learning may be added, and various modifications of the specific method are possible.

[0128] FIG. 18 is an example of a confirmation screen for allowing a developer to confirm the result of machine learning. As shown in FIG. 18, the confirmation screen displays the component name, component type, and the meaning of the label input on the basic setting screen in FIG. 14. In this way, it becomes possible to allow the developer to confirm the basic information of the created component.

[0129] Also, the confirmation screen may include the number of learning data registered using FIGS. 16 and 17, the number of data with the label "0" among the learning data, and the number of data with the label "1" among the learning data. By doing so, it becomes possible to let the developer confirm whether the number of machine learning data is sufficient and whether the bias of the learning data is not excessive. The bias of the learning data represents the difference or ratio between the number of data with the label "0" and the number of data with the label "1". If the bias of the learning data is large, the learning accuracy may decrease. In that regard, by displaying the number of data for each label, it becomes possible to let the developer confirm the bias of the data.

[0130] Also, the confirmation screen may include the accuracy rate of the learned model. For example, a part of the learning data may be used as validation data. The accuracy rate here represents the ratio at which the output data matches the correct data among the validation data when the input data among the validation data is input to the learned model. By doing so, it becomes possible to let the developer confirm the accuracy of the created learned model.

[0131] Also, the confirmation screen may display information for adjusting the learned model. For example, in the example of FIG. 18, a model (such as RNN, LSTM, etc.) for predicting future numerical changes is used, but various settings are possible depending on the situation regarding how far ahead to predict the data. For example, in FIG. 18, it includes a pull-down menu that accepts user input on how many days before to know the prediction result, in other words, how far ahead to predict the numerical value. It may also include radio buttons for selecting the filter used for preprocessing the input data. When a user input to select the recalculation button is made in the state where these are input, the first processing unit 111 executes the learning process again based on the selected parameters and performs recalculation and display processing of the accuracy rate and the like.

[0132] In addition, the confirmation screen may display the history of the correct answer rate of the learning process. For example, by recalculating while changing parameters such as the number of days and filtering described above, the first processing unit 111 obtains a plurality of correct answer rates. The first processing unit 111 may display the plurality of correct answer rates side by side on the confirmation screen. When any value is selected, the first processing unit 111 may display parameters and the like at the time when the value was obtained on the right side of the screen. In this way, the developer can confirm how the correct answer rate changes due to the change of the parameters, so that it becomes possible to easily execute the parameter adjustment.

[0133] When a user input for selecting the export button in FIG. 18 is performed, the first processing unit 111 performs a process of outputting the learned model at that time as a component. For example, the first processing unit 111 stores component information 122 including the learned model in the storage unit 120. Also, although the application in the present embodiment is assumed to be created by combining a plurality of components as a second process as described later, it is not prevented that a single component is provided to the user as an application.

[0134] FIG. 19 is an example of an application output screen displayed when a user input for selecting the export button in FIG. 18 is performed. For example, the application output screen may be pop-up displayed so as to overlap the confirmation screen shown in FIG. 18. The application output screen may include a region for selecting the type and version of the OS (Operation System) and an execution button. Here, WINDOWS, ANDROID, and iOS (WINDOWS and ANDROID are registered trademarks) are exemplified as the types of the OS, but other OSs may be selected. When a user input for selecting the execution button is performed, the first processing unit 111 performs a process of outputting an execution file or the like of the application. Also, since the application output is related to the second process, when a user input for selecting the execution button is performed, the second processing unit 112 may perform a process of outputting an execution file or the like of the application.

[0135] FIG. 20 is an example of an execution screen of an application related to swallowing prediction, and is a screen displayed on a display unit 240 of a terminal device 200 that has downloaded the application, for example. For example, the execution screen may include an area for receiving a file containing input data. In FIG. 20, a file as input data is received by a drag-and-drop operation or an address reference. Note that the input data in swallowing can be variously modified. For example, it may include the intake amount or intake ratio for each type in each meal (which may be a main dish or a side dish, or may be for each material such as meat or fish), the intake amount of water, the timing of intake, information related to diseases, information such as weight (or BMI), etc. Further, the application obtains output data by inputting the received file into a learned model, and displays the output data on the execution screen.

[0136] For example, as shown in FIG. 20, the execution screen includes a determination result as to whether it is swallowing or not for each input file. If it is swallowing, the timing of the determination and the timing when swallowing care is required may be displayed. In this way, the result of swallowing prediction for each input data can be presented clearly. Further, when any file is selected, the application may display the time-series change of the feature amount required based on the input data. The feature amount here is information calculated based on the input data, and may be the various information described above, or may be information calculated based on a plurality of information. For example, the execution screen may include a graph representing the time-series change of the measured value and the estimated value of the feature amount. Note that in FIG. 20, a graph of the moving average of these values over 7 days is illustrated. In this way, it becomes possible for an assistant to easily grasp the transition of important items in swallowing care.

[0137] Figures 21 to 23 are other examples of the component creation screen, and are screens related to components that determine positions suitable for injection based on images. Figure 21 is an example of the basic settings screen, and the displayed items are the same as those in Figure 14. Here, image determination is used as the type of component. In this case, as NN, for example, a DNN or CNN suitable for processing image data may be selected. Also, as the grouping setting, whether it is an injection point, which is a position suitable for injection, or otherwise is set.

[0138] Figure 22 is an example of the learning data settings screen. Here, when image data is added by a drag-and-drop operation, text prompting the user for annotation is displayed together with the added image data. In the example of Figure 22, a photograph of a part of a patient's body is displayed, along with the text "Please place a red circle at the feature point", and a circular object whose position and size can be changed by user input. The developer moves the circular object to the position determined by the developer to be the injection point for each image, and then selects the save button. In this way, learning data with labels that identify injection points and others is accumulated for the input image data.

[0139] Although an example of putting one red circle at the feature point has been described here, it is not limited to this, and multiple red circles can be put, and the size of each red circle can also be changed. As a result, even when the feature points projected onto the MR glass are not pinpoint (for example, even if the camera is shifted diagonally with respect to the subject), the injection points can be displayed.

[0140] Figure 23 shows an example of a confirmation screen. Since the component name, component type, meaning of the label, number of data, correct answer rate, history of the correct answer rate, etc. are the same as those in Figure 18, detailed explanations are omitted. Also, the confirmation screen in Figure 23 may include items for setting the interface when it is made into an application. For example, in Figure 23, it is possible to input comments to be displayed on the application execution screen, comments to be output as audio when the application is started, and the like.

[0141] <Another example of user input regarding the data format of the output data> In the method described above, in Figure 10, first the data format of the output data is specified, and an example of accepting user input for assigning correct answer data on the annotation screens shown in Figures 11 and 12 according to the data format was explained. However, the method of this embodiment is not limited to this. For example, since the components of this embodiment digitize the tacit knowledge of experts, it is also conceivable that the data format of the output data is not known in advance. For example, even if an expert understands that they are making a judgment based on some information in a specific situation, they may not fully grasp whether the judgment is performing classification, regression analysis (for example, predicting some numerical value), etc.

[0142] Therefore, in this embodiment, user input for specifying the data format of the output data and annotation may be performed in parallel. For example, the first processing unit 111 may perform processing of displaying sensing data in the display area RE5 of Figure 11 or the display area RE8 of Figure 12 and displaying a text box or the like for inputting the data type. As described above, an expert can make some judgment, such as the presence or absence of a risk of falling, by viewing the actual sensing data. That is, by presenting the actual sensing data to the expert, it becomes possible to appropriately determine what the pattern of the judgment result is (what the data format of the output data is). In this way, it becomes possible to realize a suitable interface when componentizing tacit knowledge.

[0143] <Creation Support> Also, in component creation, the first processing unit 111 may perform processing to support the creation method and the like using a chatbot. For example, in each scenario such as adding lines and boxes, selecting a sensor device, inputting the data format of output data, and annotation, the first processing unit 111 answers the developer's questions using an interactive interface. For example, the first processing unit 111 performs a pop-up display such as a speech bubble. As a result, even a user lacking knowledge of programming and the like can easily create components.

[0144] 2.3 Second Processing (Application Creation) When a user input for selecting an application creation button is made on the developer screen shown in FIG. 6, the processing unit 110 transitions to the state S4 in FIG. 4.

[0145] FIG. 24 is an example of a screen displayed in the state S4. The screen shown in FIG. 24 includes a new creation button and an application search button. When a user input for selecting the new creation button is made, the second processing for creating an application is started. The application search button is used when searching for existing applications from the developer's perspective. An example of the case when a user input for selecting the application search button is made will be described later with reference to FIG. 31. Also, the point that the time, login ID, mode name, etc. may be displayed is the same as in FIG. 6.

[0146] FIG. 25 is a flowchart for explaining the second processing in the second processing unit 112. First, in step S201, the second processing unit 112 identifies the components used in the application. In step S202, the second processing unit 112 identifies the acquisition method of the input data of the component. In step S203, the second processing unit 112 identifies the connection destination of the output edge of the component. The processing in steps S201 to S203 is performed based on the user input executed corresponding to the display on the display unit 240, as will be described later with reference to FIGS. 26 to 29.

[0147] In step S204, the second processing unit 112 determines whether the creation of the application is completed. For example, when a user input for selecting a completion button (not shown) is performed and it is determined that there is no error in the connection relationship, the second processing unit 112 determines that the creation of the application is completed.

[0148] When the creation of the application is completed (step S204: Yes), in step S205, after the second processing unit 112 stores the application information 123 in the storage unit 120, the process ends. When the creation of the application is not completed (step S204: No), the second processing unit 112 returns to step S201 and continues the process. Hereinafter, the details of the process will be described using a specific example.

[0149] 2.3.1 Details of the Process Hereinafter, the second process will be described in detail using an application that supports intravenous injection as an example. FIG. 26 is an example of an application creation screen displayed on the display unit 240 when a user input for selecting the new creation button in FIG. 24 is performed. The application creation screen includes display areas RE9 to RE12.

[0150] In the display area RE9, information for arranging and connecting a plurality of components constituting the application is displayed. For example, as shown in FIG. 26, in the initial state, the second processing unit 112 performs a process of displaying a text box for describing the name of the application in the display area RE9.

[0151] The display areas RE10 to RE12 are the same as the display areas RE2 to RE4 in FIG. 7, respectively. For example, in the display area RE10, parts such as boxes, text boxes, and lines are displayed. When a user input for selecting these parts is performed, the second processing unit 112 performs a process of adding the selected parts to the display area RE9. In the display area RE11, a sensor device is displayed. The display area RE12 is a search area for searching for components.

[0152] As shown in step S201 of FIG. 25, the second processing unit 112 receives a user input for adding a component. For example, a developer uses the display area RE12 to search for an existing component and makes a user input to select one of them.

[0153] A skilled person with tacit knowledge about intravenous injection recognizes the process for appropriately performing intravenous injection. For example, the skilled person accurately and quickly makes a plurality of judgments such as finding a vein from the patient's arm, determining whether the shape of the found vein is suitable for injection, and confirming that the injection needle is at an appropriate angle to the arm, thereby realizing a smooth intravenous injection. As described above, in the present embodiment, since the judgment of the skilled person is digitized as a component, a series of actions of the skilled person in assistance can be digitized by appropriately combining the components.

[0154] FIG. 27 is an example of a screen for explaining the processing of steps S201 and S202. For example, a developer first searches for components that can be used when finding a vein. In the example of FIG. 26, by using the text "vein" as a search word, a plurality of components including three components of "an algorithm for determining whether it is a vein or an artery", "an algorithm for extracting a blood vessel region from a photograph", and "an algorithm for determining whether it is abnormal by vein color determination" are searched. The developer who has seen this search result can judge that the vein can be detected from the image by first extracting the blood vessel and then determining whether it is a vein or an artery.

[0155] Therefore, the developer makes a user input to select "an algorithm for extracting a blood vessel region from a photograph". The user input here is, for example, an operation of dragging a box corresponding to "an algorithm for extracting a blood vessel region from a photograph" displayed in the display area RE12 to the display area RE9, but other operations may be performed.

[0156] As a result, an object corresponding to the selected component is displayed in the display area RE9. Here, the second processing unit 112 may determine the number of output edges to be displayed in association with the object based on the output data format associated with the component. For example, the output data format of "an algorithm for extracting a blood vessel region from a photograph" may be binary data indicating whether or not a blood vessel region is included in the image. In this case, since the second processing unit 112 can determine that the selected component has two branches, it displays a diamond-shaped object and automatically displays two output edges. In the example of FIG. 27, two output edges associated with "Yes" representing the case where the image includes a blood vessel region and "No" representing the case where the image does not include a blood vessel region are displayed.

[0157] As described above, an expert combines a plurality of tacit knowledges for assistance, and the way of combining tacit knowledges (for example, the types of tacit knowledges and the application order, etc.) is also considered as tacit knowledge. When digitizing such tacit knowledge including a plurality of tacit knowledges, it is important to appropriately reproduce the branch of the process according to the judgment. In that regard, by automatically adding output edges, it is not necessary for the developer himself / herself to consider the conditions and the number of branches of the branch process, so that setting errors and branch omissions can be suppressed. As a result, even a user with little knowledge of programming or the like can appropriately create an application. In particular, in the present embodiment, as described above with reference to FIG. 10, the data format of the output data is determined at the stage of creating the component. Therefore, by associating the data format with the number of output edges in advance, the additional processing of the output edges by the second processing unit 112 can be easily realized.

[0158] Note that the output data format of the "algorithm for extracting the blood vessel region from a photograph" is not limited to binary data only and may include accompanying data. For example, when the blood vessel region is included in the image, information for distinguishing the blood vessel region in the image from other regions may be output. The information here is, for example, binary image data in which a first value (e.g., "1") is assigned to the blood vessel region and a second value (e.g., "0") is assigned to other regions.

[0159] Also, as shown in the display area RE12, the second processing unit 112 may perform processing to display the search area of the component. Then, when the second processing unit 112 receives a user input for selecting any one of the search results displayed in the search area, the second processing unit 112 displays the object and the output edge corresponding to the selected component in the display area RE9. In this way, by linking the search for the component and the addition to the display area RE9, it becomes possible to easily use the components accumulated by the first processing described above. For example, it becomes possible to utilize the tacit knowledge of experts in various applications.

[0160] Also, as shown in step S202 of FIG. 25, when a component is added, the second processing unit 112 may perform processing to specify the acquisition method of the input data of the component. For example, in the "algorithm for extracting the blood vessel region from a photograph", as the sensor device used for acquiring the input data, a sensor device that outputs image data, such as a camera or an imaging sensor, is selected. Therefore, the second processing unit 112 performs processing to prompt a user input for specifying how to acquire the image data.

[0161] For example, in the display area RE9, as shown in FIG. 27, along with the text "What would you like to input?", two candidates, "Upload Photo" and "Web Camera", may be displayed. In this way, as input data for the component, it is possible to determine whether to use an image uploaded by the user or an image captured by the web camera. For example, when "Upload Photo" is selected, the user of the application executes the upload of some image, and the application performs a detection process for the blood vessel area on the image. When "Web Camera" is selected, a detection process for the blood vessel area is performed on the image of the web camera held by the user of the application. Note that the input data is not limited to uploaded photos and web cameras, and may be, for example, a camera mounted on the terminal (the camera of a smartphone or tablet). Hereinafter, an example where the web camera is selected will be described.

[0162] Also, as shown in step S203 of FIG. 25, the second processing unit 112 receives a user input for specifying the connection destination of the output edge of the component. FIG. 28 is an example of an application creation screen corresponding to the process of step S203. The component added in FIG. 27 has two output edges as described above. Therefore, the developer inputs the respective connection destinations.

[0163] For example, when the blood vessel area is not detected from the image, it is difficult to perform intravenous injection as it is, so it is necessary to execute the detection process for the blood vessel area again. Therefore, the developer makes a user input to connect the output edge corresponding to No to the "algorithm for extracting the blood vessel area from the photo" itself. For example, the developer selects a line from the display area RE10, connects one end of the line to the output edge corresponding to No, and connects the other end to the "algorithm for extracting the blood vessel area from the photo". In this way, loop processing when the blood vessel area is not found can be easily realized.

[0164] Also, as shown in FIG. 28, a component for displaying a comment on an image may be added. This component does not use the tacit knowledge of an expert, but is useful in creating a highly convenient application. Therefore, in this embodiment, a component that does not use tacit knowledge (hereinafter also referred to as a general-purpose component) may be used. For example, the component information 122 stored in the storage unit 120 of the server system 100 may include a library of general-purpose components in addition to information representing components registered by a developer.

[0165] In the example of FIG. 28, the output edge corresponding to No is connected to a general-purpose component representing comment display. There is one output edge of the general-purpose component, and the output edge is connected to the "algorithm for extracting the blood vessel region from the photo". In this way, when the blood vessel region is not found, it is possible to perform loop processing while displaying a comment representing a warning or advice.

[0166] Also, the developer makes a user input to specify the connection destination of the output edge corresponding to Yes. Yes in the "algorithm for extracting the blood vessel region from the photo" is a situation where the blood vessel region has been found. Therefore, the developer applies the "algorithm for determining whether it is a vein or an artery" to the blood vessel region that is the output of the "algorithm for extracting the blood vessel region from the photo".

[0167] Specifically, as shown in FIG. 28, the developer makes a user input to select the "algorithm for determining whether it is a vein or an artery" from the display area RE12. The output data format of the "algorithm for determining whether it is a vein or an artery" may be binary data representing whether the blood vessel to be processed is a vein or not. In this case, since the second processing unit 112 can determine that the selected component has two branches, it displays a diamond-shaped object and automatically displays two output edges.

[0168] The developer performs a user operation of connecting the output edge corresponding to "Yes" of the "algorithm for extracting the blood vessel region from a photograph" to the "algorithm for determining whether it is a vein or an artery". Through the above processing, for each of the two output edges automatically added to the "algorithm for extracting the blood vessel region from a photograph", the connection destination is determined. That is, it becomes possible to appropriately specify the branching process according to the output of the component.

[0169] In addition, when connecting the output edge of the first component to the second component, the second processing unit 112 may perform a process of comparing the data format of the output data of the first component with the data format of the input data of the second component. For example, the second processing unit 112 may determine whether the two data formats match, and if they do not match, perform a warning process of presenting that the first component and the second component cannot be connected. In the above-described example, in the situation corresponding to "Yes" of the "algorithm for extracting the blood vessel region from a photograph", image data for specifying the blood vessel region is output. Also, the input data of the "algorithm for determining whether it is a vein or an artery" is image data including blood vessels. Therefore, the second processing unit 112 determines that the "algorithm for extracting the blood vessel region from a photograph" and the "algorithm for determining whether it is a vein or an artery" can be connected.

[0170] In FIG. 28, the "algorithm for determining whether it is a vein or an artery", which is a new component, is added (step S201 in FIG. 25). The processing for the added component "algorithm for determining whether it is a vein or an artery" is the same, and the processing corresponding to steps S202 and S203 in FIG. 25 is executed. For example, in the display area RE9, in addition to the diamond-shaped object corresponding to the "algorithm for determining whether it is a vein or an artery" as shown in FIG. 28, two output edges corresponding to "Yes" indicating that it is a vein and "No" indicating that it is not a vein are added. Therefore, the developer performs user input for specifying the input data of the "algorithm for determining whether it is a vein or an artery" and the connection destination of the output edge.

[0171] Since the input data of the "algorithm for determining whether it is a vein or an artery" is image data, the input data is complete by obtaining the image data output from the "algorithm for extracting the blood vessel region from a photograph". Therefore, the developer does not need to separately perform user input to specify a method for obtaining input data. However, if the added component requires input data other than, for example, image data, the second processing unit 112 may perform a process of prompting the developer to perform user input to specify a method for obtaining the input data.

[0172] Also, the "No" of the "algorithm for determining whether it is a vein or an artery" corresponds to a situation where the blood vessel is an artery. Therefore, in order to perform intravenous injection, it is desirable to start over from the blood vessel region detection process again. Thus, the developer performs user input to connect the output edge corresponding to "No" to the "algorithm for extracting the blood vessel region from a photograph".

[0173] Also, the "Yes" of the "algorithm for determining whether it is a vein or an artery" corresponds to a situation where a vein is detected. Therefore, the developer performs user input to add a component that determines whether the shape of the detected vein is suitable for injection, similar to the flow of judgment in the tacit knowledge of an expert (step S201 in FIG. 25). Similarly, the second processing unit 112 performs addition of an output edge, specification of a method for obtaining input data, and specification of a connection destination of the output edge for the added component as well (steps S202 and S203).

[0174] The same applies hereinafter. The second processing unit 112 repeats addition of components, display of output edges, specification of a method for obtaining input data, and specification of an output destination of the output edge until it is determined that the application is complete.

[0175] FIG. 29 is an example of an application creation screen of an application that supports intravenous injection, and shows a state where the addition of components is completed. As shown in FIG. 29, further components are added from the state of FIG. 28, such as a component for extracting a partial region from an image, a component for determining whether the extracted region meets a given condition, and a component for superimposing and displaying an injection needle. Here, the superimposed display means, for example, displaying an image representing a virtual injection needle that does not exist in the real space over an image of the real space captured. The virtual injection needle may be displayed as a transparent object. Also, in the example shown in FIG. 29, as conditions for region extraction, a first condition for extracting a straight region and a second condition for excluding a Y-shaped shape are added. Each of these conditions may be a single component. In this case, it is possible to flexibly combine the determination conditions. Alternatively, a component for determining "whether it is a straight and non-Y-shaped region" may be used.

[0176] In this way, it becomes possible to create an application that supports intravenous injection by detecting a straight and non-Y-shaped vein from an image and superimposing and displaying an injection needle at an appropriate angle. Also, as shown in FIG. 29, even when a desired vein is not found, it is possible to realize an application that can present an appropriate comment and perform the detection process of the blood vessel region again.

[0177] The second processing unit 112 executes a process of creating an application that supports intravenous injection based on the connection relationship of the components shown in FIG. 29. For example, an application is created by automatically generating source code including conditional branches (if, switch), loops (for), etc. based on the connection relationship. Note that various methods for automatically generating source code by combining parts represented using graphics are known, and these methods can be widely applied in the present embodiment. Also, as described above with reference to FIG. 19, when outputting an application, user input for specifying the type and version of the OS may be received.

[0178] The second processing unit 112 stores application information 123 including the created application in the storage unit 120. As described above, the application information 123 is not limited to the information of the application, and may include information such as the creator of the application, the registration date, and the description of the application.

[0179] 2.3.2 Variation <Sample Screen> As described above, according to the method of the present embodiment, an application can be created using a visual interface using graphics. However, the second processing unit 112 may assist in creating the application by presenting more specific information to the developer.

[0180] For example, for an application that supports intravenous injection, as shown in FIG. 28 etc., when the blood vessel area is not detected from the photograph, a comment such as a warning is displayed. The content of the comment, the position on the screen, the font size and color, etc. can be set according to the screen of FIG. 28 or the accompanying screen. However, when actually running the application, there may also be a requirement to check how the comment looks to the user at the time of creating the application.

[0181] Therefore, the second processing unit 112 may perform a process of displaying a sample screen that is a sample of the screen presented to the user when the application is executed. The screen may be displayed on the terminal device 200 on which the application is being created. Alternatively, the second processing unit 112 creates a temporary application that executes partial processing, and the developer may confirm the sample screen by executing the temporary application on another terminal device 200 such as an MR glass. For example, the second processing unit 112 may perform a process of displaying a sample screen confirmation button (not shown) on the application creation screen. When a user input for selecting the sample screen confirmation button is performed, the second processing unit 112 performs a process of displaying the sample screen on the terminal device 200.

[0182] Figs. 30A and 30B are examples of sample screens. Fig. 30A is an example of a sample screen when the blood vessel region is not detected in the "algorithm for extracting the blood vessel region from a photograph". In the example of Fig. 30A, a comment "Veins cannot be detected. Please come closer" is displayed in the upper right of the screen. By referring to the sample screen of Fig. 30A, the developer can determine whether the position and content of the comment are appropriate.

[0183] Fig. 30B is a sample screen showing the result of region extraction corresponding to a vein and superimposed display of an injection needle. As shown in Fig. 30B, for example, the region corresponding to the vein is highlighted, and an injection needle set at a predetermined angle with respect to the arm is superimposed and displayed on the image. By referring to the sample screen of Fig. 30B, the developer can determine whether the vein is appropriately detected, whether the angle of the injection needle is appropriate, etc. If the desired display is not performed, for example, the developer may change the components used for vein extraction, adjust the display angle of the injection needle, etc.

[0184] Note that, as described later, the application created by the method of this embodiment may be used in a virtual space. For example, the sample screens shown in FIGS. 30A and 30B are not limited to the screens showing the processing results of the application for the photo of the care recipient in the real space, and may be the screens showing the processing results of the application for the virtual image of the virtual care recipient on the virtual space. Through the experience in the virtual space, the caregiver can obtain the same tactile, visual, and other sensations or tacit knowledge as in the real world. Also, the second processing unit 112 may perform a process of adding a sample screen to the application information 123. For example, the second processing unit 112 may perform a process of displaying a sample screen addition button (not shown) on the application creation screen. When a user input for selecting the sample screen addition button is made, the second processing unit 112 performs a process of adding the sample screen to the application information 123. In this way, for example, on the screen for displaying the detailed information of the application, it becomes possible to display the sample screen. For example, in the search result screen described later with reference to FIG. 31 and the user screen described later with reference to FIG. 33, information including the sample screen is presented. In this way, it becomes possible to easily present an operation example of the application to the user (developer or user in the narrow sense) who is the viewer.

[0185] <Application Search by Developer> Also, the developer may search for the applications registered in the server system 100. For example, when a user input for selecting the application search button in FIG. 24 is made, the second processing unit 112 performs a process of displaying an application search screen.

[0186] Figure 31 shows an example of an application search screen in developer mode. The application search screen includes a display area RE13 and a display area RE14. The display area RE13 includes a text box for entering a search word and displays search results based on the search word. In the example of Figure 31, based on the input of the search word "injection", the second processing unit 112 extracts and displays an application that matches "injection" from the application information 123. For example, the second processing unit 112 extracts an application that matches the search word based on a comparison process between the text representing the title and description of the applications included in the application information 123 and the search word. For example, the second processing unit 112 may determine the popularity of the applications based on the number of downloads, the number of views, user evaluations, etc., and sort the applications that match the search word in descending order of popularity.

[0187] The display area RE14 is a screen that displays detailed information of one selected application among the applications displayed in the display area RE13. The detailed information includes the usage fee, registration date, creator, application description, etc. when using the application. The detailed information may also include an example of a screen image when using the application. For example, if it is an application that supports intravenous injection, the sample screens shown in FIGS. 30A and 30B may be displayed.

[0188] When the developer is an assistant, there may be a case where not only their own tacit knowledge is componentized and applicationized, but also the tacit knowledge of other assistants is desired to be used in their own assistance. For example, when a user input is made to select the use button displayed in the display area RE14 of Figure 31, the target application may be downloaded to the developer's terminal device 200. Also, by knowing popular applications, user needs can be determined, which is useful for deciding the direction of creating new applications.

[0189] <Creation Support> Also, in application creation, the second processing unit 112 may perform processing to support the chatbot in creating methods and the like. For example, in each scenario such as component search, addition, and connection of output edges, the second processing unit 112 answers the developer's questions using an interactive interface. Also, when there is a part considered to be an error, such as an output edge whose connection destination is not specified, the second processing unit 112 may perform processing such as highlighting the error location and popping up a method for eliminating the error. As a result, even a user lacking knowledge of programming and the like can easily create an application.

[0190] 2.4 Application Usage Although the developer mode has been described above, the server system 100 of the present embodiment may operate in the user mode as shown in states S5 to S8 of FIG. 4. The user mode is a mode for using registered applications. Narrow-sense users who use the user mode are, for example, caregivers with low proficiency or family members who provide home care for care recipients. By using the application, it becomes possible for these users to perform appropriate care utilizing the tacit knowledge of experts.

[0191] 2.4.1 User Screen FIG. 32 is an example of a user screen displayed in state S5 of FIG. 4, and particularly corresponds to the home screen displayed when logging in. As shown in FIG. 32, the home screen may include display areas RE15 to RE17.

[0192] The display area RE15 is the main display area, and on the home screen, applications recommended to the user and applications with a high popularity are displayed. For example, the processing unit 110 may determine applications recommended based on the user's download history, browsing history, attributes of the care recipient assisted by the user, etc. The attributes of the care recipient include age, gender, height, weight, medical history, medication history, etc. Also, the processing unit 110 may determine the popularity of an application based on the number of downloads, number of views, evaluation by the user, etc. for each application.

[0193] The display areas RE16 and RE17 are areas for displaying various menus. When any item is selected, the display in the display area RE15 is switched to the display related to the selected item.

[0194] For example, the display area RE16 includes items such as My Apps, Shop, Account, Payment Method, Recurring Purchase, Use Code, Purchase Gift Card, Wish List, etc. When My Apps is selected, for example, the applications currently being used by the user are displayed. Shop is an item for presenting applications registered in the server system 100, and FIG. 32 shows the state where Shop is selected. When Account is selected, the account management screen of the target user is displayed. When Payment Method is selected, the screen for setting the payment method for the consideration when downloading an application that requires a consideration for use is displayed. When Recurring Purchase is selected, the setting screen when using a subscription-type application is displayed. When Use Code is selected, the screen for downloading an application using a separately issued code is displayed. When Purchase Gift Card is selected, the purchase screen for a prepaid card or the like that can be used for downloading an application is displayed. When Wish List is selected, the screen for editing the list of applications that the user wishes to use is displayed.

[0195] The display area RE17 includes items such as category, home, ranking, new works, for each assigned patient, certificate, etc. The display area RE17 may be an area for displaying sub-items corresponding to the items selected in the display area RE16, for example.

[0196] Home is in the selected state in FIG. 32, and as described above, for example, applications recommended to the user and applications with a high popularity are displayed. When a category is selected, applications are displayed separately for each category. For example, the category may be divided into medical and nursing care, or may be divided into finer units such as meal assistance, excretion assistance, transfer and boarding assistance, etc. When ranking is selected, for example, a screen is displayed in which applications are sorted in descending order of the number of downloads. When new works are selected, applications with the number of days elapsed since registration being below a predetermined value are displayed. For assigned patients and certificates, these are items that are displayed, for example, when the user is a medical professional. These two details will be described later with reference to FIG. 36 and the like.

[0197] FIG. 33 is an example of a user screen displayed when a user input for selecting any one application is performed in FIG. 32 and the like. As shown in FIG. 33, when an application is selected, detailed information of the selected application is displayed in the display area RE15. As shown in FIG. 33, the detailed information includes the name of the application, registration date, creator, usage fee, user evaluation, number of downloads, description of the application, example of the display screen, etc. Also, as shown in FIG. 33, on the display screen of the detailed information, an installation button for using the application, an addition button to the wish list considering future use, etc. may be displayed.

[0198] As shown in FIG. 33, applications similar to the selected application may be displayed in the display area RE15. The degree of similarity between applications may be determined based on, for example, comparison of texts included in the names and descriptions of applications, comparison of creators, comparison of users who downloaded the applications, etc. Also, the degree of similarity between two applications may be determined based on whether one application is likely to be used together with the other application. For example, for a patient with a specific disease, a case where it is useful to combine application A and application B, but application C is rarely used can be considered. For example, the processing unit 110 can determine whether two applications are likely to be used together by identifying combinations of applications used by each user based on the user information 121. In the above example, the degree of similarity between application A and application B is determined to be relatively high, and the degrees of similarity between application A and application C and between application B and application C are determined to be relatively low. In this way, applications that are useful to use together can be appropriately presented. Note that the above application A and application B may be used in different scenarios. For example, application A may be an application related to injection, and application B may be an application related to meal assistance. In this way, an application determined to have a low degree of similarity based on text analysis or the like can be presented as an application similar to the selected application.

[0199] By using the user screen as described above, appropriate applications can be presented for each user, so that the promotion of application usage becomes possible.

[0200] 2.4.2 Application Usage Scenarios FIG. 34 and FIG. 35 are schematic diagrams illustrating scenarios where applications installed in the terminal device 200 are used. Here, an example using the terminal device 200-3 which is the AR glasses or MR glasses shown in FIG. 1 will be described. For example, a user who is an assistant may view the user screens shown in FIGS. 32 and 33 using the MR glasses, and install an application on the MR glasses using the user screen. Alternatively, the user may view the user screens shown in FIGS. 32 and 33 on a terminal device 200 such as a smartphone or a PC, and install the application selected on the user screen on the MR glasses connected to the smartphone or the like. Also, the user who selects the application and the user who actually uses the application may be different. For example, in a hospital or the like, an administrator (e.g., a management staff or a nurse) may select an application using a management system, and the selected application may be downloaded to the MR glasses used by on-site staff (e.g., a nursing assistant).

[0201] For example, the application here may be an application that supports intervention by an assistant to suppress the fall of the person being assisted. FIG. 34 is an example of the positional relationship between an assistant who uses the application, a plurality of persons being assisted, and the screen displayed on the terminal device 200 which is the MR glasses.

[0202] As shown in FIG. 34, the assistant wears the glasses-type device which is the MR glasses as the terminal device 200. The glasses-type device has, for example, a camera that images a region corresponding to the user's field of view. In the glasses-type device, part or all of the lens portion is a display, and by transmitting light from the outside world or by displaying an image corresponding to the user's field of view imaged by the camera, the user can visually recognize the situation of the outside world. Further, by using the display, the glasses-type device additionally displays some information on the user's field of view.

[0203] For example, the application here performs a process of determining the necessity of intervention for each care recipient detected from an image, and superimposing and displaying a predetermined mark above the head of the care recipient for whom intervention is determined to be necessary. In the example of FIG. 34, among the three care recipients, the care recipients at the left and right ends are likely to shift horizontally or fall, so it is determined that intervention is necessary, and a mark including "!" is displayed above their heads. In this way, if the caregiver faces the care recipient, the necessity of intervention can be easily determined. Therefore, it becomes possible to take appropriate measures even when the number of care recipients present is large, or when the caregiver is performing other care in parallel and detailed observation is not easy.

[0204] Also, the application according to this embodiment may be an application that supports the position adjustment of the care recipient. FIG. 35 is an example of the positional relationship between the caregiver who uses the application and the care recipient who is the care target, and the screen displayed on the terminal device 200 which is the MR glass.

[0205] In a scene where a wheelchair is used, the position adjustment of the care recipient is useful for preventing slipping and bedsores. For example, the application here may determine the sitting posture of the care recipient detected from the image, and when the posture is collapsed, superimpose and display information for making the care recipient assume a more desirable posture. For example, the MR glass may superimpose and display a person in a desirable sitting posture on the actual care recipient. Alternatively, a part of the care recipient with a large difference from the desirable posture may be highlighted. Here, the sitting posture of the care recipient is determined based on the image, but it is not limited thereto. For example, the MR glass may communicate with the seat surface sensor, and determine the sitting posture of the care recipient based on the image of the MR glass and the pressure values of the seat surface sensors (each pressure sensor Se1 to Se4) in FIG. 9. Alternatively, the MR glass may determine the sitting posture of the care recipient using the determination result based on the pressure values of the seat surface sensors.

[0206] Also, as shown in FIG. 35, the MR glass may display a recommendation to use an instrument useful for assuming a desired posture. For example, in the example of FIG. 35, a virtual cushion is superimposed and displayed. By doing so, the presence or absence and type of the instrument to be used, and the arrangement and usage method of the instrument can be recommended. Therefore, it becomes possible to easily execute the position adjustment by the caregiver. Here, although the position adjustment in a wheelchair is illustrated, an application that supports the position adjustment in a bed may be used. The position adjustment in a bed is also useful for preventing pressure ulcers, and it becomes possible to prompt the caregiver to make an appropriate position adjustment by presenting a desired posture and recommending a cushion, etc.

[0207] 2.4.3 Billing Process As shown in FIG. 2, the server system 100 of the present embodiment may include an accounting processing unit 114. For example, when the first user creates a first component and the second user creates a first application including the first component, the accounting processing unit 114 may perform accounting processing to give a reward to the first user. In this way, by giving a consideration for the creation of the component, the creation of the component is promoted. That is, the digitalization of the tacit knowledge of experts can be promoted.

[0208] Also, when the first user creates a first component and the second user creates a first application including the first component, the accounting processing unit 114 may perform accounting processing to charge the second user. In this way, by collecting a consideration for the use of the component, it becomes easier for the first user to pay the consideration. For example, since it becomes possible to pay a large consideration to a user who creates a component with a large number of usage times, the digitalization of tacit knowledge is further promoted.

[0209] Also, when the first application is used by the third user, the accounting processing unit 114 may perform accounting processing to give a reward to the second user. In this way, by giving a consideration for the creation of the application, the creation of the application is promoted. That is, it becomes possible to promote the digitization of tacit knowledge that combines a plurality of tacit knowledges.

[0210] Also, when the first application is used by the third user, the accounting processing unit 114 may perform accounting processing to charge the third user. In this way, by collecting a consideration for the use of the application, it becomes easier to pay the consideration to the second user. For example, since it becomes possible to pay a large consideration to a user who created an application with a large number of downloads, the digitization of tacit knowledge is further promoted.

[0211] Also, the accounting processing related to the components described above and the accounting processing related to the application may be linked. For example, the accounting processing unit 114 may determine the consideration to be paid to the first user who is the creator of the first component included in the first application according to the sales of the first application. For example, the reward to the first user may be an amount obtained by multiplying a predetermined ratio by the sales of the first application. Other specific accounting processes can be variously modified and implemented.

[0212] Also, the right to receive the reward given to the developer by providing the component or application may be transferable. For example, the components and applications in this embodiment may be associated with non-fungible tokens (NTFs) and sold in the market as digital assets. The accounting processing unit 114 determines the legitimate owner of the component or application based on the NTF and performs processing to pay the reward to the owner. In this way, it becomes easier to capitalize the created components and the like, so the motivation for creating components and the like increases, and the digitization of tacit knowledge can be promoted.

[0213] 2.5 SNS Linkage Moreover, the information processing apparatus of this embodiment may be linked with various SNSs (Social Networking Services). For example, on the user screen illustrated in FIG. 32, applications created by video distributors frequently viewed by the target user or users followed by the target user on SNS may be preferentially displayed.

[0214] In addition, the application created by the method of this embodiment may be used in the VR (Virtual Reality) field. For example, a user who has downloaded the application to a VR device such as a VR goggle may perform assistance by a virtual assistant within the virtual space. In this way, it becomes possible to learn assistance skills without increasing the invasiveness of the person being assisted. For example, learning using VR is useful for improving the skills of novice practical nurses, family members of the person being assisted, etc. Here, the virtual space may be, for example, a metaverse. A metaverse is a virtual space constructed within a computer network. Hereinafter, an example where the virtual space is a metaverse will be described, but the method of this embodiment can be applied to any VR technology other than the metaverse.

[0215] Further, the processing unit 110 may grant points corresponding to the usage fee to the user who uses the application. As described above, when both real use and metaverse use of the application are possible, the processing unit 110 may set the points obtained by real use higher than the points obtained by metaverse use. For example, when assistance using the application is executed, by feeding back the data at that time, it becomes possible to improve the accuracy of components and applications. However, it is considered that the data obtained in reality is more useful. By relatively increasing the points when using the application in reality, it becomes possible to easily collect useful data. Note that the granted points may be used as consideration for using components or applications, or may be used to purchase assistive devices.

[0216] Also, the points here may be used for respite care, which is care for caregivers. For example, the caregiver may be able to arrange for a substitute caregiver by using the points. It is assumed that the assistance provided by the substitute caregiver will be performed in reality. However, for example, if a device used in reality (such as a suction device) can be operated in conjunction with an operation in the metaverse, the assistance provided by the substitute caregiver may be executed in the metaverse.

[0217] In the selection of the substitute caregiver, a caregiver who is compatible with the application used by the user or the highly rated application may be preferentially selected. Here, whether or not they are compatible may be determined using the number of times or the period during which the target caregiver has used the same application. Alternatively, the degree of compatibility may be determined based on whether or not the caregiver works at the same facility as the user who registered the application. Also, the substitute caregiver may be selected based on the usage experience of the device used by the target care recipient, or may be selected based on geographical conditions (transportation costs and travel time) when providing assistance in reality.

[0218] When assistance is provided by an alternative caregiver, the assistance method used by the original caregiver may be presented to the alternative caregiver. For example, the alternative caregiver may be temporarily provided with the application used by the original caregiver. Also, a video of the place where the original caregiver is providing assistance may be recorded and provided to the alternative caregiver. This video may be provided using an SNS that enables video distribution or the like. Furthermore, information necessary for communication, such as the history of the care recipient and favorite topics, may be presented to the alternative caregiver.

[0219] 3.HACs In-hospital care, hospital-acquired conditions (HACs) are known. HACs refer to the occurrence of a disease other than the original treatment purpose after hospitalization. HACs can be originally prevented and indicate that there are deficiencies in patient management. For example, in the United States, the medical expenses for HACs are generally borne by the hospital, and suppressing HACs is very important.

[0220] Examples of HACs include retained foreign bodies after surgery, air embolism, blood incompatibility, pressure ulcers, falls, trauma, fractures, dislocations, injuries within the skull, devastating injuries, burns, other injuries, inadequate blood glucose management, urinary tract infections caused by catheters, catheter-related infections, surgical site infections / mediastinitis after coronary artery bypass surgery, surgical site infections after bariatric surgery, laparoscopic gastric bypass surgery, gastric fistula augmentation surgery, limited laparoscopic gastric surgery, surgical site infections after plastic surgery, surgical site infections of implantable cardiac electronic devices, deep vein thrombosis / pulmonary embolism after plastic surgery, total knee arthroplasty, hip arthroplasty, iatrogenic pneumothorax, and other events. Note that the above is an example of HACs, and various modifications can be made for specific events.

[0221] In recent years, hospital care at home, which provides care that was previously performed in a hospital at home, has been under consideration. For example, the need for hospital care at home increases when hospitalization or frequent hospital visits become difficult due to measures against infectious diseases, etc. As a result, HACs that originally occurred in a hospital may occur at home. However, even if the above-mentioned events occur at home, it may be due to an error in the hospital, or it may be due to inappropriate behavior by the patient despite the hospital providing sufficient explanations and guidance to the patient. Therefore, when the scope of HACs is extended to hospital care at home, it is assumed that whether HACs are borne by the hospital or the patient (whether the hospital is exempted) will differ depending on whether the hospital has taken sufficient measures. In addition, if the patient has previously contracted with an insurance company, the degree of exemption of the hospital, that is, the degree of burden of the insurance company, may be determined through negotiation between the hospital and the insurance company when HACs occur.

[0222] In such cases, it is important for the hospital to prove that it has taken sufficient measures to suppress HACs. For example, a case where the hospital is exempted can be considered by proving that the hospital has provided an application effective for suppressing HACs to the patient.

[0223] FIG. 36 is a diagram for explaining the system according to the present embodiment. The system shown in FIG. 36 may include a server system 100, terminal devices 200-4, 200-5, and 200-6, and a HACs evaluation system 300. The server system 100 is the same as the example shown in FIGS. 1 and 2, and has the developer mode and the user mode as described above.

[0224] The terminal device 200-4 is a terminal device used by a developer. The developer who uses the terminal device 200-4 creates and registers an application by using the developer mode.

[0225] The terminal device 200-5 is a terminal device used by a primary user. Here, the primary user is, for example, a hospital, and the terminal device 200-5 corresponds to the accounting system of the hospital. For example, the accounting system downloads applications by using the user mode.

[0226] The terminal device 200-6 is a terminal device used by a secondary user. Here, the secondary user is, for example, a patient who receives medical treatment from the above-mentioned hospital. The patient or the insurance company pays the hospital the consideration for the medical treatment provided by the hospital. Also, the patient here may be a patient targeted for home medical care.

[0227] The HACs evaluation system 300 is a system that estimates the occurrence probability of HACs for each patient based on, for example, the electronic medical record and vital information of the patient. The HACs evaluation system 300 may obtain the occurrence probability for each of a plurality of events such as the above-mentioned postoperative foreign body residue and air embolism. The electronic medical record and vital information are provided, for example, from the hospital system. The hospital system may be the same as or different from the accounting system corresponding to the terminal device 200-5. Note that a computer system and method suitable for the determination of HACs are described in U.S. Patent Application No. 13 / 613980 filed on September 13, 2012, entitled "Clinical predictive and monitoring system and method". This patent application is hereby incorporated by reference in its entirety into this specification.

[0228] The information processing apparatus (server system 100) of the present embodiment may include a management processing unit 113 as shown in FIG. 2. When the fourth user who is a user using the application is a medical-related user, the management processing unit 113 performs a process of recommending an application determined to be related to the suppression of HACs (Hospital-Acquired Condition).

[0229] In the example of FIG. 36, the user who uses the terminal device 200-5 is the fourth user. In this case, as shown in FIG. 36 for example, the server system 100 acquires the probability of occurrence of HACs of a patient from the HACs evaluation system 300. Then, for each patient, it recommends an application related to suppressing HACs whose probability of occurrence is estimated to be high.

[0230] FIG. 37 is an example of a user screen of the fourth user who is a medical staff. The display areas RE15 to RE17 are the same as those in FIG. 32. Here, in the display area RE16, the item of "assigned patient" is selected, and information about the assigned patient is displayed in the display area RE15. Note that the user here may be a hospital administrator or an individual nurse. For example, FIG. 37 is a user screen of a nurse working in a hospital, and information about the patients assigned to the nurse is displayed in the display area RE15.

[0231] For example, the display area RE15 includes information such as patient ID, patient name, HACs with a high probability of occurrence, recommended applications, costs of using the applications, status, and certificates.

[0232] The patient ID is information that uniquely identifies a patient and is, for example, numerical data. The patient name represents the name of the patient. The HACs with a high probability of occurrence are HACs whose probability of occurrence is estimated to be high based on the patient's electronic medical record and vital information. For example, a patient named "Keny" is determined to have a high probability of developing bedsores. For example, the management processing unit 113 may acquire the probability of occurrence of each event for HACs from the HACs evaluation system 300 and display in the display area RE15 the events whose probability is equal to or higher than a predetermined threshold.

[0233] In the column of the recommended application, for example, the icon of the application is displayed. For example, the storage unit 120 may store, as application information 123, information representing the relationship between the application and HACs suppression. The management processing unit 113 determines the displayed application based on the application information 123. In the case of the example of "Keny", for example, an application that supports position adjustment in bed is recommended in order to suppress bedsores in bed. If the recommended application here has not been provided to the patient, information for providing it may be displayed. The information here corresponds to, for example, "apply" displayed to the right of the icon in FIG. 37.

[0234] The cost of using the application represents the usage fee when using the recommended application. The status indicates whether the target application has been provided to the patient or not. However, the information regarding the status is not limited to this. For example, when the application has been provided to the patient, as shown in FIG. 37, the information representing the status may include information representing the time zone and utilization rate when using the application. The time zone represents, for example, the time when the application is used within a day, and may be information representing the presence or absence and frequency of use every hour, or may be information representing the presence or absence of use within a rough classification such as morning, day, and night. Alternatively, the time zone may be information such as specifically from what time to what time it was used. Also, the information representing the time zone is not limited to this, and for example, it may be information representing the presence or absence of use for each day of the week within a week, and various modifications can be made to the specific mode. The utilization rate represents the ratio of the user's utilization of the application. The utilization rate may be the ratio of the time of using the application with respect to a reference period such as one day or one week. Alternatively, when the actions targeted by the application such as injection and medication can be counted by the number of times, the ratio of the actual number of uses to the number of times that should originally be used may be used as the utilization rate. In the column of the certificate, the issuance status of the certificate indicating that the application has been provided to the patient and, when the certificate has been issued, the information for displaying the certificate are displayed.

[0235] FIG. 38 is a diagram for explaining a method of providing an application to a target patient when a user input for selecting "apply" in FIG. 37 is performed. For example, the management processing unit 113 transmits link information for acquiring an application to the terminal device 200-5 which is the hospital's accounting system. The hospital's accounting system (terminal device 200-5) performs a process of embedding the above-described link information in, for example, a bill issued to a patient. FIG. 38 is an example of a bill issued to a patient. For example, the bill includes a two-dimensional barcode representing a link for downloading an application. When the patient reads the two-dimensional barcode using a camera or the like of the terminal device 200-6, the terminal device 200-6 starts downloading the application. Note that the method of providing an application to a patient is not limited to the method using the bill in FIG. 38, and e-mail, SNS, or the like may be used.

[0236] Note that the accounting processing unit 114 performs a process of requesting a consideration for using the application from the user of the terminal device 200-5, and does not necessarily need to request a consideration from the user of the terminal device 200-6 that downloads the application based on the link information. That is, due to the burden on the hospital, an application may be provided to a patient. This is because the burden on the hospital when HACs occur is extremely large, and if HACs can be suppressed by providing an application, the application usage fee is considered a reasonable burden for the hospital.

[0237] Through the above processing, it becomes possible for a hospital to provide an application to a patient who is the target of home medical care. As described above, since the application of the present embodiment digitizes the tacit knowledge of experts, it is possible to cause a patient who does not have medical expertise to perform appropriate actions. That is, as a hospital, sufficient measures are taken to suppress HACs, and it becomes possible to claim exemption from liability when HACs occur.

[0238] The server system 100 of this embodiment may prove that a hospital has provided an application to a patient. By doing so, it becomes possible to prove that the hospital has taken sufficient measures from the perspective of a third party different from the hospital, the patient, and the insurance company.

[0239] For example, when the management processing unit 113 provides a certificate proving that an application has been provided to the fifth user to the fourth user, who is a medical staff member, when the fourth user provides an application related to HACs to the fifth user, who is a patient, the management processing unit 113 may perform a process of issuing the certificate to the fourth user.

[0240] For example, when the management processing unit 113 provides link information to the fourth user based on a user input that selects the above-mentioned "apply", the management processing unit 113 may perform a process of issuing a certificate to the fourth user. For example, in the certificate column included in the display area RE15 of FIG. 36, information indicating that the certificate has been issued, buttons for displaying the certificate, etc. are displayed. In addition to providing the link information, the management processing unit 113 may issue a certificate to the fourth user on the condition that access using the link information has been performed and the application has been downloaded. Furthermore, the management processing unit 113 may authenticate that the subject of the download is a patient. For example, a patient creates an account as a user of the information processing system 10 according to this embodiment. Then, the management processing unit 113 determines whether the download has been performed by the patient based on the account of the user who has performed the download, and may issue a certificate when it is confirmed that the download has been performed by the patient. Alternatively, the management processing unit 113 may perform a process of prohibiting downloads by anyone other than a specific patient by embedding account-related information in the link information. In addition, various modifications of the process of issuing the certificate are possible.

[0241] In addition to the certificate regarding whether or not an application has been provided to a patient, the management processing unit 113 may provide other information to a hospital, an insurance company, or the like. For example, as described above with respect to FIG. 37, the server system 100 of the present embodiment may acquire information regarding the time zone and utilization rate at which a patient has used an application. The management processing unit 113 may provide information regarding the time zone and utilization rate to a hospital, an insurance company, or the like. For example, if a hospital not only provides an application to a patient but also fails to encourage the proper use of the application, it may be determined that the hospital has not taken sufficient measures regarding HACs suppression. In this case, a nurse or the like at the hospital checks the usage status of the application by the patient on the screen of FIG. 37 and provides guidance as necessary. In this way, the hospital can claim that it has taken sufficient measures regarding HACs suppression. On the other hand, from the perspective of an insurance company, if it is determined that a patient is not properly using an application based on the time zone and utilization rate, it can be used as a basis for the claim that the hospital is not exempt from liability because the hospital's management is insufficient. That is, the information processing apparatus of the present embodiment may provide information used in the hospital's claim (information serving as a basis for exemption), information used in the insurance company's claim (information serving as a basis for not recognizing exemption), or both of them.

[0242] Although an example of HACs suppression in the United States has been described here, it is not limited thereto, and each of the above pieces of information may be used, for example, by a care manager to manage whether or not home care is being properly provided. Also, each of the above pieces of information may be used by a family member to manage whether or not the care provided by the visiting care staff is being properly performed. Further, each of the above pieces of information may be used as an information sharing tool when the care recipient enters a nursing facility or is readmitted to the hospital.

[0243] Figure 39 is an example of an act that has conventionally been assumed to be difficult to shift to home medical care. The application of this embodiment may be an application that supports each act described in Figure 39. By doing so, it becomes possible to shift many medical acts to home medical care. That is, it also becomes possible to promote home medical care by promoting the digitization of tacit knowledge. However, the acts shown in Figure 39 are an example of the target to which the method of this embodiment is applied, and some items may be omitted due to the improvement of medical technology, or other items may be added due to the increase in difficulty, and various modifications can be made for specific examples.

[0244] Although the present embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications that do not substantially depart from the novel matters and effects of the present embodiment are possible. Therefore, all such modified examples are intended to be included in the scope of this disclosure. For example, in the specification or drawings, a term that is described at least once together with a different term that is broader or synonymous can be replaced with that different term at any location in the specification or drawings. Also, all combinations of the present embodiment and the modified examples are included in the scope of this disclosure. Further, the configuration and operation of the information processing apparatus, information processing system, terminal device, server system, etc. are not limited to those described in the present embodiment, and various modifications can be made.

[0245] For example, the method of the present embodiment can be widely applied to the digitization (component creation) of tacit knowledge related to judgment and the creation of applications by combining components, and the fields to which it is applied are not limited to assistance (including caregiving and medical care). As an example, the method of the present embodiment may digitize tacit knowledge in sports, such as body movements for winning a game. The sports here can include various sports such as baseball, soccer, golf, tennis, table tennis, etc., and may also include e-sports that use tacit knowledge related to button operations, for example. Alternatively, in recent years, attempts have been made to share know-how among institutions such as child guidance centers in order to prevent child abuse at home. Therefore, the method of the present embodiment may be applied to the digitization of tacit knowledge related to abuse suppression. By digitizing the tacit knowledge of experts in child care and introducing it into the home, it becomes possible to suppress abuse in areas where experts have difficulty reaching.

[0246] In addition, the method of the present embodiment may be provided for various arts such as painting, photography, calligraphy, sculpture, and bonsai. By doing so, tacit knowledge related to the handling of a brush, color sense, composition selection, etc. of a skilled person can be provided to beginners. Also, tacit knowledge related to risk prediction in automobile driving, etc. may be digitized. For example, it may be predicted whether an accident will occur or not, and the prediction result may be projected onto the windshield of an automobile. Also, since reliability is important for risk prediction in automobiles, etc., conditions such as not being able to be registered in the server system 100 unless evaluated in a virtual space may be set. In addition, the method of the present embodiment may be one that improves the efficiency of housework by digitizing tacit knowledge related to housework such as cooking and cleaning. Also, the method of the present embodiment may be one that improves the sales of a store by digitizing tacit knowledge related to the interior decoration of a store, product layout, etc. Also, the method of the present embodiment may be one that digitizes tacit knowledge related to the design of a screen UI, etc. or digitizes tacit knowledge related to various designs such as circuit design and mechanical design. In this case, it becomes possible to improve the quality of the output in the design.

Description of Reference Numerals

[0247] 10... Information processing system, 100... Server system, 110... Processing unit, 111... First processing unit, 112... Second processing unit, 113... Management processing unit, 114... Accounting processing unit, 120... Memory unit, 121... User information, 122... Component information, 123... Application information, 130... Communication unit, 200, 200-1 to 200-6... Terminal device, 210... Processing unit, 220... Memory unit, 230... Communication unit, 240... Display unit, 250... Operation unit, 300... HACs evaluation system, 400... Detection device, 410... Bed, 420... Mattress, 520... Wheelchair, 521... Cushion, 523... Control box, RE1 to RE17, RE5a to RE5c, RE8a to RE8c... Display area, S1 to S8... States, Se1 to Se4... Pressure sensors

Claims

1. A first processing unit that creates a component representing a learned model for performing determination in assistance; A second processing unit that creates an application for presenting information for supporting an assistant in the assistance of a person requiring assistance based on a combination of the components; comprising; The first processing unit: creates the component based on a user input associating a sensor device with input data of the learned model and a user input specifying an output data format of the learned model, in creating the component, performs a process of listing changes in a plurality of sensing data acquired using the sensor device over a given period in chronological order, and a process of receiving, on a screen where the plurality of sensing data are listed, an annotation which is a user's determination result for at least one of the plurality of listed sensing data, thereby obtaining training data in which the annotation is associated with at least one of the plurality of sensing data, The second processing unit: performs a process of displaying an object representing the component created by the first processing unit in a display area, and a process of receiving a user input for determining a connection relationship of the object in the display area, and an information processing apparatus that creates the application based on the user input.

2. In Claim 1, The second processing unit: determines the number of output edges to be displayed in association with the object based on the output data format associated with the component, generates branches for the number of output edges, arranges the object, and displays, in the display area, a suggestion for specifying an input method at the time of execution of the application based on an input format associated with the component. An information processing apparatus.

3. In Claim 2, The second processing unit: performs a process of displaying a search area for the component, and when receiving a user input for selecting any one of the components among the search results displayed in the search area, an information processing apparatus that displays the object and the output edge corresponding to the selected component in the display area.

4. In any one of Claims 1 to 3, An information processing apparatus including an accounting processing unit that performs accounting processing including at least one of a process of giving a reward to the first user and a charging process to the second user when the first user creates a first component and the second user creates a first application including the first component.

5. In claim 4, the accounting processing unit is an information processing apparatus that performs accounting processing including at least one of a process of giving a reward to the second user and a charging process to the third user when the first application is used by a third user.

6. In any one of claims 1 to 5, an HACs evaluation unit that estimates the occurrence probability of HACs (Hospital-Acquired Condition), a management processing unit that identifies a user based on the input user information, and the management processing unit is an information processing apparatus that performs a process of recommending the application determined to be related to the suppression of HACs based on the information of the care recipient and the occurrence probability of the HACs obtained from the HACs evaluation unit when the fourth user, who is a user who uses the application based on the user information, is identified as a medical-related user.

7. In claim 6, the management processing unit is an information processing apparatus that performs a process of issuing a certificate to prove that the application has been provided to the fifth user to the fourth user when it is identified based on the user information that the fourth user has provided the application related to the HACs to the fifth user who is a patient.

8. In any one of claims 1 to 5, including a management processing unit that identifies a user based on the input user information, the management processing unit is an information processing apparatus that identifies whether the user is a developer or a user who uses an application based on the user information and selects a developer mode or a user mode for using the application.

9. A computer creates a component representing the learned model that makes a determination in assistance based on a user input associating a sensor device with input data of the learned model and a user input specifying the output data format of the learned model. A process of displaying an object representing the created component in a display area, and a process of receiving a user input for determining a connection relationship of the object in the display area are performed. Based on the combination of the components determined by the user input, an application for presenting information that supports an assistant in the assistance for the assisted person is created. In creating the component, a process of listing changes in a plurality of sensing data acquired using the sensor device over a given period in chronological order, and a process of receiving an annotation, which is a user's judgment result, for at least one of the plurality of sensing data listed, on a screen where the plurality of sensing data are listed, are performed, thereby obtaining training data in which the annotation is associated with at least one of the plurality of sensing data. An information processing method.

Images