Computer program, information processing method, and information processing device

Abstract

To provide a computer program, an information processing method, and an information processing device for performing watching with improved visibility and accuracy of the display of an abnormality of an object person.SOLUTION: A computer program causes a computer to display a state image indicating a state of an object person determined on the basis of data acquired by using a sensor for each object person, set a reference value for a numerical value of a specific kind for each object person, and display an image indicating a specific abnormality as a state image when the numerical value of the specific kind is out of the set reference. The computer program causes the computer to display a first graph indicating the numerical value of the specific kind at each time point or a second graph indicating the state of the object person at each time point when an optional object person is selected, and highlight a part indicating the numerical value of the specific kind at a time point where the numerical value of the specific kind is out of the reference, in the first graph, and a part indicating a time point when the numerical value of the specific kind is out of the reference in the second graph.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device for performing monitoring. [Background technology]

[0002] Conventionally, a monitoring system for monitoring a subject has been used. The monitoring system uses a sensor to acquire data such as vital signs related to the subject's condition, and displays the subject's condition based on the acquired data on a terminal device. For example, in a nursing care facility, a sensor is attached to each bed, and care is provided by a caregiver who checks the subject's condition displayed on a terminal device. The monitoring system detects abnormalities in the subject based on the acquired data, and displays the detected abnormalities on the terminal device. Patent Document 1 discloses an example of such a monitoring system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-60710 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a monitoring system, various information is displayed on a terminal device, so the visibility of information indicating abnormalities in a subject may be poor. Also, depending on the subject or the subject's physical condition, the criteria for determining that the subject's condition is abnormal may differ, and the display of abnormalities may lack accuracy.

[0005] The present invention has been made in consideration of the above circumstances, and its object is to provide a computer program, an information processing method, and an information processing device that perform monitoring while improving the visibility and accuracy of displaying abnormalities in a target individual. [Means for solving the problem]

[0006] A computer program according to one embodiment of the present invention is characterized in that it causes a computer to execute a process of acquiring data for each subject using a sensor, displaying for each subject a status image showing the subject's condition based on the acquired data, accepting instructions for setting a standard for a specific type of numerical value based on the data acquired using the sensor for each subject, performing settings in accordance with the accepted instructions, and displaying an image showing a specific abnormality as the status image when the specific type of numerical value based on the acquired data deviates from the set standard, and when an arbitrary subject is selected, displaying at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the subject's condition based on data acquired at each time for the selected subject, and highlighting at least one of a portion in the first graph showing the specific type of numerical value at the time when the specific type of numerical value deviates from the set standard and a portion in the second graph showing the time when the specific type of numerical value deviates from the set standard.

[0007] An information processing method according to one embodiment of the present invention includes acquiring data for each subject using a sensor, displaying for each subject a status image showing the subject's condition based on the acquired data, accepting instructions for each subject to set a standard for a specific type of numerical value based on the data acquired using the sensor, performing settings in accordance with the accepted instructions, and displaying an image showing a specific abnormality as the status image when the specific type of numerical value based on the acquired data deviates from the set standard, and when an arbitrary subject is selected, displaying at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the subject's condition based on data acquired at each time for the selected subject, and highlighting at least one of a portion in the first graph showing the specific type of numerical value at the time when the specific type of numerical value deviates from the set standard and a portion in the second graph showing the time when the specific type of numerical value deviates from the set standard.

[0008] An information processing device according to one embodiment of the present invention includes a calculation unit, which acquires data for each subject using a sensor, displays a status image for each subject showing the subject's condition based on the acquired data on a display unit, accepts instructions to set a standard for a specific type of numerical value based on the data acquired using the sensor for each subject, performs settings in accordance with the accepted instructions, and when the specific type of numerical value based on the acquired data deviates from the set standard, displays an image showing a specific abnormality as the status image on the display unit, and when an arbitrary subject is selected, displays on the display unit at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the subject's condition based on data acquired at each time for the selected subject, and highlights at least one of a portion in the first graph showing the specific type of numerical value at the time when the specific type of numerical value deviates from the set standard and a portion in the second graph showing the time when the specific type of numerical value deviates from the set standard on the display unit.

[0009] In one embodiment of the present invention, for each subject, a status image showing the subject's condition based on data acquired using a sensor is displayed, and a standard for a specific type of numerical value based on the data is set. When the specific type of numerical value deviates from the standard, a status image showing a specific abnormality is displayed. For the selected subject, a first graph showing the specific type of numerical value at each time, or a second graph showing the subject's condition at each time, is displayed. In the first graph or the second graph, a portion where the specific type of numerical value deviates from the standard is highlighted. Since the standard is set to an appropriate standard for each subject, the user is accurately notified of the subject's abnormality. In addition, the highlighting improves the visibility of the subject's abnormality. Effect of the Invention

[0010] The present invention has excellent effects such as enabling monitoring to accurately confirm the condition of a subject, and enabling a user to easily confirm the condition of the subject. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram showing a configuration example of a monitoring system. [Diagram 2] 1 is a block diagram showing an example of the internal configuration of an information processing device; [Diagram 3] 2 is a block diagram showing an example of the internal configuration of a terminal device; [Figure 4] 11 is a schematic diagram showing an example in which the information processing device displays information indicating the condition of a subject on a terminal device. FIG. [Diagram 5] 11 is a flowchart illustrating an example of a procedure for a process of setting standards for a heart rate and a respiratory rate, which is performed by the information processing device. [Figure 6] FIG. 13 is a schematic diagram showing an example of a setting screen for setting standards for heart rate and respiratory rate. [Figure 7] 13 is a flowchart showing an example of a processing procedure for displaying a status image. [Figure 8] FIG. 13 is a schematic diagram showing a first example of a displayed image showing an abnormality in the respiratory rate per unit time of a subject as a status image. [Figure 9] FIG. 13 is a schematic diagram showing a second example of a displayed image showing an abnormality in the respiratory rate per unit time of a subject as a status image. [Figure 10] FIG. 13 is a schematic diagram showing a third example in which an image showing an abnormality in the respiratory rate per unit time of a subject is displayed as a status image. [Figure 11] FIG. 13 is a schematic diagram showing a fourth example in which an image showing an abnormality in the respiratory rate per unit time of a subject is displayed as a status image. [Figure 12] 13 is a flowchart showing an example of a processing procedure for displaying details of a subject's condition. [Figure 13] FIG. 13 is a schematic diagram showing an example in which a first graph, a second graph, and a state history are displayed. [Figure 14] FIG. 13 is a schematic diagram showing a first example of detailed display of an abnormality in a subject's heart rate per unit time. [Figure 15] FIG. 13 is a schematic diagram showing a second example of displaying details of an abnormality in a subject's heart rate per unit time. [Figure 16] FIG. 4 is a schematic diagram showing a display example of a captured image. [Figure 17] FIG. 13 is a schematic diagram showing a first example of an adjusted state history. [Figure 18] FIG. 13 is a schematic diagram showing a second example of an adjusted state history. [Figure 19] 13 is a schematic diagram showing an example of a first graph, a second graph, and a state history in which highlighting has been stopped; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will now be described in detail with reference to the drawings showing embodiments thereof. FIG. 1 is a schematic diagram showing a configuration example of a monitoring system 100. The monitoring system 100 is a system that performs processing for monitoring the state of a target person 51. The monitoring system 100 includes an information processing device 1, a plurality of sensor devices 2, a plurality of camera devices 7, and a plurality of terminal devices 3. The plurality of sensor devices 2 are connected to the information processing device 1 via a communication network 4 such as a LAN (Local Area Network). The sensor device 2 may communicate with the information processing device 1 without the communication network 4. The plurality of camera devices 7 are each capable of communicating with the information processing device 1 using wired communication or wireless communication. The plurality of terminal devices 3 are each capable of communicating with the information processing device 1 using wired communication or wireless communication. The terminal device 3 may communicate with the information processing device 1 via the communication network, or may communicate with the information processing device 1 without the communication network. Each of the terminal devices 3 is used by a user 53.

[0013] The sensor device 2 includes one or more sensors that measure various phenomena caused by the subject 51. The sensor measures vibration, sound, temperature, pressure change, load change, body movement, and the like as phenomena caused by the subject 51. The sensor is, for example, a piezoelectric sensor, a microphone sensor, a Doppler sensor, and the like. The sensor device 2 may include a sensor that directly measures the vital signs of the subject 51, such as heart rate or respiratory rate. The sensor device 2 transmits data related to the state of the subject 51 according to the measurement result of the sensor to the information processing device 1. The information processing device 1 determines the state of the subject 51, such as whether the subject 51 is staying in bed 52, based on the data from the sensor device 2. The information processing device 1 outputs information indicating the state of the subject 51 using the terminal device 3. The user 53 checks the state of the subject 51 using the terminal device 3.

[0014] The sensor device 2 is attached to a bed 52 used by a person 51 to be monitored. For example, a facility such as a nursing home or a hospital has multiple beds 52, and a sensor device 2 is attached to each bed 52. The sensor device 2 and the person 51 are in one-to-one correspondence. The sensor device 2 may be attached to something other than the bed 52. For example, the sensor device 2 may be attached to a futon or clothing. When the person 51 sleeps without using a bed 52, the sensor device 2 may be attached to the floor.

[0015] The camera device 7 is a camera for photographing the person 51 to be watched over. The camera device 7 may communicate with the information processing device 1 via a communication network. The camera device 7 may photograph the bed 52 or the room used by the person 51. For example, the camera device 7 is installed on the bed 52. The camera device 7 may be installed somewhere other than the bed 52. The camera device 7 has an image sensor for photographing, a storage unit for storing the photographed image created by photographing, and a communication unit for communication. The camera device 7 photographs continuously or intermittently, and stores the photographed image in association with the time when the photograph was taken. The camera device 7 may transmit the photographed image to the information processing device 1, and the photographed image created by the camera device 7 may be stored in the information processing device 1.

[0016] FIG. 2 is a block diagram showing an example of the internal configuration of the information processing device 1. The information processing device 1 is configured using a computer such as a server device, a PC (personal computer), a smartphone, or a general-purpose tablet PC. The information processing device 1 executes an information processing method. The information processing device 1 includes a calculation unit 11, a memory 12, a storage unit 13, a reading unit 14, and a communication unit 15. The calculation unit 11 is configured using, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a multi-core CPU. The calculation unit 11 may be configured using a quantum computer. The memory 12 stores temporary data generated in association with calculation. The memory 12 is, for example, a RAM (Random Access Memory). The storage unit 13 is non-volatile, for example, a hard disk or a non-volatile semiconductor memory. The reading unit 14 reads information from a recording medium 10 such as an optical disk or a portable memory. The communication unit 15 communicates with the outside of the information processing device 1 by wired communication or wireless communication. Specifically, the communication unit 15 receives data from the sensor device 2 via the communication network 4 and communicates with the terminal device 3 .

[0017] The calculation unit 11 causes the reading unit 14 to read the computer program 131 recorded on the recording medium 10, and stores the read computer program 131 in the storage unit 13. The calculation unit 11 executes processing for realizing the functions of the information processing device 1 in accordance with the computer program 131. The computer program 131 may be a program product. The computer program 131 may be stored in advance in the storage unit 13, or may be downloaded from outside the information processing device 1. In this case, the information processing device 1 does not need to include the reading unit 14.

[0018] The computer program 131 can be deployed to be executed on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network. That is, the information processing device 1 may be composed of multiple computers, and the computer program 131 may be executed on multiple computers connected via a communications network.

[0019] The storage unit 13 stores setting data in which various settings for determining and displaying the state of the subject 51 are recorded. In the setting data, a one-to-one correspondence between the sensor device 2 and the subject 51 is recorded. For example, in the setting data, subject information for identifying each subject 51 and identification information for identifying the sensor device 2 that transmits data related to the state of the subject 51 according to the measurement result of the sensor to the information processing device 1 are recorded in association with each other. The subject information is the name of the subject 51, the location of the room used by the subject 51, the location of the bed 52 used by the subject 51, etc. The identification information is the serial number of the sensor device 2, etc.

[0020] FIG. 3 is a block diagram showing an example of the internal configuration of the terminal device 3. The terminal device 3 is a computer such as a PC, a smartphone, or a tablet PC. The terminal device 3 includes a calculation unit 31, a memory 32, a storage unit 33, an operation unit 34, a display unit 35, and a communication unit 36. The calculation unit 31 is configured using, for example, a CPU, a GPU, or a multi-core CPU. The calculation unit 31 may be configured using a quantum computer. The memory 32 stores temporary data generated in association with the calculation. The memory 32 is, for example, a RAM. The storage unit 33 is non-volatile, for example, a hard disk or a non-volatile semiconductor memory.

[0021] The operation unit 34 receives instructions from the user 53 by receiving operations from the user 53, and also receives input of information. The operation unit 34 is, for example, a touch panel, a keyboard, or a pointing device. The display unit 35 displays images. The display unit 35 is, for example, a liquid crystal display or an EL display (Electroluminescent Display). The operation unit 34 and the display unit 35 may be integrated. The communication unit 36 ​​communicates with the outside of the terminal device 3 by wired or wireless communication. Specifically, the communication unit 36 ​​communicates with the information processing device 1. The terminal device 3 may be provided with an audio output unit such as a speaker.

[0022] The storage unit 33 stores a computer program 331. For example, the computer program 331 is downloaded using the communication unit 36 ​​and stored in the storage unit 33. For example, the computer program 331 may be stored in the storage unit 33 in advance. For example, the computer program 331 may be read into the terminal device 3 via an interface unit (not shown) and stored in the storage unit 33. The calculation unit 31 executes processing in accordance with the computer program 331.

[0023] 4 is a schematic diagram showing an example in which the information processing device 1 displays information indicating the status of the subject 51 on the terminal device 3. A status image 61 indicating the status of the subject 51 is displayed for each subject 51. A subject field 62 including subject information such as the name or the position of the bed 52 for identifying the subject 51 is displayed in association with the status image 61.

[0024] In the example shown in FIG. 4, six status images 61 indicating the status of six subjects 51 are displayed. One status image 61 indicates that the subject 51 is in the "rising" status of getting up from the bed 52. One status image 61 indicates that the subject 51 is in the "starting to get up" status of starting to get up from the bed 52. One status image 61 indicates that the subject 51 is in the "in bed" status of staying in the bed 52. One status image 61 indicates that the subject 51 is in the "absent" status of not being in the bed 52. One status image 61 indicates that the subject 51 is in the "body movement" status of slightly moving the body while sleeping.

[0025] As shown in Fig. 4, the information processing device 1 causes the terminal device 3 to display a status image 61 indicating a status related to the behavior of the subject 51, such as "wake up", "get up", "absent", "in bed", "body movement", or "in bed" in which the subject 51 has entered the bed 52. The information processing device 1 can also display, on the terminal device 3, a status image 61 indicating a status related to the sleep of the subject 51, such as "sleeping" in which the subject 51 is asleep, "light sleep" in which the subject 51 is in a light sleep state, or "awakening" in which the subject 51 wakes up from a sleep state. Furthermore, the information processing device 1 can cause the terminal device 3 to display a status image indicating a status according to the heart rate per unit time of the subject 51 and a status image indicating a status according to the respiratory rate per unit time of the subject 51.

[0026] The information processing device 1 judges the state of the subject 51 according to the heart rate and respiration rate based on the set criteria. The set criteria are determined by the threshold value of the subject 51's heart rate per unit time and the threshold value of the subject 51's respiration rate per unit time. The information processing device 1 performs processing to set the heart rate and respiration rate criteria for each subject 51. FIG. 5 is a flowchart showing an example of the procedure of processing to set the heart rate and respiration rate criteria performed by the information processing device 1. Hereinafter, step is abbreviated as S. The calculation unit 11 executes information processing according to the computer program 131, and the information processing device 1 executes the following processing.

[0027] The information processing device 1 causes the terminal device 3 to display a setting screen for setting the standards for the heart rate and respiratory rate (S11). In S11, the calculation unit 11 causes the communication unit 15 to transmit data for the setting screen to the terminal device 3, the terminal device 3 receives the data via the communication unit 36, and the calculation unit 31 displays the setting screen on the display unit 35 based on the data. For example, the user 53 operates the operation unit 34 to specify one subject 51 from among multiple subjects 51, input an instruction to perform setting processing, and the information processing device 1 starts setting processing for the specific subject 51.

[0028] 6 is a schematic diagram showing an example of a setting screen for setting the standards for the heart rate and respiration rate. The setting screen includes a switch for instructing whether to perform heart rate detection or respiration detection. Heart rate detection corresponds to determining an abnormality in the heart rate and displaying a status image showing a state in which the heart rate per unit time is abnormal. Respiration detection corresponds to determining an abnormality in the respiration rate and displaying a status image showing a state in which the respiration rate per unit time is abnormal.

[0029] Also, input fields for inputting the upper and lower limits of the heart rate per unit time and the upper and lower limits of the respiration rate per unit time as the thresholds of the heart rate and the respiration rate are included. The upper limit of the heart rate is a threshold at which the heart rate per unit time is determined to be increasing, and the lower limit of the heart rate is a threshold at which the heart rate per unit time is determined to be decreasing. The upper limit of the respiration rate is a threshold at which the respiration rate per unit time is determined to be increasing, and the lower limit of the respiration rate per unit time is a threshold at which the respiration rate is determined to be decreasing. In the example shown in FIG. 6, it is instructed to detect the respiration without detecting the heart rate, and the upper and lower limits of the respiration rate per unit time are specified.

[0030] The information processing device 1 receives an instruction to set the standard for the heart rate and the respiration rate (S12). In S12, the user 53 who has visually recognized the setting screen operates the operation unit 34 to input an instruction as to whether to perform heart rate detection or respiration detection to the terminal device 3. The user 53 also operates the operation unit 34 to input the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time to the terminal device 3. The calculation unit 31 causes the communication unit 36 ​​to transmit the input contents to the information processing device 1, and the information processing device 1 receives them at the communication unit 15. At this time, the calculation unit 11 receives an instruction as to whether to perform heart rate detection or respiration detection. The calculation unit 11 also receives the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time at the communication unit 15.

[0031] The information processing device 1 performs setting according to the received instruction (S13). In S13, the calculation unit 11 performs setting by recording in the setting data whether or not to perform heartbeat detection or respiration detection in association with a specific subject 51. The calculation unit 11 also performs setting by recording in the setting data the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time. If the value of the heart rate or respiration rate is included in the range from the lower limit to the upper limit, the heart rate or respiration rate meets the standard. If the value of the heart rate or respiration rate is below the lower limit or exceeds the upper limit, the heart rate or respiration rate is outside the standard. In this way, the standard for the heart rate and respiration rate is set. After S13 is completed, the information processing device 1 ends the process of setting the standard.

[0032] The information processing device 1 executes the processes of S11 to S13 for each subject 51. Through the processes of S11 to S13, the standard of the heart rate and the respiration rate is set for each subject 51. The appropriate range of the heart rate and the respiration rate may differ from person to person. By setting the standard for each subject 51, the standard of the heart rate and the respiration rate can be set to an appropriate standard that differs depending on the subject 51. Furthermore, even for the same subject 51, the appropriate standard of the heart rate and the respiration rate may differ depending on the physical condition. By changing and resetting the standard of the heart rate and the respiration rate, it is possible to set an appropriate standard according to the physical condition of the subject 51. Note that the threshold of the heart rate per unit time or the threshold of the respiration rate per unit time may be only an upper limit or only a lower limit.

[0033] Moreover, by the processes of S11 to S13, whether or not to perform heartbeat detection or respiration detection is set for each subject 51. That is, the information processing device 1 sets whether or not to display an image showing a state where there is an abnormality in the heart rate or respiration rate per unit time as a status image. Similarly, the information processing device 1 can perform a process of setting whether or not to display a status image showing a state related to the behavior of the subject 51. Similarly, the information processing device 1 can perform a process of setting whether or not to display a status image showing a state related to the sleep of the subject 51. The contents of the setting are recorded in the setting data. Whether or not to display a status image showing a state related to the behavior of the subject 51 and whether or not to display a status image showing a state related to the sleep of the subject 51 may be set in advance.

[0034] The information processing device 1 performs processing to display a plurality of status images showing the status of a plurality of subjects 51. Fig. 7 is a flowchart showing an example of a processing procedure for displaying a status image. The information processing device 1 acquires data from the sensor device 2 (S21). The sensor device 2 transmits data related to the status of the subject 51 according to the measurement result of the sensor to the information processing device 1 as needed. The sensor device 2 may transmit the value measured by the sensor as data as it is.

[0035] The sensor device 2 may transmit data obtained by processing a value measured by the sensor. For example, a piezoelectric sensor included in the sensor device 2 measures a phenomenon caused by the subject 51, such as vibration or sound, and generates a measurement signal representing a value obtained by the measurement. The sensor device 2 performs signal processing, such as noise removal, amplification, or frequency separation, on the measurement signal to generate a feature signal representing a feature of the state of the subject 51. For example, the feature signal includes a heartbeat signal indicating the heartbeat of the subject 51, a breathing signal indicating the breathing vibration of the subject 51, a body movement signal indicating the body movement of the subject 51, or a snoring signal indicating the snoring of the subject 51. The sensor device 2 transmits data including the feature signal to the information processing device 1. The data transmitted by the sensor device 2 may include the measurement signal.

[0036] The information processing device 1 acquires data from the sensor device 2 by receiving data transmitted from the sensor device 2 via the communication unit 15. The information processing device 1 receives data from the multiple sensor devices 2 at any time. The calculation unit 11 stores the received data in the storage unit 13 in association with a time. The time may include the year, month, and day.

[0037] The information processing device 1 determines the state of the subject 51 based on the acquired data (S22). In S22, the calculation unit 11 determines the state related to the behavior of the subject 51, such as "wake up" or "rising up", based on the acquired data. The calculation unit 11 also determines the state related to the sleep of the subject 51, such as "sleeping" or "awake".

[0038] For example, the calculation unit 11 determines the state of the subject 51 using a trained model that outputs information indicating the state of the subject 51 regarding the behavior and sleep when a feature signal is input. The trained model is realized by the calculation unit 11 executing processing according to the computer program 131. The storage unit 13 stores data necessary to realize the trained model. For example, the trained model is realized using a neural network.

[0039] The trained model is generated by learning using training data including a feature signal representing the characteristics of the state of the subject 51 and information indicating the state related to the behavior and sleep of the subject 51. The trained model is trained in advance so as to output information indicating the state related to the behavior and sleep of the subject 51 when a feature signal is input. The trained model may be trained by the information processing device 1. The trained model for determining the state related to the behavior of the subject 51 and the trained model for determining the state related to the sleep of the subject 51 may be different.

[0040] In S22, the calculation unit 11 inputs the body movement signal or snoring signal included in the data acquired from the sensor device 2 to the trained model. The trained model performs calculations in response to the input of the body movement signal or snoring signal, and outputs information indicating the behavior and sleep state of the subject 51. The calculation unit 11 acquires the information indicating the behavior and sleep state of the subject 51 outputted by the trained model. The calculation unit 11 determines the behavior and sleep state of the subject 51 as indicated by the acquired information.

[0041] The calculation unit 11 may determine the state of the subject 51 regarding the behavior and sleep without using a trained model. For example, the calculation unit 11 determines the state of the subject 51 regarding the behavior and sleep according to a predetermined rule. The calculation unit 11 may determine that the subject 51 is in bed when a measurement signal included in the acquired data changes significantly and exceeds a predetermined threshold. The calculation unit 11 may determine that the subject 51 is in bed when at least one of a heartbeat signal and a respiratory signal is obtained from the acquired data. The calculation unit 11 may determine that the subject 51 is in a body movement state when a body movement signal is significantly obtained.

[0042] The calculation unit 11 may determine that the state of the subject 51 is the sitting up state when a signal indicating an initial movement of the subject 51 to sit up is obtained from a body movement signal included in the acquired data. The calculation unit 11 may determine that the state of the subject 51 is the sitting up state when the body movement signal indicates a stable value below a predetermined threshold after the signal indicating the initial movement is obtained. The calculation unit 11 may determine that the state of the subject is the absent state when the body movement signal is maintained in a state significantly below the predetermined threshold after the signal indicating the initial movement is obtained.

[0043] The calculation unit 11 may determine that the state of the subject 51 is the in-bed state when the measurement signal exceeds a predetermined threshold and then falls below the threshold. The calculation unit 11 may determine that the state of the subject 51 is the in-bed state when the heartbeat signal and the respiratory signal included in the acquired data fall below a predetermined threshold. The calculation unit 11 may determine that all states other than the in-bed state, the body movement state, the sitting up state, the sitting up state, and the out-of-bed state are the in-bed state. When the calculation unit 11 determines that the state of the subject 51 is the in-bed state, it may determine that the subject 51 is in a sleeping state or an awake state according to the pattern of the characteristic signal including the heartbeat signal, the respiratory signal, the body movement signal, and the like.

[0044] The monitoring system 100 may be configured such that the sensor device 2 determines the behavior and sleep state of the subject 51 based on the measurement result of the sensor, and transmits information indicating the determined state of the subject 51 from the sensor device 2 to the information processing device 1. In this configuration, in S22, the sensor device 2 determines the behavior and sleep state of the subject 51 by performing the determination process described above based on the measurement result of the sensor, and transmits information indicating the determined state of the subject 51 to the information processing device 1. The information processing device 1 determines the behavior and sleep state of the subject 51 by receiving the information indicating the state of the subject 51 transmitted from the sensor device 2 via the communication unit 15.

[0045] Furthermore, in S22, the calculation unit 11 determines the state of the subject 51 according to the heart rate per unit time and the state of the subject 51 according to the respiratory rate per unit time. The calculation unit 11 specifies the heart rate per unit time of the subject 51 based on the acquired data. For example, the calculation unit 11 specifies the heart rate per unit time of the subject 51 based on a heart rate signal included in the acquired data. If the specified heart rate meets a set criterion, the calculation unit 11 determines that the subject 51 is in a state where the heart rate is normal. More specifically, if the specified heart rate is within a range from a lower limit to an upper limit recorded in the set data, the calculation unit 11 determines that the heart rate is normal. If the specified heart rate is outside the set criterion, the calculation unit 11 determines that the subject 51 is in a state where the heart rate is abnormal. More specifically, when the identified heart rate is a value below the lower limit or above the upper limit recorded in the setting data, the calculation unit 11 determines that the heart rate is abnormal.

[0046] The calculation unit 11 specifies the respiratory rate per unit time of the subject 51 based on the acquired data. For example, the calculation unit 11 specifies the respiratory rate per unit time of the subject 51 based on a respiratory signal included in the acquired data. If the specified respiratory rate meets the set criteria, the calculation unit 11 determines that the subject 51 is in a state where the respiratory rate is normal. More specifically, if the specified respiratory rate is within the range from the lower limit to the upper limit recorded in the set data, the calculation unit 11 determines that the respiratory rate is normal. If the specified respiratory rate falls outside the set criteria, the calculation unit 11 determines that the subject 51 is in a state where the respiratory rate is abnormal. More specifically, if the specified respiratory rate is a value below the lower limit or above the upper limit recorded in the set data, the calculation unit 11 determines that the respiratory rate is abnormal.

[0047] In S22, the calculation unit 11 judges the state of each of the multiple subjects 51. The calculation unit 11 also records the history of the judged state of the subject 51. The storage unit 13 stores history data that records the history of the state of the subject 51. In the history data, the judged state of the subject 51 is recorded in association with a time for each subject 51. The time associated with the state of the subject 51 is the time when the state of the subject 51 is judged, or the time when the data on which the state of the subject 51 is judged is acquired by the sensor device 2 or the information processing device 1. The time may include the year, month, and day. The calculation unit 11 records the history of the state of the subject 51 by associating the judged state of the subject 51 with a time in the history data for each subject 51.

[0048] The information processing device 1 causes the terminal device 3 to display a status image showing the state of the determined subject 51 (S23). The storage unit 13 stores status images showing the respective states of the subject 51. In S23, the calculation unit 11 causes the communication unit 15 to transmit the status image showing the state of the determined subject 51 to the terminal device 3. The terminal device 3 receives the status image via the communication unit 36, and the calculation unit 31 displays the received status image on the display unit 35. The information processing device 1 transmits status images for multiple subjects 51, and the terminal device 3 displays the multiple status images. The information processing device 1 may generate an image including multiple status images and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 transmits information showing each status to the terminal device 3, and the terminal device 3 may store status images showing the respective states of the subject 51 in the storage unit 33 in advance, and display a status image according to the received information on the display unit 35.

[0049] The information processing device 1 displays, among the status images, an image showing a status related to the behavior of the subject 51 and an image showing a status related to the sleep of the subject 51 on the display unit 35 of the terminal device 3 at different positions for each subject 51. FIG. 4 shows an example in which an image showing a status related to the behavior of the subject 51 or an image showing a status related to the sleep of the subject 51 is displayed as a status image 61. The information processing device 1 displays the status image 61, which is an image showing a status related to the behavior of the subject 51 or an image showing a status related to the sleep of the subject 51, in a first display area provided for each subject 51 on the display unit 35. The position where each status image 61 is arranged in FIG. 4 corresponds to the first display area.

[0050] The information processing device 1 displays an image showing an abnormality in the subject's 51 heart rate per unit time or an abnormality in the subject's 51 respiration rate per unit time in a form different from the image showing the subject's 51 behavioral state and the image showing the subject's 51 sleep state. FIG. 8 is a schematic diagram showing a first example of an image showing an abnormality in the subject's 51 respiration rate per unit time as a status image. As a status image 61, an image showing the subject's 51 behavioral state or an image showing the subject's 51 sleep state is displayed in the first display area. Furthermore, the information processing device 1 causes the display unit 35 of the terminal device 3 to display a status image 63 showing an abnormality in the subject's 51 respiration rate per unit time, superimposed on the status image 61 displayed in the first display area. At this time, the information processing device 1 makes the area of ​​the status image 63 smaller than the area of ​​the status image 61.

[0051] A status image 63 indicating that the subject 51 has an abnormality in the respiration rate per unit time is displayed superimposed on a status image 61 indicating the subject 51's status related to behavior or sleep. The user 53 can confirm that the subject 51 has an abnormality in the respiration rate per unit time by visually checking the status image 63. The subject 51 having an abnormality in the respiration rate is made clear by the status image 63 being superimposed on the status image 61. The area of ​​the status image 63 is smaller than the area of ​​the status image 61, so that the user 53 can visually check both the status image 61 and the status image 63. Therefore, the user 53 can confirm both the subject 51's status related to behavior or sleep and the subject 51's status of having an abnormality in the respiration rate per unit time.

[0052] 9 is a schematic diagram showing a second example in which an image showing an abnormality in the respiratory rate per unit time of the subject 51 is displayed as a status image. The information processing device 1 displays a status image 63 showing an abnormality in the respiratory rate per unit time of the subject 51 on the display unit 35 of the terminal device 3, superimposed on the status image 61. At this time, the status image 63 is displayed as a transparent image on the display unit 35 of the terminal device 3. The area of ​​the status image 63 may be equal to or larger than the area of ​​the status image 61.

[0053] In the second example, the user 53 can also confirm that the subject 51 has an abnormality in the respiration rate per unit time by visually checking the status image 63. The status image 63 is superimposed on the status image 61, making it clear that the subject 51 has an abnormal respiration rate. The status image 63 is transparent, so the user 53 can visually check both the status image 61 and the status image 63. Therefore, the user 53 can confirm both the behavioral or sleeping status of the subject 51 and the status of the subject 51 having an abnormality in the respiration rate per unit time.

[0054] 10 is a schematic diagram showing a third example in which an image showing an abnormality in the respiratory rate per unit time of the subject 51 is displayed as a status image. The information processing device 1 pops up an image 64 including a status image 63 showing an abnormality in the respiratory rate per unit time of the subject 51 on the display unit 35 of the terminal device 3 at a position independent of the subject 51. The information processing device 1 also causes the pop-up image 64 to include a subject field 641 including subject information for identifying the subject 51, and a warning message 642 notifying that the subject 51 has an abnormality in breathing.

[0055] Also in the third example, the user 53 can confirm that there is an abnormality in the respiratory rate per unit time of the subject 51 by visually checking the status image 63. The status image 63 is displayed as a pop-up at a position independent of the subject 51, which makes it easier for the user 53 to recognize the status image 63.

[0056] FIG. 11 is a schematic diagram showing a fourth example in which an image showing an abnormality in the respiratory rate per unit time of the subject 51 is displayed as a status image. The information processing device 1 causes the display unit 35 of the terminal device 3 to display a status image 63 showing an abnormality in the respiratory rate per unit time of the subject 51 in a second display area provided for each subject 51 different from the first display area in which the status image 61 is displayed. The position where the status image 63 is arranged in FIG. 11 corresponds to the second display area. In addition, the information processing device 1 may cause the terminal device 3 to display an image showing a normal respiratory rate per unit time of the subject 51 in the second display area. Furthermore, the information processing device 1 may cause the terminal device 3 to display an image showing the respiratory rate per unit time of the subject 51 in the second display area.

[0057] In the fourth example as well, the user 53 can confirm that the subject 51 has an abnormality in the respiratory rate per unit time by visually checking the status image 63. The status image 63 is displayed in a position different from the status image 61, so that the user 53 can visually check both the status image 61 and the status image 63. In addition, the status image 63 is displayed in a second display area provided for each subject 51, so that the subject 51 having an abnormal respiratory rate becomes clear.

[0058] The information processing device 1 also displays, on the display unit 35 of the terminal device 3, a status image showing a state in which the subject 51 has an abnormal heart rate per unit time, as in the examples shown in Figs. 8 to 11. The information processing device 1 may display, on the display unit 35 of the terminal device 3, an image showing a state in which the subject 51 has a normal heart rate per unit time, or may display, on the display unit 35 of the terminal device 3, an image showing the subject 51's respiratory rate per unit time. The user 53 can confirm that the subject 51 has an abnormal heart rate per unit time by visually checking the displayed status image. The subject 51 has an abnormal heart rate or respiratory rate per unit time, and the user 53 needs to be notified of the abnormal heart rate or respiratory rate more urgently than the subject 51's behavior or sleep status. By displaying the image showing the abnormal heart rate or respiratory rate in a form different from the subject 51's behavior or sleep status, the user 53 can be impressed with information of high urgency of notification. This improves the visibility of highly urgent information, making it possible to respond quickly to highly urgent information and provide appropriate monitoring.

[0059] Since the standard for determining that there is an abnormality in the heart rate or respiration rate per unit time is set to an appropriate standard for each subject 51, it is accurately determined that there is an abnormality in the heart rate or respiration rate for the subject 51. Even if the standard differs depending on the physical condition of the subject 51, by setting an appropriate standard according to the physical condition, it is accurately determined that there is an abnormality in the heart rate or respiration rate according to the physical condition of the subject 51. Therefore, the accuracy of the display on the terminal device 3 that there is an abnormality in the heart rate or respiration rate per unit time of the subject 51 is improved. Therefore, it is possible to monitor the subject 51 to accurately confirm his / her condition.

[0060] In S23, the information processing device 1 may cause the terminal device 3 to output an alarm in synchronization with the display of an image showing an abnormality in the heart rate or respiratory rate as a status image. For example, the terminal device 3 may output an alarm by changing the color of a part or all of the image displayed on the display unit 35 to a specific color, or by blinking a part or all of the image. For example, the terminal device 3 may be provided with an audio output unit and output an alarm by outputting audio. In this way, an alarm indicating that the subject 51 is abnormal is notified to the user 53 in synchronization with the display of the status image.

[0061] When the setting is such that heart rate detection is not performed, the information processing device 1 does not perform the process of causing the terminal device 3 to display a status image indicating that the subject 51 has an abnormality in the heart rate per unit time in S23. Similarly, when the setting is such that respiration detection is not performed, the information processing device 1 does not perform the process of causing the terminal device 3 to display a status image indicating that the subject 51 has an abnormality in the respiration rate per unit time. Furthermore, the information processing device 1 may perform the process of S23 for multiple terminal devices 3, or may perform the process of S23 for a specific terminal device 3.

[0062] The information processing device 1 waits for confirmation from the user 53 regarding the display of the status image showing the status of the subject 51 (S24). The user 53 visually confirms the status image displayed on the display unit 35 and inputs that the status of the subject 51 has been confirmed by operating the operation unit 34. In S24, the calculation unit 31 accepts the input by the user 53 operating the operation unit 34, and causes the communication unit 36 ​​to transmit information indicating that the confirmation of the status of the subject 51 has been accepted to the information processing device 1, and the information processing device 1 receives the information at the communication unit 15. If there is no acceptance of the confirmation from the user 53 (S24: NO), the information processing device 1 repeats S24. Note that the information processing device 1 may end the process if a predetermined time has elapsed without accepting the confirmation from the user 53. Also, in S24, when the information processing device 1 displays a status image different from the status image that was displayed without accepting the confirmation from the user 53, it may wait for the acceptance of the confirmation from the user 53 regarding the display of the different status image.

[0063] When confirmation is received from the user 53 (S24: YES), the information processing device 1 records that confirmation has been completed (S25). In S25, the calculation unit 11 records in the history data that confirmation has been completed in association with the state of the subject person 51. After S25 ends, the information processing device 1 ends the process of displaying the state image. The information processing device 1 executes the processes of S21 to S25 as needed.

[0064] The information processing device 1 performs a process of displaying details of the status of the subject 51. FIG. 12 is a flowchart showing an example of a procedure of a process of displaying details of the status of the subject 51. The information processing device 1 selects an arbitrary subject 51 (S301). In S301, the user 53 operates the operation unit 34, whereby the terminal device 3 accepts the selection of the arbitrary subject 51, and transmits the contents of the accepted selection to the information processing device 1. For example, the user 53 operates the operation unit 34 to specify any of the status images 61 or subject columns 62 as shown in FIG. 4, FIG. 8 to FIG. 11, thereby accepting the selection of the arbitrary subject 51 on the terminal device 3. The information processing device 1 receives the contents of the selection via the communication unit 15. The calculation unit 11 selects the subject 51 according to the received contents of the selection.

[0065] The information processing device 1 causes the terminal device 3 to display the first graph, the second graph, and the state history for the selected subject 51 (S302). FIG. 13 is a schematic diagram showing an example in which the first graph 65, the second graph 66, and the state history 67 are displayed. The first graph 65 is a scatter plot showing the heart rate and the respiration rate per unit time based on data acquired at each time for the selected subject 51. In the figure, "heart rate" refers to the heart rate, and "respiration" refers to the respiration rate. The horizontal axis of the first graph 65 indicates the time, and the vertical axis indicates the heart rate or the respiration rate per unit time. The heart rate and the respiration rate are numerical values ​​based on data acquired by the information processing device 1 from the sensor device 2 and stored in the memory unit 13.

[0066] The first graph 65 displays the standards for the heart rate and respiration rate set at each time. The first graph 65 displays a line 654 indicating the upper limit of the heart rate, a line 655 indicating the lower limit of the heart rate, a line 656 indicating the upper limit of the respiration rate, and a line 657 indicating the upper limit of the respiration rate, thereby showing the standards for the heart rate and respiration rate that have been set. In the example shown in Fig. 13, the value corresponding to the line 654 indicating the upper limit of the heart rate is different before and after the time 5:00. This indicates that the upper limit of the heart rate has been changed by the setting after the time 5:00.

[0067] The second graph 66 is a bar graph showing the state of the subject 51 at each time based on data acquired at each time for the selected subject 51. The horizontal axis of the second graph 66 shows the same time as the first graph 65. In the example shown in FIG. 13, the second graph 66 shows whether the state of the subject 51 at each time was in bed, absent, or moving. The second graph 66 may show other states of the subject 51 at each time.

[0068] The status history 67 is a history of each of a plurality of types of status of the selected subject 51. The status history 67 includes the status of the subject 51 at each time in association with a time. The time associated with the status of the subject 51 is the time associated with the status of the subject 51 in the history data. In addition, in the status history 67, whether the status of the subject 51 is unconfirmed or confirmed is associated with the status of the subject 51 at each time. In addition, a camera icon that is a link to a photographed image of the subject 51 at the time associated with the status of the subject 51 included in the status history 67 is displayed in association with the status of the subject 51 included in the status history 67. The link to the photographed image may be displayed in a form other than a camera icon. Furthermore, when a setting is made to not notify the subject 51 of a specific status so that a status image indicating the status is not displayed, the status history 67 associates the specific status of the subject 51 with being not notified.

[0069] In S302, the calculation unit 11 generates a first graph 65 based on the data obtained from the sensor device 2 and the setting data for the selected subject 51, generates a second graph 66 based on the history data, and generates a state history 67 based on the setting data and the history data. The calculation unit 11 causes the communication unit 15 to transmit the generated first graph 65, second graph 66, and state history 67 to the terminal device 3. The terminal device 3 receives the first graph 65, second graph 66, and state history 67 via the communication unit 36, and the calculation unit 31 displays the received first graph 65, second graph 66, and state history 67 on the display unit 35. The information processing device 1 may generate an image including the first graph 65, second graph 66, and state history 67 and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 may transmit data necessary for generating the first graph 65, the second graph 66, and the status history 67 to the terminal device 3, and the terminal device 3 may generate the first graph 65, the second graph 66, and the status history 67 based on the received data and display them on the display unit 35.

[0070] By displaying the first graph 65, the user 53 can confirm the heart rate or respiration rate per unit time of the subject 51 at each time, and the standard heart rate and respiration rate set at each time. By displaying the second graph 66, the user 53 can confirm the time distribution of the condition of the subject 51. By displaying the condition history 67, the selected condition of the subject 51 and the time when the subject 51 became in that condition are displayed in association with each other. The user 53 can confirm what condition the subject 51 was in at each time.

[0071] The information processing device 1 sets whether or not to display the condition history of the subject 51 for each condition category that categorizes multiple types of conditions of the subject 51. One condition category includes one or more types of conditions. In this embodiment, one of the condition categories is a first condition category that includes conditions related to the behavior of the subject 51. One of the condition categories is a second condition category that includes conditions related to the sleep of the subject 51. One of the condition categories is a third condition category that includes conditions according to the heart rate or respiratory rate per unit time of the subject 51. The setting contents are recorded in the setting data.

[0072] The state history 67 includes a setting field 671 that displays a setting of whether or not to display the state history of the subject 51 included in each state category. FIG. 13 shows an example in which the state history is set to display the state history of the subject 51's behavior, the state history of the subject 51's sleep, and the state according to the subject's heart rate or respiration rate per unit time. For this reason, the state history 67 shown in FIG. 13 includes the state history of "body movement," "getting up," "getting up," and "not being there," which are one of the states related to the behavior of the subject 51. As the state related to the behavior of the subject 51, the state history 67 may include the state of "in bed" or "in bed." As the state related to the sleep of the subject 51, the state history 67 may include the state of "sleeping" and the state of "awake." As the state related to the sleep of the subject 51, the state history 67 may include the state of "light sleep." In addition, in the setting field 671, the first state category including the state related to the behavior of the subject 51 may be displayed as "behavior," "getting out of bed," or "presence / absence." In the setting field 671, a second status category including a status related to the sleep of the subject 51 may be displayed as "Sleep". In the setting field 671, a third status category including a status according to the heart rate or respiration rate per unit time of the subject 51 may be displayed as "Heart Rate / Respiration", "Notification", "Alert", or "Abnormal".

[0073] 13 includes a state of "heart rate increase" in which the heart rate per unit time of the subject 51 is increasing as a state corresponding to the heart rate or respiratory rate per unit time of the subject 51. The state history 67 may include a state in which the heart rate per unit time is decreasing, a state in which the respiratory rate per unit time is increasing, or a state in which the respiratory rate per unit time is decreasing as a state corresponding to the heart rate or respiratory rate per unit time of the subject 51.

[0074] The information processing device 1 determines whether or not it is set to perform heartbeat detection or respiration detection (S303). In S303, the calculation unit 11 determines whether or not it is set to perform heartbeat detection or respiration detection based on the setting data. If it is set not to perform either heartbeat detection or respiration detection (S303: NO), the information processing device 1 proceeds to the process of S310 described later.

[0075] If it is set to perform heartbeat detection or respiration detection (S303: YES), the information processing device 1 judges whether it is set to display a state history according to the heart rate or respiration rate per unit time of the subject 51 in the state history 67 (S304). In S304, the calculation unit 11 makes a judgment based on the setting data. If it is set not to display a state history according to the heart rate or respiration rate in the state history 67 (S304: NO), the information processing device 1 advances the process to S310 described later.

[0076] When it is set to display the state history 67 according to the heart rate or respiration rate per unit time of the subject 51 (S304: YES), the information processing device 1 highlights the portion in the first graph 65 and the second graph 66 corresponding to the heart rate or respiration rate that deviates from the standard (S305). In S305, the calculation unit 11 adjusts the first graph 65 so as to highlight the portion in the first graph 65 that indicates the heart rate or respiration rate per unit time that deviates from the set standard. The calculation unit 11 causes the communication unit 15 to transmit the adjusted first graph 65 to the terminal device 3. The terminal device 3 receives the adjusted first graph 65 and displays it on the display unit 35. In this way, the information processing device 1 highlights the portion in the first graph 65 that indicates the heart rate or respiration rate per unit time that deviates from the set standard.

[0077] Furthermore, in S305, the calculation unit 11 adjusts the second graph 66 so as to emphasize portions showing times when the heart rate or respiration rate per unit time deviates from the set standard. The calculation unit 11 causes the communication unit 15 to transmit the adjusted second graph 66 to the terminal device 3. The terminal device 3 receives the adjusted second graph 66 and displays it on the display unit 35. In this way, the information processing device 1 emphasizes portions in the second graph 66 showing times when the heart rate or respiration rate per unit time deviates from the set standard.

[0078] In the example shown in FIG. 13, in the first graph 65, a portion showing a heart rate per unit time that deviates from the standard is an emphasized portion 651. In the second graph 66, a portion showing a time when the heart rate per unit time deviates from the standard is an emphasized portion 661. The information processing device 1 performs the emphasized display by changing the color, size, or shape of the point showing the heart rate or respiration rate that deviates from the standard in the emphasized portion 651 from that of the other portions, or by blinking the point showing the heart rate or respiration rate that deviates from the standard. The information processing device 1 performs the emphasized display by changing the color, size, or shape of the part corresponding to the heart rate or respiration rate that deviates from the standard in the emphasized portion 661 from that of the other portions, or by blinking the part corresponding to the heart rate or respiration rate that deviates from the standard.

[0079] The highlighting improves the visibility of the occurrence of an abnormality in which the heart rate or respiration rate per unit time of the subject 51 deviates from the standard, and visually notifies the user 53 that an abnormality has occurred. The user 53 can easily confirm that an abnormality has occurred in which the heart rate or respiration rate per unit time of the subject 51 deviates from the standard. The user 53 can also easily confirm the time at which the abnormality occurred.

[0080] The information processing device 1 waits for the acceptance of the designation of the highlighted portion in the first graph 65 or the second graph (S306). The user 53 operates the operation unit 34 to designate the highlighted portion in the first graph 65 or the second graph. For example, the highlighted portion 651 or 661 is designated by placing a cursor on the highlighted portion 651 or 661. In S24, the calculation unit 31 accepts the designation of the highlighted portion 651 or 661 by the user 53 operating the operation unit 34, and causes the communication unit 36 ​​to transmit information indicating that the designation of the highlighted portion 651 or 661 has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of the highlighted portion has not been accepted (S306: NO), the information processing device 1 advances the process to S310 described later.

[0081] When the designation of the highlighted portion is accepted (S306: YES), the information processing device 1 causes the terminal device 3 to display details of the state in which the subject 51 has an abnormal heart rate or respiration rate per unit time (S307). At this time, the information processing device 1 further causes the terminal device 3 to display the set reference value of the heart rate or respiration rate and a link to the captured image. FIG. 14 is a schematic diagram showing a first example of displaying details of the state in which the subject 51 has an abnormal heart rate per unit time. The highlighted portion 651 is designated by placing a cursor 652 on the highlighted portion 651 in the first graph 65, and a display field 653 of the details of the abnormality is displayed. In the display field 653, the time corresponding to the position of the cursor 652 and the value of the heart rate at that time are displayed as details of the state in which the heart rate is abnormal.

[0082] In the display field 653, the set value on which the abnormality is notified, that is, the preset reference value of the heart rate or respiration rate, is displayed. FIG. 14 shows an example in which the heart rate exceeds the upper limit, and the set upper limit value of the heart rate is displayed as the set value for notifying the abnormality. By displaying the reference value of the heart rate or respiration rate, the user 53 can confirm how much the abnormal heart rate or respiration rate deviates from the reference. In addition, a camera icon is displayed in the display field 653. The camera icon is a link to a photographed image of the subject 51 at the time corresponding to the specified highlighted portion 651. The link to the photographed image may be in a form other than the camera icon, such as a link made of text.

[0083] FIG. 15 is a schematic diagram showing a second example of displaying details of an abnormality in the subject 51's heart rate per unit time. The cursor 662 overlaps with an emphasis portion 661 in the second graph 66, thereby specifying the emphasis portion 661, and a display field 663 of the abnormality details is displayed. In the display field 663, the time corresponding to the position of the cursor 662 and the value of the heart rate at that time are displayed as details of the abnormality in the heart rate. In addition, the display field 663 displays the setting value that is the basis for notifying the abnormality. Specifically, the upper limit of the heart rate that is set is displayed. Furthermore, the display field 663 displays a camera icon that is a link to a captured image. Even if the subject 51's respiration rate per unit time is abnormal, details of the abnormality in the respiration rate are displayed in the same manner as in FIG. 14 or FIG. 15.

[0084] In addition, along with details of the state where the heart rate or respiration rate is abnormal, only one of the set value for notifying the abnormality and the reference value of the heart rate or respiration rate may be displayed. Alternatively, only details of the state where the heart rate or respiration rate is abnormal may be displayed, without displaying the set value for notifying the abnormality and the reference value of the heart rate or respiration rate.

[0085] In S307, the calculation unit 11 corrects the first graph 65 or the second graph 66 based on the data obtained from the sensor device 2 and the setting data for the selected subject 51. At this time, the calculation unit 11 corrects the first graph 65 or the second graph 66 so as to add details of the state in which the heart rate or respiration rate is abnormal, the set reference value of the heart rate or respiration rate, and a link to the captured image. The calculation unit 11 causes the communication unit 15 to transmit the corrected first graph 65 or the second graph 66 to the terminal device 3. The terminal device 3 receives the corrected first graph 65 or the second graph 66 via the communication unit 36, and the calculation unit 31 displays the corrected first graph 65 or the second graph 66 on the display unit 35. The information processing device 1 may transmit information including the time and the value of the heart rate or the respiration rate to the terminal device 3, and the terminal device 3 may correct the first graph 65 or the second graph 66 based on the received information. Through the processing of S306 to S307, the user 53 can specify the highlighted portion 651 or 661 and check the details of the condition in which the subject 51 has an abnormality in the heart rate per unit time.

[0086] The information processing device 1 waits for the acceptance of the designation of a link to a photographed image (S308). With the display field 653 or 663 displayed, the user 53 operates the operation unit 34 to designate a link to a photographed image. For example, the link to a photographed image is designated by hovering a cursor over a camera icon displayed in the display field 653 or 663. In S308, the calculation unit 31 accepts the designation of a link to a photographed image by the user 53 operating the operation unit 34, and causes the communication unit 36 ​​to transmit information indicating that the designation of a link to a photographed image has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of a link to a photographed image has not been accepted (S308: NO), the information processing device 1 advances the process to S310 described later.

[0087] When the designation of the link to the photographed image is accepted (S308: YES), the information processing device 1 displays the photographed image of the subject 51 on the terminal device 3 (S309). In S309, the calculation unit 11 transmits a request for a photographed image of the subject 51 at a time corresponding to the designated highlighted portion 651 or 661 to the camera device 7. The camera device 7 selects the requested photographed image from among the photographed images stored therein, and transmits the selected photographed image to the information processing device 1. The information processing device 1 receives the photographed image transmitted from the camera device 7 via the communication unit 15, and the calculation unit 11 causes the communication unit 15 to transmit the received photographed image to the terminal device 3. In a form in which the photographed image is stored in the information processing device 1, the information processing device 1 selects the requested photographed image from among the photographed images stored therein, and transmits the selected photographed image to the terminal device 3. The terminal device 3 receives the photographed image via the communication unit 36, and the calculation unit 31 displays the photographed image on the display unit 35.

[0088] 16 is a schematic diagram showing a display example of a captured image. A captured image of a subject 51 captured by a camera device 7 is displayed. More specifically, a captured image of the subject 51 when there is an abnormality in the heart rate or respiratory rate per unit time is displayed. A user 53 can check the state of the subject 51 when there is an abnormality in the heart rate or respiratory rate.

[0089] The monitoring system 100 may be configured such that the terminal device 3 and the camera device 7 communicate with each other in a peer-to-peer manner, and the captured image is displayed on the terminal device 3 without the intervention of the information processing device 1. In this configuration, in S308, the terminal device 3 waits for acceptance of a designation of a link to the captured image. When the designation of a link to the captured image is accepted, the calculation unit 31 transmits to the camera device 7 a request for a captured image of the subject 51 at the time corresponding to the designated highlighted portion 651 or 661. In S309, the camera device 7 transmits the requested captured image to the terminal device 3, the terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

[0090] The monitoring system 100 may perform a process of displaying the heart rate or respiratory state of the subject 51 even when a portion other than the highlighted portion of the first graph 65 or the second graph 66 is specified. For example, when a portion other than the highlighted portion of the first graph 65 or the second graph 66 is specified, a value of the heart rate or respiratory rate or a link to a photographed image may be displayed. For example, when a link to a photographed image is specified, a photographed image of the subject 51 at a time corresponding to the portion specified in the first graph 65 or the second graph 66 may be displayed.

[0091] The information processing device 1 waits for the reception of a setting instruction as to whether or not to display the condition history of the subject 51 (S310). In S310, the user 53 who visually recognizes the condition history 67 operates the operation unit 34, and the terminal device 3 receives the setting instruction. For example, as shown in FIG. 13, the setting field 671 includes a check box for each condition category, and the user 53 uses the check box to input a setting instruction as to whether or not to display the condition history included in each condition category. The terminal device 3 transmits the setting instruction to the information processing device 1, and the information processing device 1 receives the setting instruction via the communication unit 15. By receiving the setting instruction, the calculation unit 11 accepts the setting instruction. If the setting instruction is not accepted (S310: NO), the information processing device 1 advances the process to S315 described later.

[0092] When the setting instruction is received (S310: YES), the information processing device 1 performs setting according to the received instruction (S311). In S311, the calculation unit 11 performs setting by recording in the setting data whether or not to display the history of the state included in each state category according to the received instruction. The information processing device 1 then adjusts the state history 67 according to the setting (S312). In S312, the calculation unit 11 adjusts the state history 67 so as to display the history of the state set to be displayed among the multiple types of states of the subject 51, and not to display the history of the state set to not be displayed. The calculation unit 11 causes the communication unit 15 to transmit the adjusted state history 67 to the terminal device 3, the terminal device 3 receives the adjusted state history 67 via the communication unit 36, and the calculation unit 31 displays the adjusted state history 67 on the display unit 35. The information processing device 1 may generate an image including the adjusted state history 67 and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 may transmit data necessary for generating the adjusted state history 67 to the terminal device 3, and the terminal device 3 may generate the state history 67 based on the received data and display it on the display unit 35.

[0093] Fig. 17 is a schematic diagram showing a first example of an adjusted state history 67. Fig. 17 shows a state history 67 that is set to display a state history related to the behavior of the subject 51 and a state history according to the heart rate or respiratory rate of the subject 51 per unit time, and not to display a state history related to the sleep of the subject 51. The description "heart rate / respiration" included in the setting field 671 indicates a state according to the heart rate or respiratory rate of the subject 51 per unit time. The state history 67 includes a state history related to the behavior of the subject 51 and a state history according to the heart rate or respiratory rate of the subject 51 per unit time, and does not include other state histories.

[0094] Fig. 18 is a schematic diagram showing a second example of the adjusted state history 67. Fig. 18 shows the state history 67 that is set to display state history according to the heart rate or respiratory rate per unit time of the subject 51, and not to display state history related to the sleep of the subject 51 and state history related to the behavior of the subject 51. The state history 67 includes state history according to the heart rate or respiratory rate per unit time of the subject 51, and does not include other state history.

[0095] As shown in Figures 17 and 18, a status history 67 is displayed that includes the history of some types of statuses among the multiple types of statuses of the subject 51, but does not include the history of other types of statuses. If the history of all types of statuses were included in the status history 67, information with too much diversity would be displayed, and the visibility of the information would be impaired. By displaying the history of some types of statuses and not displaying the history of other types of statuses, the diversity of the displayed information is reduced and the visibility of the information is improved. By limiting the status categories that include the statuses of the subject 51 to be displayed, the user 53 can improve the visibility of the required information and easily obtain the required information.

[0096] A state according to the subject 51's heart rate or respiratory rate per unit time has a larger impact on the subject 51 and is more important than other states. As shown in FIG. 17, by reducing the types of states of the subject 51 that are displayed, the user 53 can more easily check the history of states with high importance. Also, as shown in FIG. 18, when the history of states according to the subject 51's heart rate or respiratory rate per unit time is displayed and the history of other states is not displayed, the visibility of information with high importance is improved. The user 53 can easily check the history of states with high importance.

[0097] The information processing device 1 determines whether the setting has been changed so that the history of the state according to the heart rate or respiration rate per unit time of the subject 51 is not displayed (S313). In S313, the calculation unit 11 makes a determination based on the setting data. If the setting has not been changed so that the history of the state according to the heart rate or respiration rate is not displayed (S313: NO), the information processing device 1 proceeds to S315 described later.

[0098] When the setting is changed so that the history of the state according to the heart rate or the respiratory rate is not displayed (S313: YES), the information processing device 1 stops highlighting the first graph 65 and the second graph 66 (S314). In S314, the calculation unit 11 adjusts the first graph 65 and the second graph 66 so that the first graph 65 and the second graph 66 are not highlighted. The calculation unit 11 causes the communication unit 15 to transmit the adjusted first graph 65 and second graph 66 to the terminal device 3. The terminal device 3 receives the adjusted first graph 65 and second graph 66, and displays them on the display unit 35.

[0099] FIG. 19 is a schematic diagram showing an example of a first graph 65, a second graph 66, and a state history 67 in which highlighting has been stopped. The state history 67 is set to display a state history related to the sleep of the subject 51 and a state history related to the behavior of the subject 51, and not to display a state history according to the heart rate or respiration rate per unit time of the subject 51. The state history 67 does not include a state history according to the heart rate or respiration rate per unit time of the subject 51. The first graph 65 does not include the highlighted portion 651, and is similar to the other portions in that it shows a heart rate or respiration rate that is out of the standard. The second graph 66 does not include the highlighted portion 661.

[0100] In this way, when the information processing device 1 does not display the history of the state according to the heart rate or the respiratory rate, it does not highlight the first graph 65 and the second graph 66. When the history of the state according to the heart rate or the respiratory rate is not displayed, there is no need to pay attention to the state according to the heart rate or the respiratory rate. Since information that does not require attention is not highlighted, it becomes easier for the user 53 to check other information that requires attention.

[0101] The information processing device 1 waits for the acceptance of the designation of a link to a photographed image associated with the state of the subject 51 included in the state history 67 (S315). The user 53 designates a link to a photographed image associated with the state of the subject 51 by operating the operation unit 34. For example, the designation of a link to a photographed image is performed by hovering the cursor over one of the camera icons associated with the state of the subject 51 at each time included in the state history 67. In S315, the calculation unit 31 accepts the designation of a link to a photographed image by the user 53 operating the operation unit 34, and transmits information indicating that the designation of a link to a photographed image has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of a link to a photographed image has not been accepted (S315: NO), the information processing device 1 ends the process of displaying details of the state of the subject 51.

[0102] When the designation of the link to the captured image is accepted (S315: YES), the information processing device 1 causes the terminal device 3 to display the captured image of the subject 51 (S316). In S316, the calculation unit 11 transmits a request for a captured image of the subject 51 at the time associated with the state of the subject 51 associated with the designated link to the camera device 7, and the camera device 7 transmits the requested captured image to the information processing device 1. The information processing device 1 receives the captured image, and the calculation unit 11 causes the communication unit 15 to transmit the captured image to the terminal device 3. In a form in which the captured image is stored in the information processing device 1, the information processing device 1 transmits the stored captured image to the terminal device 3. The terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

[0103] The monitoring system 100 may be configured such that the terminal device 3 and the camera device 7 communicate with each other in a peer-to-peer manner, and the captured image is displayed on the terminal device 3 without the intervention of the information processing device 1. In this configuration, in S315, the terminal device 3 waits for the acceptance of a designation of a link to a captured image. When the designation of a link to a captured image is accepted, the calculation unit 31 transmits to the camera device 7 a request for a captured image of the subject 51 at the time associated with the state of the subject 51 associated with the designated link. In S316, the camera device 7 transmits the requested captured image to the terminal device 3, the terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

[0104] Similar to the example shown in Fig. 16, a captured image is displayed. A captured image of the subject 51 captured by the camera device 7 when the subject 51 reached each of the states displayed in the status history 67 is displayed. When the subject 51 is absent, a captured image showing the bed 52 or the room used by the subject 51 may be displayed. The user 53 can check the state of the subject 51 when the subject reached each of the states displayed in the status history 67. After S316 is completed, the information processing device 1 ends the process of displaying details of the subject 51's status.

[0105] As described above in detail, in this embodiment, the information processing device 1 causes the terminal device 3 to display the status image 63 indicating the status according to the heart rate or respiratory rate per unit time of the subject 51 in a manner different from that of the status image 61 indicating other statuses of the subject 51. Meanwhile, the information processing device 1 causes the terminal device 3 to display the status history 67, thereby displaying the history of the status according to the heart rate or respiratory rate per unit time of the subject 51 and the history of other statuses of the subject 51 in the same manner. In a scene where a status image indicating the status of the subject 51 is displayed in real time, the visibility of information with a high degree of urgency of notification is improved, whereas in a scene where the status history 67 is displayed, since there is no urgency, the user 53 can equally confirm multiple types of statuses of the subject 51.

[0106] In the present embodiment, an example is shown in which a standard is set for both a state according to the heart rate per unit time of the subject 51 and a state according to the respiratory rate per unit time of the subject 51, and a status image 63, a first graph 65, and a status history 67 are displayed. The information processing device 1 may be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for only one of a state according to the heart rate per unit time of the subject 51 and a state according to the respiratory rate per unit time of the subject 51. The information processing device 1 may also be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for a specific type of numerical value other than the heart rate and respiratory rate of the subject 51 based on data acquired using the sensor device 2. For example, the information processing device 1 may be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for the pulse rate or body temperature of the subject 51.

[0107] In the present embodiment, an example has been shown in which the state of the subject 51 regarding the behavior, the state of the subject 51 regarding the sleep, and the state according to the heart rate or respiratory rate of the subject 51 are determined, and a status image is displayed. The monitoring system 100 may be configured to determine other types of states of the subject 51 and display a status image. In the present embodiment, a configuration has been shown in which the information processing device 1 causes the terminal device 3 to display various information by the processes of S11 to S13, S21 to S25, and S301 to S316. The information processing device 1 may be provided with a display unit and may display various information by itself. Alternatively, the monitoring system 100 may be configured in such a way that the terminal device 3 executes part or all of the processes of the information processing device 1 described above. In this configuration, the calculation unit 31 executes processes corresponding to part or all of the processes of the information processing device 1 according to the computer program 331.

[0108] The present invention is not limited to the contents of the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

[0109] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that cites at least one multiple claim may also be used. [Explanation of symbols]

[0110] 100 Monitoring System 1. Information processing device 10 Recording media 11 Arithmetic section 13 Storage section 131 Computer Programs 2. Sensor device 3 Terminal Equipment 31 Arithmetic section 33 Storage section 331 Computer Programs 4. Communication Network 51 Target 53 User 61, 63 Status images 65 First Graph 66 Second Graph 67 Status History

Claims

1. Data is collected from each subject using sensors, A status image showing the status of each subject based on the acquired data is displayed for each subject; receiving instructions for setting, for each subject, a specific type of numerical criterion based on data acquired using the sensor; Follow the instructions to set up the device. When the specific type of numerical value based on the acquired data falls outside a set standard, an image showing a specific abnormality is displayed as the status image; When an arbitrary subject is selected, at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the state of the selected subject based on data acquired at each time for the selected subject is displayed; At least one of a portion in the first graph showing a specific type of numerical value at a time when the specific type of numerical value deviates from a set standard and a portion in the second graph showing a time when the specific type of numerical value deviates from a set standard is highlighted. A computer program that causes a computer to execute a process.

2. The particular type of numerical value is the subject's heart rate or respiratory rate per unit time; the status image indicates a status of the subject according to a heart rate or a respiratory rate per unit time, The specific type of numerical standard is a threshold value of the subject's heart rate or respiratory rate per unit time; the first graph shows a heart rate or a respiratory rate per unit time based on data acquired at each time point for a selected subject; In the first graph, a portion showing a heart rate or a respiratory rate per unit time that deviates from a set standard is highlighted.

2. The computer program product according to claim 1, which causes a computer to execute a process.

3. The state of the subject shown in the state image includes a state related to the subject's behavior and a state according to the subject's heart rate or respiratory rate per unit time, The standard for the specific type of value is determined by a threshold value of the subject's heart rate or respiratory rate per unit time, the second graph shows a behavioral state of the selected subject at each time point; In the second graph, a portion showing a time when the heart rate or respiration rate per unit time is out of the set standard is highlighted.

2. The computer program product according to claim 1, which causes a computer to execute a process.

4. receiving an instruction for setting, for each subject, whether or not to display an image showing the specific abnormality as the status image in accordance with the specific type of numerical value; Follow the instructions to set up the device. Only when an image showing the specific abnormality is set to be displayed as the status image, a portion in the first graph showing the specific type of numerical value at the time when the numerical value deviates from the set standard, or a portion in the second graph showing the time when the specific type of numerical value deviates from the set standard is highlighted.

2. The computer program product according to claim 1, which causes a computer to execute a process.

5. In the first graph, a standard of the specific type of numerical value set at each time is displayed.

2. The computer program product according to claim 1, which causes a computer to execute a process.

6. When a designation of a highlighted portion in the first or second graph is accepted, details of the specific abnormality are displayed.

2. The computer program product according to claim 1, which causes a computer to execute a process.

7. When a designation of a highlighted portion in the first or second graph is accepted, a reference value of the specific type of numerical value that has been set is displayed.

7. A computer program according to claim 6, which causes a computer to execute a process.

8. when a designation of a highlighted portion in the first or second graph is accepted, a link to a photographed image of the subject at a time corresponding to the designated portion is displayed; When the link is specified, the image is displayed.

7. A computer program according to claim 6, which causes a computer to execute a process.

9. When an arbitrary target person is selected, a history is displayed that associates the selected target person's status with the time when the target person became in the status.

2. The computer program product according to claim 1, which causes a computer to execute a process.

10. Accepting an instruction to set whether to display, for each subject, a history of each of a plurality of types of status of the subject; Follow the instructions to set up the device. If you select "display" among multiple types of status of the target user, the history of the set status will be displayed. If you select "display" instead, the history of the set status will not be displayed.

10. The computer program product according to claim 9, which causes a computer to execute a process.

11. receiving an instruction for setting whether or not to display a history of the subject's condition included in each status category categorized to include one or more types of status among a plurality of types of status of the subject; Configure according to the received instructions 11. A computer program product according to claim 10, which causes a computer to execute a process.

12. the state of the subject represented by the state image includes a state related to the subject's behavior and a state related to the subject's sleep, displaying, as the status images, an image showing a behavioral status of the subject and an image showing a sleep status of the subject in a first display area provided for each subject; As the status image, an image showing the specific abnormality is displayed superimposed on the image displayed in the first display area.

2. The computer program product according to claim 1, which causes a computer to execute a process.

13. The image showing the specific abnormality is displayed in an area smaller than the image showing the subject's behavioral state and the image showing the subject's sleep state.

13. A computer program product according to claim 12, which causes a computer to execute a process.

14. The image showing the specific abnormality is displayed as an image having at least a part thereof transparent so that the image displayed in the first display area can be visually recognized.

13. A computer program product according to claim 12, which causes a computer to execute a process.

15. the state of the subject represented by the state image includes a state related to the subject's behavior and a state related to the subject's sleep, displaying, as the status images, an image showing a behavioral status of the subject and an image showing a sleep status of the subject in a first display area provided for each subject; As the status image, an image showing the specific abnormality is popped up and displayed at a position independent of the subject.

2. The computer program product according to claim 1, which causes a computer to execute a process.

16. the state of the subject represented by the state image includes a state related to the subject's behavior and a state related to the subject's sleep, displaying, as the status images, an image showing a behavioral status of the subject and an image showing a sleep status of the subject in a first display area provided for each subject; As the condition image, an image showing the specific abnormality is displayed in a second display area provided for each subject, which is different from the first display area.

2. The computer program product according to claim 1, which causes a computer to execute a process.

17. Data is collected from each subject using sensors, A status image showing the status of each subject based on the acquired data is displayed for each subject; receiving instructions for setting, for each subject, a specific type of numerical criterion based on data acquired using the sensor; Follow the instructions to set up the device. When the specific type of numerical value based on the acquired data falls outside a set standard, an image showing a specific abnormality is displayed as the status image; When an arbitrary subject is selected, at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the state of the selected subject based on data acquired at each time for the selected subject is displayed; At least one of a portion in the first graph showing a specific type of numerical value at a time when the specific type of numerical value deviates from a set standard and a portion in the second graph showing a time when the specific type of numerical value deviates from a set standard is highlighted.

23. An information processing method comprising:

18. A calculation unit is provided, The calculation unit is Data is collected from each subject using sensors, A status image showing the status of the subject based on the acquired data is displayed on a display unit for each subject; receiving instructions for setting, for each subject, a specific type of numerical criterion based on data acquired using the sensor; Follow the instructions to set up the device. When the specific type of numerical value based on the acquired data falls outside a set standard, an image showing a specific abnormality is displayed on the display unit as the status image; When an arbitrary subject is selected, at least one of a first graph showing the specific type of numerical value based on data acquired at each time for the selected subject and a second graph showing the state of the selected subject based on data acquired at each time for the selected subject is displayed on the display unit; At least one of a portion in the first graph showing a specific type of numerical value at a time when the specific type of numerical value deviates from a set standard and a portion in the second graph showing a time when the specific type of numerical value deviates from a set standard is highlighted on the display unit.

23. An information processing apparatus comprising:

Images