Display device, display system, display method, and display program

Abstract

To provide a display device, a display system, a display method, and a display program which allow a caring staff to analyze a state of a cared person in detail.SOLUTION: A display device includes a time series data generation section 313, a map generation section 315, and a display section 33. The time series data generation section 313 generates time series data in which indexes of walking of a cared person 70 are arranged in time series order. The map generation section 315 generates a behavioral map of the cared person 70. The display section 33 displays the time series data and the behavioral map of the cared person 70, which corresponds to the time series data.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a display device, a display system, a display method, and a display program.

Background Art

[0002] In Japan, due to the improvement of living standards, the improvement of the sanitary environment, and the improvement of the medical level accompanying the high economic growth after the war, the aging population has become remarkable. For this reason, combined with the decline in the birth rate, it has become an aging society with a high aging rate. In such an aging society, an increase in care recipients and nursing recipients (hereinafter referred to as "care target persons") who require care, nursing, etc. due to illness, injury, aging, etc. is assumed. In facilities such as hospitals and elderly welfare facilities (hereinafter simply referred to as "facilities"), care for care target persons is carried out by care staff such as nurses, nurses, physical therapists, and occupational therapists (hereinafter referred to as "care staff").

[0003] In relation to this, Patent Document 1 below discloses a rehabilitation management device that analyzes exercise information based on rehabilitation exercises performed by patients and manages the entire history of the exercise functions of multiple patients.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the rehabilitation management device of Patent Document 1 has a function of displaying a graph of a patient's walking speed over time, but there is a problem that medical staff cannot analyze the patient's condition in detail from the displayed graph over time.

[0006] This invention has been made in view of the above-mentioned problems. Accordingly, the object of this invention is to provide a display device, a display system, a display method, and a display program that enable care staff to analyze in detail the walking condition of a person receiving care. [Means for solving the problem]

[0007] The above-mentioned problems of the present invention are solved by the following means.

[0008] (1) A time-series data generation unit that generates time-series data of walking indicators of the person receiving care arranged in time series, a map generation unit that generates a behavior map of the person receiving care, and the time-series data and, Action map of the care recipient corresponding to the aforementioned time-series data and at the same time A display device having a display unit for displaying information.

[0009] (2) The display device according to (1) above, further comprising: an acquisition unit that acquires a plurality of images of the care recipient and the date and time the plurality of images were taken; and an index calculation unit that calculates an index related to the care recipient's walking based on the plurality of images and the date and time the images were taken, wherein the time series data generation unit generates time series data in which the index is arranged in time series for a predetermined time period including the date and time the images were taken; the map generation unit visualizes the care recipient's stay time for each region within the field of view of the plurality of images and generates a heat map of the care recipient's stay time for each predetermined time period as the behavior map; and the display unit displays the time series data and displays a heat map corresponding to the predetermined time period of the time series data.

[0010] (3) The display device according to (2) above, wherein the time-series data generation unit generates the time-series data for the daytime and / or nighttime periods as the predetermined time period.

[0011] (4) The display unit is the display device described in (2) above, which displays the time-series data and the heat map for each of the daytime and nighttime periods as predetermined time periods.

[0012] (5) The above-mentioned multiple images are moving images. 2 ) ~ (4) listed in any one of the above Display device.

[0013] (6) The time-series data is a graph in which the indicators are arranged in time series, as described in any one of (1) to (5) above, the display device.

[0014] (7) A display device according to any one of (2) to (4) above, further comprising a receiving unit that accepts the selection of a day or time as the unit for generating the time series data, and the selection of the indicator, wherein the display unit displays the time series data relating to the selected indicator in the selected unit, and also displays a heat map corresponding to the predetermined time period of the time series data.

[0015] (8) The above indicators include walking speed, unsteadiness, distance traveled, number of stops, and duration of stops. This is the number of times the person receiving care returned to their original area immediately after being moved. A display device according to any one of (1) to (7) above, which is either the number of repetitions or the activity time.

[0016] (9) A display system comprising an imaging unit for photographing the interior of the room of the person receiving care, and a display device as described in any one of (1) to (8) above.

[0017] (10) Computers Step (a) generates time-series data in which indicators related to the walking of the person receiving care are arranged in time series, Computers (b) A step of generating a behavior map of the person receiving care, Computers The aforementioned time series data and, Action map of the care recipient corresponding to the aforementioned time-series data and at the same time A method of display, including step (c) of displaying.

[0018] (11) Computers Before step (a), a step (d) of acquiring a plurality of images of the care recipient and the shooting date and time of the plurality of images; Computers Based on the plurality of images and the shooting date and time, further including a step (e) of calculating an index related to the walking of the care recipient, in the step (a), for a predetermined time period including the shooting date and time, generating time-series data in which the indices are arranged in time series, in the step (b), visualizing the staying time of the care recipient in each area within the visual field of the image, and generating a heat map of the staying time of the care recipient for each predetermined time period as the action map, in the step (c), displaying the time-series data and displaying a heat map corresponding to the predetermined time period of the time-series data. The display method according to (10) above.

[0019] (12) A display program for causing a computer to execute the processing included in the display method according to (10) or (11) above.

Effect of the Invention

[0020] According to the present invention, since time-series data in which indices related to the walking of the care recipient are arranged in time series is displayed, and an action map corresponding to a predetermined time period of the time-series data is displayed, it becomes possible for the care staff to analyze the walking state of the care recipient in detail.

Brief Description of the Drawings

[0021] [Figure 1] It is a diagram illustrating the overall configuration of a monitoring system according to an embodiment. [Figure 2] It is a schematic diagram illustrating a detection unit installed around the bed in the living room of the care recipient. [Figure 3] It is a block diagram illustrating the schematic configuration of the detection unit shown in FIG. 1. [Figure 4] It is a block diagram illustrating the schematic configuration of the management server shown in FIG. 1. [Figure 5] Figure 1 is a block diagram illustrating the schematic configuration of the information administrator terminal. [Figure 6] Figure 5 is a functional block diagram illustrating some of the functions of the control unit shown. [Figure 7] Figure 1 is a block diagram illustrating the schematic configuration of the care staff terminal. [Figure 8] This is a flowchart illustrating the processing procedure of a display method according to one embodiment. [Figure 9] This is a schematic diagram illustrating the screen of software used to visualize and analyze behavioral data of individuals receiving care. [Modes for carrying out the invention]

[0022] Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant explanations are omitted. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0023] Figure 1 is a diagram illustrating the overall configuration of a monitoring system according to one embodiment, and Figure 2 is a schematic diagram illustrating a detection unit installed around the bed in the living room of the person receiving care.

[0024] (Monitoring System 1) As shown in Figure 1, the monitoring system 1 functions as a display system and includes multiple detection units 10, a management server 20, an information administrator terminal 30, and one or more care staff terminals 40. These are connected to each other via a network 50, such as a LAN (Local Area Network), telephone network, or data communication network, by wire or wireless means, enabling communication between them. The network 50 may include relay devices such as repeaters, bridges, routers, or cross-connects that relay communication signals. In the example shown in Figure 1, the detection units 10, the management server 20, the information administrator terminal 30, and the care staff terminals 40 are connected to each other via a network 50, such as a wireless LAN (for example, a LAN conforming to the IEEE 802.11 standard), which includes an access point 51.

[0025] The monitoring system 1 is installed in appropriate locations depending on the care recipient 70. The care recipient 70 are, for example, patients who require nursing care due to illness or injury, elderly persons who require care due to declining physical abilities, or elderly persons living alone. In particular, from the perspective of enabling early detection and early intervention, the care recipient 70 may be persons who need to be detected if a predetermined undesirable event, such as an abnormal condition, occurs to them. For this reason, the monitoring system 1 is suitably installed in buildings such as hospitals, elderly care facilities, and residential units, depending on the type of care recipient 70. In the example shown in Figure 1, the monitoring system 1 is installed in a facility that has multiple rooms (living rooms) where multiple care recipients 70 reside, as well as multiple rooms including a care station.

[0026] The detection unit 10 is placed in each of the rooms of the care recipients 70, which are the observation areas. In the example shown in Figure 1, four detection units 10 are placed in the rooms of care recipients 70, namely A, B, C, and D. As shown in Figure 2, the observation area of ​​the detection unit 10 includes the bed 60. Care staff 80 who provide nursing or care to the care recipients 70 each carry a care staff terminal 40, which is a portable terminal. However, the location and number of each component of the monitoring system 1 are not limited to the example shown in Figure 1. For example, the management server 20 does not have to be located in the care station, but may be an external server unit connected to the network 50.

[0027] Furthermore, "care" broadly includes the care provided by 80 care staff to 70 care recipients, such as assistance with daily living activities like meals, fluid intake, excretion, bathing, and cleaning, as well as vital sign checks (measurement of temperature, blood pressure, pulse, etc.). The 80 care staff members are those who provide various forms of support (care) to the 70 care recipients according to their duties. Duties may include medical and nursing care services.

[0028] (Detection unit 10) Figure 3 is a block diagram illustrating the schematic configuration of the detection unit. As shown in the figure, the detection unit 10 comprises a control unit 11, a communication unit 12, a camera 13, a care call unit 14, and an audio input / output unit 15, which are interconnected by a bus.

[0029] The control unit 11 is composed of a CPU (Central Processing Unit) and memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and performs control and calculation processing of each part of the detection unit 10 according to the program. The control unit 11 may also be equipped with an HDD (Hard Disk Drive) or SSD (Solid State Drive) as memory.

[0030] The communication unit 12 is an interface circuit (e.g., a LAN card) for communicating with other devices via the network 50, such as the management server 20, the information administrator terminal 30, or the care staff terminal 40.

[0031] Camera 13 is positioned, for example, on the ceiling or upper part of the wall of a living room, and captures images of the bed 60 of the care recipient 70 directly below it as the observation area, outputting captured images (image data). When the care recipient 70 is in the living room, the care recipient 70 is captured from above by camera 13. These captured images include still images and moving images. Hereafter, captured images will be described as moving images at 1fps, or individual frames of such moving images. Camera 13 is a near-infrared camera, but a visible light camera may be used instead, or both may be used in combination. The captured images are transmitted to the management server 20 along with information about the date and time they were captured. Camera 13 functions as an imaging unit.

[0032] The care call unit 14 includes a push-button switch, and detects a care call (nurse call) when the switch is pressed by the person receiving care 70. Instead of a push-button switch, a voice microphone may be used to detect the care call. When the switch of the care call unit 14 is pressed, that is, when a care call is detected, the control unit 11 sends a notification that a care call has been made to the management server 20 or the like via the communication unit 12 and the network 50.

[0033] The audio input / output unit 15 is, for example, a speaker and a microphone, and enables voice communication by sending and receiving audio signals to and from the information administrator terminal 30 via the communication unit 12. The audio input / output unit 15 may also be connected to the detection unit 10 via the communication unit 12 as an external device of the detection unit 10.

[0034] (Management Server 20) Figure 4 is a block diagram showing the schematic configuration of the management server. The management server 20 comprises a control unit 21, a communication unit 22, and a database 23. This management server 20 may be located in the same building as the living quarters of the care recipients 70, or it may be located in a remote location and accessible via a network. For example, the management server 20 may be a cloud server virtually constructed by multiple servers located on a network such as the Internet. Each component is connected to the others via a bus, enabling them to communicate with each other.

[0035] Since the control unit 21 and the communication unit 22 have the same functions as the components of the detection unit 10, a detailed explanation will be omitted.

[0036] The database 23 stores images captured by the camera 13 of the detection unit 10, an event list, information about the care recipient 70, information about the care staff 80, care records, and the like.

[0037] The event list includes information on various events, including designated events that could lead to accidents involving the care recipient 70 (e.g., falls and slips). Information on the care recipient 70 includes, for example, the care recipient 70's name, date of birth, ID number, room number, usual food intake, fluid intake other than food, medical history, level of care required, and current medical visit status. Information on the care staff 80 includes, for example, the care staff 80's name, staff ID, and date of birth.

[0038] A care record is a document that records information about the care provided to the facility's care recipients 70. For example, care staff 80 provide various daily living assistance to each care recipient 70, such as meals, fluid intake, excretion, bathing, and cleaning, as well as vital sign checks (measurement of temperature, blood pressure, pulse, etc.), based on a care plan prepared for each care recipient 70. The assistance and vital sign checks performed by care staff 80 to care recipient 70, as well as the care recipient 70's behavior and condition, are recorded in the care record. The care record is updated when care staff 80 input information about the care provided to care recipient 70. Input into the care record is done, for example, through the input unit 44 of the care staff terminal 40.

[0039] The management server 20, either independently or in cooperation with the detection unit 10, determines (identifies) which care recipient 70 an event detected by the detection unit 10, such as getting up, leaving bed, falling, or making a care call. This determination is made by identifying the care recipient 70 (i.e., the occupant of the room) associated with the room number where the detection unit 10 that detected the event is installed. The determined event type and the care recipient 70 are then associated and added to the event list in the database 23. In this embodiment, if the care recipient 70 is carrying an IC tag, the care recipient 70 may be identified by reading the IC tag with an RFID reader installed in each room. In cases where there are multiple care recipients 70 in one room, such as in a shared room, the care recipients 70 may be identified by placing a detection unit 10 for each bed 60.

[0040] (Information administrator terminal 30) Figure 5 is a block diagram showing the schematic configuration of the information administrator terminal, and Figure 6 is a functional block diagram illustrating some of the functions of the control unit shown in Figure 5. The information administrator terminal 30 is a so-called PC (Personal Computer) and functions as a display device. The information administrator terminal 30 comprises a control unit 31, a communication unit 32, a display unit 33, an input unit 34, and an audio input / output unit 35, which are interconnected by a bus. The control unit 31 has a configuration similar to the control unit 11 of the detection unit 10, and includes a CPU, RAM, ROM, etc.

[0041] As shown in Figure 6, the control unit 31 functions as an acquisition unit 311, an index calculation unit 312, a time-series data generation unit 313, a reception unit 314, and a heatmap generation unit 315. These functions are realized by the CPU executing a display program.

[0042] The acquisition unit 311, in cooperation with the communication unit 32, acquires multiple images captured by the camera 13 and information regarding the date and time the images were taken from the database 23 of the management server 20.

[0043] The index calculation unit 312 calculates an index related to the walking of the person receiving care 70 based on the captured image and the date and time of capture. For example, the index calculation unit 312 calculates the movement trajectory of the person receiving care 70 in the room from the captured image, and calculates an index related to the walking of the person receiving care 70 (hereinafter also simply referred to as the "index") based on the movement trajectory and the date and time of capture of the image.

[0044] The movement trajectory of the 70 care recipients can be calculated, for example, by the following method: From the captured images, human silhouettes (hereinafter referred to as "human silhouettes") are detected using background subtraction or time subtraction, and the rectangular region containing the human silhouette (hereinafter referred to as "human rectangle") is detected for each frame of the captured images. The human rectangle may be detected using a neural network trained with training data of combinations of captured images and the correct labels for the human rectangle.

[0045] Next, a tracking process is performed to track the movement of the person rectangle detected from each frame of the captured image. The tracking process can employ known methods, such as determining that person rectangles that overlap in multiple adjacent frames in a time series belong to the same person. By performing the tracking process, even if multiple person rectangles are detected in adjacent frames in the time series of the captured image, it is possible to associate the person rectangles of each of the multiple person rectangles in each frame with those of the same person.

[0046] For example, the movement trajectory of the person being cared for 70 can be calculated as the movement of the center of the associated person rectangle across multiple frames. Alternatively, the system may be configured to calculate the movement trajectory of the person being cared for 70 as the movement of the center of gravity of the person silhouette across multiple frames. The calculation of the movement trajectory may be performed by a program processed by the CPU of the control unit 31, or by an embedded processing circuit.

[0047] Indicators related to walking for the 70 care recipients may include, but are not limited to, walking speed (m / min), unsteadiness (mm), distance traveled (m), number of stops (times), duration of stops (min), number of repetitions (times), and activity time (min).

[0048] Walking speed and activity time can serve as indicators of the motor function of the care recipient 70, for example. Similarly, unsteadiness can serve as an indicator of the care recipient 70's physical condition. Furthermore, the number of times the person stops, the duration of each stop, and the number of repetitions can serve as indicators of the severity of the dementia symptoms of the care recipient 70, for example.

[0049] Walking speed, as an indicator, can be calculated as the distance traveled per unit time along the movement trajectory of the 70 care recipients at each point in time.

[0050] Fluctuation as an indicator can be calculated, for example, as the difference between the positional information at each point in the movement trajectory of the 70 care recipients and the value obtained by smoothing the positional information (e.g., by a moving average).

[0051] The distance traveled as an indicator can be calculated, for example, as the distance traveled along the movement trajectory of the care recipients 70 during a predetermined time period. The predetermined time period can be, for example, daytime hours (7:00 to 19:00) or nighttime hours (19:00 to 7:00 the following day). The time settings for daytime and nighttime hours can be changed as appropriate by the information manager 90 at each facility. Furthermore, within a facility, these can also be set for each floor of the building, each unit, or individually.

[0052] The indicator of stopping count is the number of times the care recipient 70 stopped during a specified time period. For example, if a walking speed remains below a specified speed for a specified period of time, it may be judged as stopping.

[0053] The indicator of "standing time" is the amount of time that the care recipient (70 people) stood still during a specified time period. For example, it is the amount of time during which their walking speed remained below a specified speed.

[0054] The repetition count, used as an indicator, is the number of times the care recipient 70 moved from one area within their room to another adjacent area, and then immediately returned to their original area.

[0055] The activity time used as an indicator is the total time that the care recipient 70 spends moving around after getting up. In other words, the activity time is the total time that the care recipient 70 is out of bed. Recognition of getting up, getting out of bed, etc., is performed by recognizing the care recipient 70's posture (e.g., standing, sitting, lying down, etc.) from the detected person silhouette, and their relative position to objects in the room such as the bed 60, to determine whether they are getting up or getting out of bed.

[0056] The time-series data generation unit 313 generates time-series data in which indicators are arranged in chronological order for a predetermined time period including the date and time of shooting. As described later, the user (care staff 80, information manager 90, etc.) inputs a day or time as the unit for generating the time-series data, and the reception unit 314 accepts the input day or time selection. The user also inputs indicators for generating the time-series data, and the reception unit 314 accepts the input indicator selection.

[0057] The heatmap generation unit 315 visualizes the time spent by the care recipient 70 in each area within the field of view of the captured image and generates a heatmap of the care recipient 70's time spent in each predetermined time period. The heatmap (behavior map) can be used to understand how the care recipient 70 is behaving within the room. Not limited to heatmaps, a map of the care recipient 70's movement trajectory can also be used for the above purpose. The heatmap generation unit 315 functions as a map generation unit. Details of the functions of the time-series data generation unit 313, the reception unit 314, and the heatmap generation unit 315 will be described later.

[0058] The communication unit 32 is an interface for various local connections, such as a wired communication network interface based on standards like Ethernet (registered trademark), and a wireless communication interface based on standards like Bluetooth (registered trademark) and IEEE 802.11, and communicates with each terminal connected to the network 50.

[0059] The display unit 33 is, for example, a liquid crystal display and displays various information. The display unit 33 displays, for example, a graph visualizing the behavioral data of the care recipients 70, or a heat map of the care recipients 70's stay time.

[0060] The input unit 34 is equipped with a keyboard, numeric keypad, mouse, etc., for inputting various types of information. The audio input / output unit 35 is, for example, a headset equipped with a speaker and microphone.

[0061] In this embodiment, the information manager terminal 30 is used as a terminal for the information manager 90, displaying various information within the facility to the information manager 90 and receiving instructions. Furthermore, voice communication with care staff 80, etc., is possible by sending and receiving voice signals with the detection unit 10 or the care staff terminal 40 via the communication unit 32. Here, the information manager 90 is a manager who oversees multiple care staff 80. The information manager 90 may manage care staff 80 working in multiple facilities, not just one.

[0062] Furthermore, care staff 80 or technical staff, etc., can use the information management terminal 30 to associate the detection unit 10 with the room number when it is installed in each room (resident's room), and to calibrate and specify the position information of objects installed in the resident's room, such as the bed 60, i.e., the contour information of the overhead view from the ceiling camera 13. They can also associate the identification information of the hospitalized or residing care recipients 70, such as their names and ID numbers, with each room number.

[0063] (Care staff terminal 40) Figure 7 is a block diagram illustrating the schematic configuration of the care staff terminal. The care staff terminal 40 comprises a control unit 41, a wireless communication unit 42, a display unit 43, an input unit 44, and an audio input / output unit 45, which are interconnected by a bus. The control unit 41 has the same configuration as the control unit 11 of the detection unit 10, and includes a CPU, RAM, ROM, etc. The wireless communication unit 42 enables wireless communication using standards such as Wi-Fi and Bluetooth (registered trademark), and communicates wirelessly with each device via the access point 51 or directly. The display unit 43 and the input unit 44 are touch panels, with a touch sensor superimposed on the display surface of the display unit 43, which is made of liquid crystal or the like. The display unit 43 and the input unit 44 display various operation screens to the care staff 80, which list multiple events included in the event list, and accept various operations through the operation screens. In addition, when the care staff 80 provides care to the person being cared for 70, they input information about the care provided to the person being cared for 70 into the care record via the input unit 44.

[0064] The audio input / output unit 45 is, for example, a speaker and a microphone, and enables voice communication by the care staff 80 with other care staff terminals 40 via the wireless communication unit 42. The care staff terminal 40 is a device that functions as the user interface of the monitoring system 1 and can be composed of, for example, a portable communication terminal device such as a tablet computer, smartphone, or mobile phone.

[0065] When starting work, care staff 80 perform login authentication through their assigned care staff terminal 40. Care staff 80 enters their staff ID and password through the touch panel (display unit 43, input unit 44) of the care staff terminal 40 and sends this information to the management server 20. The management server 20 compares the authentication information stored in the database 23 and sends the authentication result corresponding to the care staff 80's authority to the care staff terminal 40, thereby completing the login authentication.

[0066] Furthermore, the detection unit 10, management server 20, information administrator terminal 30, and care staff terminal 40 may include components other than those described above, or may not include some of the components described above.

[0067] (Display method) Next, with reference to Figures 8 and 9, an overview of the processing procedure for the display method of this embodiment will be described. Figure 8 is a flowchart illustrating the processing procedure for the display method of this embodiment. The processing in the flowchart in the figure is realized by the CPU of the control unit 31 executing the display program.

[0068] Figure 9 is a schematic diagram illustrating the screen of the software that visualizes and analyzes the behavioral data of the care recipient 70. The control unit 31 displays the screen 200 of the software that visualizes and analyzes the behavioral data of the care recipient 70 on the display unit 33. When "Behavioral Details" 211 is selected by the user from the menu area 210, the control unit 31 displays the care recipient selection area 220 and the behavioral display area 230 on the screen 200.

[0069] The care recipient selection area 220 is an area for selecting the care recipients 70 whose behaviors will be displayed, and includes a floor selection unit 221 and a user search unit 222. The floor selection unit 221 allows the user to select the care recipients 70 for each floor of the facility building in a pull-down format. The names of the care recipients 70 on the selected floor are displayed in a list in the user name display area 231. In the example shown in the figure, names from Mr. / Ms. A in room 101 to Mr. / Ms. M in room 113 are displayed. The user selects the care recipients 70 whose behaviors will be displayed by selecting one of the user names displayed in the user name display area 231. The selected user name is displayed in the user display area 232. In the example shown in the figure, "Mr. / Ms. A, Room 101" is displayed.

[0070] The user search unit 222 searches for the care recipients 70 using the user name (name of the care recipient 70) entered by the user, and selects the care recipients 70 corresponding to the user name as the care recipients 70 whose behaviors will be displayed. The user can select the care recipients 70 whose behaviors will be displayed using either the floor selection unit 221 or the user search unit 222.

[0071] As shown in Figure 8, first, the captured image and the date and time the image was taken are acquired (step S101). The acquisition unit 311 acquires from the management server 20's database 23 multiple captured images (videos) of the care recipient 70 whose actions are to be displayed, and information regarding the date and time those images were taken. If the care recipient 70 was in their room when the image was taken, there is a high probability that the care recipient 70 is in the captured image.

[0072] Next, an index related to the walking of the care recipient 70 is calculated (step S102). The index calculation unit 312 calculates an index related to the walking of the care recipient 70 based on the captured images and the date and time of capture. For example, the index calculation unit 312 calculates the movement trajectory of the care recipient 70 in the room from the captured images, and calculates an index related to the walking of the care recipient 70 based on the movement trajectory and the date and time of capture of the images.

[0073] The indicator setting area 233 is an area for selecting an indicator to generate time-series data. The indicator can be selected from, for example, walking speed, unsteadiness, distance traveled, number of stops, duration of stops, number of repetitions, and activity time, but the options are not limited to these. In the example shown in the figure, "walking speed" is selected. The reception unit 314 receives the selected indicator.

[0074] Next, time-series data is generated by arranging the indicators in chronological order (step S103). The time-series data generation unit 313 generates time-series data in which the walking indicators of the care recipient 70 are arranged in chronological order for a predetermined time period (for example, daytime and nighttime) that includes the date and time the captured images were taken. The time-series data is automatically generated up to the day before it is displayed.

[0075] The period setting area 234 is an area where the user can set the period for which time-series data will be displayed. In the example shown in the figure, "September 6, 2020 - October 3, 2020" is set.

[0076] The day setting button 235 is for selecting "day" as the unit for generating time-series data, and the time setting button 236 is for selecting "hour" as the unit for generating time-series data. When the user presses the day setting button 235, the reception unit 314 accepts "day" as the unit for generating time-series data and sets the horizontal axis of the graph to date. On the other hand, when the user presses the time setting button 236, the reception unit 314 accepts "hour" as the unit for generating time-series data and sets the horizontal axis of the graph to hour. When the horizontal axis of the graph is set to hour, the user must specify a date. The time on the horizontal axis of the graph is, for example, the time during daytime hours (e.g., 7:00 to 19:00) or nighttime hours (19:00 to 7:00 the next day).

[0077] For example, if "day" is selected as the unit for generating time-series data, the walking speed during a given time period may be the median of the movement speeds of the care recipient 70 calculated at each point in time during that period. If the care recipient 70 stops frequently while walking, the average value of the distance traveled during the given time period will be very small, so it is preferable to use the median of the distance traveled as the walking speed during the given time period. On the other hand, if the care recipient 70 stops infrequently while walking, the walking speed during the given time period may be used as the average value of the distance traveled after appropriate processing.

[0078] Furthermore, if "time" is selected as the unit for generating time-series data, the walking speed at each time point may be the median of the movement speeds of the 70 care recipients calculated at each point in the predetermined time period.

[0079] Next, a heat map is generated (step S104). The heat map generation unit 315 visualizes the time spent by the care recipient 70 in each region within the field of view of the captured image and generates a heat map of the time spent by the care recipient 70 for each predetermined time period.

[0080] Next, time-series data and a heatmap are displayed (step S105). The behavior display area 230 is an area for displaying the behavior of the care recipient 70 (behavioral data), and includes an area for displaying time-series data and an area for displaying a heatmap corresponding to a predetermined time period of the time-series data. In the example shown in Figure 9, screen 200 includes time-series data and a heatmap for both daytime and nighttime periods. The time-series data may be, for example, a graph (hereinafter simply referred to as "graph") in which indicators are arranged in time series. The figure shows examples of graphs 237 for daytime and 238 for nighttime, with the vertical axis being walking speed (m / min) and the horizontal axis being date. Although not shown, graphs with the horizontal axis set to time can also be displayed in a similar manner.

[0081] The daytime heatmap 239 corresponds to the daytime graph 237 and is the daytime heatmap for the date specified by the user in the daytime graph 237. Similarly, the nighttime heatmap 240 corresponds to the nighttime graph 238 and is the nighttime heatmap for the date specified by the user in the nighttime graph 238. For example, in the daytime graph 237, the date where the horizontal axis and cursor C1 overlap (October 3, 2020) is specified, and in the nighttime graph 238, the date where the horizontal axis and cursor C2 overlap (October 3, 2020) is specified. Note that even when "hours" is selected as the unit for generating time series data, the heatmaps for each specified time period on the specified date will be displayed in the same way as when "days" is selected as the unit for generating time series data.

[0082] In this way, graphs 237, 238, and heatmaps 239, 240 are displayed simultaneously, allowing users to easily compare graph 237 and heatmap 239 for daytime hours with graph 238 and heatmap 240 for nighttime hours.

[0083] In the daytime heatmap 239 and the nighttime heatmap 240, the color, density, and pattern of each area are displayed differently depending on the length of time the care recipient 70 stays. For example, the longer the care recipient 70 stays, the darker the color (e.g., dark red) is displayed, and the shorter the stay, the lighter the color (e.g., light red) is displayed. In this specification, for the sake of displaying the figures, the patterns of each area are displayed differently depending on the length of stay (the correspondence between each pattern and the length of stay is shown on the far right of Figure 9). In the example shown in the figure, the case is illustrated with three levels of stay: long, medium, and short, but there may be four or more levels. Furthermore, it is possible to display heatmaps for time periods specified by the user, not just for predetermined time periods.

[0084] As described above, the flowchart in Figure 8 acquires images of the care recipient 70 and the date and time the images were taken. Based on the images and their date and time, an index related to the care recipient 70's walking is calculated. Next, time-series data is generated by arranging the walking index in chronological order for a predetermined time period including the date and time the images were taken. The time spent by the care recipient 70 in each region within the field of view of the images is visualized, and a heatmap of the care recipient 70's time spent is generated for each predetermined time period. The time-series data is then displayed, and the heatmap corresponding to the predetermined time period of the time-series data is displayed on the display unit 33.

[0085] Specifically, the control unit 31 controls the display on the screen 200 to simultaneously show a graph of a predetermined time period for an indicator selected by the user, and a heatmap of the same predetermined time period on the same day as the graph. This allows the user to easily check the behavior of the care recipient 70 during the predetermined time period on the same day as the graph using the heatmap.

[0086] For example, by displaying a graph of indicators related to the walking of the care recipient 70 in their room (walking speed, unsteadiness, etc.) alongside a heat map of the corresponding time period, care staff 80 can easily grasp the relationship between the indicators and the care recipient 70's behavior. For instance, care staff 80 can easily understand which areas of the room the care recipient 70 spends time in, and which areas of the room they move around in, at what walking speed and with what degree of unsteadiness.

[0087] The display device, display system, display method, and display program of this embodiment described above have the following effects.

[0088] The system displays time-series data showing walking indicators for 70 care recipients, along with behavioral maps (heatmaps) corresponding to specific time periods in the time-series data. This allows care staff 80 to analyze the condition of 70 care recipients in detail.

[0089] The present invention is not limited to the embodiments described above, and can be modified in various ways within the scope of the claims.

[0090] For example, in the embodiment described above, the case in which the information administrator terminal 30 functions as a display device was explained, but the present invention is not limited to such a case, and the care staff terminal 40 may be configured to function as a display device.

[0091] Furthermore, in the above-described embodiment, the case in which the control unit 31 of the information administrator terminal 30 calculates the movement trajectory of the care recipient 70 and recognizes its actions was explained. However, the system is not limited to this case, and the control unit 11 of the detection unit 10 may be configured to calculate the movement trajectory of the care recipient 70 and recognize its actions. In this case, the control unit 11 of the detection unit 10 and the information administrator terminal 30 function as display devices.

[0092] Furthermore, the control unit 31 or control unit 11 may be configured to learn from machine learning how a person moves within a room, including their walking speed and stride length, and to determine whether the person in the room is a care staff member 80 or a person receiving care 70.

[0093] Furthermore, the display program may be provided on a computer-readable recording medium such as a USB memory stick, flexible disk, or CD-ROM, or it may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in memory or storage. This display program may also be provided, for example, as a standalone application software, or it may be incorporated into the software of each device as a function of the server.

[0094] Furthermore, in the embodiment, some or all of the processing performed by the program may be replaced and executed by hardware such as circuits. [Explanation of Symbols]

[0095] 1. Monitoring system, 10 Detection unit, 11 Control unit, 12 Communications Department, 13 cameras, 14 Care Call Department, 15. Voice input section, 20 management servers, 21 Control unit, 22 Communications Department, 23 databases, 30 Information administrator terminals, 40 care staff terminals, 41 Control unit, 42 Wireless Communication Section, 43 Display section, 44 Input section, 45 Audio input / output section, 50 networks, 51 access points, 60 beds, 70 people receiving care, 80 care staff.

Claims

1. A time-series data generation unit generates time-series data in which indicators related to the walking of the person receiving care are arranged in chronological order, A map generation unit that generates a behavior map of the person receiving care, A display device having a display unit that simultaneously displays the aforementioned time-series data and the behavior map of the care recipient corresponding to the time-series data side by side.

2. An acquisition unit that acquires multiple images of the person receiving care and the date and time the multiple images were taken, The system further includes an index calculation unit that calculates an index related to the walking of the person receiving care based on the plurality of images and the date and time of the images taken, The time-series data generation unit generates time-series data in which the indicators are arranged in time series for a predetermined time period including the date and time of the photograph, The map generation unit visualizes the time spent by the care recipient in each region within the field of view of the multiple images, and generates a heatmap of the care recipient's time spent for each predetermined time period as the behavior map. The display unit displays the time-series data and also displays a heat map corresponding to the predetermined time period of the time-series data, as described in claim 1.

3. The display device according to claim 2, wherein the time-series data generation unit generates the time-series data for daytime hours and / or nighttime hours as the predetermined time period.

4. The display device according to claim 2, wherein the display unit displays the time-series data and the heat map for each of the predetermined time periods: daytime and nighttime.

5. The display device according to any one of claims 2 to 4, wherein the plurality of images are moving images.

6. The display device according to any one of claims 1 to 5, wherein the time-series data is a graph in which the indicators are arranged in time series.

7. The system further includes a receiving unit that accepts the selection of days or hours as the unit for generating the aforementioned time series data, and the selection of the aforementioned indicator. The display unit displays time-series data relating to the selected indicator in a selected unit, and also displays a heat map corresponding to the predetermined time period of the time-series data, according to any one of claims 2 to 4.

8. The display device according to any one of claims 1 to 7, wherein the indicator is one of walking speed, unsteadiness, distance traveled, number of stops, duration of stops, number of repetitions (number of times the person receiving care returned to the original area before moving immediately after moving), and activity time.

9. An imaging unit for taking pictures of the room of the person receiving care, A display device according to any one of claims 1 to 8, A display system having the following features.

10. A computer generates time-series data in which indicators related to the walking of the person receiving care are arranged in time series (a), (b) The computer generates a behavior map of the person receiving care, A display method comprising the step (c) in which a computer simultaneously displays the time-series data and a behavior map of the person receiving care that corresponds to the time-series data side by side.

11. The computer, prior to step (a), obtains a plurality of images of the person receiving care and the date and time the plurality of images were taken, The computer further includes the step (e) of calculating an index relating to the gait of the person receiving care based on the plurality of images and the date and time of the images taken, In step (a) above, time-series data is generated by arranging the indicators in chronological order for a predetermined time period including the date and time of the photograph, In step (b) above, the time spent by the care recipient in each region within the field of view of the image is visualized, and a heatmap of the care recipient's time spent in each predetermined time period is generated as the behavior map. The display method according to claim 10, wherein in step (c), the time series data is displayed and a heat map corresponding to the predetermined time period of the time series data is displayed.

12. A display program for causing a computer to perform a process included in the display method according to claim 10 or 11.

Images