Information processing device, program, computer-readable recording medium, information processing system, and information processing method

Abstract

To provide an information processing apparatus configured to display video image data to support appropriate review of a rehabilitation plan, a program, a computer-readable recording medium, an information processing system, and an information processing method.SOLUTION: An information processing apparatus includes: an acquisition unit which acquires first video image data obtained by imaging daily activity of a cared person, and second video image data obtained by imaging rehabilitation activity of the cared person; and a display unit which displays the first video image data and the second video image data so as to be compared to each other.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] This technology relates to an information processing apparatus, a program, a computer-readable recording medium, an information processing system, and an information processing method.

Background Art

[0002] In the field of rehabilitation, it has been conventionally practiced to record the movements of a care recipient in a moving image (for example, Patent Document 1). Depending on the degree of recovery by rehabilitation, the rehabilitation plan is also appropriately reviewed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the rehabilitation plan is reviewed by referring to the moving image data of the care recipient, if there are unusual circumstances such as the physical condition of the care recipient (target person) not being good during the implementation of rehabilitation, the rehabilitation plan may not be appropriately reviewed even by referring only to the moving image data during the implementation of rehabilitation.

[0005] An object of this technology is to provide an information processing apparatus, a program, a computer-readable recording medium, an information processing system, and an information processing method that enable the display of moving image data for assisting in appropriately reviewing a rehabilitation plan.

Means for Solving the Problems

[0006] According to one aspect of this technology, the information processing device includes an acquisition unit that acquires a first motion image data obtained by imaging the daily movements of the person receiving care and a second motion image data obtained by imaging the rehabilitation movements of the person receiving care, and a display unit that displays the first motion image data and the second motion image data in a comparative manner.

[0007] Preferably, the display unit includes a display device having a first display area and a second display area. Displaying in a comparative manner includes displaying a first video data in the first display area and simultaneously displaying a second video data in the second display area.

[0008] Preferably, the acquisition unit acquires a third video data image capturing the care recipient's daily activities at a different time than the first video data image. The information processing device further includes an input unit that allows the user to select the content to be displayed on the display unit. Based on the selection result of the content to be displayed on the input unit, the display unit displays the third video data image in a second display area.

[0009] Preferably, the acquisition unit acquires a third video data image capturing the rehabilitation movements of the person receiving care at a different time than the second video data image. The information processing device further includes an input unit that allows the user to select the content to be displayed on the display unit. Based on the selection result of the content to be displayed on the input unit, the display unit displays the third video data image in the first display area.

[0010] Preferably, the first motion image data includes information obtained by capturing at least one of the following actions of the person being cared for: walking, getting up, and standing up.

[0011] Preferably, the first and second video data are associated with information about specific behavioral indicators of the person receiving care. The information processing device further includes an input unit from which the content to be displayed on the display unit can be selected. The display unit displays the first and second video data based on the selection of a specific behavioral indicator in the input unit.

[0012] Preferably, the first and second video data each include time information. The information processing device further includes an input unit from which the content to be displayed on the display unit can be selected. Based on the selection of a specific time on the input unit, the display unit displays the first and second video data at a specific time.

[0013] Preferably, the information processing device is a terminal device. The acquisition unit acquires a first video data set and a second video data set from a server device that is communicatively connected to the terminal device.

[0014] In accordance with other aspects of this disclosure, the program causes the processor of the information processing device to perform the steps of acquiring first motion image data obtained by imaging the daily movements of the person being cared for and second motion image data obtained by imaging the rehabilitation movements of the person being cared for, and displaying the first motion image data and the second motion image data in a comparative manner.

[0015] In accordance with yet another aspect of this disclosure, the recording medium is computer-readable and stores the aforementioned program.

[0016] According to further aspects of this disclosure, the information processing system comprises an imaging device for imaging the movements of a person receiving care, a server device for acquiring motion image data from the imaging device, and a terminal device capable of communicating with the server device. The terminal device acquires first motion image data obtained by imaging the daily movements of the person receiving care and second motion image data obtained by imaging the rehabilitation movements of the person receiving care from the server device, and displays the first motion image data and the second motion image data in a comparative manner.

[0017] According to still another aspect of the present disclosure, an information processing method includes: obtaining first moving image data obtained by imaging the daily actions of a care recipient and second moving image data obtained by imaging the rehabilitation actions of the care recipient; and displaying the first moving image data and the second moving image data in a comparable manner.

Effect of the Invention

[0018] According to the present technology, by displaying the first moving image data obtained by imaging the daily actions of a care recipient and the second moving image data obtained by imaging the rehabilitation actions of the care recipient in a comparable manner, it is possible to support an appropriate review of the rehabilitation plan.

Brief Description of the Drawings

[0019] [Figure 1] It is a diagram showing an example of the configuration of a rehabilitation support system. [Figure 2] It is a diagram showing an example of an outline of the action record or activity record of the user 800 in the living room 900. [Figure 3] It is a diagram showing an example of the hardware configuration of the sensor box 100 and the portable terminal 300A. [Figure 4] It is a diagram showing an example of the hardware configuration of the server device 200. [Figure 5] It is a diagram showing an example of the action record of the user 800 in the living room 900. [Figure 6] It is a diagram showing an example of the action record of the user 800 in the rehabilitation room 950. [Figure 7] It is a diagram showing an example of a method for estimating the posture (estimating the action type) of the user 800 in the rehabilitation support system. [Figure 8] It is a functional block diagram for explaining the functional configurations of the sensor box 100, the server device 200, and the terminal device 300 that constitute the rehabilitation support system. [Figure 9]FIG. (1) showing a screen displayed on the display 305 of the terminal device 300 in a certain situation. [Figure 10] FIG. (2) showing a screen displayed on the display 305 of the terminal device 300 in a certain situation. [Figure 11] FIG. (3) showing a screen displayed on the display 305 of the terminal device 300 in a certain situation. [Figure 12] FIG. (4) showing a screen displayed on the display 305 of the terminal device 300 in a certain situation. [Figure 13] SEQUENCE DIAGRAM showing a processing flow executed in the rehabilitation support system. [Figure 14] FLOWCHART for explaining the processing flow executed in the terminal device 300.

MODE FOR CARRYING OUT THE INVENTION

[0020] Hereinafter, embodiments of the present technology will be described. In the following description, the same or corresponding parts may be denoted by the same reference numerals, and the description thereof may not be repeated.

[0021] In the embodiments described below, when referring to the number etc., unless otherwise specified, the scope of the present technology is not necessarily limited to the number etc. Further, in the following embodiments, each component is not necessarily essential for the present technology, unless otherwise specified.

[0022] <SYSTEM CONFIGURATION> Referring to FIGS. 1 and 2, the overall configuration of the rehabilitation support system (information processing system) will be described. The rehabilitation support system according to the present embodiment is mainly assumed to be used in residences where elderly people live, special nursing homes, care facilities, etc. (hereinafter, these are collectively referred to as "facilities"). In the following description, the care recipient who receives care in the facility is referred to as "user", and the caregiver who performs care work in the facility is referred to as "staff".

[0023] Figure 1 shows an example of the configuration of a rehabilitation support system according to this embodiment. Referring to Figure 1, the rehabilitation support system includes a sensor box 100, a server device 200, and a terminal device 300.

[0024] Facility 10 comprises at least a living room 900 and a rehabilitation room 950. The living room 900 is a room where users 800 spend their daily lives. The rehabilitation room 950 is a room where users 800 undergo rehabilitation (including physical fitness testing) aimed at improving their motor skills. In the example shown in Figure 1, user 800A spends their daily life in the living room 900, and user 800B receives rehabilitation in the rehabilitation room 950.

[0025] The sensor box 100 can acquire records of the user 800's movements (referred to as "daily movements" in this specification) or activities (referred to as "daily activities" in this specification) within the living room 900 using the camera and sensors built into the sensor box 100, as well as various other sensors. The sensor box 100 transmits the acquired records of the user 800's daily movements to the server device 200 via the internal network. The sensor box 100 may also be installed in the rehabilitation room 950. The sensor box 100 in the rehabilitation room 950 can acquire records of the user 800's movements (referred to as "rehabilitation movements" in this specification) or activities (referred to as "rehabilitation activities" in this specification) within the rehabilitation room 950.

[0026] In this specification, "actions" of user 800 refer to actions involving physical movement of user 800. "Activities" of user 800 refer to all actions of user 800, regardless of whether physical movement is involved. For example, "sleeping" and "reading" are included in "activities," even if they do not necessarily involve physical movement of user 800. In this specification, "daily activities" and "rehabilitation activities" are collectively referred to as "actions," and "daily activities" and "rehabilitation activities" are collectively referred to as "activities." Furthermore, records of actions or activities acquired by cameras, sensors, etc., are referred to as "action records" or "activity records."

[0027] The user 800's motion record includes, for example, video data captured by a camera. The user 800's activity record includes, for example, information such as the user 800's sleep time and heart rate acquired by sensors.

[0028] The actions of user 800 include, as an example, "walking," in which user 800's position changes while standing; "getting up," in which user 800's posture changes from lying down to sitting; and "standing up," in which user 800's posture changes from sitting to standing. The action record of user 800 may also include any other actions besides those described above.

[0029] The rehabilitation support system may further include a gateway server 11, a switching device 12, and an access point 13.

[0030] The gateway server 11 interconnects the internal network of facility 10 with the external network of facility 10. The internal network may be a local network consisting of wired and wireless connections. The external network may be, for example, the Internet or a public telephone network. Devices connected to the internal or external network send and receive data using communication protocols such as TCP (Transmission Control Protocol) / IP (Internet Protocol), UDP (User Datagram Protocol), and ICMP (Internet Control Message Protocol).

[0031] The gateway server 11 may have security features. For example, the gateway server 11 may use a firewall or filtering function to prevent devices connected to the external network from accessing devices on the internal network. Conversely, the gateway server 11 may impose restrictions on devices connected to the internal network, such as preventing them from accessing specific web pages on the external network.

[0032] The switching device 12 interconnects each device on the internal network of the facility 10. In some cases, a router or switch may be used as the switching device 12. The switching device 12 may have multiple Ethernet® communication ports and may be connected to each device via Ethernet cables. Furthermore, the switching device 12 may have a wireless antenna compatible with Wi-Fi® (Wireless Fidelity), etc. In that case, the switching device 12 can communicate with each device by both wired and wireless means. In the example shown in Figure 1, there is one switching device 12, but the number of switching devices 12 is not limited to this. The internal network of the facility 10 may be composed of a combination of multiple switching devices 12.

[0033] The access point 13 is equipped with a wireless antenna and connects each device to the internal network via wireless communication. In some cases, a Wi-Fi router may be used as the access point 13. In some cases, the switching device 12 and the access point 13 may be integrated into a single unit.

[0034] The sensor box 100 transmits the acquired user 800 operation records or activity records to the server device 200 via the internal network.

[0035] The server device 200 manages the rehabilitation support system's programs and various data. The server device 200 also stores the user's (800) movement or activity records in their room 900 and the user's (800) movement or activity records in their rehabilitation room 950.

[0036] The server device 200 may be a cloud server on an external network. In that case, the server device 200 will communicate with each device on the internal network via the gateway server 11.

[0037] The terminal device 300 may include mobile terminals 300A used by individual staff members and an administrator terminal 300B used by the administrator of facility 10. The administrator terminal 300B may be used as a terminal for the administrator of facility 10 to log in to the server device 200. In the example shown in Figure 1, two mobile terminals 300A1, 300A2 and one administrator terminal 300B are illustrated, but the number of mobile terminals 300A and administrator terminals 300B is not particularly limited.

[0038] The sensor box 100 can analyze the motion or activity records of the user 800 in the living room 900 or rehabilitation room 950 and transmit the analysis results to the server device 200. The analysis results may include, for example, the estimated type of motion for each motion included in the user 800's motion record, and information such as the duration or speed of each motion. Furthermore, the sensor box 100 can estimate whether the user 800 is in a sleeping state or an awake state.

[0039] The sensor box 100 may further transmit location information to the server device 200. This location information may include information about the living room 900, the rehabilitation room 950, the dining room, and other shared spaces. In other aspects, the sensor box 100 may transmit all or part of the acquired information to the mobile terminal 300A. In that case, the mobile terminal 300A may transmit various information acquired from the sensor box 100 to the server device 200.

[0040] The sensor box 100 incorporates sensors for detecting the movements of the user 800 within the living room 900. An example of a sensor is a Doppler sensor for detecting the movement of an object.

[0041] The mobile terminal 300A is used by staff engaged in caregiving at facility 10. Staff can use the mobile terminal 300A to transmit records of user 800's rehabilitation activities to the server device 200. Staff use the mobile terminal 300A to perform rehabilitation for user 800. For example, staff can use the mobile terminal 300A as a stopwatch to measure the time taken for activities such as walking, getting up, and standing up. In this case, the mobile terminal 300A can transmit timestamps (start time and end time) for each activity to the server device 200.

[0042] In the example shown in Figure 1, the facility 10 contains one resident room 900 and one rehabilitation room 950. However, the rehabilitation support system according to this embodiment can be used in a facility 10 with any number of resident rooms 900 and any number of rehabilitation rooms 950. Similarly, the number of other devices, such as the mobile terminal 300A, access point 13, and switching device 12, is not limited to the example shown in Figure 1.

[0043] The living room 900 includes, for example, a bed 901 as part of its facilities. The user 800 is wearing a vital sensor 400 that detects the user's vital information. One example of vital information to be detected is the user's body temperature. Another example is the user's respiration. Yet another example is the user's heart rate. Yet another example is two or more types of information from these.

[0044] Figure 2 shows an example of an overview of the movement or activity record of user 800 in room 900. The sensor box 100, the server device 200, and the mobile terminal 300A can be connected to each other via a wired or wireless network. Although the example shown in Figure 2 is for room 900, the rehabilitation support system can acquire movement or activity records of user 800 in the rehabilitation room 950 using a similar procedure.

[0045] The sensor box 100 includes a camera 105 (imaging device) and an optional sensor 110 (such as a microphone, infrared sensor, Doppler sensor, or a combination thereof). For example, the sensor box 100 may only capture the user 800's movements with the camera 105 when the sensor 110 detects the user 800's movements, and transmit the motion data to the server device 200 as a record of those movements. As another example, the sensor box 100 may continuously capture the user 800's movements and transmit the motion data to the server device 200 with detection values ​​acquired from the sensor 110 at regular intervals added.

[0046] The server device 200 can analyze the received operation records and store the analysis results along with the user 800's identification information. The terminal device 300 can display the information obtained from the server device 200 on its display.

[0047] Camera 105 photographs the interior of the living room 900. Camera 105 may be a movable camera with two or more movable axes. Camera 105 may be an infrared camera. Furthermore, camera 105 may be a movable infrared camera. In that case, the sensor box 100 can photograph the interior of the living room 900 even during the lights-out period at night without disturbing the sleep of the user 800. Furthermore, in other aspects, camera 105 may include both a normal camera and an external camera. In that case, the sensor box 100 can photograph the interior of the living room 900 during the day with the normal camera and the interior of the living room 900 at night with the infrared camera.

[0048] Sensor 110 can acquire data for detecting or analyzing the user 800's movements. Sensor 110 can acquire, for example, the start time of the user 800's movements, the end time of the movements, and location information. Sensor box 100 or server device 200 can analyze the user 800's movements with high accuracy based on the motion image data and the information acquired from sensor 110.

[0049] Motion data of the user's (rehabilitation) movements in the rehabilitation room 950 can be acquired by the mobile terminal 300A. Motion data of rehabilitation movements may also be acquired by the camera 105 of the sensor box 100, similar to movements (daily activities) in the living room 900.

[0050] <Hardware Configuration> Next, the hardware configuration of the sensor box 100, server device 200, and mobile terminal 300A will be described with reference to Figures 3 and 4. Figure 3 is a diagram showing an example of the hardware configuration of the sensor box 100 and mobile terminal 300A.

[0051] Referring to Figure 3, the sensor box 100 records the movements or activities of the user 800 in the living room 900 or rehabilitation room 950, and transmits the acquired information as a movement record or activity record to the server device 200. The sensor box 100 comprises a control device 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a camera 105, a Doppler sensor 106, a wireless communication device 107, and a storage device 108.

[0052] The control device 101 controls the sensor box 100. The control device 101 is comprised of, for example, at least one integrated circuit. The integrated circuit is comprised of, for example, at least one CPU (Central Processing Unit), at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or a combination thereof.

[0053] ROM 102 can store various programs and settings executed by the control device 101. RAM 103 temporarily stores programs loaded from ROM 102 or storage device 108. The control device 101 executes the programs loaded into RAM 103.

[0054] The communication interface 104 is connected to an antenna (not shown) or the like. The sensor box 100 exchanges data with other communication devices via this antenna. Other communication devices include, for example, a server device 200, a mobile terminal 300A, an access point 13, and other communication terminals. In some cases, the communication interface 104 may be connected to other communication devices by a wired connection.

[0055] In one implementation, camera 105 is a near-infrared camera. The near-infrared camera includes an IR (Infrared) illuminator that emits near-infrared light and can capture images and / or photographs of the interior of the living room 900 even at night. In another implementation, camera 105 is a surveillance camera that receives only visible light. In yet another implementation, a 3D sensor or a thermographic camera may be used as camera 105. The sensor box 100 and camera 105 may be configured as an integrated unit or as separate units.

[0056] The Doppler sensor 106 is, for example, a microwave Doppler sensor that emits and receives radio waves to detect the behavior (movement) of objects in the living room 900. This allows for the detection of biometric information of the user 800 in the living room 900. In one example, the Doppler sensor 106 emits 24GHz microwaves toward the beds 901 in each living room 900 and receives the microwaves reflected by the user 800 (hereinafter referred to as "reflected waves"). The reflected waves undergo a Doppler shift due to the user 800's movements. From these reflected waves, the Doppler sensor 106 can detect the user 800's respiratory state and heart rate. The Doppler sensor 106 corresponds to the sensor 110 that detects the user 800's movements shown in Figure 2. In some cases, the sensor box 100 may also include any other sensors that detect the user 800's movements besides the Doppler sensor 106.

[0057] The wireless communication device 107 receives each signal from the vital sensor 400 and transmits the signal to the control device 101. The vital sensor 400 transmits its respective detection output to the wireless communication device 107.

[0058] The storage device 108 is, for example, a storage medium such as a hard disk or an external storage device. The storage device 108 can store the motion image data acquired by the camera 105 and the detection information from the Doppler sensor 106. In some cases, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) may be used as the storage device 108.

[0059] The mobile terminal 300A displays the user 800's movement records or activity records. The mobile terminal 300A also receives input of the user 800's rehabilitation movement records and transmits the received rehabilitation movement records to the server device 200. The mobile terminal 300A includes a control device 301, RAM 303, communication interface 304, display 305, input device 306, camera 307, and internal memory 308.

[0060] The control device 301 controls the mobile terminal 300A. The control device 301 is composed of, for example, at least one integrated circuit. The integrated circuit is composed of, for example, at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof. The RAM 303 temporarily stores programs loaded from the internal memory 308. The control device 301 executes the programs loaded into the RAM 303.

[0061] The communication interface 304 is connected to an antenna (not shown) or the like. The mobile terminal 300A exchanges data with external communication equipment via the antenna and access point 13. External communication equipment includes, for example, a sensor box 100 and a server device 200.

[0062] The display 305 is implemented, for example, by a liquid crystal display or an organic EL (Electro-Luminescence) display. The input device 306 is implemented, for example, by a touch sensor superimposed on the display 305. The touch sensor accepts various operations on the mobile terminal 300A via touch operation and outputs the content of the operation to the control device 301. The display 305 may also display a screen for inputting care records and accept input of care records from staff. The camera 307 (imaging device) can acquire moving image data as information on the user 800's movements.

[0063] The internal memory 308 is, for example, a storage medium such as an eMMC (Embedded Multimedia Card). As an example, the internal memory 308 stores a program executed by the control device 301, but this program may be stored in a storage device other than the internal memory 308, as long as it is a storage device accessible by the control device 301. For example, the program may be stored in a recording medium readable by the terminal device 300 (computer).

[0064] Figure 4 shows an example of the hardware configuration of server device 200. Referring to Figure 4, the hardware configuration of server device 200 and the function of each component will be explained. Server device 200 may, for example, be a PC (Personal Computer) or a workstation.

[0065] The server device 200 includes a control unit 201, a ROM 202, a RAM 203, a communication interface 204, a display interface 205, an operation interface 206, and a storage device 207.

[0066] The control device 201 controls the server device 200. The control device 201 is comprised of, for example, at least one integrated circuit. The integrated circuit is comprised of, for example, at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

[0067] ROM 202 can store various programs and settings executed by the control unit 201. RAM 203 temporarily stores programs loaded from ROM 202 or storage device 207. The control unit 201 executes the programs loaded into RAM 203.

[0068] An antenna (not shown) and the like are connected to the communication interface 204. The server device 200 exchanges data with external communication equipment via this antenna. External communication equipment includes, for example, a sensor box 100 and a mobile terminal 300A.

[0069] The display interface 205 is connected to the display 208 and sends image signals to the display 208 for displaying images according to commands from the control device 201 or the like. In certain situations, the display 208 may display the expected improvement in each of the user 800's movements, a rehabilitation plan, and rehabilitation records.

[0070] The operating interface 206 is, for example, a USB (Universal Serial Bus) terminal and is connected to the input device 209. The operating interface 206 receives signals indicating user operation from the input device 209. The input device 209 is, for example, a mouse, keyboard, touch panel, or other device capable of accepting user input.

[0071] The storage device 207 is, for example, a storage medium such as a hard disk or an external storage device. The storage device 207 stores, as an example, records of daily activities, records of rehabilitation activities, results of analysis of activity records, expected values ​​of improvement in each activity for each user 800, and records of rehabilitation for each user 800. In some cases, an HDD or SSD may be used as the storage device 207.

[0072] <Acquisition of motion or activity records> Next, with reference to Figures 5 and 6, the process for acquiring user 800's movement records or activity records in the rehabilitation support system will be explained.

[0073] Figure 5 shows an example of a record of daily activities. The example shown in Figure 5 is a view of the living room 900 from a camera 105 in a sensor box 100 installed on the ceiling of the living room 900. The movement path 500 represents the trajectory of the user 800's movements.

[0074] The sensor box 100 uses the camera 105 to capture images of the living room 900. The sensor box 100 adds a timestamp and detection data from the sensor 110 to the video data to generate a record of daily movements or activities. The sensor box 100 periodically transmits the record of daily movements or activities to the server device 200.

[0075] The server device 200 analyzes the motion image data and detection data from the sensor 110 included in the record of daily movements or activities to extract the movement path 500. Next, the server device 200 calculates the walking speed of the user 800 in daily life based on the amount of change in the user 800's position based on the movement path 500 and the timestamp when the user 800's position changes. In a certain situation, the sensor box 100 may calculate the walking speed of the user 800 in daily life and transmit the result of this calculation to the server device 200.

[0076] Figure 6 shows an example of a record of rehabilitation activities. In the example shown in Figure 6, user 800 is receiving walking rehabilitation under the guidance of staff. The sensor box 100 is installed on the ceiling of the rehabilitation room 950, similar to the living room 900.

[0077] User 800 walks from the starting point 601 to the finish point 602, following the instructions of the staff. In one example, user 800's rehabilitation movements are captured by the camera 307 of the mobile terminal 300A. In another example, user 800's rehabilitation movements are captured by the camera 105 of the sensor box 100. A timestamp and detection data from the sensor 110 are added to these video images to generate a record of the rehabilitation movements. The generated record of the rehabilitation movements is stored in the server device 200.

[0078] <Estimation of operation type> Figure 7 shows an example of a method for estimating the posture (estimated type of movement) of user 800 in a rehabilitation support system according to this embodiment. The server device 200 can estimate the type of movement of user 800 by estimating the posture of user 800 based on the movement record and detecting changes in the posture of user 800. The method for estimating the posture of user 800 includes the following four steps.

[0079] In step 810, the server device 200 extracts multiple images in a continuous time series from the operation record. In some cases, the server device 200 may extract video footage from a certain period of time.

[0080] In step 820, the server device 200 identifies the user 800 included in each image extracted in step 810. In some cases, the server device 200 may use existing image recognition technology to identify the user 800 included in the image. In step 830, the server device 200 identifies the size of the user 800's head and the positions of the joints of the head and body included in each image extracted in step 810.

[0081] In step 840, the server device 200 compares adjacent images in chronological order and estimates the user's movements based on changes in the size of the user's head and changes in the positional relationship of the joints of the head and the entire body. Each image is associated with a timestamp, and the server device 200 can also calculate the speed of the user's movements based on the timestamp.

[0082] In a given scenario, the server device 200 may store in the storage device 207 first related information that associates the change in the size of the user 800's head with pattern information relating the positional relationship between the head and the joints of the entire body, and the user 800's posture. In this case, the server device 200 can estimate the user 800's posture in each image based on the first related information. Furthermore, the server device 200 can estimate the type of action performed by the user 800 based on the change in the user 800's posture in each image. For example, if the user 800's posture changes from sitting to standing, the server device 200 estimates that the user 800 performed the action "standing up".

[0083] <Functional configuration> Figure 8 is a functional block diagram illustrating the functional configuration of the sensor box 100, server device 200, and terminal device 300 that constitute the rehabilitation support system 1. As shown in Figure 8, the server device 200 includes an acquisition unit 210 and an output unit 220. The terminal device 300 (information processing device) includes an acquisition unit 310, a display unit 320, and an input unit 330.

[0084] The acquisition unit 210 of the server device 200 acquires video data from the sensor box 100 and the terminal device 300. The output unit 220 outputs the video data acquired by the acquisition unit 210 to the terminal device 300 in response to a request from the terminal device 300. The acquisition unit 210 and the output unit 220 of the server device 200 are included in the control device 201.

[0085] In this embodiment, the acquisition unit 210 acquires motion image data (first motion image data) obtained by imaging the daily activities of the user 800 (care recipient) in the living room 900 using the camera 105 of the sensor box 100. The acquisition unit 210 acquires motion image data (second motion image data) obtained by imaging the rehabilitation activities of the user 800 in the rehabilitation room 950 using the camera 307 of the terminal device 300. The motion image data acquired by the acquisition unit 210 is stored in the storage device 207.

[0086] The acquisition unit 310 of the terminal device 300 acquires the video data output by the output unit 220 of the server device 200. The video data acquired by the acquisition unit 310 is sent to the display unit 320.

[0087] The display unit 320 includes a display control unit 320A and a display 305. The display control unit 320A controls the display content of the display 305. The display control unit 320A receives motion image data from the output unit 220 of the server device 200. The display 305 displays motion image data obtained by imaging daily activities in the living room 900 (first motion image data) and motion image data obtained by imaging rehabilitation activities in the rehabilitation room 950 (second motion image data) in a way that allows for visual comparison. The meaning of "comparable" here will be explained later.

[0088] The input unit 330 includes an input receiving unit 330A and an input device 306. The input receiving unit 330A receives information input via the input device 306. Based on the information input to the input device 306, the input receiving unit 330A sends a request signal to the acquisition unit 210 of the server device 200. As a response signal to this request signal, moving image data is transmitted from the output unit 220 of the server device 200 to the acquisition unit 310 of the terminal device 300. Based on the information input to the input device 306, the input receiving unit 330A sends a display selection signal to the display control unit 320A. Based on the moving image data received from the output unit 220 of the server device 200 and the display selection signal received from the input receiving unit 330A, the display control unit 320A controls the display content of the display 305.

[0089] The acquisition unit 310, display control unit 320A, and input reception unit 330A of the terminal device 300 are included in the control device 301.

[0090] <User Interface (displayed under "Comparable")> Figures 9 to 12 show examples of screens displayed on the display 305. As shown in Figures 9 to 11, the display 305 includes a first display area 305A and a second display area 305B. The first display area 305A and the second display area 305B are provided adjacent to or close to each other.

[0091] In the example shown in Figure 9, video data 1000 (first video data) obtained by capturing footage of the waking up activity (daily activity) in room 900 at 5:32 AM on August 1, 2021 (first time), is displayed. Simultaneously, video data 2000 (second video data) obtained by capturing footage of walking training activity (rehabilitation activity) in rehabilitation room 950 at 1:15 PM on August 2, 2021 (second time), is displayed in the second display area 305B. In this way, video data 1000, which records daily activities, and video data 2000, which records rehabilitation videos, are displayed in a way that allows for visual comparison.

[0092] However, "display in a comparative manner" is not limited to this; for example, video data 1000 and video data 2000 may be displayed alternately on the display 305. Also, three or more video data may be displayed in a comparative manner. Furthermore, "display in a comparative manner" is not necessarily limited to displaying on a single display 305, but may also include displaying on multiple display devices simultaneously.

[0093] In the example shown in Figure 10, video data 1000A (the third video data) obtained by capturing a wake-up action (daily activity) at a different time (5:32 AM on August 2, 2021) than video data 1000 is displayed in the second display area 305B. The display state shown in Figure 10 is achieved by selecting the display content in the input unit 330. The result of the display content selection is sent to the display control unit 320A as a display selection signal. Based on the display content selection result in the input unit 330, the display unit 320 displays video data 1000A in the second display area 305B instead of video data 2000 in Figure 9.

[0094] In the example shown in Figure 11, video data 2000A (third video data) obtained by capturing a gait training movement (rehabilitation movement) at a different time (13:15 on August 1, 2021) than video data 2000 is displayed in the first display area 305A. The display state shown in Figure 11 is achieved by selecting the display content in the input unit 330. The result of the display content selection is sent to the display control unit 320A as a display selection signal. Based on the display content selection result in the input unit 330, the display unit 320 displays video data 2000A in the first display area 305A instead of video data 1000 in Figure 9.

[0095] In the example shown in Figure 12, motion image data 1000B obtained by imaging walking movements in the living room 900 and motion image data 2000B obtained by imaging walking training movements in the rehabilitation room 950 are displayed side by side on the display 305.

[0096] In the example shown in Figure 12, the video data 1000B and 2000B are displayed alongside graphs showing the changes over time in "walking speed in the living room" and "walking speed during rehabilitation."

[0097] The video data 1000B and 2000B are each associated with a specific behavioral indicator called "walking." When the input unit 330 receives input to display video data for "walking" (display mode selection), the video data 1000B and 2000B associated with the "walking" motion are displayed on the display 305. The result of the display mode selection is sent as a display selection signal from the input reception unit 330A to the display control unit 320A.

[0098] The video data 1000B and 2000B each contain time information. By specifying a particular time (C1, C2) on the graph, the video data 1000B and 2000B at that specific time are displayed on the display 305. The time specification information is sent from the input reception unit 330A to the display control unit 320A as a display selection signal.

[0099] <Processing flow> Figure 13 is a sequence diagram showing the flow of information processing methods performed in the rehabilitation support system. As shown in Figure 13, motion image data is transmitted from the sensor box 100 and the terminal device 300 to the server device 200. In sequence SQ1, the terminal device 300 receives input to identify the display target (user ID, date and time, operation type, etc.).

[0100] In sequence SQ2, terminal device 300 sends a request signal to server device 200 based on an input that identifies the display target. In sequence SQ3, server device 200 extracts the video data to be displayed based on the request signal. In sequence SQ4, server device 200 sends a response signal (a signal containing video data) to terminal device 300. In sequence SQ5, terminal device 300 displays the video data corresponding to the input.

[0101] Figure 14 is a flowchart illustrating the processing flow performed in the terminal device 300. As shown in Figure 14, in step S10, the terminal device 300 acquires video data output from the server device 200.

[0102] Next, in step S20A, the input device 306 selects whether or not to display a graph related to a specific behavioral indicator (for example, "walking motion").

[0103] If "Display graph" is selected in step S20A, then in step S21A, as shown in the example in Figure 12, motion image data 1000B and 2000B associated with "walking" will be displayed along with graphs showing the changes over time of "walking speed in the living room" and "walking speed during rehabilitation".

[0104] If "Do not display graph" is selected in step S20A, then in step S20B, the video display mode is selected via the input device 306.

[0105] If "Mode 1" is selected in step S20B, then in step S21B, as shown in the example in Figure 9, video data 1000 recording the activity in the living room 900 and video data 2000 recording the activity in the rehabilitation room 950 are displayed.

[0106] If "Mode 2" is selected in step S20B, then in step S22B, video data 1000 and 1000A recording the activity in the living room 900 will be displayed, as shown in the example in Figure 10.

[0107] If "Mode 3" is selected in step S20B, then in step S23B, video data 2000 and 2000A recording the movements in the rehabilitation room 950 will be displayed, as shown in the example in Figure 11.

[0108] <Program> The internal memory 308 of the terminal device 300 stores a program for executing the above-mentioned information processing in the control device 301. The program stored in the internal memory 308 causes the control device 301 (processor) to execute the following steps: acquiring motion image data 1000 obtained by imaging the daily movements of the user 800 and motion image data 2000 obtained by imaging the user's rehabilitation movements, and displaying the motion image data 1000 and 2000 in a comparative manner.

[0109] <Summary> To summarize the exemplary content described above, it is as follows: The rehabilitation support system 1 comprises cameras 105 and 307 (imaging devices) that capture images of the user 800's movements, a server device 200 that acquires motion image data from the cameras 105 and 307, and a terminal device 300 that can communicate with the server device 200. The terminal device 300 acquires motion image data 1000 obtained by capturing images of the user 800's daily movements and motion image data 2000 obtained by capturing images of the user 800's rehabilitation movements from the server device 200.

[0110] The terminal device 300 (information processing device) includes an acquisition unit 310 that acquires motion image data 1000 (first motion image data) obtained by imaging the daily movements of the user 800 (person receiving care) and motion image data 2000 (second motion image data) obtained by imaging the rehabilitation movements of the user 800, and a display unit 320 that displays the motion image data 1000 and 2000 in a comparative manner.

[0111] The acquisition unit 310 of the terminal device 300 acquires motion image data 1000 and 2000 from a server device 200 that is communicatively connected to the terminal device 300. The display unit 320 includes a display 305 (display device) having a first display area 305A and a second display area 305B.

[0112] In the example shown in Figure 9, by displaying the video data 1000 in the first display area 305A and simultaneously displaying the video data 2000 in the second display area 305B, the video data 1000 and 2000 are displayed in a way that allows for comparison.

[0113] The terminal device 300 further includes an input unit 330 that allows the user to select the display content for the display unit 320.

[0114] In the example shown in Figure 10, the acquisition unit 310 acquires video data 1000A (third video data) which captures the daily activities of user 800 at a different time than video data 1000. The display unit 320 displays video data 1000A in the second display area 305B based on the selection result of the display content in the input unit 330.

[0115] In the example shown in Figure 11, the acquisition unit 310 acquires video data 2000A (third video data) which captures the rehabilitation movements of user 800 at a different time than video data 2000. The display unit 320 displays video data 2000A in the first display area 305A based on the selection result of the display content in the input unit 330.

[0116] The video data 1000, which is a record of daily activities, includes information obtained by capturing images of at least one of the following actions of the user 800: walking, getting up, and standing up.

[0117] In the example shown in Figure 12, the video data 1000B and 2000B are associated with information about a specific behavioral indicator of the user 800 (for example, "walking motion"). The display unit 320 displays the video data 1000B and 2000B based on the selection of a specific behavioral indicator in the input unit 330.

[0118] In the example shown in Figure 12, the video data 1000B and 2000B each contain time information. The display unit 320 displays the video data 1000B and 2000B at specific times (C1 and C2) based on the selection of specific times (C1 and C2) in the input unit 330.

[0119] According to this embodiment, by visually comparing and displaying the video data 1000 obtained by imaging the daily activities of user 800 and the video data 2000 obtained by imaging the rehabilitation activities of user 800, it becomes easier to compare and confirm the daily activities of user 800 in the living room 900 with the rehabilitation activities of user 800 in the rehabilitation room 950, thereby facilitating appropriate revisions to the rehabilitation plan.

[0120] While embodiments of the present technology have been described above, the embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present technology is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0121] 1 Rehabilitation support system, 10 Facilities, 11 Gateway server, 12 Switching device, 13 Access point, 100 Sensor box, 101 Control device, 102 ROM, 103 RAM, 104 Communication interface, 105 Camera, 106 Doppler sensor, 107 Wireless communication device, 108 Storage device, 110 Sensor, 200 Server device, 201 Control device, 202 ROM, 203 RAM, 204 Communication interface, 205 Display interface, 206 Operation interface, 207 Storage device, 208 Display, 209 Input device, 210 Acquisition unit, 220 Output unit, 300 Terminal device, 300A, 300A1, 300A2 Mobile terminal, 300B Administrator terminal, 301 Control device, 303 RAM, 304 Communication interface, 305 Display, 305A First display area, 305B Second display area, 306 Input device, 307 Camera, 308 Internal memory, 310 Acquisition unit, 320 Display unit, 320A Display control unit, 330 Input unit, 330A Input reception unit, 400 Vital sensor, 500 Movement path, 601 Starting point, 602 Goal point, 800, 800A, 800B User, 900 Living room, 901 Bed, 950 Rehabilitation room, 1000, 1000A, 1000B, 2000, 2000A, 2000B Moving image data.

Claims

1. An acquisition unit that acquires first motion image data obtained by imaging the daily activities of the person receiving care, and second motion image data obtained by imaging the rehabilitation activities of the person receiving care, The system includes a display unit that displays the first video data and the second video data in a comparative manner, The first and second video data are associated with information about specific behavioral indicators of the person receiving care, respectively. The display unit further includes an input unit that allows selection of the display content, The display unit is an information processing device that displays the first and second video data associated with the specific behavior indicator based on the selection of the specific behavior indicator in the input unit.

2. The display unit includes a display device having a first display area and a second display area. The aforementioned display in a comparable manner means The information processing apparatus according to claim 1, comprising displaying the first video data in the first display area and simultaneously displaying the second video data in the second display area.

3. The acquisition unit acquires a third video data image of the care recipient's daily activities at a different time than the first video data image, The information processing device further comprises an input unit that allows selection of the display content in the display unit, The information processing apparatus according to claim 2, wherein the display unit displays the third moving image data in the second display area based on the selection result of the display content in the input unit.

4. The acquisition unit acquires a third video data image of the care recipient's rehabilitation movements at a different time than the second video data image, The information processing device further comprises an input unit that allows selection of the display content in the display unit, The information processing apparatus according to claim 2, wherein the display unit displays the third moving image data in the first display area based on the selection result of the display content in the input unit.

5. The information processing apparatus according to claim 1 or claim 2, wherein the first motion image data includes information obtained by capturing at least one of the following actions of the person being cared for: walking, getting up, and standing up.

6. The first video data and the second video data each include time information, The aforementioned information processing device is The display unit further includes an input unit that allows selection of the display content, The information processing apparatus according to claim 1 or 2, wherein the display unit displays the first video data and the second video data at the specified time based on the input unit selecting a specific time.

7. The aforementioned information processing device is a terminal device, The information processing apparatus according to claim 1 or 2, wherein the acquisition unit acquires the first video data and the second video data from a server device that is communicably connected to the terminal device.

8. The steps include acquiring a first motion image data obtained by imaging the daily activities of the person receiving care, and a second motion image data obtained by imaging the rehabilitation activities of the person receiving care, A program for causing the processor of an information processing device to perform the step of displaying the first video data and the second video data in a comparative manner, The first and second video data are associated with information about specific behavioral indicators of the person receiving care, respectively. A program that displays the first and second video data associated with the specific behavior indicator, based on the selection of the specific behavior indicator in the input unit of the information processing device.

9. A computer-readable recording medium storing the program described in Claim 8.

10. An imaging device that captures the movements of the person receiving care, A server device that acquires moving image data from the aforementioned imaging device, The server device is equipped with a terminal device capable of communicating with it, The aforementioned terminal device is The server device acquires a first video data set obtained by imaging the daily activities of the person receiving care, and a second video data set obtained by imaging the rehabilitation activities of the person receiving care. An information processing system that displays the first video data and the second video data in a comparative manner, The first and second video data are associated with information about specific behavioral indicators of the person receiving care, respectively. The terminal device further includes an input section that allows the user to select the display content, An information processing system that, based on the selection of a specific behavior indicator in the input unit, displays the first and second video data associated with the specific behavior indicator.

11. An information processing device acquires a first video data image of the daily activities of a person receiving care and a second video data image of the rehabilitation activities of the person receiving care, An information processing method comprising the step of displaying the first video data and the second video data in a comparative manner using the information processing device, The first and second video data are associated with information about specific behavioral indicators of the person receiving care, respectively. An information processing method that, based on the selection of a specific behavior indicator in the input unit of the information processing device, displays the first and second video data associated with the specific behavior indicator.

Images