Information processing method, program, and information processing device

Abstract

To provide an information processing method or the like capable of outputting (estimating) fall information of a subject according to an algorithm based on sleep information and the fall information of the subject.SOLUTION: In an information processing method according to an aspect, an algorithm based on sleep information of a subject W1 obtained through a bed sensor 3 that detects a sleep state and fall information related to falling of the subject W1 imaged by an imaging device 4 is read, the sleep information of the subject W1 is input to the read algorithm, and thereby, the fall information of the subject W1 is output.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] In recent years, there has been active development of technology for detecting critical situations such as a person's fall. For example, Patent Document 1 discloses a fall detection device that detects a person's fall by detecting a change in capacitance caused by a difference in the contact area between a sensor and the human body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-152475 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention of Patent Document 1 detects a fall by detecting the capacitance that changes according to the contact area of ​​the human body after the subject falls, but has the problem that it cannot estimate (predict) a fall before it occurs.

[0005] One aspect of the present invention is to provide an information processing method or the like that is capable of outputting (estimating) information about a subject's fall using an algorithm based on the subject's sleep information and fall information. [Means for solving the problem]

[0006] An information processing method according to one aspect reads out an algorithm based on sleep information of a subject obtained through a bed sensor that detects the sleep state and fall information regarding the subject's fall captured by an imaging device, and inputs the sleep information of the subject into the read out algorithm, thereby outputting fall information of the subject. [Effects of the Invention]

[0007] In one aspect, an algorithm based on sleep information and fall information of a subject can output fall information of the subject. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing an overview of a fall detection and prediction system. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a server. [Figure 3] FIG. 10 is an explanatory diagram showing an example of the record layout of a subject DB and an inference result DB. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a record layout of a training data DB. [Figure 5] FIG. 2 is a block diagram illustrating an example of the configuration of a terminal. [Figure 6] FIG. 10 is an explanatory diagram illustrating an operation of outputting information about a subject's fall. [Figure 7] FIG. 10 is an explanatory diagram illustrating an example of an alert display screen. [Figure 8] 10 is a flowchart showing a processing procedure for outputting information about a subject's fall. [Figure 9] 10 is a flowchart showing a processing procedure for outputting information about a subject's fall after taking a nap. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a server in the second embodiment. [Figure 11] FIG. 10 is an explanatory diagram showing an example of a record layout of a training data DB in the second embodiment. [Figure 12] FIG. 10 is an explanatory diagram illustrating the operation of outputting information about a subject's fall in the second embodiment. [Figure 13] 10 is a flowchart showing a processing procedure for outputting information about a subject's fall in the second embodiment. [Figure 14] 10 is a flowchart showing a processing procedure for estimating a fall of a subject. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below with reference to the drawings showing embodiments thereof.

[0010] (Embodiment 1) The first embodiment relates to a form in which fall information of a subject is output based on the subject's sleep information. Fig. 1 is an explanatory diagram showing an overview of a fall detection and prediction system. The system of this embodiment includes an information processing device 1, an information processing terminal 2, a bed sensor 3, and an imaging device 4, and each device transmits and receives information via a network N such as the Internet.

[0011] The information processing device 1 is an information processing device that processes, stores, and transmits / receives various types of information. The information processing device 1 is, for example, a server device, a personal computer, or a general-purpose tablet PC (personal computer). In this embodiment, the information processing device 1 is assumed to be a server device, and for simplicity, will be referred to as server 1 below.

[0012] The information processing terminal 2 is a terminal device that receives and displays information about the fall of the subject W1. The information processing terminal 2 is, for example, an information processing device such as a smartphone, tablet, mobile phone, personal computer terminal, or wearable device such as a smart watch. For simplicity, the information processing terminal 2 will be referred to as terminal 2 below.

[0013] The bed sensor 3 is a device used in nursing homes, hospitals, etc., that detects the breathing state, body movement, sleeping state, etc. of the subject W1 based on vibrations emitted by the subject W1. The bed sensor 3 is placed, for example, between the floor and mattress of the bed, and detects the breathing state, body movement, etc. by detecting the air pressure in the air mattress.

[0014] The imaging device 4 is an imaging device for detecting a fall of the subject W1. The imaging device 4 is installed in a location where it can capture an image of the entire subject W1 (for example, diagonally above the bed). By installing the imaging device 4 from a viewpoint that can cover the entire subject W1, it is possible to accurately record the movements or activities (behavior) of the subject W1. Note that in this embodiment, the fall information obtained by the imaging device 4 is used in the learning process of a fall estimation algorithm, which will be described later, but the imaging device 4 is not essential when estimating a fall of the subject W1 using this algorithm.

[0015] The imaging device 4 of this embodiment includes a wireless communication unit. The wireless communication unit is a wireless communication module for performing communication-related processing, and transmits image data or video data of the captured subject W1 to the server 1 via a network. Note that the imaging device 4 may be replaced by a smartphone capable of capturing images, a personal computer, or a mobile surveillance robot capable of capturing images of the subject W1.

[0016] Typically, after a person W1 in need of care falls, a notification is sent to the caregiver via a fall detection system or the like, but this does not prevent injury from the accident. To solve this problem, in this embodiment, a fall of the person W1 is estimated (predicted) based on sleep information. Therefore, the caregiver can recognize the risk of a fall in advance and take appropriate measures, thereby reducing the risk of injury from a fall.

[0017] The server 1 according to this embodiment receives the response obtained through the bed sensor 3 that detects the sleeping state. The server 1 reads out an algorithm based on the sleep information of the subject W1 and the fall information relating to the fall of the subject W1 captured by the imaging device 4. The server 1 inputs the sleep information of the subject W1 into the read algorithm, and outputs the fall information of the subject W1. The algorithm, sleep information, and fall information will be described later.

[0018] 2 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a control unit 11, a storage unit 12, a communication unit 13, a reading unit 14, and a large-capacity storage unit 15. Each component is connected by a bus B.

[0019] The control unit 11 includes an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or a quantum processor. The control unit 11 reads and executes a control program 1P (program product) stored in the storage unit 12, thereby performing various information processing, control processing, and the like related to the server 1.

[0020] The control program 1P can be deployed to run on a single computer, or on multiple computers located at one site, or distributed across multiple sites and interconnected by a communications network. While the control unit 11 is illustrated in FIG. 2 as a single processor, it may also be a multiprocessor.

[0021] The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the terminal 2, the bed sensor 3, the imaging device 4, etc. via the network N.

[0022] The reading unit 14 reads a portable storage medium 1a including a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 14 and store it in the mass storage unit 15. Alternatively, the control unit 11 may download the control program 1P from another computer via a network N or the like and store it in the mass storage unit 15. Furthermore, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0023] The mass storage unit 15 includes a recording medium such as a hard disk drive (HDD) or a solid state drive (SSD). The mass storage unit 15 includes a fall information output model 151, a subject DB (database) 152, an estimation result DB 153, and a training data DB 154.

[0024] The fall information output model 151 is an output device (estimator) that outputs fall information of the subject W1 based on the sleep information of the subject W1 obtained by the bed sensor 3, and is a trained model generated by machine learning. The subject DB 152 stores information about the subject whose fall is to be estimated. The estimation result DB 153 stores estimation results of the fall of the subject W1. The training data DB 154 stores training data for constructing (generating) the fall information output model 151.

[0025] In this embodiment, the storage unit 12 and the mass storage unit 15 may be configured as an integrated storage device. The mass storage unit 15 may also be configured by a plurality of storage devices. Furthermore, the mass storage unit 15 may also be an external storage device connected to the server 1. .

[0026] The server 1 may execute various information processing and control processing on a single computer, or may execute the processing in a distributed manner on multiple computers. The server 1 may also be realized by multiple virtual machines provided in a single server, or may be realized by using a cloud server.

[0027] FIG. 3 is an explanatory diagram showing an example of the record layout of the subject DB 152 and the estimation result DB 153. The subject DB 152 includes a subject ID column, a name column, and a room number column. The subject ID column stores a unique ID of the subject W1 to identify each subject W1. The name column stores the name of the subject W1. The room number column stores the room number of the subject W1.

[0028] The inference result DB153 includes a result ID column, a subject ID column, a fall information column, and an estimated date and time column. The result ID column stores a uniquely specified ID of the inference result in order to identify the inference result of a fall for each subject W1. The subject ID column stores a subject ID for identifying the subject W1. The fall information column stores fall information for the subject W1. The fall information includes, for example, the probability of a fall or information indicating whether or not a fall has occurred. The estimated date and time column stores date and time information when the fall of the subject W1 is estimated.

[0029] 4 is an explanatory diagram showing an example of the record layout of the training data DB 154. The training data DB 154 includes a training ID column, a model type column, an input data column, and an output data column. The training ID column stores a unique ID of each training data item to identify the training data item. The model type column stores the type of learning model, including the fall information output model 151 or a second fall information output model 155 in the second embodiment described below.

[0030] The input data is sleep information of the subject W1 at a first time. The sleep information is obtained by the bed sensor 3. The output data is fall information regarding a fall at a second time after the first time. The fall information is obtained by the imaging device 4.

[0031] For example, the first time period may be from 10 PM the previous day to 6 AM the next day, and the second time period may be from 10 PM the current day to 6 AM the next day. Alternatively, the first time period may be late at night (e.g., from midnight to 3 AM) and the second time period may be early in the morning (4 AM to 7 AM). Furthermore, the first time period may be nighttime sleep (e.g., from 10 PM to 6 AM) and the second time period may be a nap (1 PM to 3 AM).

[0032] The sleep information includes the time of getting into bed, the time of getting out of bed, the time of waking up during the night, the heart rate, the respiratory rate, the amount of body movement, the sleep duration, or the quality of sleep. The time of getting into bed is the time when the subject W1 gets into bed and lies down to sleep. The time of getting out of bed is the time when the subject W1 gets up from the bed. The time of getting into bed and the time of getting out of bed can be obtained, for example, by the bed sensor 3 detecting the weight or body movement of the subject W1.

[0033] The time of awakening during the night is the time when the subject W1 wakes up during the night. The bed sensor 3 detects the body movement of the subject W1 based on changes in pressure on the mattress caused by the subject W1 turning over while sleeping, etc. When the bed sensor 3 detects the body movement of the subject W1, it records the time of awakening during the night. Note that, because the heart rate increases when waking up, the bed sensor 3 may also record the time of awakening during the night based on changes in the heart rate.

[0034] The heart rate is a value that indicates how many times the subject W1's heart beats per minute. It is a numerical value indicating how many times the subject W1 breathes per minute. The amount of body movement is an index indicating how much the subject W1 moves his / her body while sleeping (such as the number of times or strength of turning over). The bed sensor 3 detects minute vibrations while the subject W1 is lying in bed. The bed sensor 3 calculates the heart rate, respiratory rate or amount of body movement by analyzing the detected vibration data.

[0035] The sleep time is the time that the subject W1 actually sleeps during the night, calculated by subtracting the time of awakening during the night from the time between the time of getting into bed and the time of getting out of bed. For example, the bed sensor 3 measures body movement or heart rate during sleep and calculates the time period when body movement increases or the time period when the heart rate increases as the time of awakening during the night. The bed sensor 3 calculates the sleep time by subtracting the calculated time of awakening during the night from the time between the time of getting into bed and the time of getting out of bed.

[0036] Sleep quality is an index that represents the depth of sleep (sleep depth). Sleep quality includes, for example, light sleep, deep sleep, and wakefulness. For example, when time-series data including heart rate, respiratory rate, or body movement (e.g., turning over in sleep) is input, the server 1 inputs the time-series data including the heart rate, respiratory rate, or body movement of the subject W1 into a learning model that has been trained to output a recognition result that recognizes the sleep quality, and outputs the sleep quality of the subject W1.

[0037] The fall information includes, for example, information indicating whether or not a fall occurred, the type of fall (falling over, slipping off the bed, etc.), the time the fall occurred, or the elapsed time.

[0038] The image capture device 4 detects a fall of the subject W1 by monitoring the movement or posture (e.g., a posture that is likely to lead to a fall or a fall) of the subject W1, for example, through a fall detection model installed on the image capture device 4. The fall detection model is a detector that detects information about the fall of the subject W1 based on the movement or posture of the subject W1, and is a trained model generated by machine learning.

[0039] The fall detection model is generated by learning using training data that combines image data of continuously captured images of the movements or posture, etc. of the subject W1 and a label indicating whether the subject W1 has fallen and the type of fall (for example, normal, fall, or slip). When the imaging device 4 acquires image data of continuously captured images of the movements or posture, etc. of the subject W1, it inputs the acquired image data into the fall detection model and outputs fall information about the subject W1.

[0040] As shown in the figure, the output data string stores a label indicating normal, fall, slippage (slippage off the bed), or the like detected by the imaging device 4 at the second time.

[0041] The storage format of each DB described above is an example, and other storage formats may be used as long as the relationships between the data are maintained.

[0042] 5 is a block diagram showing an example of the configuration of the terminal 2. The terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, and a display unit 25.

[0043] The control unit 21 includes a processing unit such as a CPU or an MPU, and performs various information processing and control processing related to the terminal 2 by reading and executing a control program 2P (program product) stored in the storage unit 22.

[0044] 5, the control unit 21 is described as a single processor, but it may be a multi-processor. The control unit 21 may execute various information processes or control processes by the same processor in the terminal 2, or may execute various information processes or control processes by different processors in the terminal 2. It can also be executed by the processor.

[0045] The storage unit 22 includes memory elements such as RAM or ROM, and stores the control program 2P or data required for the control unit 21 to execute processing. The storage unit 22 also temporarily stores data required for the control unit 21 to execute arithmetic processing.

[0046] The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the server 1, etc. via the network N. The input unit 24 may be a keyboard, a mouse, or a touch panel integrated with the display unit 25. The display unit 25 is a liquid crystal display, an organic EL (electroluminescence) display, or the like, and displays various information according to instructions from the control unit 21.

[0047] 6 is an explanatory diagram illustrating the operation of outputting fall information of subject W1. By inputting sleep information of subject W1 at a first time into fall information output model 151, fall information of subject W1 at a second time after the first time can be output.

[0048] The fall information output model 151 is used as a program module that is part of artificial intelligence software. The fall information output model 151 is an output device constructed by machine learning that receives sleep information of the subject W1 as input and outputs information about the subject W1's fall. The fall information output model 151 is constructed, for example, by a support vector machine (SVM).

[0049] The server 1 acquires training data used for learning the fall information output model 151 from the training data DB 154. The training data is a data set having a response variable (label) and an explanatory variable (feature). The response variable (output data) is an object to be estimated (predicted) or classified for the second time period, and is, for example, a label indicating normal, fall, or slipping off (slipping off the bed), etc.

[0050] The explanatory variables (input data) are input data used to estimate or classify the objective variables, and are data representing the state before (the first time) an incident (e.g., a fall or a slip) occurs. The input data is sleep information of the subject W1 obtained by the bed sensor 3. The sleep information includes the time of getting into bed, the time of getting out of bed, the time of waking up during the night, heart rate, respiratory rate, amount of body movement in bed, sleep time, sleep quality, etc., or a combination of these. The sleep information may also include body temperature, blood pressure, etc.

[0051] The server 1 uses the acquired objective variables and explanatory variables to construct (generate) the fall information output model 151. Specifically, the server 1 binarizes each objective variable and converts it into a value of "0" or "1" that indicates whether an incident has occurred. For example, the value may be 1 if an incident has occurred and 0 if it has not occurred.

[0052] The server 1 uses the binarized objective variable and each explanatory variable (sleep information) before the incident to find a decision boundary (hyperplane) that best divides the data. The server 1 trains the fall information output model 151 by finding a decision boundary that maximizes the margin, which is the distance from the decision boundary to the nearest data point (support vector).

[0053] The server 1 acquires the sleep information of the subject W1 for a first hour from the bed sensor 3. The sleep information is time-series data including the subject W1's time of getting into bed, time of getting out of bed, time of awakening during sleep, heart rate, respiratory rate indicating breathing frequency or rhythm, amount of body movement in bed (body movement), sleep time, or sleep quality (for example, sleep depth). The server 1 inputs the acquired sleep information of the subject W1 for the first hour to the fall information output model 151, and outputs fall information of the subject W1 for a second hour.

[0054] The fall information may be, for example, a probability of a fall (e.g., a value ranging from "0" to "1"), a label indicating the type of fall (normal, fall, slippage, etc.), or a value indicating discretely whether a fall has occurred (e.g., a value of "0" or "1"), etc. As shown in the figure, an estimated result that the probability of a fall is "0.86" is output for the sleep information of subject W1.

[0055] In this embodiment, the fall information output model 151 is described as being an SVM; however, the fall information output model 151 is not limited to an SVM, and other models such as a random forest, an isolation forest, a decision tree, a support vector regression (SVR), a local outlier factor (LOF), a Bayesian network, a regression tree, a neural network related to a long-short term memory (LTSM), a transformer, a convolutional neural network (CNN), or a recurrent neural network (RNN) may also be used.

[0056] In this embodiment, an example has been described in which the algorithm for outputting the fall information is the fall information output model 151, but the present invention is not limited to this. The algorithm for outputting the fall information may be a regression equation, a rule base, or the like.

[0057] The regression equation is an algorithm that analyzes a specific mathematical pattern or trend based on data obtained from the bed sensor 3 and detects a fall based on the analysis. For example, the server 1 analyzes the movement pattern of the subject W1 using the time series data (sleep information) of the subject W1 obtained from the bed sensor 3. If a specific pattern (e.g., a change in the value of body movement) is detected based on the analysis results, the server 1 may output fall information of the subject W1.

[0058] The rule base is a set of rules for detecting a fall when a specific condition is met. For example, when a body movement value in the time-series data of the subject W1 obtained by the bed sensor 3 exceeds a predetermined threshold, the server 1 may output fall information of the subject W1.

[0059] The server 1 assigns a result ID to the estimation result output from the fall information output model 151. The server 1 stores the subject ID of the subject W1, the probability of a fall, and the estimated date and time in the estimation result DB 153 as one record in association with the assigned result ID.

[0060] The server 1 determines whether the probability of a fall output from the fall information output model 151 exceeds a predetermined threshold (for example, 0.85). If the probability of a fall exceeds the predetermined threshold, the server 1 transmits (issues) an alert about a fall to the terminal 2.

[0061] 7 is an explanatory diagram showing an example of an alert display screen, which includes a subject information display field 11a, a sleep information display field 11b, a message display field 11c, and an image display field 11d.

[0062] The subject information display field 11a is a display field that displays information about the subject W1, including the subject W1's name, room number, etc. The sleep information display field 11b is a display field that displays sleep information about the subject W1 obtained by the bed sensor 3. The message display field 11c is a display field that displays a message corresponding to an alert. The image display field 11d is a display field that displays one or more images before and after a fall occurs. The images before and after the fall include an image showing the situation before the subject W1 fell and an image showing the situation after the subject W1 fell.

[0063] The server 1 acquires sleep information of the subject W1 for a first time period from the bed sensor 3. The first time period may be, for example, from 10:00 PM on the previous day to 6:00 AM on the next day. The acquired sleep information for the first hour is input to the fall information output model 151, and fall information for the subject W1 for the second hour is output (estimated). The second hour may be, for example, from 10 PM on the current day to 6 AM the next day. The fall information is, for example, the probability of a fall (e.g., 0.86). The server 1 determines whether the probability of a fall exceeds a predetermined threshold (e.g., 0.85).

[0064] When the probability of a fall exceeds a predetermined threshold, the server 1 transmits alert information about the fall to the terminal 2. Specifically, based on the subject ID, the server 1 acquires the name and room number of the subject W1 from the subject DB 152. Based on the estimated time of the fall by the fall information output model 151, the server 1 acquires images from the imaging device 4 before and after the fall that show the state of the subject W1's fall.

[0065] The server 1 sends an alert including the name, room number, sleep information, message (e.g., "Fall forecast: There is an increasing possibility of falling! Probability of falling: 0.86"), and images taken before and after the fall to the terminal 2. The terminal 2 receives the alert sent from the server 1 and displays the received alert on its screen.

[0066] Specifically, the terminal 2 displays the received name and room number of the subject W1 in the subject information display field 11a. The terminal 2 displays the sleep information of the subject W1 (heart rate, respiratory rate, etc.) acquired from the bed sensor 3 in the sleep information display field 11b. The terminal 2 displays the message in the message display field 11c. The terminal 2 may use TTS (Text-To-Speech) technology to output audio data of the message read aloud from a speaker.

[0067] The terminal 2 displays images before and after the fall in the image display field 11d. Note that, although the image before the fall is displayed in FIG. 7, this is not limiting. For example, the server 1 may acquire images before and after the subject W1 falls from the imaging device 4 in real time and transmit the acquired images before and after the fall to the terminal 2. The terminal 2 displays an image showing the current situation of the subject W1 in real time on the alert display screen.

[0068] 8 is a flowchart showing the processing steps for outputting fall information of the subject W1. The control unit 11 of the server 1 acquires the sleep information of the subject W1 at a first time, which is time-series data, from the bed sensor 3 via the communication unit 13 (step S101). The control unit 11 reads out the fall information output model 151 from the mass storage unit 15 (step S102). The control unit 11 inputs the acquired sleep information of the subject W1 at the first time into the read fall information output model 151 (step S103), and outputs fall information of the subject W1 at a second time (for example, the probability of falling) (step S104).

[0069] The control unit 11 stores the estimation result including the fall information output from the fall information output model 151 in the estimation result DB 153 of the mass storage unit 15 (step S105). Specifically, the control unit 11 assigns a result ID to the estimation result. The control unit 11 stores the subject ID of the subject W1, the probability of falling, and the estimated date and time as one record in the estimation result DB 153, in association with the assigned result ID.

[0070] The control unit 11 determines whether the probability of falling has exceeded a predetermined threshold (step S106). If the probability of falling has not exceeded the predetermined threshold (NO in step S106), the control unit 11 ends the process. If the probability of falling has exceeded the predetermined threshold (YES in step S106), the control unit 11 acquires the subject information (such as name and room number) of the subject W1 from the subject DB 152 of the mass storage unit 15 based on the subject ID (step S107).

[0071] The control unit 11 receives, via the communication unit 13, the estimated time of the fall by the fall information output model 151. Based on the above, images before and after the fall are acquired from the imaging device 4 (step S108). The control unit 11 transmits an alert including the name, room number, sleep information, message, and images before and after the fall of the subject W1 to the terminal 2 via the communication unit 13 (step S109).

[0072] The control unit 21 of the terminal 2 receives the alert transmitted from the server 1 via the communication unit 23 (step S201). The control unit 21 displays the received alert on the display unit 25 (step S202). The control unit 21 then ends the process.

[0073] According to this embodiment, by using an algorithm based on sleep information and fall information related to falls, it is possible to output fall information about the subject W1.

[0074] According to this embodiment, it is possible to output an alert when the probability of a fall exceeds a predetermined threshold.

[0075] According to this embodiment, by inputting the sleep information of the subject W1 at the first hour into the fall information output model 151, it is possible to output the fall information of the subject W1 at the second hour after the first hour without using the imaging device 4.

[0076] According to this embodiment, by outputting fall information for the second hour, which is later than the first hour, caregivers and others can recognize the risk of the subject W1 falling in advance and take appropriate measures, thereby reducing the risk of injury due to a fall.

[0077] <Variation 1> The following describes a process for outputting fall information about subject W1 after a nap by inputting sleep information about the subject W1's nap into the fall information output model 151. The sleep information about the nap is time-series data for the nap time period (e.g., 1:00 PM to 3:00 PM) including the subject W1's time of getting into bed, time of getting out of bed, time of awakening during the nap, heart rate, respiratory rate indicating breathing frequency or rhythm, amount of body movement in bed (body movement), sleep duration, or sleep quality (e.g., sleep depth).

[0078] In this modification, the fall information output model 151 is trained using training data including a dependent variable and an explanatory variable. The dependent variable is a label indicating a fall, a slip, or the like in the time period after a nap. The explanatory variable is sleep information related to the nap of the subject W1 obtained by the bed sensor 3. The learning process of the fall information output model 151 is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0079] When the server 1 acquires sleep information related to the afternoon nap of the subject W1 from the bed sensor 3, it inputs the acquired sleep information related to the afternoon nap to the fall information output model 151 and outputs fall information (e.g., probability of falling) after the afternoon nap of the subject W1 (e.g., 3:00 PM to 6:00 PM). The server 1 performs a determination process for alert output based on the probability of fall output from the fall information output model 151. Note that the determination process for alert output is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0080] 9 is a flowchart showing the processing steps for outputting information about a fall of the subject W1 after an afternoon nap. The same reference numerals are used to designate the same parts as in FIG. 8, and the description thereof will be omitted. The control unit 11 of the server 1 acquires sleep information about the subject W1's afternoon nap, which is time-series data, from the bed sensor 3 via the communication unit 13 (step S111).

[0081] The control unit 11 executes the process of step S102. The control unit 11 inputs the acquired sleep information about the afternoon nap into the read fall information output model 151 (step S112). The control unit 11 outputs information about the fall of the subject W1 after the nap (step S113). The control unit 11 executes the process of step S105.

[0082] According to this modification, by outputting information about the subject W1's fall after taking a nap, it becomes possible to detect a change in the subject W1's condition after taking a nap early on and provide appropriate medical care.

[0083] (Embodiment 2) The second embodiment relates to a form in which fall information of the subject W1 is output based on activity information relating to the activity of the subject W1. Note that a description of the contents that overlap with the first embodiment will be omitted.

[0084] Fig. 10 is a block diagram showing an example of the configuration of the server 1 in embodiment 2. Note that the same reference numerals are used to denote the same parts as in Fig. 2, and descriptions thereof will be omitted. The large-capacity storage unit 15 includes a second fall information output model 155. The second fall information output model 155 is an output device (estimator) that outputs fall information about the subject W1 based on activity information about the subject W1's activity captured by the imaging device 4, and is a trained model generated by machine learning.

[0085] 11 is an explanatory diagram showing an example of a record layout of the training data DB 154 in embodiment 2. Note that a description of the contents that overlap with FIG. 4 will be omitted.

[0086] The input data string is activity information related to the activity (physical activity) of the subject W1 obtained by the imaging device 4. The activity information is time-series data including the activity amount, activity time distribution, activity pattern, etc. of the subject W1.

[0087] The activity amount is data indicating the physical activity of the subject W1 (e.g., stride length, walking speed, standing up or sitting down), etc. The activity amount includes, for example, the activity amount before a fall occurs, the activity amount before a slip and fall occurs, the activity amount before lying down occurs, the activity amount before getting up, the activity amount before a boundary position (sitting on the edge of bed) occurs, the activity amount indoors, or the activity amount before getting out of bed.

[0088] The amount of activity before a fall occurs is the amount of bodily movement or activity of the subject W1 immediately before the subject W1 falls when standing up from the bed. For example, this is time series data such as the speed or smoothness of the movement when standing up from the bed, or the distance traveled, speed, or walking pattern after standing up from the bed. The amount of activity before a slip-off occurs is the amount of bodily movement or activity of the subject W1 immediately before the subject W1 slips off the bed. For example, this is time series data such as the movement or change in body position while sitting before the subject W1 slips off the bed.

[0089] The amount of activity before lying down is the amount of bodily movement or activity immediately before the subject W1 lies down. For example, it is time series data of movements while standing or walking before lying down on the bed. The amount of activity before getting up is the amount of bodily movement or activity immediately before the subject W1 stands up from a sitting or lying state. For example, it is time series data of movements or changes in body position while sitting before getting up from the bed.

[0090] The amount of activity before the occurrence of the boundary position is the amount of body movement or activity immediately before the subject W1 transitions to the edge-sitting position (sitting on the edge of the seating space). For example, it is time-series data such as the standing state, movement while walking, and the speed or smoothness of the sitting motion before transitioning to the edge-sitting position, or the movement distance, speed, or change in body position after transitioning to the edge-sitting position.

[0091] The amount of indoor activity is the overall amount of physical movement or activity that the subject W1 performs indoors. For example, it is time-series data such as the distance traveled indoors, the time spent standing, or the time spent sitting. The amount of activity before getting out of bed is the amount of body movement or activity immediately before the subject W1 gets up from the bed. For example, it is time-series data of movements or changes in body position while lying down before getting up from the bed.

[0092] The activity time distribution is data showing the frequency of individual activities over a specific period of time, such as the time periods during which subject W1's physical activities (such as lying on the floor or getting up from bed) are performed. Activity patterns include, for example, falling on the floor, lying on the floor, getting up from bed, boundary position movements, sliding off the bed, or getting out of bed.

[0093] For example, the image capturing device 4 monitors the movement or posture of the subject W1 (for example, lying on the floor, getting up from the bed, or falling), and acquires activity data of the subject W1. The image capturing device 4 analyzes the acquired activity data to calculate the activity time distribution of the subject W1 (a distribution indicating whether the subject W1 is active during a given time period).

[0094] For example, when the imaging device 4 inputs the acquired activity data of the subject W1, it inputs the activity data of the subject W1 into an activity pattern identification model that has been trained to output a recognition result that identifies the activity pattern of the subject W1, and outputs the activity pattern of the subject W1.

[0095] 12 is an explanatory diagram illustrating the operation of outputting fall information of subject W1 in embodiment 2. The server 1 acquires activity information (such as the amount of activity, distribution of activity time, or activity pattern) related to the activity of subject W1 from the imaging device 4. The server 1 inputs the acquired activity information of subject W1 to the second fall information output model 155 and outputs fall information of the subject W1. The fall information may be, for example, a label indicating the probability of a fall, the type of fall (such as a fall or a slip and fall), or a value that discretely indicates whether or not a fall has occurred.

[0096] The second fall information output model 155 is used as a program module that is part of artificial intelligence software. The second fall information output model 155 is an output device constructed by machine learning that receives activity information of the subject W1 as input and outputs information about the subject W1's fall. The second fall information output model 155 is constructed, for example, using SVM.

[0097] The server 1 acquires training data used for training the second fall information output model 155 from the training data DB 154. The training data is a data set having a response variable and an explanatory variable. The response variable (output data) is a label indicating, for example, normal, fall, or slippage.

[0098] The explanatory variables (input data) are activity information of the subject W1 at the first time point obtained by the imaging device 4. Specifically, the explanatory variables include the amount of activity before a fall occurs, the amount of activity before a slip occurs, the time and elapsed time of lying down on the floor, the amount of activity before lying down, the time of getting up from the bed, the amount of activity before getting up, the time of boundary position occurrence, the amount of activity before boundary position occurrence, the amount of activity indoors, the time of getting out of bed, or the amount of activity before getting out of bed.

[0099] The server 1 uses the acquired objective variables and explanatory variables to construct a second fall information output model 155. Note that the construction process for the second fall information output model 155 is similar to the construction process for the fall information output model 151 in the first embodiment, and therefore a description thereof will be omitted.

[0100] In this embodiment, the second fall information output model 155 is described as being an SVM. However, the second fall information output model 155 is not limited to an SVM, and may be, for example, a random forest, an isolation forest, a decision tree, an SVR, a LOF, a Bayesian network, a regression tree, a neural network related to LTSM, a transformer, a CNN, or the like. Alternatively, RNN or the like may be used.

[0101] The server 1 assigns a result ID to the estimation result output from the second fall information output model 155. The server 1 stores the subject ID of the subject W1, the probability of a fall, and the estimated date and time in the estimation result DB 153 as one record in association with the assigned result ID.

[0102] The server 1 performs a process of determining whether to output an alert based on the fall information output from the fall information output model 151. Note that the process of determining whether to output an alert is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0103] Fig. 13 is a flowchart showing the processing procedure for outputting fall information of the subject W1 in embodiment 2. Note that the same reference numerals are used to designate the same contents as in Fig. 8, and the description thereof will be omitted.

[0104] The control unit 11 of the server 1 acquires the activity information of the subject W1 for the first time period, which is time-series data, from the imaging device 4 via the communication unit 13 (step S121). The control unit 11 reads out the second fall information output model 155 from the mass storage unit 15 (step S122). The control unit 11 inputs the acquired activity information of the subject W1 for the first time period into the read out second fall information output model 155 (step S123), and outputs fall information of the subject W1 (for example, the probability of falling) (step S124). The control unit 11 executes the process of step S105.

[0105] According to this embodiment, it is possible to output fall information about the subject W1 using the second fall information output model 155 based on the activity information about the subject W1.

[0106] (Embodiment 3) The third embodiment relates to an embodiment in which a fall of a subject W1 is estimated using a fall information output model 151 (first fall information output model 151) and a second fall information output model 155. Note that a description of the contents that overlap with the first and second embodiments will be omitted.

[0107] The first fall information output model 151 is an example, and is not limited to this. For example, the first fall information output model 151 can be similarly applied to an algorithm for outputting fall information, such as a regression equation or rule base, as described in the first embodiment.

[0108] 14 is a flowchart showing the processing steps for estimating a fall of the subject W1. The control unit 11 of the server 1 acquires sleep information of the subject W1, which is time-series data, from the bed sensor 3 via the communication unit 13 (step S131). The control unit 11 inputs the acquired sleep information to the fall information output model 151 (step S132), and outputs first fall information of the subject W1 (step S133).

[0109] The control unit 11 acquires activity information of the subject W1, which is time-series data, from the imaging device 4 via the communication unit 13 (step S134). The control unit 11 inputs the acquired sleep information to the second fall information output model 155 (step S135), and outputs second fall information of the subject W1 (step S136).

[0110] The control unit 11 estimates a fall of the subject W1 based on the first fall information output from the first fall information output model 151 and the second fall information output from the second fall information output model 155 (step S137). Note that Fig. 14 illustrates an example in which the first fall information and the second fall information are the probability of a fall.

[0111] For example, the control unit 11 may estimate a fall of the subject W1 based on an average value of the probabilities. Specifically, the control unit 11 averages the fall probability output from the first fall information output model 151 and the fall probability output from the second fall information output model 155. The control unit 11 estimates a fall of the subject W1 based on whether or not the average value obtained by the averaging process exceeds a predetermined threshold.

[0112] For example, if the average value exceeds a predetermined threshold, the control unit 11 estimates that a fall has occurred for the subject W1. If the average value does not exceed the predetermined threshold, the control unit 11 estimates that a fall has not occurred for the subject W1.

[0113] Alternatively, the control unit 11 may estimate a fall of the subject W1 based on the maximum value of the probabilities. Specifically, the control unit 11 obtains the maximum probability of a fall from the fall probabilities output from the first fall information output model 151 and the fall probabilities output from the second fall information output model 155. The control unit 11 estimates a fall of the subject W1 based on whether the obtained maximum probability of a fall exceeds a predetermined threshold.

[0114] For example, if the maximum probability of a fall exceeds a predetermined threshold, the control unit 11 presumes that a fall has occurred for the subject W1. If the maximum probability of a fall does not exceed the predetermined threshold, the control unit 11 presumes that a fall has not occurred for the subject W1.

[0115] The control unit 11 transmits the estimation result of the fall estimation to the terminal 2 via the communication unit 13 (step S138). The control unit 21 of the terminal 2 receives the estimation result transmitted from the server 1 via the communication unit 23 (step S231). The control unit 21 displays the received estimation result on the display unit 25 (step S232). The control unit 21 ends the process.

[0116] Note that the present invention is not limited to the above-described fall estimation process. For example, both the fall information output from first fall information output model 151 and the fall information output from second fall information output model 155 may be displayed on the screen.

[0117] The first fall information output model 151 estimates fall information of the subject W1 at a second time (e.g., the current day) based on the subject W1's sleep information at a first time (e.g., the previous day). The second fall information output model 155 estimates fall information of the subject W1 in the most recent time period based on the subject W1's activity information.

[0118] For example, if it is determined that the subject W1 has an increased risk of falling based on the fall information estimated by the first fall information output model 151, the caregiver will warn the subject W1 to monitor him / her. In this case, by further monitoring the subject W1 through the second fall information output model 155, the risk of injury due to a fall can be reduced.

[0119] According to this embodiment, the first fall information output model 151 and the second fall information output model 155 can be used to estimate a fall of the subject person W1.

[0120] According to this embodiment, by combining the first fall information output model 151 and the second fall information output model 155, it is possible to improve the accuracy of estimating a fall.

[0121] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0122] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0123] 1. Information processing device (server) 11 Control section 12 Storage section 13 Communications Department 14 Reading unit 15 Mass storage 151 Fall information output model (first fall information output model) 152 Target Person DB 153 Estimation result DB 154 Training Data DB 155 Second Fall Information Output Model 1a Portable storage media 1b semiconductor memory 1P control program 2. Information processing terminal (terminal) 21 Control Unit 22 Memory section 23 Communications Department 24 Input section 25 Display section 2P control program 3 Bed Sensor 4. Imaging device

Claims

1. reading out an algorithm based on sleep information of the subject obtained through a bed sensor that detects a sleep state and fall information regarding a fall of the subject captured by an imaging device; The sleep information of the subject is input into the read algorithm, and the fall information of the subject is output. Information processing methods.

2. the algorithm is a learning model; The learning model is trained using training data including sleep information of the subject obtained by the bed sensor and fall information regarding a fall of the subject captured by the imaging device. The information processing method according to claim 1 .

3. the fall information indicates a probability of a fall; Output an alert if the probability of falling exceeds a predetermined threshold.

3. The information processing method according to claim 1 or 2.

4. Sleep information of the subject at a first time is input to the learning model, and fall information of the subject at a second time after the first time is output. The information processing method according to claim 2 .

5. Sleep information regarding the subject's afternoon nap is input into the learning model, and information regarding the subject's fall after the afternoon nap is output. The information processing method according to claim 2 .

6. The sleep information includes the time of getting into bed, the time of getting out of bed, the time of waking up during sleep, the heart rate, the respiratory rate, the amount of body movement, the sleep duration, or the quality of sleep.

3. The information processing method according to claim 1 or 2.

7. When activity information about the activity of the subject captured by the imaging device is input, the acquired activity information is input to a second learning model that outputs fall information about the subject's fall, thereby outputting fall information about the subject.

3. The information processing method according to claim 1 or 2.

8. A fall of the subject is estimated based on the fall information of the subject output from the algorithm and the fall information of the subject output from the second learning model. The information processing method according to claim 7.

9. The activity information includes the subject's activity amount, activity time distribution, or activity pattern. The information processing method according to claim 7.

10. reading out an algorithm based on sleep information of the subject obtained through a bed sensor that detects a sleep state and fall information regarding a fall of the subject captured by an imaging device; The sleep information of the subject is input into the read algorithm, and the fall information of the subject is output. A program that causes a computer to perform a process.

11. An information processing device including a control unit, The control unit reading out an algorithm based on sleep information of the subject obtained through a bed sensor that detects a sleep state and fall information regarding a fall of the subject captured by an imaging device; The sleep information of the subject is input into the read algorithm, and the fall information of the subject is output. Information processing device.

Images