Prediction device, prediction method, and program

Abstract

A prediction device according to one embodiment of the present invention comprises: a position information generation unit for generating the position information of a subject; a loitering degree calculation unit for calculating a loitering degree concerning the subject that indicates the degree of a stereotypical behavior regarding ambulation, on the basis of the position information of the subject; a prediction unit for predicting the occurrence of a prescribed symptom to the subject, on the basis of the loitering degree of the subject; and a notification unit for outputting a notification in response to a prediction that the symptom will occur to the subject.

Description

[Technical field]

[0001] The present invention relates to a technique for predicting the occurrence of symptoms such as wandering. [Background technology]

[0002] Dementia sufferers may experience behavioral and psychological symptoms of dementia (BPSD), such as wandering and sleep disorders. For example, when a dementia sufferer experiences symptoms of wandering, unexpected events such as the patient going missing may occur. To prevent unexpected events from occurring, it is important to be able to predict the occurrence of BPSD in dementia sufferers. For example, Non-Patent Document 1 discloses a method for detecting the occurrence of wandering using image analysis using a surveillance camera, RFID (Radio Frequency Identification), BLE (Bluetooth (registered trademark) Low Energy) beacons, and the like. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Hiroaki Ito, “Measures for Dementia in the Elderly Using IoT in an Aging Society”, [online], November 22, 2019, [Retrieved September 10, 2021], Internet<URL:https: / / www.nttpc.co.jp / technology / longevity.html> Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a technique capable of predicting the occurrence of symptoms such as wandering. [Means for solving the problem]

[0005] A prediction device according to one aspect of the present invention includes a location information generation unit that generates location information of a subject, a wandering degree calculation unit that calculates a wandering degree indicating a degree of stereotypic behavior related to walking for the subject based on the location information of the subject, a prediction unit that predicts the occurrence of a specified symptom in the subject based on the wandering degree of the subject, and a notification unit that outputs a notification in response to a prediction that the symptom will occur in the subject. Effect of the Invention

[0006] According to the present invention, a technique can be provided that can predict the occurrence of symptoms such as wandering. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing a prediction system including a prediction device according to an embodiment. [Diagram 2] FIG. 2 is a diagram for explaining the wandering degree calculation according to the embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating a hardware configuration of the prediction device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing a prediction method according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] [composition] FIG. 1 is a schematic diagram of a prediction system 10 including a prediction device 100 according to an embodiment of the present invention. The prediction system 10 is a system for monitoring dementia patients, and the prediction device 100 predicts the occurrence of wandering in dementia patients. Wandering is one of the behavioral and psychological symptoms of dementia (BPSD). In the following, a dementia patient who is the target of monitoring is referred to as a subject. Note that a dementia patient is a non-limiting example of a subject.

[0010] In the example shown in Fig. 1, the prediction system 10 includes a depth camera 20 and a smartphone 25 in addition to the prediction device 100. The prediction device 100 may be a computer such as a server. The prediction device 100 is communicatively connected to the depth camera 20 and the smartphone 25. The prediction device 100 may communicate with the depth camera 20 and the smartphone 25 via a network such as the Internet.

[0011] The depth camera 20 generates RGBD data including a depth image indicating depth information of the target space and a color image of the target space, and transmits the RGBD data to the prediction device 100. As an example, the depth camera 20 includes a depth sensor that generates a depth image and an RGB camera that generates a color image. The depth sensor may measure depth information based on, for example, ToF (Time of Flight) or stereo vision. The depth camera 20 periodically generates pairs of depth images and color images and transmits them in real time. For example, the depth camera 20 may generate five image pairs per second.

[0012] Typically, the depth camera 20 is placed in a building (e.g., a house or a hospital) where the subject lives. The target space is, for example, the subject's bedroom. Although one depth camera 20 is shown in the example shown in FIG. 1, multiple depth cameras 20 may be distributed to obtain a wide observation range. Some depth cameras 20 may be placed inside the building and some depth cameras 20 may be placed outside the building.

[0013] The prediction device 100 includes an acquisition unit 101 , a position information generation unit 102 , a position information storage unit 107 , a wandering degree calculation unit 108 , a wandering degree storage unit 109 , a wandering occurrence prediction unit 110 , and a notification unit 111 .

[0014] The acquisition unit 101 acquires RGBD data from the depth camera 20. The RGBD data includes a depth image and a color image.

[0015] The position information generating unit 102 generates position information indicating the position of a target person based on the RGBD data acquired by the acquiring unit 101. In the example shown in FIG. 1, the position information generating unit 102 includes a skeleton estimating unit 103, a person identifying unit 104, a position measuring unit 105, and a time measuring unit 106.

[0016] The skeleton estimation unit 103 estimates the skeleton of a person existing in the target space from the RGBD data to generate a skeleton estimation result. The skeleton estimation result includes, for example, three-dimensional positions of multiple feature points included in the skeleton. Examples of the feature points include joints. The three-dimensional positions may be relative positions to a reference point (for example, the position of the depth camera 20). The skeleton estimation unit 103 performs skeleton estimation using a depth image alone or using both a depth image and an RGB image. Since well-known skeleton estimation techniques can be used, a detailed description of skeleton estimation will be omitted.

[0017] The person identification unit 104 receives the skeleton estimation result from the skeleton estimation unit 103, and identifies a person existing in the target space based on the skeleton estimation result. The prediction device 100 further includes a skeleton information storage unit (not shown) that stores a database in which skeleton information for each subject is recorded, and the person identification unit 104 refers to the database to identify the person. In the database, the skeleton information is associated with a unique ID of the subject. The unique ID corresponds to identification information for identifying the subject. For example, the person identification unit 104 calculates the degree of match between the skeleton estimation result and the skeleton information of each subject, and obtains the unique ID associated with the skeleton information with the highest degree of match. There may be a person other than the subject in the target space. If the highest degree of match is below a predetermined threshold, the person identification unit 104 determines that the person existing in the target space is a person other than the subject. The person identification unit 104 may perform person identification by performing image processing on a color image included in the RGBD data.

[0018] The position measurement unit 105 receives the skeleton estimation result from the skeleton estimation unit 103, and measures the position of the person existing in the target space based on the skeleton estimation result. In one example, the three-dimensional position of the person's waist is used as the position of the person. The position measurement unit 105 calculates the three-dimensional position of the waist from the skeleton estimation result.

[0019] The time measurement unit 106 measures the time when the RGBD data is acquired. When the RGBD data includes time information, the time measurement unit 106 may use the time indicated in the time information as the acquisition time of the RGBD data. Alternatively, the time measurement unit 106 may use the time when the prediction device 100 receives the RGBD data from the depth camera 20 as the acquisition time of the RGBD data.

[0020] The position information generating unit 102 stores in the position information storage unit 107 the unique ID of the subject identified by the person identification unit 104, position information indicating the position measured by the position measuring unit 105, and time information indicating the time measured by the time measuring unit 106. In the position information storage unit 107, the position information is associated with the unique ID and the time information. As described above, the depth camera 20 periodically generates image pairs. Therefore, the position information generating unit 102 generates position information indicating the position of the subject at multiple consecutive times for each subject.

[0021] The wandering degree calculation unit 108 calculates the wandering degree of the subject based on the position information of the subject stored in the position information storage unit 107. The wandering degree indicates the degree of stereotypic behavior regarding walking. The wandering degree is calculated based on the predetermined time length T 1 The wandering degree is evaluated using the position information of the subject in a first time period having the following formula (1):

number

[0022] FIG. 2 shows a schematic diagram of the result of clustering a number of positions. For example, the x-means method can be used as the clustering method. The x-means method determines the number of clusters according to the data to be clustered, and classifies the data into the determined number of clusters. In the example shown in FIG. 2, the positions are classified into two clusters 201 and 202. The standard deviation S 1 is calculated from the positions belonging to cluster 201, and the standard deviation S 2 is calculated from the positions belonging to cluster 202.

[0023] Referring again to FIG. 1, the wandering degree calculation unit 108 calculates the time tT 1 The wandering degree calculation unit 108 calculates the position information of the target person from time tT 1 The wandering degree calculation unit 108 calculates the moving distance D of the subject during the period from time k to time t. The wandering degree calculation unit 108 clusters the positions indicated by the acquired position information, and calculates the standard deviation S k The wandering degree calculation unit 108 calculates the moving distance D with the standard deviation S as shown in the above formula (1). k The wandering degree W is calculated by dividing by the sum of

[0024] The wandering degree calculation unit 108 stores the wandering degree obtained by the calculation in the wandering degree storage unit 109. In the wandering degree storage unit 109, the wandering degree is associated with the subject's unique ID and time information. The time information may be information indicating the above-mentioned time t. The wandering degree calculation unit 108 stores the wandering degree obtained by the calculation in the wandering degree storage unit 109. 0 The wandering degree of the subject is calculated at intervals of, for example, one minute. Thus, time-series data indicating the wandering degree at a plurality of consecutive times is stored in the wandering degree storage unit 109 for each subject.

[0025] The wandering occurrence prediction unit 110 predicts the occurrence of wandering in a subject based on the subject's wandering degree. For example, the wandering occurrence prediction unit 110 calculates the wandering occurrence probability of a subject based on the subject's wandering degree. The wandering occurrence probability represents the probability (possibility) of wandering occurring in a subject. The wandering occurrence prediction unit 110 determines that wandering will occur when the wandering occurrence probability exceeds a threshold, and determines that wandering will not occur when the wandering occurrence probability is equal to or less than the threshold.

[0026] In one example, the wandering occurrence prediction unit 110 uses one wandering degree (for example, the latest wandering degree at the time of making the prediction) as the wandering occurrence probability. 2 The wandering occurrence prediction unit 110 calculates the probability of occurrence of wandering based on time series data including a plurality of wandering degrees in a second time period having a time t For example, the wandering occurrence prediction unit 110 may use a machine learning model such as a recurrent neural network (RNN). The machine learning model is configured to receive the time series data of the wandering degrees as an input, and to output the probability of occurrence of wandering. The model is trained in advance using training data. The training data is prepared by actually observing a plurality of persons with dementia. The training data includes time series data of the wandering degrees immediately before the occurrence of wandering, and time series data of the wandering degrees under normal circumstances. For example, the time series data of the wandering degrees immediately before the occurrence of wandering is obtained from a time t 0 -T 1 From time t 0t 0 is the time a predetermined time (e.g., 30 minutes) before the time when wandering actually occurred. By using such training data for learning, it is possible to generate a model that outputs the probability of wandering occurring after a predetermined time (e.g., 30 minutes).

[0027] In response to the wandering occurrence prediction unit 110 predicting that the subject is likely to wander, the notification unit 111 notifies a caregiver set for the subject. For example, the notification unit 111 transmits a notification indicating that there is a sign of wandering to a communication terminal (e.g., smartphone 25) used by the caregiver. The smartphone 25 may be set to display a message by push notification.

[0028] Fig. 3 shows an outline of a hardware configuration example of the prediction device 100. As shown in Fig. 3, the prediction device 100 includes, as hardware components, a processor 151, a RAM (Random Access Memory) 152, a program memory 153, a storage device 154, a communication interface 155, and a bus 156. The processor 151 is connected to the RAM 152, the program memory 153, the storage device 154, and the communication interface 155 via the bus 156 so as to be able to communicate with them.

[0029] The processor 151 includes a general-purpose circuit such as a CPU (Central Processing Unit). The RAM 152 includes a volatile memory such as an SDRAM. The RAM 152 is used by the processor 151 as a working memory. The program memory 153 stores programs executed by the processor 151, including a detection program. The programs include computer-executable instructions. For example, a ROM is used as the program memory 153. A portion of the storage device 154 may be used as the program memory 153.

[0030] Processor 151 loads the program stored in program memory 153 into RAM 152, and interprets and executes the program. When the detection program is executed by processor 151, it causes processor 151 to perform a series of processes described herein. In other words, processor 151 is configured to operate as acquisition unit 101, position information generation unit 102, wandering degree calculation unit 108, wandering occurrence prediction unit 110, and notification unit 111 in accordance with the detection program.

[0031] The program may be provided to the prediction device 100 in a state where it is stored in a computer-readable recording medium. In this case, the prediction device 100 includes a drive for reading data from the recording medium and acquires the program from the recording medium. Examples of the recording medium include a magnetic disk, an optical disk (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), a magneto-optical disk (MO, etc.), and a semiconductor memory. The program may also be distributed through a network. Specifically, the program may be stored in a server on the network, and the prediction device 100 may download the program from the server.

[0032] The storage device 154 includes a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD). The storage device 154 stores data such as the above-mentioned skeleton information, database, position information, wandering degree, and machine learning model. The storage device 154 implements the position information storage unit 107 and the wandering degree storage unit 109.

[0033] The communication interface 155 is an interface for communicating with external devices such as the depth camera 20 and the smartphone 25. The communication interface 155 includes a wired module and / or a wireless module.

[0034] Note that the processor 151 may include a dedicated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) instead of or in addition to the general-purpose circuit. In other words, a part or all of the series of processes described here may be performed by a dedicated circuit.

[0035] [Operation] Next, the operation of the prediction device 100 will be described.

[0036] Fig. 4 shows an outline of an example of a prediction method executed by the prediction device 100. For simplicity, a case will be described in which there is one depth camera and one subject. When multiple depth cameras are provided, the processes shown in steps S401 to S405 in Fig. 4 are executed for each depth camera. When multiple subjects are handled, the processes shown in steps S406 to S411 in Fig. 4 are executed for each subject.

[0037] In step S401, the acquisition unit 101 acquires RGBD data from the depth camera 20. The RGBD data includes a depth image of the object space and a color image of the object space.

[0038] In step S402, the skeleton estimation unit 103 estimates the skeleton of a person existing in the target space based on the RGBD data, and generates a skeleton estimation result including the three-dimensional positions of feature points in the skeleton.

[0039] In step S403, the person identification unit 104 identifies a person existing in the target space based on the skeleton estimation result obtained by the skeleton estimation unit 103. For example, the person identification unit 104 determines whether or not the person existing in the target space is a target person.

[0040] In step S404, the position measurement unit 105 measures the three-dimensional position of the person existing in the target space based on the skeleton estimation result obtained by the skeleton estimation unit 103.

[0041] In step S405, the time measurement unit 106 measures the time when the RGBD data is acquired. For example, the time measurement unit 106 obtains the time indicated by the time information included in the RGBD data as the data acquisition time.

[0042] If the person identification unit 104 determines in step S403 that the person present in the target space is the target person, the target person's unique ID, the position measured in step S404, and the time measured in step S405 are stored in the position information storage unit 107 in association with each other.

[0043] A series of processes shown in steps S401 to S405 is performed every time RGBD data is generated in the depth camera 20. Thus, time-series data indicating the position of the subject is recorded.

[0044] In step S406, the wandering degree calculation unit 108 calculates the distance traveled by the subject during the first time period based on the position information of the subject stored in the position information storage unit 107. 1 Then, the first time period is time t 1 -T 1 From time t 1 The wandering degree calculation unit 108 calculates the moving distance of the subject during the first time period from the position information of the subject during the first time period.

[0045] In step S407, the wandering degree calculation unit 108 calculates the wandering degree of the subject in the first time period. For example, the wandering degree calculation unit 108 clusters the positions indicated by the position information of the subject in the first time period, and calculates the standard deviation of the positions belonging to each cluster. The wandering degree calculation unit 108 calculates the wandering degree based on the movement distance calculated in step S406 and the standard deviation of each cluster. For example, as shown in the above formula (1), the wandering degree calculation unit 108 calculates the sum of the standard deviations of the clusters as the movement range of the subject in the first time period, and calculates the wandering degree by dividing the movement distance in the first time period by the movement range of the subject in the first time period.

[0046] In step S408, the wandering degree calculation unit 108 stores the calculated wandering degree in the wandering degree storage unit 109 in association with the time t and the subject's unique ID.

[0047] The series of processes shown in steps S406 to S408 is performed at a predetermined time interval T 0 Thus, time series data indicating the subject's wandering degree is recorded.

[0048] In step S409, the wandering occurrence prediction unit 110 calculates the wandering occurrence probability of the target based on the wandering degree of the target stored in the wandering degree storage unit 109. For example, the wandering occurrence prediction unit 110 acquires time series data including a plurality of wandering degrees in the second time period from the wandering degree storage unit 109, and calculates the wandering occurrence probability of the target based on this time series data. A reference time (for example, the current time) is set as t 2 Then, the second time period is, for example, time t 2 -T 2 From time t 2 The wandering occurrence prediction unit 110 inputs time series data including a plurality of wandering degrees in the second time period to the machine learning model, and obtains the wandering occurrence probability output from the machine learning model as the wandering occurrence probability of the target person.

[0049] In step S410, the wandering occurrence prediction unit 110 judges whether the wandering occurrence probability obtained in step S409 exceeds a predetermined threshold value. If the wandering occurrence probability is equal to or less than the threshold value (step S410; No), the wandering occurrence prediction unit 110 judges that the possibility of the subject wandering occurring is low, and the process returns to step S401.

[0050] If the wandering occurrence probability exceeds the threshold (step S410; Yes), the wandering occurrence prediction unit 110 determines that the subject is highly likely to wander, and the process proceeds to step S411. In step S411, the notification unit 111 outputs a notification indicating that the subject will wander. For example, the notification unit 111 transmits the notification to the smartphone 25 used by the caregiver set for the subject.

[0051] The series of processes shown in steps S409 to S411 is performed over a time interval T 0 or time interval T 0 It is executed at longer intervals.

[0052] [effect] The prediction device 100 calculates a wandering degree indicating the degree of stereotypic behavior regarding walking for the subject based on the position information of the subject, predicts the occurrence of behavioral and psychological symptoms in the subject based on the wandering degree of the subject, and outputs a notification in response to the prediction that the behavioral and psychological symptoms will occur in the subject. The stereotypic behavior regarding walking is quantified. For example, the wandering degree may be defined as the moving distance of the subject in the first time period divided by the moving range of the subject in the first time period. For example, the prediction device 100 may cluster the positions of the subject in the first time period, calculate the size of at least one cluster obtained by the clustering, and calculate the moving range of the subject in the first time period based on the size of the at least one cluster. By quantifying the stereotypic behavior regarding walking, it is possible to predict the occurrence of wandering in advance.

[0053] The prediction device 100 may perform prediction using a trained model configured to receive time series data of wandering degrees as input and output a probability of occurrence of wandering. The prediction device 100 may input time series data including a plurality of wandering degrees of the subject in a second time period to the trained model, obtain a probability output from the trained model, and predict the occurrence of wandering in the subject based on a comparison between the probability output from the trained model and a predetermined threshold. It is possible to perform prediction with higher accuracy than rule-based prediction.

[0054] The prediction device 100 measures the position of a subject based on a depth image and a color image of a target space in which the subject exists. This makes it possible to capture the fine movements of the subject, and enables highly accurate position measurement. For example, an indoor space such as a room is assumed as the target space. When the target space is an indoor space, positioning using a GPS (Global Positioning System) system reduces accuracy, but position measurement using a depth image does not reduce accuracy.

[0055] [Variations] In the above-described embodiment, the occurrence of wandering is predicted based on the degree of wandering. Prediction based on the degree of wandering can also be applied to symptoms (abnormal behavior) other than wandering.

[0056] In the above-described embodiment, the depth camera 20 generates a depth image and a color image. In other embodiments, the depth camera 20 may generate a depth image without generating a color image. In an embodiment in which the depth camera 20 generates only a depth image, the position information generator 102 generates position information of the subject based on the depth image.

[0057] In the above-described embodiment, the depth camera 20 is used to obtain position information of the subject. Any device may be used as long as it is capable of obtaining position information of the subject.

[0058] The present invention is not limited to the above-mentioned embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. The embodiments may be implemented in appropriate combination, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0059] 10. Prediction system 20…Depth camera 25. Smartphone 100…Prediction device 101…Acquisition Department 102...Position information generation unit 103…Bone structure estimation section 104...Person identification section 105... Position measurement unit 106...Time measurement section 107...Position information storage unit 108...Wandering degree calculation section 109...Wandering Memory Section 110…Wandering Prediction Department 111…Notification department 151…Processor 152…RAM 153...Program memory 154...Storage device 155...Communication interface 156…Bus

Claims

1. A position information generating unit that generates position information of a subject; a wandering degree calculation unit that calculates a moving distance of the subject in a first time period based on the position information of the subject, calculates a moving range of the subject in the first time period based on the position information of the subject, and calculates a wandering degree for the subject, the wandering degree indicating a degree of stereotypic behavior regarding walking, by dividing the moving distance of the subject in the first time period by the moving range of the subject in the first time period; a prediction unit that uses the degree of wandering of the subject as a probability of occurrence of a predetermined symptom, and predicts occurrence of the predetermined symptom in the subject based on a comparison between the probability and a predetermined threshold; a notification unit that outputs a notification in response to a prediction that the subject will experience the predetermined symptom; A prediction device comprising:

2. The wandering degree calculation unit performs clustering on the positions of the subject in the first time period indicated by the position information of the subject, calculates a size of at least one cluster obtained by the clustering, and calculates the movement range of the subject in the first time period based on the size of the at least one cluster. The prediction device according to claim 1 .

3. A location information generating unit that generates location information of a subject; a wandering degree calculation unit that calculates a moving distance of the subject in a first time period based on the position information of the subject, calculates a moving range of the subject in the first time period based on the position information of the subject, and calculates a wandering degree for the subject, the wandering degree indicating a degree of stereotypic behavior regarding walking, by dividing the moving distance of the subject in the first time period by the moving range of the subject in the first time period; a prediction unit that inputs time series data including the multiple wandering degrees of the subject during a second time period into a model that has been generated in advance by machine learning so as to calculate a probability of occurrence of a predetermined symptom from the time series data of the wandering degrees, and predicts the occurrence of the predetermined symptom in the subject based on a comparison between the probability output from the model and a predetermined threshold; a notification unit that outputs a notification in response to a prediction that the subject will experience the predetermined symptom; A prediction device comprising:

4. The position information generating unit acquires a depth image indicating depth information of a target space in which the target person exists, measures a position of the target person based on the depth image, and generates the position information of the target person. The prediction device according to claim 1 .

5. The prediction device according to claim 1 , wherein the symptom is wandering.

6. A computer implemented prediction method comprising: generating location information of the subject; calculating a moving distance of the subject in a first time period based on the position information of the subject, calculating a moving range of the subject in the first time period based on the position information of the subject, and calculating a wandering degree indicating a degree of stereotypic behavior regarding walking for the subject by dividing the moving distance of the subject in the first time period by the moving range of the subject in the first time period; predicting the occurrence of a predetermined symptom in the subject based on a comparison of the probability with a predetermined threshold using the wandering degree of the subject as a probability of occurrence of the predetermined symptom; outputting a notification in response to the prediction that the subject will experience the predetermined symptom; A prediction method comprising:

7. A computer implemented prediction method comprising: generating location information of the subject; calculating a moving distance of the subject in a first time period based on the position information of the subject, calculating a moving range of the subject in the first time period based on the position information of the subject, and calculating a wandering degree indicating a degree of stereotypic behavior regarding walking for the subject by dividing the moving distance of the subject in the first time period by the moving range of the subject in the first time period; inputting time series data including the multiple wandering degrees of the subject during a second time period into a model previously generated by machine learning so as to calculate a probability of occurrence of a predetermined symptom from the time series data of the wandering degrees, and predicting the occurrence of the predetermined symptom in the subject based on a comparison between the probability output from the model and a predetermined threshold; outputting a notification in response to the prediction that the subject will experience the predetermined symptom; A prediction method comprising:

8. A program for causing a computer to function as each unit included in the prediction device according to claim 1 .

Images