Determination device and determination method
The system uses image analysis to detect frailty or dementia by monitoring changes in scattering, posture, and facial expressions, providing timely notifications, thus addressing the challenge of undetected frailty or dementia in elderly individuals.
Patent Information
- Application Number
- JP2021040610
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-12
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-03-12
AI Technical Summary
Existing technologies fail to effectively detect early signs of frailty or dementia in elderly individuals, particularly those living alone, leading to increased societal care demands and economic burdens.
A determination system that analyzes images from a living space to detect changes in scattering, posture, and facial expressions over time, using machine learning models to identify abnormalities indicative of frailty or dementia, and notifies relevant parties when such changes are detected.
Enables early detection of frailty or dementia by objectively assessing individual characteristics, reducing the risk of undetected deterioration and associated societal costs.
Smart Images

Figure 0007591953000001 
Figure 0007591953000002 
Figure 0007591953000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a technique for making a judgment regarding frailty or dementia. [Background technology]
[0002] Japan is said to be an aging society, with approximately 30% of the population currently aged 65 or over. As the elderly population increases, the number of elderly people living alone (so-called elderly people living alone) is also on the rise, exceeding 6 million as of 2015. There is a problem with these elderly people living alone in that even if they show signs of frailty or dementia, it is not immediately noticeable.
[0003] In order to solve this problem, some local government and private organizations try to check for signs by calling or visiting elderly people living alone. However, the burden on each staff member of such organizations is very large, and it may not be possible to check sufficiently. In particular, in recent years, there has been an increase in telephone crimes such as "it's me" scams, and many elderly people do not answer the phone in the first place. In addition, from the perspective of preventing infectious diseases, it may be difficult to visit. This problem is not limited to elderly people living alone, but can also occur in facilities such as elderly care facilities that are premised on living in a single space. In addition, even if there are relatives or other people living together, there is also the problem that it is difficult to notice signs of frailty or dementia if they live together on a regular basis. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2013-206012 A Summary of the Invention [Problem to be solved by the invention]
[0005] The worsening of frailty and dementia leads to an increase in the number of elderly people requiring nursing care. This leads to an increase in the national social security expenditure and an increase in the number of people in their prime who leave their jobs to care for their loved ones, resulting in significant losses to society. Thus, as one of the social challenges, it is becoming increasingly important to quickly detect and deal with signs of frailty and dementia. For example, as in Patent Document 1, a technology has been proposed to monitor elderly people by installing cameras in their living spaces, but this technology is not sufficient to detect signs of frailty and dementia. This problem is not limited to the elderly, but is a common problem for those at risk of frailty and dementia.
[0006] In view of the above circumstances, the present invention aims to provide a technology that can determine the risk of frailty or dementia at an earlier stage. [Means for solving the problem]
[0007] One aspect of the present invention is a judgment device that includes an abnormality judgment unit that judges the presence or absence of an abnormality indicating that the subject is showing signs of frailty or dementia based on images captured in the living space of the subject, including images captured in the past and current images captured thereafter, and a notification unit that notifies other devices of the presence of an abnormality when the abnormality judgment unit judges that there is an abnormality.
[0008] One aspect of the present invention is the above-mentioned judgment device, wherein the abnormality judgment unit acquires a degree of scattering indicating the degree to which objects are scattered in the living space of the subject of judgment, and judges the presence or absence of an abnormality based on the degree of scattering in a past image and the degree of scattering in a current image.
[0009] One aspect of the present invention is the above-mentioned judgment device, wherein the abnormality judgment unit acquires posture information that is information indicative of the posture of the subject to be judged, acquires symptom information that is information indicative of signs of frailty or dementia in the subject to be judged based on the posture information, and judges the presence or absence of an abnormality based on the symptom information in past images and the symptom information in current images.
[0010] One aspect of the present invention is the above-mentioned judgment device, wherein the abnormality judgment unit acquires facial expression information that is information indicative of the facial expression of the subject to be judged, acquires symptom information that is information indicative of signs of frailty or dementia in the subject to be judged based on the facial expression information, and judges the presence or absence of an abnormality based on the symptom information in past images and the symptom information in current images.
[0011] One aspect of the present invention is a method for determining whether or not a subject is exhibiting an abnormality indicating signs of frailty or dementia based on images captured in the living space of the subject, including images captured in the past and current images captured thereafter, and a notification step of notifying another device of the presence of an abnormality if an abnormality is determined to be present in the abnormality determination step. Effect of the Invention
[0012] The present invention aims to provide technology that can detect risk of frailty or dementia at an earlier stage. [Brief description of the drawings]
[0013] [Figure 1] 1 is a schematic block diagram showing a system configuration of a determination system 100 according to the present invention. [Diagram 2] 2 is a schematic block diagram showing a specific example of a functional configuration of the imaging terminal 10. FIG. [Diagram 3] FIG. 13 is a diagram showing a specific example of a background image. [Figure 4] FIG. 13 is a diagram showing a specific example of a background image. [Diagram 5] FIG. 13 is a diagram showing a specific example of a determination target image. [Figure 6] FIG. 13 is a diagram showing a specific example of a determination target image. [Figure 7] FIG. 13 is a diagram showing a specific example of a determination target image. [Figure 8] 13 is a diagram showing a specific example of a notification image transmitted by a notification unit 164. FIG. [Figure 9] 2 is a schematic block diagram showing a specific example of the functional configuration of a control device 20. FIG. [Figure 10] 4 is a sequence chart showing a specific example of the flow of operations of the determination system 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, specific configuration examples of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the system configuration of a determination system 100 of the present invention. The determination system 100 is a system for determining the risk of frailty or dementia for a subject active in a facility 90 to be determined. The facility 90 to be determined may be, for example, a residence where the subject lives, an apartment building or facility (e.g., a nursing home) where multiple subjects live, or a facility (e.g., a prison) where one or multiple subjects live. The determination system 100 includes an imaging terminal 10, a control device 20, and a user terminal 30. The imaging terminal 10, the control device 20, and the user terminal 30 are communicatively connected via a network 40. The network 40 may be a network using wireless communication or a network using wired communication. The network 40 may be configured by combining multiple networks.
[0015] The imaging terminal 10 is installed in the facility 90 to be judged. The imaging terminal 10 may be installed so as to capture an image of a specific area. For example, the imaging terminal 10 may be installed in a specific location with specific camera parameters (angle of view and direction, etc.). In this embodiment, the imaging terminal 10 functions as a judgment device. The imaging terminal 10 captures an image of the judgment target in the judgment target facility 90, and judges the presence or absence of an abnormality (a sign of frailty, a sign of dementia) in the judgment target. When the imaging terminal 10 judges that there is an abnormality, it notifies another device (for example, the control device 20 or the user terminal 30) that the abnormality has been judged. The user terminal 30 is operated by a person who is scheduled to respond to the abnormality when it is judged that an abnormality has occurred in the judgment target. For example, the user terminal 30 may be used by a relative or acquaintance of the judgment target, or may be used by a person who performs the task of judging (watching over) the judgment target, or may be used by a plurality of them. With such a mechanism, the judgment system 100 makes it possible to judge the signs of frailty and dementia at an early timing and respond to the risks. The determination system 100 will be described in detail below.
[0016] 2 is a schematic block diagram showing a specific example of a functional configuration of the imaging terminal 10. The imaging terminal 10 is configured using information devices such as a smartphone, a tablet, a personal computer, a portable game machine, a stationary game machine, a dedicated device, etc. The imaging terminal 10 includes a communication unit 11, an imaging unit 12, an audio input unit 13, an audio output unit 14, a display unit 15, a control unit 16, and a storage unit 17.
[0017] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 performs data communication with other devices via the network 40 in response to the control of the control unit 16. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.
[0018] The imaging unit 12 is configured using an imaging device such as a camera or a LIDAR (Light Detection and Ranging). The imaging unit 12 may be configured as an imaging device itself, or may be configured as an interface for connecting the imaging device to the imaging terminal 10 as an external device. The imaging device images a predetermined space in the facility 90 to be determined. The imaging by the imaging device may be realized as a process of forming an image by receiving visible light, may be realized as a process of forming an image using a radar, or may be realized by other implementation methods. It is desirable that the predetermined space imaged by the imaging device is a space in which the determination target is likely to be located. For example, the predetermined space imaged may be a space where the person to be determined usually lives (living space: for example, a living room, a hallway, a bedroom, etc.). One imaging device may be provided, or multiple imaging devices may be provided. The imaging unit 12 outputs data of the image captured by the imaging device to the control unit 16.
[0019] The audio input unit 13 is configured using a microphone. The audio input unit 13 may be configured as a microphone itself, or may be configured as an interface for connecting the microphone to the imaging terminal 10 as an external device. The microphone acquires audio from a predetermined space (e.g., a living space) within the facility 90 to be determined. The predetermined space from which audio is acquired by the microphone is preferably a space in which the determination target may be located. For example, it is preferably a space that is the imaging target of the imaging device described above. The audio input unit 13 outputs data of the audio acquired by the microphone to the control unit 16.
[0020] The audio output unit 14 is configured using a speaker. The audio output unit 14 may be configured as a speaker itself, or may be configured as an interface for connecting a speaker to the imaging terminal 10 as an external device. The speaker outputs audio to a predetermined space in the facility 90 to be determined. The predetermined space from which audio is output by the speaker is preferably a space in which the determination target may be located. For example, it is preferably a space that is the imaging target of the imaging device described above. The audio output unit 14 outputs audio corresponding to the audio signal output by the control unit 16.
[0021] The display unit 15 displays an image to the subject under the control of the control unit 16. The display unit 15 may display an image to prompt the subject to react, for example. For example, the display unit 15 may display a character string such as "I'm going to take a picture" or an image of a person related to the subject (for example, a relative, an acquaintance, or a pet).
[0022] The control unit 16 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 16 functions as an imaging control unit 161, a human body region estimation unit 162, an abnormality determination unit 163, and a notification unit 164 by the processor executing a program. All or part of the functions of the control unit 16 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (for example, an SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.
[0023] The imaging control unit 161 controls the imaging process by the imaging unit 12. The imaging control unit 161 operates the imaging unit 12 at a predetermined timing, for example, to capture an image. The imaging control unit 161 may capture an image that does not include a person such as a determination target (hereinafter referred to as a "background image") and an image that includes a determination target (hereinafter referred to as a "determination target image").
[0024] The imaging control unit 161 may capture a background image at a predetermined time every day (generally a time when the subject is likely to be inactive). For example, the imaging control unit 161 may output a sound or image from the audio output unit 14 or the display unit 15 to notify the subject not to stand in front of the imaging device because the background image will be captured, and then capture the background image.
[0025] The imaging control unit 161 may capture the judgment target image at a predetermined time every day (generally a time when the judgment target is likely to be active). For example, the imaging control unit 161 may output a sound or an image on the sound output unit 14 or the display unit 15 to notify the judgment target that the judgment target should stand in front of the imaging device to capture the judgment target image, and then capture the judgment target image.
[0026] The imaging control unit 161 records the captured images (for example, the background image and the determination target image) in the storage unit 17 in association with the date and time of capture.
[0027] The human body region estimation unit 162 estimates a human body region in the determination target image. The human body region estimation unit 162 estimates information (hereinafter referred to as "human body region information") indicating a region (human body region) in which the whole or part of the determination target is located in the image obtained from the imaging unit 12. The human body region may be, for example, a rectangle including the region in which the whole or part of the determination target is located. In this case, the human body region information may be information indicating the coordinates of the four corners of the rectangle. The human body region is not limited to a rectangle and may be another geometric shape (for example, a circle, an ellipse, a triangle, a square, a pentagon, etc.). The human body region may be a shape indicated by a plurality of straight lines and curves along the shape of the determination target.
[0028] The human body region estimation unit 162 may perform a process of estimating a human body region (hereinafter referred to as "human body region estimation") by using a trained model obtained by performing machine learning processing in advance using a teacher image, for example. Note that the technology used to implement the human body region estimation unit 162 does not need to be limited to the above-mentioned supervised learning, and any technology may be used. The human body region estimation unit 162 records human body region information in the storage unit 17 in association with the determination target image used for the human body region estimation.
[0029] The abnormality determination unit 163 determines the presence or absence of an abnormality based on an image (for example, a background image or an image to be determined) captured by the imaging unit 12. The abnormality of the object determined by the abnormality determination unit 163 is the presence or absence of a sign of frailty or a sign of dementia in the object to be determined. The abnormality determination unit 163 may determine the presence or absence of an abnormality based on a difference between a past image and a current image. More specifically, the abnormality determination unit 163 may determine the presence or absence of an abnormality based on, for example, a time-series change in the image. More specifically, the abnormality determination unit 163 may determine that an abnormality exists when a change between the state of the living space or the object to be determined obtained based on a past image and the state of the living space or the object to be determined obtained based on a current image satisfies a predetermined condition. A specific example of the process of the abnormality determination unit 163 will be described below.
[0030] [1. Deterioration of scattering] The abnormality determination unit 163 may obtain the degree of scattering in the life space to be determined based on the image captured by the imaging unit 12, and determine the presence or absence of an abnormality based on the obtained degree of scattering. The degree of scattering is a value indicating the degree to which objects are scattered in the life space. When many objects are scattered, a higher value of the degree of scattering is obtained. When objects are not scattered but are tidy, a lower value of the degree of scattering is obtained.
[0031] The image used to acquire the degree of scattering may be, for example, a background image or a judgment target image. When a judgment target image is used, for example, an image of a region excluding a human body region estimated by the human body region estimation unit 162 may be used. The abnormality determination unit 163 may estimate the degree of scattering by using a trained model obtained by performing machine learning processing in advance using a teacher image. For example, data to which an image in a living space and a label indicating the degree of scattering in the image are assigned may be used as a data set of teacher images.
[0032] The abnormality determination unit 163 records the estimated scattering degree in the storage unit 17 in association with information indicating date and time. The abnormality determination unit 163 determines that there is an abnormality in the determination target when the past scattering degree and the subsequent (current) scattering degree satisfy a predetermined condition. The predetermined condition is a condition indicating that the current scattering degree is higher than the past. For example, it may be determined that there is an abnormality when a scattering degree that is high enough to satisfy a predetermined condition (ratio or threshold) compared to a past scattering degree going back a predetermined time continues for a predetermined period (e.g., three days, one week, etc.).
[0033] [2. Poor posture] The abnormality determination unit 163 may estimate information indicating the posture of the determination target (hereinafter referred to as "posture information") based on an image of a human body region in the determination target image captured by the imaging unit 12, and may determine the presence or absence of an abnormality based on the acquired posture information. The abnormality determination unit 163 may estimate the posture information by using a trained model obtained in advance by performing machine learning processing using a teacher image, for example. For example, data to which an image of the determination target or another person and a label indicating posture information in the image are assigned may be used as a data set of the teacher image.
[0034] The posture information may be acquired as, for example, skeletal information of the object to be determined. The skeletal information is, for example, information including the positions of a plurality of feature points predetermined for the object to be determined and links connecting the feature points. The feature points of the skeletal information may be configured to include, for example, the nose, both eyes, neck, both shoulders, both elbows, both wrists, waist, the bases of both feet, both knees, both heels, and both toes. The abnormality determination unit 163 may perform skeletal estimation by using a trained model obtained by performing machine learning processing in advance using a teacher image, for example. The technology used to implement the skeletal estimation processing does not need to be limited to the above-mentioned supervised learning, and may be any technology.
[0035] The abnormality determination section 163 may acquire skeletal information by performing the above-mentioned processing, and determine the posture of the object to be determined based on the acquired skeletal information.
[0036] The abnormality determination unit 163 may acquire information indicating a sign of frailty or dementia in the subject based on the posture information acquired by performing the above-mentioned processing (hereinafter referred to as "sign information": for example, a value indicating the likelihood that a sign of frailty or dementia is present), and may determine the presence or absence of an abnormality based on the acquired sign information. The abnormality determination unit 163 may estimate the sign information by using, for example, a trained model acquired by performing machine learning processing in advance using training data. For example, data to which posture information and a label indicating the sign information in the posture information are assigned may be used as a dataset of training data.
[0037] The abnormality determination unit 163 records the estimated symptom information in the storage unit 17 in association with information indicating date and time. The abnormality determination unit 163 determines that there is an abnormality in the determination target when the past symptom information and the current symptom information satisfy a predetermined condition. The predetermined condition is a condition indicating that the current symptom information is more likely to show the above symptom than the past symptom information. For example, it may be determined that there is an abnormality when symptom information that is high enough to satisfy a predetermined condition (ratio or threshold) compared to past symptom information going back a predetermined time continues for a predetermined period (e.g., three days, one week, etc.).
[0038] [3. Deterioration of facial expression] The abnormality determination unit 163 may estimate information indicating the facial expression of the subject (hereinafter referred to as "facial expression information") based on an image of a human body region in the subject image captured by the imaging unit 12, and determine the presence or absence of an abnormality based on the acquired facial expression information. The abnormality determination unit 163 may estimate the facial expression information by using, for example, a trained model obtained by performing machine learning processing in advance using a teacher image. For example, data to which an image of the subject or another person and a label indicating the facial expression information in the image are assigned may be used as a data set of the teacher image.
[0039] The facial expression information may be acquired as information on a plurality of feature points of the face of the determination target (hereinafter referred to as "facial expression feature information"). The facial expression feature information is, for example, information including the positions of a plurality of feature points predetermined for the face and links connecting the feature points. The feature points of the facial expression feature information may be configured to include, for example, the nose, the outer corners of the left and right eyes, the center of the pupil, both ends of the eyebrows, the forehead, both ends of the lips, and a plurality of points along the chin contour. The abnormality determination unit 163 may acquire the facial expression feature information by using a trained model obtained by performing machine learning processing in advance using a teacher image, for example. The technology used to implement the acquisition of the facial expression feature information does not need to be limited to the above-mentioned supervised learning, and may be any technology.
[0040] The abnormality determination unit 163 may acquire information (signal information) indicating a sign of frailty or dementia in the subject based on the facial expression feature information acquired by performing the above-mentioned processing, and may determine the presence or absence of an abnormality based on the acquired sign information. The abnormality determination unit 163 may estimate the sign information by using, for example, a trained model acquired by performing machine learning processing in advance using teacher data. For example, data to which the facial expression feature information and a label indicating the sign information in the facial expression feature information are assigned may be used as a dataset of teacher data.
[0041] The abnormality determination unit 163 records the estimated symptom information in the storage unit 17 in association with information indicating date and time. The abnormality determination unit 163 determines that there is an abnormality in the determination target when the past symptom information and the current symptom information satisfy a predetermined condition. The predetermined condition is a condition indicating that the current symptom information is more likely to show the above symptom than the past symptom information. For example, it may be determined that there is an abnormality when symptom information that is high enough to satisfy a predetermined condition (ratio or threshold) compared to past symptom information going back a predetermined time continues for a predetermined period (e.g., three days, one week, etc.).
[0042] When the notification condition is satisfied, the notification unit 164 transmits notification information indicating that it has been determined that there is an abnormality to another device (e.g., the control device 20) via the communication unit 11. The notification condition may be, for example, that it has been determined that there is an abnormality by the abnormality determination unit 163. The notification condition may also be defined as another condition.
[0043] The notification unit 164 may transmit a notification image to the user terminal 30. The notification image may include, for example, an image of the person to be determined. More specifically, the notification image may include an image of the past determination target and an image of the current determination target. The notification unit 164 may be configured to record the generated notification image in the storage unit 17, and transmit the notification image to the user terminal 30 that is the request source when a request for a notification image is received from the user terminal 30. The notification unit 164 may also transmit the notification image to the user terminal 30 that is the predetermined destination.
[0044] Next, a specific example of the image captured by the imaging unit 12 and the processing of the control unit 16 will be described. Fig. 3 and Fig. 4 are diagrams showing specific examples of background images. In the background image of Fig. 3, the floor is not littered with objects, and the desk on which the device is placed is also neatly arranged without objects being scattered. In the background image of Fig. 4, the floor is littered with objects (piles of books), and the desk on which the device is placed is also littered with objects. When such a background image is acquired, a low value of the degree of scattering is acquired in the background image of Fig. 3, and a high value of the degree of scattering is acquired in the background image of Fig. 4.
[0045] 5, 6, and 7 are diagrams showing specific examples of judgment target images. In the judgment target image of FIG. 5, the judgment target has a smiling face, the floor is not littered, and the desk on which the judgment target is placed is also neatly arranged without any scattered objects. In the judgment target image of FIG. 6, the judgment target does not have a smiling face, the hair is not neatly arranged, the floor is littered (books are piled up), and objects are scattered on the desk on which the judgment target is placed. In the judgment target image of FIG. 7, the judgment target does not have a smiling face, but has a blank face, the judgment target is not standing but sitting on the floor, the floor is littered (books are piled up), and objects are scattered on the desk on which the judgment target is placed. When such judgment target images are acquired, the judgment target image of FIG. 5 has a low scattering degree value and low symptom information, and the judgment target image of FIG. 6 has a high scattering degree value and high symptom information. Moreover, the judgment target image of FIG. 7 has even higher symptom information than that of FIG. 6.
[0046] Fig. 8 is a diagram showing a specific example of a notification image transmitted by the notification unit 164. In the example of Fig. 8, an image captured two months ago and an image captured one month ago are included as specific examples of past images of the determination target. Also, in the example of Fig. 8, an image of the determination target captured on the day the notification is made is included as an image of the current determination target. By including a past image and a current image in this way, a person who sees the notification image can easily understand changes in the state (e.g., facial expression and posture) of the determination target.
[0047] 9 is a schematic block diagram showing a specific example of a functional configuration of the control device 20. The control device 20 is configured using information equipment such as a personal computer, a server device, etc. The control device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.
[0048] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 performs data communication with other devices via the network 40 in response to control by the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.
[0049] The storage unit 22 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. For example, the storage unit 22 may store a destination information table. The destination information table is a table that associates identification information (hereinafter referred to as "target identification information") assigned to the imaging terminal 10 with information on the destination of user notification information generated based on notification information transmitted from the imaging terminal 10 (hereinafter referred to as "notification destination information").
[0050] The control unit 23 is configured using a processor such as a CPU and a memory. The control unit 23 functions as a notification control unit 231 and a data control unit 232 by the processor executing a program. All or part of the functions of the control unit 23 may be realized using hardware such as an ASIC, a PLD, or an FPGA. The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (e.g., SSD), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.
[0051] When the notification control unit 231 receives the notification information from the imaging terminal 10, it generates user notification information based on the received notification information. The user notification information includes, for example, a character string indicating that an abnormality has occurred in the determination target (for example, "An abnormality has occurred. Please check.") and notification address information indicating a storage area in which a notification image is stored. The character string included in the user notification information may be, for example, a fixed character string that is determined in advance regardless of the imaging terminal 10 or the user terminal 30. The character string included in the user notification information may be a character string defined in association with the imaging terminal 10 or the user terminal 30. These character strings may be stored in the storage unit 22.
[0052] The notification address information may be generated by the notification control unit 231, or may be generated by another server that publishes the notification image. For example, the notification control unit 231 may upload the notification image to a storage area accessible from other information devices. In this case, address information (e.g., a URL) indicating the storage area of the upload destination may be used as the notification address information.
[0053] The notification control unit 231 determines the notification destination based on the identification information included in the received notification information. The notification control unit 231 may determine, as the notification destination, the notification destination information associated with the target identification information that is the sender of the notification information, based on, for example, a destination information table stored in the storage unit 22. The notification control unit 231 transmits the user notification information to the determined notification destination.
[0054] The data control unit 232 registers the data transmitted from the imaging terminal 10 in the storage unit 22 in association with the target identification information of the imaging terminal 10 that is the transmission source. The data control unit 232 may register the data transmitted from the imaging terminal 10 in the storage unit 22 in association with the identification information (e.g., terminal ID) of the imaging terminal 10 that is the transmission source.
[0055] FIG. 10 is a sequence chart showing a specific example of the flow of the operation of the determination system 100. When the imaging unit 12 of the imaging terminal 10 captures an image, the control unit 16 acquires image data captured by the imaging unit 12 (step S101). The human body region estimation unit 162 estimates the human region of the determination target in the captured image (step S102). The abnormality determination unit 163 determines whether or not there is an abnormality in the determination target in the captured image (step S103). If there is no abnormality, that is, if there is a normality (step S104-NO), the process of the control unit 16 returns to the process of step S101. On the other hand, if there is an abnormality, that is, if there is no normality (step S104-YES), the notification unit 164 generates a notification image (step S105). The notification unit 164 generates notification information including the notification image and transmits the notification information to the control device 20 (step S106).
[0056] When the notification control unit 231 of the control device 20 receives the notification information, it generates user notification information according to the notification information. Furthermore, the notification control unit 231 determines a transmission destination of the user notification information according to the received notification information. Then, the notification control unit 231 transmits the user notification information to the user terminal 30 (step S107). Furthermore, the notification control unit 231 records the notification information in the storage unit 22 (step S108).
[0057] When the user terminal 30 receives the user notification information, it outputs information indicating that the user notification information has been received. For example, an image or text indicating that the user notification information has been received may be displayed in a predetermined area on the screen of the user terminal 30, or audio output or activation of a vibrator may be performed according to a notification method preset in the user terminal 30. When a user instructs the output of the user notification information by operating the user terminal 30, the user terminal 30 outputs the contents of the user notification information. For example, the user terminal 30 may display on the screen a character string indicating that an abnormality has occurred in the determination target and a character string indicating the notification address information (step S109). When a user operates the user terminal 30 to instruct the display of a notification image, the user terminal 30 accesses the notification address information and requests the transmission of the notification image (step S110).
[0058] When the control device 20 receives a request to transmit a notification image to the notification address information, the control device 20 transmits the notification image in response to the request to the user terminal 30 (step S111). When the user terminal 30 receives the notification image, the control device 20 displays the received notification image on the screen (step S112).
[0059] In the assessment system 100 configured in this manner, the signs of frailty and dementia are assessed based on images of the subject to be assessed and images of the subject's living space. This makes it possible to assess the risk of frailty and dementia at an earlier stage. For example, if the subject to be assessed lives alone, there is a possibility that the subject has few opportunities to come into contact with relatives, acquaintances, or monitoring companies and government officials. However, even in such cases, the signs of frailty and dementia can be assessed based on the above images, so that the risk can be assessed at an earlier stage even if the subject has few opportunities to actually come into contact with others. In addition, for example, if the subject to be assessed lives with relatives or acquaintances, it may be difficult for those who are in daily contact with the subject to notice changes in the subject. Even in such cases, the signs of frailty and dementia can be objectively assessed based on the above images, so that the risk can be assessed at an earlier stage.
[0060] In addition, in the determination system 100 configured in this manner, a determination is made on the signs of frailty or dementia based on the past image and the current image. Therefore, it is possible to perform a more accurate determination without being affected by differences in the individual characteristics of the subject. For example, when abnormality is determined simply based on the value of the degree of scattering, a subject who is originally not good at organizing may be determined to be abnormal even though there are no signs of frailty or dementia, resulting in an erroneous determination result. Conversely, a subject who is originally very good at organizing may not be determined to be abnormal even though there are signs of frailty or dementia, even if the degree of scattering has increased somewhat. However, as described above, a determination on signs of frailty or dementia is performed based on the past image and the current image, and a more accurate determination can be performed by taking into account the above-mentioned characteristics specific to the subject.
[0061] (Modification) The user notification information may include notification image data instead of notification address information. In this case, the user terminal 30 may display the notification image more smoothly. Furthermore, when the user terminal 30 receives the notification image data, the user terminal 30 may display the notification image data on the screen without receiving any operation from the user. By configuring in this manner, it is possible to more quickly notify the user of the abnormality being judged.
[0062] The control device 20 may include an abnormality determination unit 163. In this case, the imaging control unit 161 of the imaging terminal 10 is configured to transmit the captured image to the control device 20 via the communication unit 11. When the control device 20 determines that there is an abnormality, it notifies the user terminal 30. In this case, the control device 20 functions as a determination device.
[0063] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]
[0064] 100...judgment system, 10...imaging terminal, 20...control device, 30...user terminal, 40...network, 90...facility to be judged, 11...communication unit, 12...imaging unit, 13...audio input unit, 14...audio output unit, 15...display unit, 16...control unit, 161...imaging control unit, 162...human body region estimation unit, 163...abnormality judgment unit, 164...notification unit, 17...storage unit, 21...communication unit, 22...storage unit, 23...control unit, 231...notification control unit, 232...data control unit
Claims
1. an abnormality determination unit that determines the presence or absence of an abnormality indicating that the subject is showing a sign of frailty or a sign of dementia based on a past image and a current image captured thereafter among images captured in the living space of the subject; a notification unit that notifies another device that an abnormality has occurred when the abnormality determination unit determines that an abnormality has occurred; Equipped with The notification unit transmits a notification image including a past image and a current image of the person to be determined to the other device.
2. The determination device according to claim 1 , wherein the abnormality determination unit obtains a degree of scattering indicating the degree to which objects are scattered in the living space to be determined, and determines whether or not an abnormality exists based on the degree of scattering in a past image and the degree of scattering in a current image.
3. The abnormality determination unit obtains posture information that is information indicating the posture of the subject to be determined, obtains symptom information that is information indicating signs of frailty or dementia in the subject to be determined based on the posture information, and determines the presence or absence of an abnormality based on the symptom information in a past image and the symptom information in a current image.
4. The abnormality determination unit obtains facial expression information that is information indicating a facial expression of the subject to be determined, obtains symptom information that is information indicating a sign of frailty or a sign of dementia in the subject to be determined based on the facial expression information, and determines the presence or absence of an abnormality based on the symptom information in a past image and the symptom information in a current image.
5. An abnormality determination step in which a computer determines the presence or absence of an abnormality indicating that the subject is showing signs of frailty or dementia based on images captured in the living space of the subject, the images having been captured in the past and current images captured thereafter; a notification step of notifying another device that an abnormality exists when the abnormality is determined to exist in the abnormality determination step; having In the notification step, a notification image including a past image and a current image of the person to be determined is transmitted to the other device.
Citation Information
Patent Citations
Monitoring system and monitoring method
JP2013206012A
Monitoring method of changing tendency in disorderliness, monitoring program of changing tendency in disorderliness and monitor system of tendency changing tendency in disorderliness
JP2016128977A
Health management assist device and health management assist method
JP2016147006A
Diagnosis support system, operation method of diagnosis support system, and program
JP2018175052A
Obstacle determination method, obstacle determination program and information processing apparatus
JP2021033622A
