Information processing device, information processing method, and program

Abstract

To detect a variation in a plurality of items concerning physical conditions of a person without contact.SOLUTION: An information processing device 80 comprises: estimation units 802 and 803 that estimate in time series a plurality of items concerning physical conditions of a person on the basis of information obtained by detecting the person with a radar 10; a detection unit 806 that detects a variation in the physical conditions of the person on the basis of estimation results obtained in time series concerning the plurality of items; and an output unit 807 that outputs information concerning the detected variation in the physical conditions.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] There is a technique for estimating the walking speed of a person based on an image of the person captured by a surveillance camera and evaluating the cognitive function of the person based on the change in the estimated walking speed. In addition, there is a technique for monitoring (watching over) the behavior of a person in a dwelling based on the combination or order of electrical devices operated by the person, or based on the position information of a wireless tag attached to the person's clothing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regarding the technology for detecting changes in a person's physical condition, there is room for further consideration.

[0005] Non-limiting embodiments of the present disclosure contribute to providing an information processing apparatus, an information processing method, and a program capable of non-contact detection of changes in a plurality of items related to a person's physical condition.

Means for Solving the Problems

[0006] An information processing apparatus according to an embodiment of the present disclosure includes an estimation unit that estimates a plurality of items related to the physical state of a person in time series based on information obtained by detecting the person with a radar, a detection unit that detects a change in the physical state of the person based on the estimation results obtained in time series for the plurality of items, and an output unit that outputs information regarding the detected change in the physical state of the person.

[0007] These general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0008] A non-limiting example of the present disclosure can detect changes in a plurality of items related to a person's physical state without contact.

[0009] Further advantages and effects in an embodiment of the present disclosure will be clarified from the specification and drawings. Such advantages and / or effects are provided by some embodiments and the features described in the specification and drawings, respectively, but it is not necessarily required that all of them be provided in order to obtain one or more identical features.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same function are denoted by the same reference numerals, and redundant description is omitted.

[0012] <One Embodiment> It is important to detect changes in a person's physical condition at an early stage and take countermeasures in order to maintain health. For example, detecting changes in the sleep state or walking speed can lead to an understanding of the physical condition such as a person's concentration, motivation, appetite, and motor ability, and thus can be applied to the prevention of depression or injury, and further to the detection of dementia.

[0013] Examples of changes in a person's physical state include changes in the sleep state and walking speed (e.g., decrease). Examples of changes in the sleep state include changes such as increases or decreases in the time spent in non-REM sleep (deep sleep) state, increases or decreases in the time spent in apnea sleep state, and increases or decreases in the frequency of middle awakenings.

[0014] To detect changes in the physical state as described above, for example, the methods shown in the following (1) to (3) are considered.

[0015] (1) Detection of REM sleep or non-REM sleep state, or apnea sleep state using detection of pulse or respiration by a wearable device (2) Detection of REM sleep or non-REM sleep state, or middle awakening using detection of body movement, respiration, or pulse by a sensor mat (3) Detection of walking speed using an image of the person to be detected taken by a camera (hereinafter sometimes referred to as a "camera image").

[0016] In the following description, the terms "detection" or "detection" may be mutually read as terms such as "estimation" or "measurement", for example.

[0017] Here, in method (1), since the device is worn on the person to be detected, for example, it may give a sense of discomfort or discomfort to the person to be detected. Also, in method (2), items other than the sleep state (e.g., walking speed) are not measured.

[0018] In method (3), since a camera image of the person to be detected is used, for example, privacy infringement may be a problem. Also, in method (3), when estimating the position change of the person to be detected based on the change in the size of the person to be detected in the camera image, the size in the camera image may not change depending on the movement path of the person to be detected. Therefore, it may be difficult to estimate the walking speed.

[0019] In one embodiment of the present disclosure, for example, temporal changes (in other words, changes over time) of a plurality of items related to the physical state of a person who is the detection target are detected and presented non-contactingly by a single radio wave sensor (for example, a millimeter wave radar). A non-limiting example of the plurality of items to be detected is "change in sleep state" and "decrease in walking speed".

[0020] For example, since a millimeter wave radar can three-dimensionally detect the position of a person non-contactingly, it can calculate the moving distance of the person without depending on the moving path of the person, and thus can estimate the walking speed of the person.

[0021] In addition, for example, since a millimeter wave radar can detect body movements (including, for example, fluctuations in the body surface accompanying heartbeat or breathing) of the detected person, it can estimate the sleep state (for example, the depth of sleep such as non-REM sleep and REM sleep) based on body movements during sleep. In addition, it is possible to estimate the apnea sleep state based on the presence or absence of breathing during sleep.

[0022] <System configuration example> FIG. 1 is a diagram showing a configuration example of a physical state detection system 1 according to an embodiment. As shown in FIG. 1, the physical state detection system 1 may include, for example, a millimeter wave radar 10, a personal computer (PC) 20, servers 30A and 30B, and databases (DBs) 40A and 40B.

[0023] These components 10, 20, 30A, 30B, 40A, and 40B may be communicably connected to each other via a network 50 such as the Internet or a local area network. The connection form of the components 10, 20, 30A, 30B, 40A, and 40B may be either wired or wireless, or a mixture of wired and wireless. Each of the PC 20, the server 30A, and the server 30B is an example of an information processing device.

[0024] The millimeter-wave radar 10 is an example of a radio wave sensor. For example, it is installed in a living room within a building 60 and non-contact detects a person P located within the living room by radio waves (radar waves). Non-limiting examples of the building 60 include a residence, a store, a dormitory, a condominium, an office, a hotel, a restaurant, a school building, a research institute, a hospital, a clinic, and the like.

[0025] "Detection" may be read as, for example, "detection" or "sensing". The frequency of the millimeter-wave radar 10 may be, as a non-limiting example, a frequency in the range of several tens of gigahertz (GHz) to several hundreds of GHz, and may be, for example, a frequency in the 60 GHz band.

[0026] The PC 20 acquires (or receives), for example, a detection result by the millimeter-wave radar 10 (hereinafter may be referred to as "radar detection result", "radar detection information", or "radar data") from the millimeter-wave radar 10.

[0027] The radar detection result may be transmitted, for example, from the PC 20 to one or both of the servers 30A and 30B via the network 50. Note that when the millimeter-wave radar 10 has a connection (or access) function to the network 50 (in other words, a communication function), the radar detection result may be transmitted to one or both of the servers 30A and 30B without passing through the PC 20.

[0028] Further, the PC 20 may display, for example, information received from one or both of the servers 30A and 30B via the network 50 on a display unit (display) of the PC 20. For example, information processed or generated based on the radar detection result in one or both of the servers 30A and 30B (for example, information regarding changes in the physical state of the person P) may be displayed on the PC 20. Information received via the network 50 may be printed on a printer connected to the PC 20.

[0029] Note that instead of or in addition to the PC 20, a mobile terminal such as a smartphone may perform at least some of the functions equivalent to those of the above-described PC 20.

[0030] One of the servers 30A and 30B (for example, server 30A) detects (or estimates or measures) information such as the position, body movement, and respiration rate (or heart rate) of the person P in the room based on, for example, the radar detection result. The detected information may be sequentially (or in other words, time-series) stored (or accumulated) in, for example, the DB 40A.

[0031] Further, the server 30A reads out information for a predetermined period (for example, one day's worth) from the DB 40A, for example, and detects (or estimates or measures) information such as the sleep state of the person P (for example, at least one time zone of non-REM sleep, apnea sleep, and intermediate awakening), and walking speed. The detected information may be stored (or accumulated) in the DB 40B in time series (for example, daily).

[0032] The server 30B determines (or detects) whether there is a change in the physical state of the person P based on, for example, the time-series information (hereinafter sometimes referred to as "time-series data") accumulated in the DB 40B. When it is determined that there is a change in the physical state of the person P, the server 30B provides (or notifies) information indicating that there is a change in the physical state of the person P (hereinafter sometimes referred to as "physical state change information") to the PC 20 via the network 50. The PC 20 displays the received physical state change information on, for example, the display of the PC 20 or prints it by a printer. Note that the display or printing of the physical state change information is not limited to the PC 20 and may be performed in the server 40A or 40B.

[0033] Note that the detection (or estimation or measurement) of various information such as the position, body movement, respiration rate (or heart rate), sleep state, and walking speed of the person P may be performed centrally or dispersedly based on the radar detection result in one or both of the servers 30A and 30B. In other words, it is not necessary that a specific relationship exists in which specific information among various information (which may also be referred to as "measurement items") is detected in a specific server.

[0034] For example, the servers 30A and 30B may be integrated into, for example, one server. Also, the processing related to the detection of the above-described measurement items may be performed distributively by three or more servers.

[0035] The DBs 40A and 40B may be provided inside the servers 30A and 30B, respectively. In other words, the storage units or storage devices provided in the servers 30A and 30B may correspond to the DBs 40A and 40B. The DBs 40A and 40B may be integrated into, for example, one DB. In three or more DBs, the above-described measurement items may be stored and managed distributively.

[0036] FIG. 2 is a block diagram showing a configuration example of the physical state detection system 1 according to an embodiment. In FIG. 2, the millimeter-wave radar 10 may include, for example, a detection unit 101, a processing unit 102, and an output unit 103.

[0037] The detection unit 101 transmits, for example, radar waves and receives and detects the reflected waves reflected by the detection target (for example, the person P).

[0038] The processing unit 102 detects information such as the position, speed, and angle of the detection target with respect to the millimeter-wave radar 10 based on, for example, the point cloud data corresponding to the detection target obtained from the reflected waves.

[0039] The output unit 103 transmits, for example, the information obtained by the processing unit 102 (in other words, the radar data) to the radar control device 70.

[0040] The radar control device 70 may include, for example, a radar communication unit 701 and a radar data storage unit 702. Note that the radar control device 70 may correspond to, for example, the PC 20 illustrated in FIG. 1, or may correspond to one or both of the servers 30A and 30B illustrated in FIG. 1.

[0041] The radar communication unit 701 communicates with, for example, the millimeter-wave radar 10 either wired or wirelessly, receives radar data, and stores (memorizes) it in the radar data storage unit 702. Further, the radar communication unit 701 transmits, for example, the radar data to the state change detection device 80 either wired or wirelessly.

[0042] The state change detection device 80 detects, for example, a change in the physical state of a detection target (e.g., person P) based on the radar data. The state change detection device 80 is an example of an information processing device and may correspond to, for example, one or both of the servers 30A and 30B illustrated in FIG. 1.

[0043] The state change detection device 80 may include, for example, a radar data reading unit 801, a sleep state estimation unit 802, a walking speed estimation unit 803, an estimation result storage unit 804, an estimation result reading unit 805, a state change detection processing unit 806, a communication unit 807, a radar data storage unit 808, and an estimation result information storage unit 809.

[0044] The radar data reading unit 801 reads, for example, the radar data stored in the radar data storage unit 808 and outputs it to the sleep state estimation unit 802. Note that the radar data storage unit 808 may correspond to, for example, the DB40A illustrated in FIG. 1.

[0045] The sleep state estimation unit 802 estimates, for example, the sleep state of person P (e.g., non-REM sleep, apnea sleep, and time zones of waking up midway) who is the detection target based on the radar data.

[0046] The walking speed estimation unit 803 estimates, for example, the walking speed of person P based on the radar data. The estimation of the walking speed may not be executed, for example, during the sleep time zone estimated by the sleep state estimation unit 802, and thus may be executed for the time zones excluding the estimated sleep time zone of person P.

[0047] The sleep state estimation unit 802 and the walking speed estimation unit 803 may be integrated into, for example, one estimation unit.

[0048] The estimation result storage unit 804 stores (memorizes) each estimation result by, for example, the sleep state estimation unit 802 and the walking speed estimation unit 803 in the estimation result information storage unit 809. Note that the estimation result information storage unit 809 may be integrated with, for example, the radar data storage unit 808. Further, the estimation result information storage unit 809 may correspond to the DB40B illustrated in FIG. 1.

[0049] The estimation result reading unit 805 reads out, for example, the estimation result stored in the estimation result information storage unit 809 and outputs it to the state change detection processing unit 806.

[0050] The estimation result storage unit 804 and the estimation result reading unit 805 may be integrated into, for example, one write and read control unit.

[0051] The state change detection processing unit 806 detects, for example, a change in the physical state of the person P based on the estimation result input from the estimation result reading unit 805. The detection result may be output to, for example, the communication unit 807.

[0052] The communication unit 807 is an example of an output unit, and communicates with, for example, the display terminal device 90 (for example, the communication unit 901) wirelessly or by wire, and transmits the detection result regarding the change in the physical state of the person P to the display terminal device 90.

[0053] The display terminal device 90 may include, for example, a communication unit 901 and a display unit 902. Note that the display terminal device 90 may correspond to, for example, the PC20 illustrated in FIG. 1.

[0054] The communication unit 901 communicates with, for example, the state change detection device 80 (for example, the communication unit 807) wirelessly or by wire, receives the detection result regarding the change in the physical state of the person P, and outputs the received information to the display unit 902.

[0055] The display unit 902 displays, for example, the information input from the communication unit 901.

[0056] The configuration example of the physical state detection system 1 according to one embodiment has been described above.

[0057] <Operation Example> Next, an operation example of the body state detection system 1 having the above-described configuration will be described with reference to the flowcharts of FIGS. 3 and 4.

[0058] FIG. 3 is a flowchart showing an example of the overall operation of the body state detection system 1 according to an embodiment. FIG. 4 is a flowchart showing an example of the sleep time zone detection process (S803) illustrated in FIG. 3.

[0059] As shown in FIG. 3, radar data is detected in the millimeter-wave radar 10 (S101), and the detected radar data is stored (memorized) in, for example, the radar data storage unit 808 of the state change detection device 80 (S102).

[0060] The state change detection device 80 reads radar data from the radar data storage unit 808 regularly by, for example, the radar data reading unit 801. For example, the radar data reading unit 801 reads one day's worth of radar data from the radar data storage unit 808 at 1:00 every day (S801, S802).

[0061] The radar data read from the radar data storage unit 808 is output to, for example, the sleep state estimation unit 802, and the sleep state estimation unit 802 estimates the sleep state of the person P who is the detection target based on the radar data (S803).

[0062] For example, the sleep state estimation unit 802 determines the presence or absence of a sleep time zone based on, for example, one day's worth of radar data (S804). An example of the sleep time zone detection process will be described later with reference to FIG. 4. As a result of the determination, if there is no sleep time zone (S804; NO), for example, the estimation by the sleep state estimation unit 802 may end and the estimation by the walking speed estimation unit 803 may be executed (S808).

[0063] If there is a sleep time zone (S804; YES), the sleep state estimation unit 802 estimates, for example, the time zones of non-REM sleep, apnea sleep, and intermediate awakening (S805 to S807).

[0064] Next, for example, the walking speed estimation unit 803 estimates the walking speed of the person P based on the radar data (S808).

[0065] The estimation results obtained by each of the processes of S805 to S808 are stored (memorized) in the estimation result information storage unit 809 by, for example, the estimation result storage unit 804 (S809).

[0066] Note that the processes of S805 to S808 may be performed serially as described above, or may be performed in parallel. When performed serially, the processing order of S805 to S807 may be appropriately changed.

[0067] The estimation result stored in the estimation result information storage unit 809 is read out by, for example, the estimation result reading unit 805 and output to the state change detection processing unit 806 (S810).

[0068] The state change detection processing unit 806 detects, for example, a change in the physical state of the person P based on the estimation result input from the estimation result reading unit 805 (S811). For example, the state change detection processing unit 806 determines whether there is a change in the physical state of the person P based on each estimation result of the sleep state and the walking speed (S812).

[0069] As a result of the determination, if there is no change in the physical state of the person P (S812; NO), the state change detection processing unit 806 may end the processing. On the other hand, if there is a change in the physical state of the person P (S812; YES), the state change detection processing unit 806 transmits, for example, information regarding the change in the physical state of the person P to the display terminal device 90 by the communication unit 807 (S813).

[0070] When the presentation terminal device 90 receives information regarding a change in the physical state of the person P from the state change detection device 80 (S901), it displays the received information on, for example, the screen of the display unit 902 (S902).

[0071] <Estimation of Sleep State> Next, the estimation (or detection) of the sleep state of the person P (for example, each time zone of sleep time zone, non-REM sleep, apnea sleep, and intermediate awakening) in the sleep state estimation unit 802 will be described item by item.

[0072] <Detection of Sleep Time Zone (S803 in FIG. 3)> As illustrated in FIG. 4, the sleep state estimation unit 802 reads, for example, the setting of the sleep location definition area 501 of the person P (see FIG. 5) (S831). The setting of the sleep location definition area 501 may be performed, for example, by the sleep state estimation unit 802 or may be performed by a setting unit (not shown) in the state change detection device 80.

[0073] The setting data of the sleep location definition area 501 may be stored in, for example, either one of the storage units 808 and 809, or may be stored in another storage unit (not shown) in the state change detection device 80.

[0074] The sleep location definition area 501 may be set as a three-dimensional (3D) space for the place where the person P goes to bed (for example, bedding such as a bed or futon) in the living room, as shown in the upper part of FIG. 5.

[0075] In the example in the upper part of FIG. 5, a sleep location definition area 501 defined by XYZ coordinates is set in the upper space of the place where the person P goes to bed. The size (area) of the XY plane of the sleep location definition area 501 may be set to a size corresponding to the size of bedding such as a bed, mattress, or futon.

[0076] For example, the size of the XY plane of the sleep area definition area 501 may be set to a size that matches the size of the bedding, or may be set to a size slightly smaller or slightly larger than the size of the bedding. Alternatively, the sleep area definition area 501 may be set to a spatial size corresponding to the three-dimensional space occupied by at least one of the supine position, prone position, and lateral lying position of the person P in the sleeping position.

[0077] For example, the length (or height) of the sleep area definition area 501 in the Z-axis direction may be set to a height corresponding to the height (or body height) of the person P in the sleeping posture (or body position). For example, it may be set according to the highest body height among the body heights of the person P in the supine position, prone position, and lateral lying position of the person P.

[0078] According to such a setting, for example, when the person P raises the upper body while staying in the sleep area definition area 501, at least a part of the upper body of the person P comes out of the sleep area definition area 501. Therefore, the raising of the upper body of the person P can be detected by the millimeter wave radar 10.

[0079] Note that the height of the sleep area definition area 501 may be set based on the upper surface of bedding such as a bed, a mattress, and a futon as illustrated in FIG. 5, or may be set based on the floor surface on which the bedding is placed. In other words, the sleep area definition area 501 may be set as a space covering the bedding, for example, or may be set as a space not covering the bedding.

[0080] In the lower part of FIG. 5, for example, as illustrated in the upper part of FIG. 5, examples are shown in which the position (XYZ coordinate position) of the person P is detected as radar data by the millimeter wave radar 10 when the person P is located inside and outside the sleep area definition area 501, respectively.

[0081] By setting the sleep location definition area 501, for example, it is possible to detect based on radar data (e.g., the position information of person P) whether person P has entered the sleep location definition area 501 for bedtime or whether person P has woken up (including intermediate awakenings) and exited outside the sleep location definition area 501.

[0082] Therefore, for example, as follows, by setting the sleep start determination condition and the sleep end determination condition based on the entry detection and exit detection of person P with respect to the sleep location definition area 501, the sleep time zone of person P can be detected (or estimated).

[0083] (1) Sleep start determination condition: Detection of entry into the sleep location definition area 501 (2) Sleep end determination condition: Detection of exit from the sleep location definition area 501 and no detection of entry into the sleep location definition area 501 within the subsequent N minutes

[0084] N may be set to, for example, a time when it can be determined that person P will not return to the bedtime location, for example, a time of about several tens of minutes such as 30 minutes. In other words, if entry into the sleep location definition area 501 is (re)detected within N minutes or less, the sleep end determination condition is not satisfied, so it may be determined that the sleep time of person P has not ended and continues. Note that the time zone of N minutes or less may be determined as the time zone when person P has an intermediate awakening.

[0085] Returning to FIG. 4, hereinafter, the detection process of the sleep time zone based on the above-mentioned conditions (1) and (2) will be described.

[0086] After reading the setting of the sleep location definition area 501 as described above, the sleep state estimation unit 802, for example, acquires the data at the start time among the radar data (S832), and determines whether the data satisfies the sleep start determination condition (1) (S833).

[0087] If, as a result of the determination, it is determined that the data at the start time does not satisfy the sleep start determination condition (1) (S833; NO), the sleep state estimation unit 802 acquires the data at the next time (S835) and determines whether the data at the next time satisfies the sleep start determination condition (1), as long as the data at the next time exists (S834; YES).

[0088] If data that satisfies the sleep start determination condition (1) exists (S833; YES), the sleep state estimation unit 802 determines whether the data at the next time satisfies the sleep end determination condition (2) (S836).

[0089] If, as a result of the determination, it is determined that the data at the next time does not satisfy the sleep end determination condition (2) (S836; NO), the sleep state estimation unit 802 acquires the data at the next time (S838) and determines whether the data at the next time satisfies the sleep end determination condition (2), as long as the data at the next time exists (S837; YES).

[0090] If, as a result of the determination, it is determined that the data at the next time satisfies the sleep end determination condition (2) (S836; YES), the sleep state estimation unit 802 records, for example, the time period between the time of the data that satisfies the sleep start determination condition (1) and the time of the data that satisfies the sleep end determination condition (2) as the sleep time period of the person P (S839).

[0091] Thereafter, the sleep state estimation unit 802 checks whether there is data at the next time (S840). If there is data at the next time (S840; YES), the sleep state estimation unit 802 acquires the data at the next time (S841) and determines whether the data at the next time satisfies the sleep start determination condition (1) (S833).

[0092] In this way, the sleep state estimation unit 802 searches for data that satisfies the sleep start determination condition (1) and then data that satisfies the sleep end determination condition (2) in the order of the times of the radar data obtained in time series, and detects and records the time period between the times of both pieces of data as the sleep time period.

[0093] In such data search, for example, when data for the next time does not exist (NO in S834, S837, and S840), the sleep state estimation unit 802 may end the detection process for the sleep time zone. In response to the end of the detection process for the sleep time zone, for example, the processes after S804 illustrated in FIG. 3 are executed.

[0094] FIG. 6 shows an example of the detection of the sleep time zone. In FIG. 6, illustratively, within one day (24 hours from 0:00:00), the sleep time zones of person P are detected in the time zones of 00:00:00 to 07:00:00 (7 hours) and 22:00:00 to 24:00:00 (4 hours), and an example is shown where a mid-awakening is presumed to have occurred to person P in the time zone of 01:10:20 to 01:20:20 (20 minutes < N).

[0095] The estimation of the time zones of non-REM sleep, apnea sleep, and mid-awakening may be performed for the sleep time zone detected as described above. Hereinafter, the estimation of the time zones of non-REM sleep, apnea sleep, and mid-awakening will be described.

[0096] <Estimation of non-REM sleep time zone (S805 in FIG. 3)> As an example of an index that can determine (or estimate) whether a person is in non-REM sleep, the respiratory rate and the pulse rate can be mentioned. For example, during non-REM sleep, the respiratory rate and the pulse rate tend to be stable and regular (in other words, periodic). In contrast, during non-REM sleep, the heart rate and the respiratory rate tend to increase and become irregular compared to non-REM sleep.

[0097] Therefore, the sleep state estimation unit 802 may, for example, based on the radar data, measure the respiratory state (for example, the respiratory rate and the pulse rate) of person P in the detected sleep time zone, and estimate the time zone in which a periodic waveform is continuously observed as non-REM sleep. For example, as shown in FIG. 7, the time zone of 01:10:25 to 01:50:14 may be estimated as the time zone of non-REM sleep. Note that the vertical axis in FIG. 7 represents the phase change of the radio wave signal due to the movement of the chest, and the unit is, for example, [rad]. The same applies to the vertical axis in FIG. 8 as in FIG. 7.

[0098] <Estimation of apnea sleep time zone (S806 in FIG. 3)> The sleep state estimation unit 802 may measure, for example, the breathing state (e.g., respiratory rate and pulse rate) of the person P in the detected sleep time zone based on the radar data, and estimate the time zone during which breathing detection is interrupted as the apnea sleep state. For example, as shown in FIG. 8, the time zone from 03:20:27 to 03:20:36 may be estimated as the time zone of apnea sleep.

[0099] <Estimation of intermediate awakening time zone (S807 in FIG. 3)> For example, when the sleep state estimation unit 802 detects, based on the radar data, an act in which the person P temporarily leaves the sleeping place (sleeping place definition area 501) during sleep, or an act in which the person P raises the upper body, the sleep state estimation unit 802 may estimate the duration as the time when intermediate awakening occurs.

[0100] For example, as shown in the upper part of FIG. 9, assume that the person P leaves the sleeping place (sleeping place definition area 501) at a certain time (e.g., 01:10:20) in the sleep time zone and returns to the sleeping place at 01:20:20, 10 minutes later.

[0101] In this case, the movement (or rather, the position change) of the person P in the radar data is represented as shown in the lower part of FIG. 9. For example, the departure of the person P from the sleeping place is detected at the time 01:10:20, and the entry of the person P into the sleeping place is detected at the time 01:20:20. Therefore, the sleep state estimation unit 802 may estimate, for example, the 10-minute time zone from 01:10:20 to 01:20:20 as the time zone of intermediate awakening.

[0102] In addition, as shown in the upper part of FIG. 10, when the person P raises the upper body while staying in the sleeping place, at least a part of the upper body moves out of the sleeping place definition area 501 as described above. Therefore, as shown in the lower part of FIG. 10, the raising of the upper body of the person P can be detected based on the radar data. Therefore, the sleep state estimation unit 802 may include, for example, the time zone in which the raising of the upper body of the person P is detected in this way in the time zone of intermediate awakening.

[0103] <Estimation of walking speed (S808 in FIG. 3)> Next, the estimation of the moving speed (e.g., walking speed) of the person P by the walking speed estimation unit 803 (see FIG. 2) will be described. The walking speed estimation unit 803 calculates, for example, the distance (D) that the person P has moved in the room based on radar data, and estimates the walking speed (V) of the person P by dividing the distance by the time (T) taken for the movement.

[0104] For example, as shown in the upper part of FIG. 11, when the person P moves from time t1 to time t4, as shown in the lower part of FIG. 11, the walking speed estimation unit 803 calculates the moving distance D of the person P as D = d1 + d2 + d3 based on radar data, for example. Therefore, the walking speed V of the person P is estimated as V = (d1 + d2 + d3) / (t4 - t1), where the time taken for the movement is T = t4 - t1.

[0105] Note that the estimation of the walking speed V may be performed for a time period excluding the sleep time period detected in the above-described sleep time period detection process (S803 in FIG. 3) (in other words, a time period determined not to be the sleep time period).

[0106] In this way, by narrowing down (or limiting) the time period for estimating the walking speed V to the non-sleep time period, the amount of radar data used for estimating the walking speed can be reduced, and thus the processing amount of the state change detection device 80 can be reduced.

[0107] <Reduction of the influence of radar detection position error on the moving distance> When the acquisition frequency of radar data by the millimeter-wave radar 10 is, for example, several tens of milliseconds to several hundreds of milliseconds, or when there is an error in the detection position of several centimeters (cm) to several tens of cm, fluctuations or deviations in the trajectory that do not match the actual movement of the person P (e.g., the upper part of FIG. 12) may occur in the radar radar (e.g., the middle part of FIG. 12).

[0108] Therefore, when simply integrating the detection positions based on the radar data to calculate the moving distance D, an error outside the allowable range may occur with respect to the actual moving distance of the person P. Thus, the walking speed estimation unit 803 may perform correction (for example, moving average filtering) on the detection positions based on the radar data, as shown in the lower part of FIG. 12, for example. Thereby, the temporal change (in other words, the trajectory) of the detection position of the person P can be made to coincide with the actual movement trajectory of the person P, or brought closer to the minimum error, and the estimation accuracy of the walking speed V can be improved.

[0109] <State change detection (S811 in FIG. 3)> Next, an example of the state change detection process (S811 in FIG. 3) by the state change detection processing unit 806 (see FIG. 2) will be described.

[0110] The state change detection processing unit 806 detects a change point in the trend variation (in other words, the trend) of the physical state of the person P based on, for example, the sleep state (for example, each time zone of non-REM sleep, apnea sleep, and intermediate awakening) estimated by the sleep state estimation unit 802, and the time series data of the walking speed estimated by the walking speed estimation unit 803.

[0111] For example, the state change detection processing unit 806 determines the presence or absence of a change point in the trend variation of the time series data separately for the sleep state and the walking speed. When a change point is detected, the state change detection processing unit 806 creates a prediction model for the time series data after the change point based on the time series data before the change point, for example. For creating the prediction model, for example, an autoregressive (AR) model may be used.

[0112] For example, as shown in FIG. 13(a), when a change in the trend variation is detected in the time series data regarding non-REM sleep on January 5, as shown in FIG. 13(e), a prediction model for after January 5 is created based on the time series data before January 4 (for example, the time series data for several days such as the 3rd).

[0113] Note that the prediction model may be individually created when a change point in the trend variation is detected for each time series data of apnea sleep, middle awakening, and walking speed, for example, illustrated in FIGS. 13(b) to 13(d).

[0114] Then, the state change detection processing unit 806 determines, for example, whether the abnormality degree is equal to or greater than a threshold value using the difference between the measured value and the predicted value by the prediction model as an index of the abnormality degree, and detects the timing at which it is determined that the abnormality degree is equal to or greater than the threshold value as a change point in the trend variation. For example, FIG. 13(e) shows an example in which a change (in other words, an abnormality) in the trend variation is detected on January 5th for the time zone of non-REM sleep.

[0115] The state change detection processing unit 806 transmits, for example, information on the measurement item for which a change point in the trend variation is detected, separately from the measurement items of non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, to the display terminal device 90 as information regarding the change in the physical state of the person P.

[0116] The display terminal device 90 displays the information received from the state change detection device 90 on the screen of, for example, the display unit 902 of the display terminal device 90. Thereby, information regarding the change in the physical state of the person P is presented to, for example, the person P himself / herself or other persons involved in the health management of the person P (for example, the family, caregiver, administrator, attending doctor, etc. of the person P).

[0117] FIG. 14 shows an example of the screen display (GUI) of the display unit 902. FIG. 14 shows an example in which the display screen of the display unit 902 is divided into a state display area 921 and a notification area 922. As illustrated in FIG. 14, the time series data respectively illustrated in FIGS. 13(a) to 13(d) are displayed in the state display area 921, and changes in the physical state of the person P, for example, detection of a decrease in non-REM sleep time and detection of an increase in apnea sleep time may be displayed in the notification area 922.

[0118] Note that the screen display example of the display unit 902 is not limited to the example shown in FIG. 14. For example, either the status display area 921 or the notification area 922 may be selectively displayed. In the status display area 921, any one or more of the time-series data illustrated in FIGS. 13(a) to 13(d) may be selectively displayed. For example, in the status display area 921, the time-series data that contributed to the detection of the change in the physical state may be selectively displayed. Alternatively or additionally, the time-series data that contributed to the detection of the change in the physical state may be highlighted in the status display area 921. The mode of highlighting is not limited. Also, the display mode in the notification area 922 is not limited to the example in FIG. 14. In combination with the display on the notification area 922, a sound such as an alarm sound may be used to prompt attention to the detection of the change in the physical state.

[0119] As described above, according to the above-described embodiment, changes in the physical state of the person P, such as non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, can be detected and presented non-contact by the millimeter-wave radar 10.

[0120] Therefore, for example, since the physical state of the person P, such as concentration, motivation, appetite, and motor ability, corresponding to changes in the sleep state and walking speed of the person P can be grasped early, it becomes possible to take measures for maintaining the health of the person P at an early stage. For example, it can be applied to the prevention of depression or injury of the person P, and further to the detection of dementia.

[0121] Also, according to the above-described embodiment, based on time-series data with a unified time axis (in other words, timing-synchronized), such as daily data, a plurality of measurement items related to the physical state of the person P, such as non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, are detected or estimated. Therefore, the timing at which a change occurs in the physical state of the person P can be easily grasped for each measurement item.

[0122] Therefore, for example, based on the change points (e.g., the degree of match or mismatch) of the trend variations for each measurement item related to the physical state, it may be easier to identify the factors presumed to have caused a change in the physical state of person P. Also, for example, based on the correlation between the change points among multiple measurement items, the factors of the change in the physical state of person P can be analyzed and estimated comprehensively.

[0123] <Overall supplement> The sleep location definition area 501 (see FIG. 5) does not have to be set as a three-dimensional space. For example, it may be set in two dimensions (e.g., the X - Y axis excluding the Z - axis direction).

[0124] The items to be measured by the millimeter - wave radar 10 do not have to be all of non - REM sleep time, apnea sleep time, middle - awakening time, and walking speed. For example, it may be a combination of some of the measurement items. Also, the items to be measured by the millimeter - wave radar 10 are not limited to the four items of non - REM sleep time, apnea sleep time, middle - awakening time, and walking speed.

[0125] Some of the four items may be replaced by other items related to the physical state of person P that can be detected by the millimeter - wave radar 10. Also, five or more items related to the physical state of person P that can be detected by the millimeter - wave radar 10 may be the measurement targets by the millimeter - wave radar 10.

[0126] Also, the person to be detected by the millimeter - wave radar 10 is not limited to one person and may be two or more persons. According to the distance resolution of the millimeter - wave radar 10, for example, body movements and movements can be detected for each person, so the changes in the physical state described above can be detected separately for multiple persons.

[0127] The embodiments have been described above with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various modifications or alterations within the scope described in the claims, and it is naturally understood that they also belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the disclosure, the components in the above embodiments may be arbitrarily combined.

[0128] Moreover, the specific examples of the present disclosure are merely illustrative and do not limit the claims. The technology described in the claims includes those obtained by variously modifying and changing the specific examples exemplified above.

[0129] Also, the notation “··· part” in the above-described embodiments may be mutually replaced with other notations such as “··· circuitry”, “··· device”, “··· unit”, or “··· module”.

[0130] Also, the present disclosure can be realized by software, hardware, or software in cooperation with hardware. For example, the functions of the above-described system can be realized by a computer program.

[0131] FIG. 15 is a diagram showing the hardware configuration of a computer (or information processing device) that realizes the functions of each device constituting the above-described passage management system 1 by a program. In FIG. 15, a computer 1100 includes an input device 1101 such as a keyboard, mouse, or touch pad, an output device 1102 such as a display or speaker, a CPU (Central Processing Unit) 1103, a GPU (Graphics Processing Unit) 1104, a ROM (Read Only Memory) 1105, a RAM (Random Access Memory) 1106, a storage device 1107 such as a hard disk device or SSD (Solid State Drive), a reading device 1108 that reads information from a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or USB (Universal Serial Bus) memory, and a transmission / reception device 1109 that communicates via a network. Each unit is connected by a bus 1110.

[0132] The reading device 1108 reads the program from a recording medium storing a program for realizing the functions of the terminal PC 20, for example, and stores the program in the storage device 1107. Alternatively, the transmission / reception device 1109 communicates with a server device connected to the network (which may be the server 30 or a server device different from the server 30), and stores the program for realizing the functions of each of the above devices downloaded from the server device in the storage device 1107.

[0133] The CPU 1103 copies the program stored in the storage device 1107 to the RAM 1106, and sequentially reads and executes the instructions included in the program from the RAM 1106, thereby realizing the functions of the terminal PC 20 according to the above-described embodiment.

[0134] Each functional block used in the description of the above-described embodiments may be realized, partially or wholly, as an integrated circuit LSI, and each process described in the above embodiments may be controlled, partially or wholly, by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include some or all of the functional blocks. The LSI may be provided with data input and output. Depending on the degree of integration, the LSI may also be referred to as an IC, a system LSI, a super LSI, or an ultra LSI.

[0135] The method of integrating into an integrated circuit is not limited to LSI, and it may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after manufacturing the LSI, an FPGA (Field Programmable Gate Array) that can be programmed, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0136] Furthermore, if a technology for integrating into an integrated circuit that replaces the LSI appears due to the progress of semiconductor technology or another derived technology, naturally, the integration of functional blocks may be performed using that technology. The application of biotechnology, etc. may be possible as a possibility.

[0137] The present disclosure can be implemented in any type of apparatus, device, system having a communication function (collectively referred to as a communication device). The communication device may include a wireless transceiver (transceiver) and a processing / control circuit. The wireless transceiver may include a receiving unit and a transmitting unit, or may include them as functions. The wireless transceiver (transmitting unit, receiving unit) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of the communication device include a telephone (mobile phone, smartphone, etc.), a tablet, a PC (laptop, desktop, notebook, etc.), a camera (digital still / video camera, etc.), a digital player (digital audio / video player, etc.), a wearable device (wearable camera, smartwatch, tracking device, etc.), a game console, a digital book reader, a telehealth / telemedicine (remote healthcare / medical prescription) device, a vehicle or mobile transportation means with a communication function (automobile, airplane, ship, etc.), and combinations of the various devices described above.

[0138] The communication device is not limited to being portable or movable, and includes any type of apparatus, device, system that is not portable or is fixed, such as a smart home device (home appliance, lighting device, smart meter or measuring device, control panel, etc.), a vending machine, and any "Thing" that can exist on the IoT (Internet of Things) network.

[0139] In recent years, in IoT (Internet of Things) technology, CPS (Cyber Physical Systems), which is a new concept of creating new added value through information cooperation between the physical space and the cyber space, has attracted attention. This CPS concept can also be adopted in the above embodiments.

[0140] That is, as a basic configuration of CPS, for example, an edge server arranged in the physical space and a cloud server arranged in the cyber space are connected via a network, and the processing can be distributed and processed by processors mounted on both servers. Here, each piece of processing data generated in the edge server or the cloud server is preferably generated on a standardized platform. By using such a standardized platform, it is possible to improve the efficiency when constructing a system including various types of sensor groups and IoT application software.

[0141] For communication, in addition to data communication by a cellular system, a wireless LAN system, a communication satellite system, etc., data communication by a combination of these is also included.

[0142] In addition, the communication device also includes devices such as a controller and a sensor that are connected or linked to a communication device that executes the communication function described in the present disclosure. For example, a controller and a sensor that generate a control signal or a data signal used by the communication device that executes the communication function of the communication device are included.

[0143] In addition, the communication device also includes infrastructure facilities such as a base station, an access point, and any other devices, apparatuses, and systems that communicate with or control the above-described various non-limiting devices.

Industrial Applicability

[0144] One embodiment of the present disclosure is suitable for detecting changes in a person's physical state, for example.

Explanation of Signs

[0145] 1 Body state detection system 10 Millimeter-wave radar 20 Personal computer (PC) 30A, 30B Server 40A, 40B Database (DB) 70 Radar control device 80 State change detection device 90 Display terminal device 101 Detection unit 102 Processing unit 701 Radar communication unit 702 Radar data storage unit 801 Radar data reading unit 802 Sleep state estimation unit 803 Walking speed estimation unit 804 Estimation result storage unit 805 Estimation result reading unit 806 State change detection processing unit 807, 901 Communication unit 808 Radar data memory unit 809 Estimation result information memory unit 902 Display unit 921 State display area 922 Notification area 1100 Computer 1101 Input device 1102 Output device 1103 CPU 1104 GPU (Graphics Processing Unit) 1105 ROM (Read Only Memory) 1106 RAM (Random Access Memory) 1107 Storage device 1108 Reading device 1109 Transceiver device 1110 Bus P Person

Claims

1. An estimation unit that estimates items related to the sleep state and walking speed of the person in time series based on information detected by a radar; A detection unit that detects changes in the physical state of the person based on the estimation results obtained in time series for the plurality of items; An output unit that outputs information related to the detected changes in the physical state of the person; A display terminal device; Comprising: The detection unit applies an autoregressive model to the estimation results obtained in the time series, and detects, as a change point in the trend variation of the estimation results obtained in the time series, a timing at which the difference between the prediction model obtained and the estimation results is equal to or greater than a threshold value; The output unit displays, on the display terminal device in one screen, as information related to changes in the physical state of the person, the change point in the trend variation of the sleep state and the change point in the trend variation of the walking speed; An information processing device.

2. The output unit displays, as time series data, the change point in the trend variation of the sleep state and the change point in the trend variation of the walking speed, and displays, within the one screen, a message indicating the change in the trend variation of the sleep state and / or the walking speed; The information processing device according to Claim 1.

3. The radar is a millimeter wave radar; The information processing device according to Claim 1.

4. By one or more information processing devices, Based on information detected by a radar, estimate items related to the sleep state and walking speed of the person in time series, Based on the estimation results obtained in time series for the plurality of items, detect changes in the physical state of the person, Output information related to the detected changes in the physical state of the person, Apply an autoregressive model to the estimation results obtained in the time series, and detect, as a change point in the trend variation of the estimation results obtained in the time series, a timing at which the difference between the prediction model obtained and the estimation results is equal to or greater than a threshold value, Display, in one screen, as information related to changes in the physical state of the person, the change point in the trend variation of the sleep state and the change point in the trend variation of the walking speed; An information processing method.

5. By one or more information processing devices, A process of estimating items related to the sleep state and walking speed of the person in time series based on information detected by a radar; A process of detecting changes in the physical state of the person based on the estimation results obtained in time series for the plurality of items; A process of outputting information regarding a change in the physical state of the detected person, A process of detecting, as a change point in the trend variation of the estimation result obtained in the time series, a timing at which the difference between the prediction model obtained by applying an autoregressive model to the estimation result obtained in the time series and the estimation result is equal to or greater than a threshold value, A process of displaying, on a single screen, as information regarding a change in the physical state of the person, the change point in the trend variation of the sleep state and the change point in the trend variation of the walking speed, Causing to execute, Program.

Images