Information processing device, information processing method, and program

Abstract

To detect a variation in a plurality of items concerning physical conditions of a person without contact.SOLUTION: An information processing device 80 comprises: estimation units 802 and 803 that estimate in time series a plurality of items concerning physical conditions of a person on the basis of information obtained by detecting the person with a radar 10; a detection unit 806 that detects a variation in the physical conditions of the person on the basis of estimation results obtained in time series concerning the plurality of items; and an output unit 807 that outputs information concerning the detected variation in the physical conditions.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] There is a technique for estimating a person's walking speed based on an image of the person captured by a surveillance camera and evaluating the person's cognitive function based on changes in the estimated walking speed. Further, there is a technique for monitoring (watching over) the behavior of a person in a dwelling based on the combination or order of electrical devices operated by the person, or based on the position information of a wireless tag attached to the person's clothing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regarding the technique for detecting changes in a person's physical condition, there is room for further consideration.

[0005] Non-limiting embodiments of the present disclosure contribute to providing an information processing apparatus, an information processing method, and a program capable of non-contact detection of changes in a plurality of items related to a person's physical condition.

Means for Solving the Problems

[0006] An information processing apparatus according to an embodiment of the present disclosure includes an estimation unit that estimates a plurality of items related to the physical state of a person in time series based on information obtained by detecting the person with a radar, a detection unit that detects a change in the physical state of the person based on the estimation results obtained in time series for the plurality of items, and an output unit that outputs information related to the detected change in the physical state of the person.

[0007] Note that these general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0008] A non-limiting example of the present disclosure can detect changes in a plurality of items related to a person's physical state non-contact.

[0009] Further advantages and effects in an embodiment of the present disclosure will be clarified from the specification and drawings. Such advantages and / or effects are provided by several embodiments and the features described in the specification and drawings respectively, but not all are necessarily provided to obtain one or more of the same features.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same function are denoted by the same reference numerals, and redundant description is omitted.

[0012] <An embodiment> Discovering changes in a person's physical condition early and taking countermeasures is also important for maintaining health. For example, detecting changes in sleep state or walking speed can lead to an understanding of a person's physical condition such as concentration, motivation, appetite, and motor ability, and thus can be applied to the prevention of depression or injury, and further to the discovery of dementia.

[0013] Examples of changes in a person's physical condition include changes in sleep state and walking speed (e.g., decrease). Examples of changes in sleep state include changes such as increases or decreases in the time spent in non-REM sleep (deep sleep) state, increases or decreases in the time spent in apnea sleep state, and increases or decreases in the frequency of middle awakenings.

[0014] To detect changes in physical condition as described above, for example, the methods shown in the following (1) to (3) are considered.

[0015] (1) Detection of REM sleep or non-REM sleep state, or apnea sleep state, using detection of pulse or respiration by a wearable device (2) Detection of REM sleep or non-REM sleep state, or middle awakening, using detection of body movement, respiration, or pulse by a sensor mat (3) Detection of walking speed using an image of the detection target person taken by a camera (hereinafter sometimes referred to as a "camera image").

[0016] In the following description, the terms "detection" or "detection" may be mutually read as terms such as "estimation" or "measurement", for example.

[0017] Here, in method (1), since the device is worn on the detection target person, it may give, for example, a sense of discomfort or discomfort to the detection target person. Also, in method (2), items other than the sleep state (e.g., walking speed) are not measured.

[0018] In method (3), since a camera image of the detection target person is used, for example, privacy infringement may become a problem. Also, in method (3), when estimating the position change of the detection target person based on the change in the size of the detection target person in the camera image, the size in the camera image may not change depending on the movement path of the detection target person. Therefore, it may be difficult to estimate the walking speed.

[0019] In one embodiment of the present disclosure, for example, temporal changes (in other words, changes over time) of a plurality of items related to the physical state of a person who is the detection target are detected and presented non-contactlessly by a single radio wave sensor (for example, a millimeter wave radar). A non-limiting example of the plurality of items to be detected is "change in sleep state" and "decrease in walking speed".

[0020] Since a millimeter wave radar can, for example, detect the position of a person three-dimensionally in a non-contact manner, it can calculate the moving distance of the person without depending on the moving path of the person, and thus can estimate the walking speed of the person.

[0021] In addition, since a millimeter wave radar can, for example, detect the body movement of the detected person (including, for example, the movement of the body surface accompanying heartbeat or respiration), it can estimate the sleep state (for example, the depth of sleep such as non-REM sleep and REM sleep) based on the body movement during sleep. In addition, it is possible to estimate the apnea sleep state based on the presence or absence of respiration during sleep.

[0022] <System configuration example> FIG. 1 is a diagram showing a configuration example of a physical state detection system 1 according to an embodiment. As shown in FIG. 1, the physical state detection system 1 may include, for example, a millimeter wave radar 10, a personal computer (PC) 20, servers 30A and 30B, and databases (DBs) 40A and 40B.

[0023] These components 10, 20, 30A, 30B, 40A, and 40B may be communicably connected to each other via a network 50 such as the Internet or a local area network. The connection form of the components 10, 20, 30A, 30B, 40A, and 40B may be either wired or wireless, or a mixture of wired and wireless. Each of the PC 20, the server 30A, and the server 30B is an example of an information processing device.

[0024] The millimeter-wave radar 10 is an example of a radio wave sensor. For example, it is installed in a living room within a building 60 and non-contact detects a person P located within the living room by radio waves (radar waves). Non-limiting examples of the building 60 include a private house, a store, a dormitory, an apartment, an office, a hotel, a restaurant, a school building, a research institute, a hospital, a clinic, and the like.

[0025] "Detection" may be read as, for example, "detection" or "sensing". The frequency of the millimeter-wave radar 10 may be, as a non-limiting example, a frequency in the range of several tens of gigahertz (GHz) to several hundreds of GHz, and may be, for example, a frequency in the 60 GHz band.

[0026] The PC 20 acquires (or receives), for example, the detection result by the millimeter-wave radar 10 (hereinafter may be referred to as "radar detection result", "radar detection information", or "radar data") from the millimeter-wave radar 10.

[0027] The radar detection result may be transmitted, for example, from the PC 20 to one or both of the servers 30A and 30B via the network 50. When the millimeter-wave radar 10 has a connection (or access) function to the network 50 (in other words, a communication function), the radar detection result may be transmitted to one or both of the servers 30A and 30B without going through the PC 20.

[0028] Also, the PC 20 may display, for example, information received from one or both of the servers 30A and 30B via the network 50 on the display unit (display) of the PC 20. For example, information processed or generated based on the radar detection result in one or both of the servers 30A and 30B (for example, information regarding changes in the physical state of the person P) may be displayed on the PC 20. Information received via the network 50 may be printed on a printer connected to the PC 20.

[0029] Note that instead of or in addition to the PC 20, a mobile terminal such as a smartphone may perform at least some functions equivalent to those of the above-described PC 20.

[0030] One of the servers 30A and 30B (for example, server 30A) detects (or estimates or measures) information such as the position, body movement, and respiratory rate (or heart rate) of the person P in the room based on, for example, the radar detection result. The detected information may be sequentially (in other words, in time series) stored (or accumulated) in, for example, the DB 40A.

[0031] Also, the server 30A reads out information for a predetermined period (for example, one day's worth) from the DB 40A, and detects (or estimates or measures) information such as the sleep state of the person P (for example, at least one time zone of non-REM sleep, apnea sleep, and intermediate awakening), and walking speed. The detected information may be stored (or accumulated) in the DB 40B in time series (for example, daily).

[0032] The server 30B determines (or detects) whether there is a change in the physical state of the person P based on, for example, the time-series information accumulated in the DB 40B (hereinafter sometimes referred to as "time-series data"). When it is determined that there is a change in the physical state of the person P, the server 30B provides (or notifies) information indicating that there is a change in the physical state of the person P (hereinafter sometimes referred to as "physical state change information") to the PC 20 via the network 50. The PC 20 displays the received physical state change information on, for example, the display of the PC 20 or prints it by a printer. Note that the display or printing of the physical state change information is not limited to the PC 20 and may be performed in the server 40A or 40B.

[0033] Note that the detection (or estimation or measurement) of various information such as the position, body movement, respiratory rate (or heart rate), sleep state, and walking speed of the person P may be performed intensively or dispersively in one or both of the servers 30A and 30B based on the radar detection result. In other words, it is not necessary that a specific relationship exists such that specific information among various information (which may also be referred to as "measurement items") is detected in a specific server.

[0034] For example, servers 30A and 30B may be integrated into one server. Also, the processing related to the detection of the above-described measurement items may be distributedly performed by three or more servers.

[0035] DBs 40A and 40B may be respectively provided inside servers 30A and 30B. In other words, the storage units or storage devices provided in servers 30A and 30B may correspond to DBs 40A and 40B. DBs 40A and 40B may be integrated into one DB, for example. In three or more DBs, the above-described measurement items may be distributedly accumulated and managed.

[0036] FIG. 2 is a block diagram showing a configuration example of the body state detection system 1 according to an embodiment. In FIG. 2, the millimeter-wave radar 10 may include, for example, a detection unit 101, a processing unit 102, and an output unit 103.

[0037] The detection unit 101 transmits, for example, a radar wave and receives and detects a reflected wave reflected by the radar wave at a detection target (for example, person P).

[0038] The processing unit 102 detects information such as the position, speed, and angle of the detection target with respect to the millimeter-wave radar 10 based on, for example, the point cloud data corresponding to the detection target obtained from the reflected wave.

[0039] The output unit 103 transmits, for example, the information obtained by the processing unit 102 (in other words, radar data) to the radar control device 70.

[0040] The radar control device 70 may include, for example, a radar communication unit 701 and a radar data storage unit 702. Note that the radar control device 70 may correspond to, for example, the PC 20 illustrated in FIG. 1, or may correspond to one or both of the servers 30A and 30B illustrated in FIG. 1.

[0041] The radar communication unit 701 communicates with the millimeter-wave radar 10, for example, either wired or wirelessly, receives radar data, and stores (memorizes) it in the radar data storage unit 702. Further, the radar communication unit 701 transmits the radar data to the state change detection device 80, for example, either wired or wirelessly.

[0042] The state change detection device 80 detects changes in the physical state of a detection target (e.g., person P) based on radar data, for example. The state change detection device 80 is an example of an information processing device and may correspond to one or both of the servers 30A and 30B illustrated in FIG. 1, for example.

[0043] The state change detection device 80 may include, for example, a radar data reading unit 801, a sleep state estimation unit 802, a walking speed estimation unit 803, an estimation result storage unit 804, an estimation result reading unit 805, a state change detection processing unit 806, a communication unit 807, a radar data storage unit 808, and an estimation result information storage unit 809.

[0044] The radar data reading unit 801 reads the radar data stored in the radar data storage unit 808 and outputs it to the sleep state estimation unit 802, for example. Note that the radar data storage unit 808 may correspond to the DB40A illustrated in FIG. 1, for example.

[0045] The sleep state estimation unit 802 estimates the sleep state of person P, who is the detection target (e.g., non-REM sleep, apnea sleep, and the time zones of waking up in the middle of sleep), based on radar data, for example.

[0046] The walking speed estimation unit 803 estimates the walking speed of person P based on radar data, for example. The estimation of the walking speed may not be performed, for example, during the sleep time zone estimated by the sleep state estimation unit 802, and thus may be performed for the time zones excluding the estimated sleep time zone of person P.

[0047] The sleep state estimation unit 802 and the walking speed estimation unit 803 may be integrated into one estimation unit, for example.

[0048] The estimation result storage unit 804 stores (memorizes) each estimation result by, for example, the sleep state estimation unit 802 and the walking speed estimation unit 803 in the estimation result information storage unit 809. Note that the estimation result information storage unit 809 may be integrated with, for example, the radar data storage unit 808. Further, the estimation result information storage unit 809 may correspond to the DB40B illustrated in FIG. 1.

[0049] The estimation result reading unit 805 reads out, for example, the estimation result stored in the estimation result information storage unit 809 and outputs it to the state change detection processing unit 806.

[0050] The estimation result storage unit 804 and the estimation result reading unit 805 may be integrated into, for example, one write and read control unit.

[0051] The state change detection processing unit 806 detects, for example, a change in the physical state of the person P based on the estimation result input from the estimation result reading unit 805. The detection result may be output to, for example, the communication unit 807.

[0052] The communication unit 807 is an example of an output unit, and communicates with, for example, the display terminal device 90 (for example, the communication unit 901) wirelessly or by wire, and transmits the detection result regarding the change in the physical state of the person P to the display terminal device 90.

[0053] The display terminal device 90 may include, for example, a communication unit 901 and a display unit 902. Note that the display terminal device 90 may correspond to, for example, the PC20 illustrated in FIG. 1.

[0054] The communication unit 901 communicates with, for example, the state change detection device 80 (for example, the communication unit 807) wirelessly or by wire, receives the detection result regarding the change in the physical state of the person P, and outputs the received information to the display unit 902.

[0055] The display unit 902 displays, for example, the information input from the communication unit 901.

[0056] The configuration example of the physical state detection system 1 according to one embodiment has been described above.

[0057] <Operation Example> Next, an operation example of the body state detection system 1 having the above-described configuration will be described with reference to the flowcharts of FIGS. 3 and 4.

[0058] FIG. 3 is a flowchart showing an example of the overall operation of the body state detection system 1 according to an embodiment. FIG. 4 is a flowchart showing an example of the sleep time zone detection process (S803) illustrated in FIG. 3.

[0059] As shown in FIG. 3, radar data is detected in the millimeter-wave radar 10 (S101), and the detected radar data is stored (memorized) in, for example, the radar data storage unit 808 of the state change detection device 80 (S102).

[0060] The state change detection device 80 periodically reads radar data from the radar data storage unit 808 by, for example, the radar data reading unit 801. For example, the radar data reading unit 801 reads one day's worth of radar data from the radar data storage unit 808 at 1:00 every day (S801, S802).

[0061] The radar data read from the radar data storage unit 808 is output to, for example, the sleep state estimation unit 802, and the sleep state estimation unit 802 estimates the sleep state of the person P to be detected based on the radar data (S803).

[0062] For example, the sleep state estimation unit 802 determines the presence or absence of a sleep time zone based on, for example, one day's worth of radar data (S804). An example of the sleep time zone detection process will be described later with reference to FIG. 4. As a result of the determination, if there is no sleep time zone (S804; NO), for example, the estimation by the sleep state estimation unit 802 may end and the estimation by the walking speed estimation unit 803 may be executed (S808).

[0063] If there is a sleep time zone (S804; YES), the sleep state estimation unit 802 estimates, for example, time zones of non-REM sleep, apnea sleep, and intermediate awakening (S805 to S807).

[0064] Next, for example, the walking speed estimation unit 803 estimates the walking speed of the person P based on the radar data (S808).

[0065] The estimation results obtained by each of the processes of S805 to S808 are stored (memorized) in the estimation result information storage unit 809 by, for example, the estimation result storage unit 804 (S809).

[0066] Note that the processes of S805 to S808 may be performed serially as described above, or may be performed in parallel. When performed serially, the processing order of S805 to S807 may be appropriately changed.

[0067] The estimation result stored in the estimation result information storage unit 809 is read out by, for example, the estimation result reading unit 805 and output to the state change detection processing unit 806 (S810).

[0068] The state change detection processing unit 806 detects, for example, a change in the physical state of the person P based on the estimation result input from the estimation result reading unit 805 (S811). For example, the state change detection processing unit 806 determines whether there is a change in the physical state of the person P based on each estimation result of the sleep state and the walking speed (S812).

[0069] As a result of the determination, if there is no change in the physical state of the person P (S812; NO), the state change detection processing unit 806 may end the process. On the other hand, if there is a change in the physical state of the person P (S812; YES), the state change detection processing unit 806 transmits, for example, information regarding the change in the physical state of the person P to the display terminal device 90 by the communication unit 807 (S813).

[0070] When the presentation terminal device 90 receives information regarding a change in the physical state of the person P from the state change detection device 80 (S901), it displays the received information on the screen of, for example, the display unit 902 (S902).

[0071] <Estimation of sleep state> Next, the estimation (or detection) of the sleep state of the person P (for example, each time zone of sleep time zone, non-REM sleep, apnea sleep, and mid-awakening) in the sleep state estimation unit 802 will be described item by item.

[0072] <Detection of sleep time zone (S803 in FIG. 3)> As illustrated in FIG. 4, the sleep state estimation unit 802 reads, for example, the setting of the sleep location definition area 501 of the person P (see FIG. 5) (S831). The setting of the sleep location definition area 501 may be performed, for example, by the sleep state estimation unit 802 or by a setting unit (not shown) in the state change detection device 80.

[0073] The setting data of the sleep location definition area 501 may be stored in, for example, either one of the storage units 808 and 809, or may be stored in another storage unit (not shown) in the state change detection device 80.

[0074] The sleep location definition area 501 may be set as a three-dimensional (3D) space with respect to the location where the person P goes to bed (for example, bedding such as a bed or futon) in the room, as shown in the upper part of FIG. 5, for example.

[0075] In the example in the upper part of FIG. 5, a sleep location definition area 501 defined by XYZ coordinates is set in the upper space of the place where the person P goes to bed. The size (area) of the XY plane of the sleep location definition area 501 may be set to a size corresponding to the size of bedding such as a bed, mattress, or futon, for example.

[0076] For example, the size of the XY plane of the sleep area definition area 501 may be set to a size that matches the size of the bedding, or may be set to a size slightly smaller or slightly larger than the size of the bedding. Alternatively, the sleep area definition area 501 may be set to a spatial size corresponding to the three-dimensional space occupied by at least one of the supine position, prone position, and lateral lying position of the person P in the sleeping position.

[0077] For example, the length (or height) of the sleep area definition area 501 in the Z-axis direction may be set to a height corresponding to the height (or body height) of the person P in the sleeping posture (or body position). For example, it may be set according to the highest body height among the body heights of the person P in the supine position, prone position, and lateral lying position of the person P.

[0078] According to such a setting, for example, when the person P raises the upper body while staying in the sleep area definition area 501, at least a part of the upper body of the person P comes out of the sleep area definition area 501. Therefore, the raising of the upper body of the person P can be detected by the millimeter-wave radar 10.

[0079] Note that the height of the sleep area definition area 501 may be set based on the upper surface of bedding such as a bed, a mattress, and a futon as illustrated in FIG. 5, or may be set based on the floor surface on which the bedding is placed. In other words, the sleep area definition area 501 may be set as a space covering the bedding, for example, or may be set as a space not covering the bedding.

[0080] In the lower part of FIG. 5, for example, as illustrated in the upper part of FIG. 5, examples are shown in which the position (XYZ coordinate position) of the person P is detected as radar data by the millimeter-wave radar 10 when the person P is located inside and outside the sleep area definition area 501, respectively.

[0081] By setting the sleep location definition area 501, for example, it is possible to detect whether a person P has entered the sleep location definition area 501 for bedtime or whether the person P has woken up (including intermediate awakenings) and exited outside the sleep location definition area 501 based on radar data (e.g., the position information of the person P).

[0082] Therefore, for example, by setting the sleep start determination condition and the sleep end determination condition based on the entry detection and exit detection of the person P with respect to the sleep location definition area 501 as follows, the sleep time zone of the person P can be detected (or estimated).

[0083] (1) Sleep start determination condition: Detection of entry into the sleep location definition area 501 (2) Sleep end determination condition: Detection of exit from the sleep location definition area 501 and no detection of entry into the sleep location definition area 501 within the next N minutes

[0084] N may be set to, for example, a time when it can be determined that the person P will not return to the bedtime location, for example, a time of several tens of minutes such as 30 minutes. In other words, if entry into the sleep location definition area 501 is (re)detected within N minutes or less, the sleep end determination condition is not satisfied, so it may be determined that the sleep time of the person P has not ended and continues. Note that the time zone of N minutes or less may be determined as the time zone when the person P has an intermediate awakening.

[0085] Returning to FIG. 4, the detection process of the sleep time zone based on the above-described conditions (1) and (2) will be described below.

[0086] After reading the setting of the sleep location definition area 501 as described above, the sleep state estimation unit 802, for example, acquires the data at the start time among the radar data (S832), and determines whether the data satisfies the sleep start determination condition (1) (S833).

[0087] If, as a result of the determination, it is determined that the data at the start time does not satisfy the sleep start determination condition (1) (S833; NO), the sleep state estimation unit 802 acquires the data at the next time (S835) and determines whether the data at the next time satisfies the sleep start determination condition (1) as long as there is data at the next time (S834; YES).

[0088] If there is data that satisfies the sleep start determination condition (1) (S833; YES), the sleep state estimation unit 802 determines whether the data at the next time satisfies the sleep end determination condition (2) (S836).

[0089] If, as a result of the determination, it is determined that the data at the next time does not satisfy the sleep end determination condition (2) (S836; NO), the sleep state estimation unit 802 acquires the data at the next time (S838) and determines whether the data at the next time satisfies the sleep end determination condition (2) as long as there is data at the next time (S837; YES).

[0090] If, as a result of the determination, it is determined that the data at the next time satisfies the sleep end determination condition (2) (S836; YES), the sleep state estimation unit 802 records, for example, the time period between the time of the data that satisfies the sleep start determination condition (1) and the time of the data that satisfies the sleep end determination condition (2) as the sleep time period of the person P (S839).

[0091] Thereafter, the sleep state estimation unit 802 checks whether there is data at the next time (S840). If there is data at the next time (S840; YES), the sleep state estimation unit 802 acquires the data at the next time (S841) and determines whether the data at the next time satisfies the sleep start determination condition (1) (S833).

[0092] In this way, the sleep state estimation unit 802 searches for the data that satisfies the sleep start determination condition (1) and the data that satisfies the sleep end determination condition (2) thereafter in the order of the time of the radar data obtained in time series, and detects and records the time period between the times of both data as the sleep time period.

[0093] In such data search, for example, when data for the next time does not exist (NO in S834, S837, and S840), the sleep state estimation unit 802 may end the detection process for the sleep time zone. In response to the end of the detection process for the sleep time zone, for example, the processes after S804 illustrated in FIG. 3 are executed.

[0094] FIG. 6 shows an example of the detection of the sleep time zone. In FIG. 6, illustratively, in one day (24 hours from 0:00:00 to 24:00:00), the sleep time zone of person P is detected in the time zones of 00:00:00 to 07:00:00 (7 hours) and 22:00:00 to 24:00:00 (4 hours), and an example is shown where a mid-awakening is presumed to have occurred to person P in the time zone of 01:10:20 to 01:20:20 (20 minutes < N).

[0095] The estimation of the time zones of non-REM sleep, apnea sleep, and mid-awakening may be performed for the sleep time zone detected as described above. Hereinafter, the estimation of the time zones of non-REM sleep, apnea sleep, and mid-awakening will be described.

[0096] <Estimation of non-REM sleep time zone (S805 in FIG. 3)> As an example of an index that can determine (or estimate) whether a person is in non-REM sleep, the respiratory rate and the pulse rate can be cited. For example, during non-REM sleep, the respiratory rate and the pulse rate tend to be stable and regular (in other words, periodic). In contrast, during non-REM sleep, the heart rate and the respiratory rate tend to increase and become irregular compared to non-REM sleep.

[0097] Therefore, the sleep state estimation unit 802 may measure, for example, based on the radar data, the respiratory state (for example, the respiratory rate and the pulse rate) of person P in the detected sleep time zone, and estimate the time zone in which a periodic waveform is continuously observed as non-REM sleep. For example, as shown in FIG. 7, the time zone of 01:10:25 to 01:50:14 may be estimated as the non-REM sleep time zone. Note that the vertical axis in FIG. 7 represents the phase change of the radio wave signal due to the movement of the chest, and the unit is, for example, [rad]. The same applies to the vertical axis in FIG. 8 as in FIG. 7.

[0098] <Estimation of apnea sleep time zone (S806 in FIG. 3)> The sleep state estimation unit 802 may, for example, based on radar data, measure the breathing state (e.g., respiratory rate and pulse rate) of the detected sleep time zone of the person P, and estimate the time zone during which the detection of breathing is interrupted as the apnea sleep state. For example, as shown in FIG. 8, the time zone from 03:20:27 to 03:20:36 may be estimated as the time zone of apnea sleep.

[0099] <Estimation of mid-awakening time zone (S807 in FIG. 3)> For example, when the sleep state estimation unit 802 detects, based on radar data, for example, an act of the person P temporarily leaving the sleeping place (sleeping place definition area 501) during sleep, or an act of the person P raising the upper body, the sleep state estimation unit 802 may estimate the duration thereof as the time when mid-awakening occurred.

[0100] For example, as shown in the upper part of FIG. 9, assume that the person P leaves the sleeping place (sleeping place definition area 501) at a certain time (e.g., 01:10:20) during the sleep time zone and returns to the sleeping place at 01:20:20, 10 minutes later.

[0101] In this case, the movement (or rather, the position change) of the person P in the radar data is represented as shown in the lower part of FIG. 9. For example, the departure of the person P from the sleeping place is detected at the time 01:10:20, and the entry of the person P into the sleeping place is detected at the time 01:20:20. Therefore, the sleep state estimation unit 802 may, for example, estimate the 10-minute time zone from 01:10:20 to 01:20:20 as the mid-awakening time zone.

[0102] In addition, as shown in the upper part of FIG. 10, when the person P raises the upper body while staying at the sleeping place, since at least a part of the upper body goes out of the sleeping place definition area 501 as described above, the raising of the upper body of the person P can be detected based on the radar data as shown in the lower part of FIG. 10. Therefore, the sleep state estimation unit 802 may, for example, also include the time zone in which the raising of the upper body of the person P is detected in this way in the mid-awakening time zone.

[0103] <Estimation of walking speed (S808 in FIG. 3)> Next, the estimation of the moving speed (e.g., walking speed) of the person P by the walking speed estimation unit 803 (see FIG. 2) will be described. The walking speed estimation unit 803 calculates, for example, the distance (D) that the person P has moved in the room based on radar data, and estimates the walking speed (V) of the person P by dividing the distance by the time (T) taken for the movement.

[0104] For example, as shown in the upper part of FIG. 11, when the person P moves from time t1 to time t4, as shown in the lower part of FIG. 11, the walking speed estimation unit 803 calculates the moving distance D of the person P as D = d1 + d2 + d3 based on radar data, for example. Therefore, the walking speed V of the person P is estimated as V = (d1 + d2 + d3) / (t4 - t1), where the time taken for the movement is T = t4 - t1.

[0105] Note that the estimation of the walking speed V may be performed for a time period excluding the sleep time period detected in the above-described sleep time period detection process (S803 in FIG. 3) (in other words, a time period determined not to be the sleep time period).

[0106] In this way, by narrowing down (or limiting) the time period for estimating the walking speed V to the non-sleep time period, the amount of radar data used for estimating the walking speed can be reduced, and thus the processing amount of the state change detection device 80 can be reduced.

[0107] <Reduction of the influence of radar detection position error on the moving distance> When the acquisition frequency of radar data by the millimeter-wave radar 10 is, for example, several tens of milliseconds to several hundreds of milliseconds, or when there is an error in the detection position of several centimeters (cm) to several tens of cm, fluctuations or deviations in the trajectory that do not match the actual movement of the person P (e.g., the upper part of FIG. 12) may occur in the radar radar (e.g., the middle part of FIG. 12).

[0108] Therefore, when the detection positions based on the radar data are simply integrated to calculate the moving distance D, an error outside the allowable range may occur with respect to the actual moving distance of the person P. Thus, the walking speed estimation unit 803 may perform correction (for example, moving average filtering) on the detection positions based on the radar data, as shown in the lower part of FIG. 12, for example. Thereby, the temporal change (in other words, the trajectory) of the detection position of the person P can be made to coincide with the actual movement trajectory of the person P, or brought closer to the minimum error, and the estimation accuracy of the walking speed V can be improved.

[0109] <State change detection (S811 in FIG. 3)> Next, an example of the state change detection process (S811 in FIG. 3) by the state change detection processing unit 806 (see FIG. 2) will be described.

[0110] The state change detection processing unit 806 detects a change point in the tendency variation (in other words, the trend) of the physical state of the person P based on, for example, the sleep state (for example, each time zone of non-REM sleep, apnea sleep, and intermediate awakening) estimated by the sleep state estimation unit 802 and the time-series data of the walking speed estimated by the walking speed estimation unit 803.

[0111] For example, the state change detection processing unit 806 determines the presence or absence of a change point in the tendency variation of the time-series data separately for the sleep state and the walking speed. When a change point is detected, the state change detection processing unit 806 creates a prediction model for the time-series data after the change point based on, for example, the time-series data before the change point. For creating the prediction model, for example, an autoregressive (AR) model may be used.

[0112] For example, as shown in FIG. 13(a), when a change in the tendency variation is detected in the time-series data regarding non-REM sleep on January 5, a prediction model for after January 5 is created based on the time-series data before January 4 (for example, the time-series data for several days such as the 3rd), as shown in FIG. 13(e).

[0113] Note that the prediction model may be individually created when a change point in the trend variation is detected for each time series data of apnea sleep, middle awakening, and walking speed, for example, illustrated in FIGS. 13(b) to 13(d).

[0114] Then, for example, the state change detection processing unit 806 determines whether or not the abnormality degree is equal to or greater than a threshold value using the difference between the measured value and the predicted value by the prediction model as an index of the abnormality degree, and detects the timing at which it is determined that the abnormality degree is equal to or greater than the threshold value as a change point in the trend variation. For example, FIG. 13(e) shows an example in which a change (in other words, an abnormality) in the trend variation is detected on January 5th for the time zone of non-REM sleep.

[0115] The state change detection processing unit 806 transmits, as information regarding a change in the physical state of the person P, information about the measurement item for which a change point in the trend variation is detected, separately from the measurement items of non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, to the display terminal device 90.

[0116] The display terminal device 90 displays the information received from the state change detection device 90, for example, on the screen of the display unit 902 of the display terminal device 90. Thereby, information regarding a change in the physical state of the person P is presented to, for example, the person P himself / herself or another person (for example, the family member, caregiver, administrator, attending doctor, etc.) involved in the health management of the person P.

[0117] FIG. 14 shows an example of the screen display (GUI) of the display unit 902. FIG. 14 shows an example in which the display screen of the display unit 902 is divided into a state display area 921 and a notification area 922. As illustrated in FIG. 14, the time series data respectively illustrated in FIGS. 13(a) to 13(d) is displayed in the state display area 921, and a change in the physical state of the person P, for example, detection of a decrease in non-REM sleep time and detection of an increase in apnea sleep time may be displayed in the notification area 922.

[0118] Note that the screen display example of the display unit 902 is not limited to the example shown in FIG. 14. For example, either the status display area 921 or the notification area 922 may be selectively displayed. In the status display area 921, any one or more of the time-series data exemplified in FIGS. 13(a) to 13(d) may be selectively displayed. For example, in the status display area 921, the time-series data that contributed to the detection of changes in the physical state may be selectively displayed. Alternatively or additionally, the time-series data that contributed to the detection of changes in the physical state may be highlighted in the status display area 921. The highlighting mode is not limited. Also, the display mode in the notification area 922 is not limited to the example in FIG. 14. In combination with the display in the notification area 922, a sound such as an alarm sound may be used to prompt attention to the detection of a change in the physical state.

[0119] As described above, according to the above-described embodiments, changes in the physical state of the person P, such as non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, can be detected and presented non-contact by the millimeter-wave radar 10.

[0120] Therefore, for example, since the physical states of the person P, such as concentration, motivation, appetite, and motor ability, can be grasped early according to changes in the sleep state and walking speed of the person P, it becomes possible to take measures for maintaining the health of the person P at an early stage. For example, it can be applied to the prevention of depression or injury of the person P, and further to the detection of dementia.

[0121] Also, according to the above-described embodiments, based on time-series data with a unified time axis (in other words, timing-synchronized), such as daily data, a plurality of measurement items related to the physical state of the person P, such as non-REM sleep time, apnea sleep time, middle awakening time, and walking speed, are detected or estimated. Therefore, it is possible to easily grasp the timing when a change occurs in the physical state of the person P for each measurement item.

[0122] Therefore, for example, based on the change points (e.g., the degree of agreement or disagreement) of the tendency fluctuations for each measurement item related to the physical state, it may be easier to investigate the factors presumed to have caused a change in the physical state of person P. Also, for example, based on the correlation between the change points among multiple measurement items, the factors of the change in the physical state of person P can be analyzed and estimated comprehensively.

[0123] <Overall supplement> The sleep location definition area 501 (see Fig. 5) does not have to be set as a three-dimensional space. For example, it may be set in two dimensions (e.g., the X-Y axis excluding the Z-axis direction).

[0124] The items to be measured by the millimeter-wave radar 10 do not have to be all of the non-REM sleep time, apnea sleep time, middle awakening time, and walking speed. For example, it may be a combination of some measurement items. Also, the items to be measured by the millimeter-wave radar 10 are not limited to the four items of non-REM sleep time, apnea sleep time, middle awakening time, and walking speed.

[0125] Some of the four items may be replaced by other items related to the physical state of person P that can be detected by the millimeter-wave radar 10. Also, five or more items related to the physical state of person P that can be detected by the millimeter-wave radar 10 may be the measurement targets by the millimeter-wave radar 10.

[0126] Also, the person to be detected by the millimeter-wave radar 10 is not limited to one person and may be two or more people. According to the distance resolution of the millimeter-wave radar 10, for example, body movement and movement can be detected for each person, so the above-described changes in the physical state can be detected separately for multiple people.

[0127] The embodiments have been described above with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope described in the claims, and it is naturally understood that they also belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the disclosure, the components in the above embodiments may be arbitrarily combined.

[0128] Moreover, the specific examples of the present disclosure are merely illustrative and do not limit the claims. The technology described in the claims includes those obtained by variously modifying and changing the specific examples illustrated above.

[0129] Also, the notation "··· part" in the above-described embodiments may be mutually replaced with other notations such as "··· circuitry", "··· device", "··· unit", or "··· module".

[0130] In addition, the present disclosure can be realized by software, hardware, or software in cooperation with hardware. For example, the functions of the above-described system can be realized by a computer program.

[0131] FIG. 15 is a diagram showing the hardware configuration of a computer (or information processing apparatus) that realizes the functions of each device constituting the above-described passage management system 1 by a program. In FIG. 15, a computer 1100 includes an input device 1101 such as a keyboard, mouse, or touch pad, an output device 1102 such as a display or speaker, a CPU (Central Processing Unit) 1103, a GPU (Graphics Processing Unit) 1104, a ROM (Read Only Memory) 1105, a RAM (Random Access Memory) 1106, a storage device 1107 such as a hard disk device or SSD (Solid State Drive), a reading device 1108 that reads information from a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or USB (Universal Serial Bus) memory, and a transceiver device 1109 that communicates via a network. Each part is connected by a bus 1110.

[0132] The reading device 1108 reads, for example, a program for realizing the functions of the terminal PC 20 from a recording medium on which the program is recorded, and stores the program in the storage device 1107. Alternatively, the transceiver device 1109 communicates with a server device connected to the network (which may be the server 30 or a server device different from the server 30), and stores in the storage device 1107 a program for realizing the functions of the above-described devices downloaded from the server device.

[0133] The CPU 1103 copies the program stored in the storage device 1107 to the RAM 1106, and sequentially reads and executes the instructions included in the program from the RAM 1106, thereby realizing the functions of the terminal PC 20 according to the above-described embodiment.

[0134] Each functional block used in the description of the above-described embodiments may be realized, in part or in whole, as an integrated circuit LSI, and each process described in the above embodiments may be controlled, in part or in whole, by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include part or all of the functional blocks. The LSI may be provided with data input and output. Depending on the degree of integration, the LSI may also be referred to as an IC, a system LSI, a super LSI, or an ultra LSI.

[0135] The method of integrating into an integrated circuit is not limited to LSI, and it may also be realized by an application-specific circuit, a general-purpose processor, or a dedicated processor. Further, after manufacturing the LSI, an FPGA (Field Programmable Gate Array) that can be programmed, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0136] Furthermore, if an integrated circuit technology that replaces the LSI appears due to the progress of semiconductor technology or another derived technology, of course, the integration of functional blocks may be performed using that technology. The application of biotechnology or the like is a possible example.

[0137] The present disclosure can be implemented in any type of device, apparatus, system having a communication function (collectively referred to as a communication device). The communication device may include a wireless transceiver (transceiver) and a processing / control circuit. The wireless transceiver may include a receiving unit and a transmitting unit, or may include them as functions. The wireless transceiver (transmitting unit, receiving unit) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of the communication device include a telephone (mobile phone, smartphone, etc.), a tablet, a PC (laptop, desktop, notebook, etc.), a camera (digital still / video camera, etc.), a digital player (digital audio / video player, etc.), a wearable device (wearable camera, smartwatch, tracking device, etc.), a game console, a digital book reader, a telehealth / telemedicine (remote healthcare / medical prescription) device, a vehicle or mobile transportation means with a communication function (automobile, airplane, ship, etc.), and combinations of the various devices described above.

[0138] The communication device is not limited to being portable or mobile, and includes any type of device, apparatus, system that is not portable or is fixed, such as a smart home device (home appliance, lighting device, smart meter or measuring device, control panel, etc.), a vending machine, and any "thing" that can exist on the IoT (Internet of Things) network.

[0139] In recent years, in IoT (Internet of Things) technology, CPS (Cyber Physical Systems), which is a new concept of creating new added value through information cooperation between the physical space and the cyber space, has attracted attention. This CPS concept can also be adopted in the above embodiments.

[0140] That is, as a basic configuration of CPS, for example, an edge server arranged in the physical space and a cloud server arranged in the cyber space are connected via a network, and the processing can be distributed and processed by the processors mounted on both servers. Here, each processing data generated in the edge server or the cloud server is preferably generated on a standardized platform. By using such a standardized platform, it is possible to improve the efficiency when constructing a system including various diverse sensor groups and IoT application software.

[0141] For communication, in addition to data communication by a cellular system, a wireless LAN system, a communication satellite system, etc., data communication by a combination of these is also included.

[0142] In addition, the communication device also includes devices such as a controller and a sensor that are connected or linked to a communication device that executes the communication function described in the present disclosure. For example, a controller and a sensor that generate a control signal or a data signal used by the communication device that executes the communication function of the communication device are included.

[0143] In addition, the communication device also includes infrastructure facilities such as base stations, access points, and any other devices, apparatuses, and systems that communicate with or control the above-described various non-limiting devices.

Industrial Applicability

[0144] One embodiment of the present disclosure is suitable for detecting changes in a person's physical state, for example.

Explanation of Signs

[0145] 1 Body state detection system 10 Millimeter-wave radar 20 Personal computer (PC) 30A, 30B Server 40A, 40B Database (DB) 70 Radar control device 80 State change detection device 90 Display terminal device 101 Detection unit 102 Processing unit 701 Radar communication unit 702 Radar data storage unit 801 Radar data reading unit 802 Sleep state estimation unit 803 Walking speed estimation unit 804 Estimation result storage unit 805 Estimation result reading unit 806 State change detection processing unit 807, 901 Communication unit 808 Radar data memory unit 809 Estimation result information memory unit 902 Display unit 921 State display area 922 Notification area 1100 Computer 1101 Input device 1102 Output device 1103 CPU 1104 GPU (Graphics Processing Unit) 1105 ROM (Read Only Memory) 1106 RAM (Random Access Memory) 1107 Storage device 1108 Reading device 1109 Transceiver device 1110 Bus P Person

Claims

1. An estimation unit that estimates items related to the sleep state and walking speed of the person in time series based on information detected by a radar; A detection unit that detects changes in the physical state of the person based on the estimation results obtained in time series for the plurality of items; An output unit that outputs information regarding the detected changes in the physical state of the person; Comprising: Setting an area corresponding to the sleeping place of the person; Estimating the time zone of the sleep state based on the position of the person with respect to the area; Estimating a walking section by performing correction to approximate the movement trajectory of the person on the radar data of the person in a time zone other than the time zone; Estimating the walking speed in the walking section; An information processing device.

2. The area has a spatial size corresponding to a three-dimensional space occupied by at least one of the supine position, prone position, and lateral lying position at the sleeping place of the person, The information processing device according to claim 1.

3. The estimation unit estimates the walking speed based on the change in the position of the person indicated in the information and the time taken for the change in the position, The information processing device according to claim 1.

4. The radar is a millimeter-wave radar, The information processing device according to claim 1.

5. By one or more information processing devices, Based on information detected by a radar, estimating items related to the sleep state and walking speed of the person in time series, Detecting changes in the physical state of the person based on the estimation results obtained in time series for the plurality of items, Outputting information regarding the detected changes in the physical state of the person, Setting an area corresponding to the sleeping place of the person, Estimating the time zone of the sleep state based on the position of the person with respect to the area, Estimating a walking section by performing correction to approximate the movement trajectory of the person on the radar data of the person in a time zone other than the time zone, Estimating the walking speed in the walking section, An information processing method.

6. By one or more information processing devices, A process of estimating items related to the sleep state and walking speed of the person in time series based on information detected by a radar; A process of detecting changes in the physical state of the person based on the estimation results obtained in time series for the plurality of items; A process of outputting information regarding the detected changes in the physical state of the person; A process of setting an area corresponding to the sleeping place of the person, A process of estimating the time zone of the sleeping state based on the position of the person with respect to the area, A process of estimating a walking section by correcting the radar data of the person in the time zone other than the time zone to approach the movement trajectory of the person, A process of estimating the walking speed in the walking section, To execute, Program.

Images