Sleep estimation system, sleep estimation method, and program

Abstract

The present disclosure addresses the problem of accurately estimating a sleep state without depending on the temperature of a sleeping environment. A sleep estimation system (1) comprises a thermal image acquisition unit (301), a temperature acquisition unit (302), a body movement analysis unit (303), and a sleep estimation unit (304). The thermal image acquisition unit (301) acquires a thermal image of a sleeper. The temperature acquisition unit (302) acquires the temperature of the sleeper in a sleeping environment. The body movement analysis unit (303) analyzes the body movement during sleep of the sleeper on the basis of the thermal image to derive a body movement feature amount. The sleep estimation unit (304) estimates the sleep state of the sleeper using the body movement feature amount. The body movement analysis unit (303) corrects the body movement feature amount on the basis of the temperature acquired by the temperature acquisition unit (302). The sleep estimation unit (304) estimates the sleep state of the sleeper using the corrected body movement feature amount.

Description

Sleep estimation system, sleep estimation method, and program

[0001] The present disclosure generally relates to a sleep estimation system, a sleep estimation method, and a program, and more particularly to a sleep estimation system, a sleep estimation method, and a program that estimate the sleep state of a sleeper.

[0002] Patent Document 1 describes an air conditioner that determines the sleeping state of a user and controls air conditioning in accordance with the sleeping state.

[0003] The air conditioner in Patent Document 1 identifies a person's head, torso, limbs from a thermal pixel image, which is a temperature distribution detected by an infrared sensor, analyzes the magnitude and frequency of body movements of each identified part, as well as the surface temperature of each part, and predicts and judges the person's sleeping state based on the analysis results.

[0004] When using thermal images obtained from infrared sensors, changes in the temperature of the sleeping environment can affect the detection results of human body parts, which can lead to inaccurate estimation of the sleep state.

[0005] JP 2010-133692 A

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a sleep estimation system, a sleep estimation method, and a program that can accurately estimate a sleep state without depending on the temperature of the sleep environment.

[0007] A sleep estimation system according to one aspect of the present disclosure includes a thermal image acquisition unit, a temperature acquisition unit, a body movement analysis unit, and a sleep estimation unit. The thermal image acquisition unit acquires a thermal image of a sleeping person. The temperature acquisition unit acquires the temperature in the sleeping environment of the sleeping person. The body movement analysis unit analyzes the body movement of the sleeping person during sleep based on the thermal image to obtain a body movement feature. The sleep estimation unit estimates the sleep state of the sleeping person using the body movement feature. The body movement analysis unit corrects the body movement feature based on the temperature acquired by the temperature acquisition unit. The sleep estimation unit estimates the sleep state of the sleeping person using the corrected body movement feature.

[0008] A sleep estimation method according to one aspect of the present disclosure includes a thermal image acquisition step, a temperature acquisition step, a body movement analysis step, and a sleep estimation step. In the thermal image acquisition step, a thermal image of a sleeper is acquired from a thermal image sensor. In the temperature acquisition step, the temperature in the sleep environment of the sleeper is acquired from a temperature sensor. In the body movement analysis step, body movement during sleep of the sleeper is analyzed based on time-series changes in each of a plurality of pixels of the thermal image to obtain a body movement feature. In the sleep estimation step, a sleep state of the sleeper is estimated from the number of occurrences of body movement per predetermined time obtained using the body movement feature. In the body movement analysis step, the body movement feature is corrected based on the temperature acquired in the temperature acquisition step. In the sleep estimation step, the sleep state of the sleeper is estimated using the corrected body movement feature.

[0009] A program according to one aspect of the present disclosure is a program for causing a computer system to execute the sleep estimation method.

[0010] FIG. 1 is a block diagram showing a configuration of a sleep estimation system according to an embodiment of the present disclosure. FIG. 2 is a system diagram showing an example of use of the sleep estimation system. FIG. 3 is a flowchart showing the operation of the sleep estimation system. FIG. 4 is a graph illustrating a relational expression used for correction by the sleep estimation system. FIG. 5 is a graph illustrating a range of the slope of the relational expression used for correction by the sleep estimation system. FIG. 6 is a graph illustrating a graph in which the slope of the relational expression used for correction by the sleep estimation system is at its maximum and minimum values. FIG. 7A is an image diagram showing a thermal image at a reference temperature. FIG. 7B is a waveform diagram showing body movement feature amounts obtained from the thermal image shown in FIG. 7A. FIG. 8A is an image diagram showing a thermal image at a temperature higher than the reference temperature. FIG. 8B is a waveform diagram showing body movement feature amounts obtained from the thermal image shown in FIG. 8A. FIG. 8C is a waveform diagram showing body movement feature amounts after correction of the body movement feature amounts shown in FIG. 8B.

[0011] The embodiments and modifications described below are merely examples of the present disclosure, and the present disclosure is not limited to the embodiments and modifications. Various modifications other than the following embodiments and modifications are possible depending on the design, etc., as long as they do not deviate from the technical concept of the present disclosure.

[0012] (Embodiment) Hereinafter, a sleep estimation system 1 according to this embodiment will be described with reference to Figs. 1 to 8C.

[0013] (1) Overview As shown in FIG. 1 , the sleep estimation system 1 according to the embodiment includes a thermal image acquisition unit 301, a temperature acquisition unit 302, a body movement analysis unit 303, and a sleep estimation unit 304. The thermal image acquisition unit 301 acquires a thermal image of a sleeper u1 (see FIG. 2 ). The temperature acquisition unit 302 acquires the temperature in the sleep environment of the sleeper u1. The body movement analysis unit 303 analyzes the body movement of the sleeper u1 during sleep based on the thermal image to determine a body movement feature amount. The sleep estimation unit 304 estimates the sleep state of the sleeper u1 from the body movement feature amount. The body movement analysis unit 303 corrects the body movement feature amount based on the temperature acquired by the temperature acquisition unit 302. The sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the corrected body movement feature amount.

[0014] According to this configuration, the body movement feature is corrected based on the temperature acquired by the temperature acquisition unit 302, and the sleep state of the sleeper u1 is estimated using the corrected body movement feature, so that the sleep state can be accurately estimated without depending on the temperature of the sleeping environment.

[0015] (2) Configuration (2.1) Sleep Estimation System The sleep estimation system 1 estimates the sleep state of a person (sleeper u1) sleeping in a space SP1 (see FIG. 2 ). The sleep estimation system 1 transmits the estimation result to an information terminal 40. Here, the information terminal 40 is, for example, a smartphone or a tablet terminal.

[0016] As shown in FIG. 1 , the sleep estimation system 1 includes a thermal image sensor 10 , a temperature sensor 20 , and a signal processing device 30 .

[0017] The thermal image sensor 10 receives infrared light emitted from a person. The thermal image sensor 10 outputs a thermal image obtained based on the received infrared light. Specifically, the thermal image sensor 10 receives infrared light emitted from a sleeping person (sleeping person u1) and outputs a thermal image based on the received infrared light to the signal processing device 30. More specifically, the thermal image sensor 10 continuously receives infrared light emitted from the sleeping person u1 and outputs multiple continuous thermal images, i.e., a moving image of thermal images, to the signal processing device 30.

[0018] The temperature sensor 20 detects the temperature in the sleeping environment of the sleeper u1. Specifically, the temperature sensor 20 detects the room temperature (temperature) of the space SP1 in which the sleeper u1 is sleeping. The temperature sensor 20 outputs the detected temperature to the signal processing device 30.

[0019] The signal processing device 30 analyzes the body movement of the sleeping person u1 based on the moving image of the thermal image, obtains a body movement feature amount, and corrects the obtained body movement feature amount. The signal processing device 30 estimates the sleep state of the sleeping person u1 based on the corrected body movement feature amount. As shown in FIG. 1 , the signal processing device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0020] The signal processing device 30 includes, for example, a computer system having a processor and a memory. The processor executes a program stored in the memory, causing the computer system to function as the control unit 33. The program executed by the processor is pre-recorded in the memory of the computer system in this example, but may also be provided by being recorded on a recording medium such as a memory card, or via a telecommunications line such as the Internet.

[0021] The communication unit 31 has a communication interface for communicating with the information terminal 40. For example, the communication unit 31 is configured to be able to communicate with the information terminal 40 via wireless communication. Note that the communication unit 31 may also be configured to be able to communicate with the information terminal 40 via wired communication.

[0022] The storage unit 32 is configured by a device selected from a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), and the like.

[0023] The storage unit 32 stores information used to correct the body movement feature amount. The storage unit 32 stores a relational expression defining the relationship between the body movement feature amount and temperature [K] as information used to correct the body movement feature amount. The relational expression is a linear function expressing the relationship between the body movement feature amount and temperature. Here, the linear function expresses the relationship between the temperature and the body movement feature amount normalized using the body movement feature amount (representative feature amount) obtained at the reference temperature as the reference. The linear function has a negative value as the slope and passes through a reference point. The reference point is a coordinate point formed by the reference temperature and the reference feature amount [AU]. Here, the reference feature amount is a feature amount obtained by normalizing the representative feature amount and has a value of "1." Furthermore, the slope is within a range of -0.122 or more and -0.0262 or less. The storage unit 32 also stores the representative feature amount described above.

[0024] As shown in FIG. 1 , the control unit 33 includes a thermal image acquisition unit 301 , a temperature acquisition unit 302 , a body movement analysis unit 303 , a sleep estimation unit 304 , and a transmission processing unit 305 .

[0025] The thermal image acquisition unit 301 acquires a thermal image of the sleeping person u1. The thermal image acquisition unit 301 acquires a thermal image (moving image) of the sleeping person u1 output from the thermal image sensor 10. That is, the thermal image acquisition unit 301 acquires a thermal image based on the detection result of the thermal image sensor 10 from the thermal image sensor 10.

[0026] The temperature acquisition unit 302 acquires the temperature in the sleeping environment of the sleeper u1. Specifically, the temperature acquisition unit 302 acquires the temperature in the sleeping environment of the sleeper u1 from the temperature sensor 20. More specifically, the temperature acquisition unit 302 acquires the room temperature (temperature) of the space where the sleeper is sleeping from the temperature sensor 20. That is, the temperature acquisition unit 302 acquires the temperature output by the temperature sensor 20.

[0027] The body movement analysis unit 303 analyzes the body movement of the sleeper u1 during sleep based on the thermal image to determine body movement feature amounts. The body movement analysis unit 303 determines the body movement feature amounts based on time-series heat changes obtained from the thermal image motion picture. The thermal image includes a plurality of pixels. The body movement analysis unit 303 determines the body movement feature amounts based on time-series heat changes for each of the plurality of pixels. More specifically, the body movement analysis unit 303 determines the body movement feature amounts of the sleeper u1 based on time-series heat changes for each first predetermined period (e.g., one second).

[0028] Furthermore, the body movement analysis unit 303 corrects the body movement feature amount based on the temperature acquired by the temperature acquisition unit 302. That is, the body movement analysis unit 303 corrects the body movement feature amount of the sleeping person u1, which is calculated based on the thermal change along the time series, based on the temperature acquired by the temperature acquisition unit 302.

[0029] Specifically, the body movement analysis unit 303 corrects the body movement feature amount using the temperature and the relational expression stored in the storage unit 32. Here, the body movement analysis unit 303 corrects the body movement feature amount calculated based on thermal changes over time using the temperature and the relational expression stored in the storage unit 32. More specifically, the body movement analysis unit 303 corrects the body movement feature amount using the temperature and the linear function stored in the storage unit 32. The body movement analysis unit 303 corrects the body movement feature amount by multiplying the body movement feature amount by the correction value obtained by substituting the temperature into the linear function and raising the value obtained to the power of −1. In other words, the body movement analysis unit 303 corrects the body movement feature amount by multiplying the body movement feature amount by the reciprocal of the value obtained by substituting the temperature into the linear function.

[0030] The sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the body movement feature amount. That is, the sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the corrected body movement feature amount. For example, the sleep estimation unit 304 estimates the sleep state of the sleeper u1 using a learning model that inputs the corrected body movement feature amount and outputs the sleep state of the sleeper u1. Specifically, during estimation, the sleep estimation unit 304 uses the corrected body movement feature amount to calculate a body movement density that represents the number of body movement occurrences per predetermined time. The sleep estimation unit 304 estimates the sleep state of the sleeper u1 based on the body movement density for each predetermined time. That is, the sleep estimation unit 304 estimates the sleep state of the sleeper u1 from the number of body movement occurrences per predetermined time obtained using the body movement feature amount.

[0031] The transmission processing unit 305 notifies the information terminal 40 of the estimation result of the sleep estimation unit 304, i.e., the sleep state of the sleeper u1 estimated by the sleep estimation unit 304. Specifically, the transmission processing unit 305 transmits the estimation result of the sleep estimation unit 304 via the communication unit 31.

[0032] (2.2) Information Terminal The information terminal 40 has, for example, a computer system having a processor and a memory. The processor executes a program stored in the memory, causing the computer system to realize the functions of the information terminal 40. The program executed by the processor is pre-recorded in the memory of the computer system here, but it may also be provided by being recorded on a recording medium such as a memory card, or via a telecommunications line such as the Internet.

[0033] The information terminal 40 is configured to be able to communicate with the sleep estimation system 1. Specifically, the information terminal 40 is configured to be able to communicate with the signal processing device 30.

[0034] The information terminal 40 acquires the estimation result of the signal processing device 30 from the signal processing device 30. Upon acquiring the estimation result, the information terminal 40 displays the estimation result on a display unit included in the information terminal 40. Here, the display unit is a thin display device such as a liquid crystal display or an organic electroluminescence (EL) display.

[0035] (3) Operation Here, the operation of the sleep estimation system 1 will be described with reference to the flowchart shown in FIG.

[0036] The thermal image acquisition unit 301 acquires a thermal image of the sleeping person u1 (step S1). The thermal image acquisition unit 301 acquires a thermal image (a moving image) from the thermal image sensor 10 based on the detection result of the thermal image sensor 10.

[0037] The temperature acquisition unit 302 acquires the temperature in the sleeping environment of the sleeper u1 (step S2). The temperature acquisition unit 302 acquires the temperature output by the temperature sensor 20.

[0038] The body movement analysis unit 303 performs a body movement analysis process (step S3).

[0039] The body movement analysis process will be described in detail below.

[0040] The body movement analysis unit 303 analyzes the body movement of the sleeper u1 during sleep based on the thermal image to acquire body movement feature amounts (step S31). More specifically, the body movement analysis unit 303 acquires the body movement feature amounts of the sleeper u1 based on time-series thermal changes for each first predetermined period (e.g., one second).

[0041] The body movement analysis unit 303 acquires the relational expression (linear function) stored in the storage unit 32 from the storage unit 32 (step S32).

[0042] The body movement analysis unit 303 calculates a correction value (step S33). The body movement analysis unit 303 determines the correction value by raising the obtained value to the power of −1 by substituting the temperature detected by the temperature sensor into the obtained linear function.

[0043] The body movement analysis unit 303 performs a correction process (step S34). The body movement analysis unit 303 corrects the body movement feature amounts acquired in step S31. The body movement analysis unit 303 corrects the body movement feature amounts by multiplying the body movement feature amounts acquired in step S31 by a correction value. More specifically, the body movement analysis unit 303 corrects the multiple body movement feature amounts by multiplying the multiple body movement feature amounts acquired for each first predetermined period by the correction value.

[0044] When the body movement analysis unit 303 finishes the body movement analysis process, the sleep estimation unit 304 performs a sleep estimation process (step S4). The sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the body movement feature amount. That is, the sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the corrected body movement feature amount.

[0045] The transmission processing unit 305 performs a notification process (step S5). The transmission processing unit 305 notifies the information terminal 40 of the estimation result of the sleep estimation unit 304.

[0046] (4) Relational Expression Here, a relational expression (linear function) used for correcting the body movement feature amount will be described.

[0047] The relational expression (linear function) representing the relationship between the body movement feature and temperature, used to correct the body movement feature, is determined, for example, through experiments. The distance between the thermal image sensor 10 and the surface of the bedding is kept constant, and the body movement feature is determined at each temperature when the temperature of the space is changed. Then, each body movement feature is normalized so that the body movement feature is "1" at a reference temperature (e.g., 300 K), and the normalized body movement feature is determined. The normalized body movement feature is determined using the formula 1 "Bm(T)_n = Bm(T) / B(T0)." Here, Bm(T)_n represents the normalized feature, Bm(T) represents the body movement feature at temperature T, and B(T0) represents the body movement feature (representative feature) at the reference temperature T0. An example of the normalized body movement feature is shown in FIG. 4 at points P1 to P6.

[0048] Next, by performing linear approximation using points P1 to P6, a relational expression (linear function) that passes through reference point P0 (300[K], 1) shown in FIG. 4 and has a negative slope is found. For example, the linear function is expressed as Bm = -0.0739T + 23.175. The straight line G1 shown in FIG. 4 represents the linear function "Bm = -0.0739T + 23.175". Hereinafter, the straight line G1 may be referred to as the linear function G1.

[0049] When performing the correction process, if a linear function G1 is used, the reciprocal (Bm_n) of the normalized body movement feature amount Bm_n obtained by substituting the temperature T into the linear function G1 is －１ is a correction value that is multiplied by the body movement feature amount at temperature T. The curve G11 shown in FIG.－１ and temperature. That is, the function "(Bm_n) －１ = 1 / (-0.0739T + 23.175)" can also be used to find the correction value.

[0050] Furthermore, because the linear function G1 is determined experimentally, error must be taken into consideration. Therefore, the error range of the body movement feature Bm is set to ±2σ, where σ is 0.478. The effective temperature range is set to 290 to 305 K. 290 K corresponds to the lowest temperature that can be set on an air conditioning device. 305 K corresponds to the limit at which a thermal image sensor can distinguish between a person's body surface temperature and the ambient temperature. Point P11 in FIG. 5 is the maximum error value for the value obtained by the linear function G1 when the temperature is 290 K. Point P12 is the minimum error value for the value obtained by the linear function G1 when the temperature is 290 K. Point P13 is the maximum error value for the value obtained by the linear function G1 when the temperature is 305 K. Point P14 is the minimum error value for the value obtained by the linear function G1 when the temperature is 305 K.

[0051] The relational expression (linear function) used in the correction process preferably passes through a region surrounded by a straight line R1 passing through points P11 and P12, a straight line R2 passing through points P13 and P14, a straight line R3 passing through points P11 and P13, and a straight line R4 passing through points P12 and P14, and also passes through the reference point P0. In other words, the relational expression (linear function) used in the correction process is preferably a straight line that intersects with the straight line R1 when the temperature is 290 [K], intersects with the straight line R2 when the temperature is 305 [K], and passes through the reference point P0.

[0052] Of the multiple linear functions that have a negative slope, intersect with the lines R1 and R2, and pass through the reference point P0, the linear function with the smallest slope is Bm_n = -0.122T + 37.5 (see line G2 in FIG. 5). Of the multiple linear functions that have a negative slope, intersect with the lines R1 and R2, and pass through the reference point P0, the linear function with the largest slope is Bm_n = -0.0262T + 8.85 (see line G3 in FIG. 5). In other words, the slope of the linear function that has a negative slope, intersects with the lines R1 and R2, and passes through the reference point P0 is a value within the range of -0.122 or more and -0.0262 or less.

[0053] When performing the correction process, if the linear function "Bm_n=-0.122T+37.5" is used, the function "(Bm_n) －１ = 1 / (-0.122T + 37.5)" (see curve G21 in FIG. 6). When performing the correction process, if the linear function "Bm_n = -0.0262T + 8.85" is used, the function "(Bm_n) －１ =1 / (-0.0262T+8.85)" (see curve G31 shown in FIG. 6). At this time, the curves G21 and G31 pass through the reference point P0.

[0054] The storage unit 32 may store one of a plurality of linear functions whose slope is a value within a range of not less than −0.122 and not more than −0.0262 and which passes through the reference point P0.

[0055] Alternatively, the storage unit 32 may store the range of the slope and the reference point P0. In this case, the signal processing device 30 of the sleep estimation system 1 can receive from the user the value of the slope of the linear function used for correction, and calculate the linear function from the received value of the slope and the reference point P0.

[0056] (5) Effectiveness of Correction FIG. 7A shows a thermal image Im10 when the temperature is the reference temperature (300 [K]). Region R10 in the thermal image Im10 represents the region detected as the sleeping person u1. FIG. 7B shows a waveform W10 of the body movement feature when the temperature is the reference temperature (300 [K]). FIG. 8A shows a thermal image Im11 when the temperature is 304 [K]. Region R11 in the thermal image Im11 represents the region detected as the sleeping person u1. FIG. 8B shows a waveform W11 of the body movement feature when the temperature is 304 [K].

[0057] Comparing Figures 7A and 8A, region R11 is smaller than region R10. That is, as the temperature increases, the area considered to be the sleeper u1 (i.e., the area of ​​the sleeper u1 in the thermal image acquired by the thermal image acquisition unit 301) becomes smaller due to the relative relationship between the temperature in the sleeping environment and the body temperature of the sleeper u1. Therefore, the change in the thermal image due to body movement also becomes smaller, and the body movement feature amount becomes smaller. As a result, comparing Figures 7B and 8B, the amplitude of waveform W11 is smaller than the amplitude of waveform W10. Therefore, when the body movement density is calculated using waveform W11, the calculated body movement density is smaller than the body movement density obtained from waveform W10.

[0058] Therefore, the body movement analysis unit 303 corrects the body movement feature obtained from the thermal image using, for example, the linear function G1 described above. As shown in Fig. 4, when the temperature exceeds the reference temperature (300 [K]), the normalized feature (normalized body movement feature) becomes a value smaller than 1. In other words, when the temperature exceeds the reference temperature (300 [K]), the correction value, which is the reciprocal of the normalized feature, becomes a value larger than 1. Therefore, by correcting the waveform W11 with the correction value, the amplitude of the waveform W11 increases, and the waveform W21 shown in Fig. 8C can be obtained.

[0059] Therefore, by performing the correction, it is possible to obtain a waveform having an amplitude equivalent to the amplitude of the waveform of the body movement feature obtained at the reference temperature, and therefore it is possible to obtain a body movement density value similar to that at the reference temperature.

[0060] We also verified the effectiveness of the correction based on the evaluation index of the prediction results used to evaluate the learning model.

[0061] Evaluation indices for prediction results include Accuracy, Precision, Recall, F-score, and AUC (Area Under the Curve).

[0062] Accuracy is an index based on the accuracy rate and is an index showing how correct the prediction was. Precision is an index based on the precision rate and is an index showing how correct the predictions were. Recall is an index based on the recall rate and is an index showing how many of the actual correct results were predicted as correct. The F value is the harmonic mean of Precision and Recall. AUC is an index that uses the area under a certain curve as an evaluation index.

[0063] Since the methods for determining these indices are well known, the explanations thereof will be omitted here.

[0064] The indices of Accuracy, Precision, Recall, F-score, and AUC obtained through the experiment for the estimation results when the body movement feature amount was not corrected were 0.777, 0.810, 0.314, 0.453, and 0.642, respectively. On the other hand, the indices of Accuracy, Precision, Recall, F-score, and AUC obtained through the experiment for the estimation results when the body movement feature amount was corrected were 0.801, 0.770, 0.458, 0.574, and 0.701, respectively. Each index when the body movement feature amount was corrected was larger than each index when the body movement feature amount was not corrected. In other words, each index demonstrated that estimating a sleep state by correcting the body movement feature amount is effective.

[0065] (6) Advantages As described above, the sleep estimation system 1 of this embodiment includes a thermal image acquisition unit 301, a temperature acquisition unit 302, a body movement analysis unit 303, and a sleep estimation unit 304. The thermal image acquisition unit 301 acquires a thermal image of the sleeper u1. The temperature acquisition unit 302 acquires the temperature in the sleep environment of the sleeper u1. The body movement analysis unit 303 analyzes the body movement of the sleeper u1 during sleep based on the thermal image to determine body movement feature amounts. The sleep estimation unit 304 estimates the sleep state of the sleeper u1 from the body movement feature amounts. The body movement analysis unit 303 corrects the body movement feature amounts based on the temperature acquired by the temperature acquisition unit 302. The sleep estimation unit 304 estimates the sleep state of the sleeper u1 using the corrected body movement feature amounts.

[0066] According to this configuration, the body movement feature is corrected based on the temperature acquired by the temperature acquisition unit 302, and the sleep state of the sleeper u1 is estimated using the corrected body movement feature, so that the sleep state can be accurately estimated without depending on the temperature of the sleeping environment.

[0067] (7) Modifications Modifications are listed below. The modifications described below can be applied in appropriate combination with the above-described embodiment.

[0068] (7.1) Modification 1 The temperature acquisition unit 302 is configured to acquire the detection result of the temperature sensor 20, but is not limited to this configuration.

[0069] The temperature acquisition unit 302 may estimate the surface temperature of the bedding from the radiation temperature of the surface of the bedding based on the thermal image, and acquire the estimated surface temperature as the temperature in the sleeping environment of the sleeper u1 (environmental temperature). In this case, the sleep estimation system 1 does not necessarily need to include the temperature sensor 20.

[0070] In the first modification, the temperature acquisition unit 302 acquires the radiation temperature of the bedding surface using a thermal image. The temperature acquisition unit 302 acquires the surface temperature of the bedding corresponding to the acquired radiation temperature as the environmental temperature. More specifically, the storage unit 32 stores a database (temperature database) that associates the surface temperatures of the bedding with preset reference radiation temperatures. The temperature acquisition unit 302 uses the temperature database to acquire, as the environmental temperature, the surface temperature corresponding to the reference radiation temperature obtained from the thermal image according to the radiation temperature of the bedding surface. Here, the radiation temperature of the bedding surface is the average value of the entire surface of the bedding.

[0071] In the first modification, when the sleep estimation system 1 is manufactured or shipped, the storage unit 32 stores a temperature database for each distance between the thermal image sensor 10 and the surface of the bedding.

[0072] Once the sleep estimation system 1 is installed, the sleep estimation system 1 receives an input of the distance between the installed thermal image sensor 10 and the surface of the bedding through a user operation, and stores the distance in the storage unit 32. The sleep estimation system 1 also identifies the bedding area using a thermal image output by the thermal image sensor 10 in a situation where no sleeper is present.

[0073] When estimating a sleep state, the temperature acquisition unit 302 of the sleep estimation system 1 acquires a temperature database corresponding to the distance stored in the storage unit 32. The temperature acquisition unit 302 estimates the radiation temperature in the bedding area identified when the sleep estimation system 1 was installed, using the thermal image acquired from the thermal image sensor 10. The temperature acquisition unit 302 uses the acquired temperature database to acquire the environmental temperature corresponding to the estimated radiation temperature.

[0074] As another variation, the memory unit 32 may store a temperature database for each combination of the distance between the thermal image sensor 10 and the surface of the bedding and the material of the bedding when the sleep estimation system 1 is manufactured or shipped. In this case, when the sleep estimation system 1 is installed, the sleep estimation system 1 receives input of the distance between the installed thermal image sensor 10 and the surface of the bedding and the material of the bedding through a user operation, and stores the information in the memory unit 32. When estimating a sleep state, the temperature acquisition unit 302 acquires a temperature database corresponding to the combination of the distance and the material of the bedding stored in the memory unit 32. The temperature acquisition unit 302 uses the acquired temperature database to acquire an environmental temperature corresponding to the estimated radiation temperature.

[0075] (7.2) Modification 2 The temperature acquisition unit 302 may acquire the temperature in the sleeping environment of the sleeper u1 from a temperature sensor provided in a device installed in the space where the sleeper u1 sleeps.

[0076] In this case, the sleep estimation system 1 does not necessarily need to include the temperature sensor 20 .

[0077] (7.3) Modification 3 The sleep estimation system 1 is configured to transmit the estimation result to the information terminal 40, but is not limited to this configuration.

[0078] If the sleep estimation system 1 includes a display unit, the sleep estimation system 1 may display the estimation result on the display unit of the sleep estimation system 1 .

[0079] (7.4) Modification 4 The sleep estimation system 1 may notify the information terminal 40 of the estimation result via a server using a network such as the Internet.

[0080] (7.5) Modification 5 Although the sleep inference system 1 is configured to include the thermal image sensor 10, the configuration is not limited to this. The thermal image sensor 10 is not an essential component of the sleep inference system 1. In other words, the sleep inference system 1 does not have to include the thermal image sensor 10. In short, it is sufficient for the sleep inference system 1 to include at least the signal processing device 30. In this case, the signal processing device 30 of the sleep inference system 1 acquires a thermal image from a thermal image sensor provided outside the sleep inference system 1.

[0081] (7.6) Modification 6 The body movement analysis unit 303 may normalize the body movement feature amount of the sleeping person u1 obtained using the thermal image, and multiply the normalized feature amount by a correction value.

[0082] In this case, the body movement analysis unit 303 performs normalization processing between steps S31 and S32 in the flowchart shown in Fig. 3. Specifically, the body movement analysis unit 303 calculates a normalized body movement feature amount using the body movement feature amount at the temperature detected by the temperature sensor 20 and the body movement feature amount at the reference temperature. For example, the body movement analysis unit 303 calculates the normalized body movement feature amount using the above-mentioned Equation 1. The body movement analysis unit 303 multiplies the calculated normalized body movement feature amount by a correction value to calculate a corrected body movement feature amount.

[0083] (7.7) Modification 7 In the embodiment, the relational expression used in the correction process is a linear function that intersects with the lines R1 and R2 and passes through the reference point P0, but is not limited to this.

[0084] The relational expression used in the correction process may be any function that intersects with the lines R1 and R2 and passes through the reference point P0, or may be a function that represents a curve that intersects with the lines R1 and R2 and passes through the reference point P0.

[0085] In this case, the body movement analysis unit 303 corrects the body movement feature amount by using the reciprocal of the value obtained by a function (for example, a function representing a curve) that intersects with the lines R1 and R2 and passes through the reference point P0 as the correction value.

[0086] (Other Modifications) The above embodiment is merely one of various embodiments of the present disclosure. Various modifications can be made to the above embodiment depending on the design, etc., as long as the object of the present disclosure can be achieved. Furthermore, functions similar to those of the sleep estimation system 1 may be embodied in a sleep estimation method, a computer program, a non-transitory recording medium on which a program is recorded, or the like. The sleep estimation method of the sleep estimation system 1 according to one aspect includes a thermal image acquisition step, a temperature acquisition step, a body movement analysis step, and a sleep estimation step. In the thermal image acquisition step, a thermal image of the sleeper u1 is acquired from the thermal image sensor 10. In the temperature acquisition step, the temperature in the sleep environment of the sleeper u1 is acquired from the temperature sensor 20. In the body movement analysis step, the body movement of the sleeper u1 during sleep is analyzed based on time-series changes in each of multiple pixels in the thermal image to obtain body movement feature values. In the sleep estimation step, the sleep state of the sleeper is estimated from the number of body movement occurrences per predetermined time obtained using the body movement feature values. In the body movement analysis step, the body movement feature values ​​are corrected based on the temperature acquired in the temperature acquisition step. In the sleep estimation step, the sleep state of the sleeper is estimated using the corrected body movement feature amount. A program according to one aspect is a program for causing a computer system to function as the sleep estimation system 1 or the sleep estimation method of the sleep estimation system 1 described above.

[0087] The sleep estimation system 1 or the sleep estimation method of the sleep estimation system 1 according to the present disclosure includes a computer system. The computer system has a processor and a memory as hardware. The processor executes a program stored in the memory of the computer system to realize the function of the sleep estimation system 1 or the sleep estimation method of the sleep estimation system 1 according to the present disclosure. The program may be pre-stored in the memory of the computer system or may be provided via a telecommunications line. The program may also be provided by being recorded on a non-transitory recording medium readable by the computer system, such as a memory card, an optical disk, or a hard disk drive. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integration (VLSIs), or ultra-large-scale integration (ULSIs). Furthermore, a field-programmable gate array (FPGA) that is programmed after the LSI is manufactured, or a logic device that allows the reconfiguration of the connections within the LSI or the reconfiguration of the circuit partitions within the LSI, can also be employed as a processor. Multiple electronic circuits may be integrated into a single chip or distributed across multiple chips. Multiple chips may be integrated into a single device or distributed across multiple devices.

[0088] Furthermore, it is not essential for the sleep estimation system 1 that multiple functions of the sleep estimation system 1 are concentrated in one housing, and the components of the sleep estimation system 1 may be distributed across multiple housings. Furthermore, at least some of the functions of the sleep estimation system 1 may be realized by the cloud (cloud computing) or the like.

[0089] (Summary) As described above, the sleep estimation system (1) of the first aspect includes a thermal image acquisition unit (301), a temperature acquisition unit (302), a body movement analysis unit (303), and a sleep estimation unit (304). The thermal image acquisition unit (301) acquires a thermal image of a sleeper (u1). The body movement analysis unit (303) analyzes the body movement of the sleeper (u1) during sleep based on the thermal image to determine a body movement feature amount. The sleep estimation unit (304) estimates the sleep state of the sleeper (u1) using the body movement feature amount. The body movement analysis unit (303) corrects the body movement feature amount based on the temperature acquired by the temperature acquisition unit (302). The sleep estimation unit (304) estimates the sleep state of the sleeper (u1) using the corrected body movement feature amount.

[0090] According to this aspect, the body movement feature is corrected based on the temperature acquired by the temperature acquisition unit (302), and the sleep state of the sleeper (u1) is estimated using the corrected body movement feature, so that the sleep state can be accurately estimated without depending on the temperature of the sleeping environment.

[0091] In the sleep estimation system (1) of the second aspect, in the first aspect, the body movement analysis unit (303) corrects the body movement feature amount using a relational expression that defines the relationship between the temperature and the body movement feature amount and the temperature.

[0092] According to this aspect, the body movement feature amount is corrected using the temperature and the relational expression, so that the body movement feature amount can be calculated with high accuracy, and as a result, the sleep state can be estimated with high accuracy without depending on the temperature of the sleeping environment.

[0093] In the sleep estimation system (1) of the third aspect, in the second aspect, the relational expression is a linear function expressing the relationship between the body movement feature amount and the temperature. The linear function has a negative slope and passes through the reference point.

[0094] According to this embodiment, the body movement feature amount can be calculated with high accuracy, and as a result, the sleep state can be estimated with high accuracy without depending on the temperature of the sleep environment.

[0095] In the sleep estimation system (1) of the fourth aspect, in the third aspect, the slope is in the range of not less than −0.122 and not more than −0.0262.

[0096] According to this aspect, when the body movement feature amount is calculated using the linear function, an error can be taken into consideration, so that the body movement feature amount can be calculated with high accuracy.

[0097] In the sleep estimation system (1) of the fifth aspect, in the fourth aspect, the body movement analysis unit (303) corrects the body movement feature by multiplying the body movement feature by a correction value obtained by substituting the temperature into a linear function and raising the value to the power of −1.

[0098] According to this embodiment, the body movement feature amount can be obtained with high accuracy.

[0099] In the sleep estimation system (1) of the sixth aspect, in any of the first to fifth aspects, the temperature acquisition unit (302) acquires the radiation temperature of the surface of the bedding using a thermal image, and acquires the surface temperature of the bedding according to the acquired radiation temperature as a temperature.

[0100] According to this embodiment, the temperature of the sleeping environment can be obtained without providing a temperature sensor for detecting the temperature.

[0101] In the sleep estimation system (1) of the seventh aspect, in the sixth aspect, the temperature acquisition unit (302) acquires, as a temperature, a surface temperature corresponding to a reference radiation temperature according to the radiation temperature obtained from the thermal image, using a database in which the surface temperature of the bedding is associated with a preset reference radiation temperature.

[0102] According to this aspect, the temperature of the sleeping environment can be obtained using a database that associates the surface temperature of the bedding with a preset reference radiation temperature.

[0103] In the eighth aspect of the sleep estimation system (1), in any of the first to fifth aspects, the temperature acquisition unit (302) acquires the room temperature of the space (SP1) in which the sleeper (u1) is sleeping from the temperature sensor (20).

[0104] According to this aspect, the temperature of the sleeping environment is acquired from the temperature sensor (20), so that the temperature can be acquired with high accuracy.

[0105] The sleep estimation system (1) of a ninth aspect is any one of the first to eighth aspects, further comprising a thermal image sensor (10). The thermal image sensor (10) receives infrared light emitted from the sleeper (u1). The thermal image acquisition unit (301) acquires a thermal image based on the detection result of the thermal image sensor (10) from the thermal image sensor (10).

[0106] According to this aspect, a thermal image can be acquired from the thermal image sensor (10).

[0107] A tenth aspect of the sleep estimation method includes a thermal image acquisition step, a temperature acquisition step, a body movement analysis step, and a sleep estimation step. In the thermal image acquisition step, a thermal image of a sleeper (u1) is acquired from a thermal image sensor (10). In the temperature acquisition step, the temperature in the sleep environment of the sleeper (u1) is acquired from a temperature sensor (20). In the body movement analysis step, the body movement of the sleeper (u1) during sleep is analyzed based on time-series changes in each of multiple pixels in the thermal image to determine a body movement feature. In the sleep estimation step, the sleep state of the sleeper (u1) is estimated from the number of body movement occurrences per predetermined time period obtained using the body movement feature. In the body movement analysis step, the body movement feature is corrected based on the temperature acquired in the temperature acquisition step. In the sleep estimation step, the sleep state of the sleeper (u1) is estimated using the corrected body movement feature.

[0108] According to this embodiment, the sleep state can be estimated with high accuracy without depending on the temperature of the sleep environment.

[0109] A program according to an eleventh aspect is a program for causing a computer system to execute the sleep estimation method according to the tenth aspect.

[0110] According to this embodiment, the sleep state can be estimated with high accuracy without depending on the temperature of the sleep environment.

[0111] REFERENCE SIGNS LIST 1 Sleep estimation system 10 Thermal image sensor 20 Temperature sensor 30 Signal processing device 301 Thermal image acquisition unit 302 Temperature acquisition unit 303 Body movement analysis unit 304 Sleep estimation unit G1 Linear function SP1 Space u1 Sleeper

Claims

1. A sleep estimation system comprising: a thermal image acquisition unit that acquires a thermal image of a sleeping person; a temperature acquisition unit that acquires the temperature in the sleeping environment of the sleeping person; a body movement analysis unit that analyzes the body movement of the sleeping person during sleep based on the thermal image to determine body movement features; and a sleep estimation unit that estimates the sleeping state of the sleeping person using the body movement features, wherein the body movement analysis unit corrects the body movement features based on the temperature acquired by the temperature acquisition unit, and the sleep estimation unit estimates the sleeping state of the sleeping person using the corrected body movement features.

2. The sleep estimation system according to claim 1, wherein the body movement analysis unit corrects the body movement feature amount using the temperature and a relational expression that defines the relationship between the body movement feature amount and the temperature.

3. The sleep estimation system according to claim 2, wherein the relational expression is a linear function that expresses the relationship between the body movement feature amount and the temperature, and the linear function has a negative slope and passes through a reference point.

4. The sleep estimation system according to claim 3, wherein the slope is within a range of not less than −0.122 and not more than −0.0262.

5. The sleep estimation system of claim 4, wherein the body movement analysis unit corrects the body movement feature by multiplying the body movement feature by a correction value obtained by substituting the temperature into the linear function to the power of -1.

6. A sleep estimation system according to any one of claims 1 to 5, wherein the temperature acquisition unit acquires the radiation temperature of the surface of the bedding using the thermal image, and acquires the surface temperature of the bedding according to the acquired radiation temperature as the temperature.

7. The sleep estimation system of claim 6, wherein the temperature acquisition unit uses a database that associates the surface temperature of the bedding with a predetermined reference radiation temperature to acquire the surface temperature corresponding to the reference radiation temperature according to the radiation temperature obtained from the thermal image as the temperature.

8. The sleep estimation system according to any one of claims 1 to 5, wherein the temperature acquisition unit acquires the room temperature of the space in which the sleeper is sleeping as the temperature from a temperature sensor.

9. A sleep estimation system as described in any one of claims 1 to 5, further comprising a thermal image sensor that receives infrared light emitted from the sleeper, and wherein the thermal image acquisition unit acquires the thermal image from the thermal image sensor based on the detection result of the thermal image sensor.

10. A sleep estimation method comprising: a thermal image acquisition step of acquiring a thermal image of a sleeping person from a thermal image sensor; a temperature acquisition step of acquiring the temperature in the sleeping environment of the sleeping person from a temperature sensor; a body movement analysis step of analyzing the body movement of the sleeping person during sleep based on time-series changes in each of a plurality of pixels of the thermal image to obtain body movement features; and a sleep estimation step of estimating the sleeping state of the sleeping person from the number of body movement occurrences per predetermined time obtained using the body movement features, wherein in the body movement analysis step, the body movement features are corrected based on the temperature acquired in the temperature acquisition step, and in the sleep estimation step, the corrected body movement features are used to estimate the sleeping state of the sleeping person.

11. A program for causing a computer system to execute the sleep estimation method according to claim 10.

Images