Sleep state measurement system, sleep state measurement method, sleep state measurement program

Abstract

Provided is a contactless-type sleep state measurement system 100 capable of accurately measuring the sleep state of a subject irrespective of the age, disorder, or the like of the subject. The contactless-type sleep state measurement system 100 is provided with: a frame image acquisition unit that acquires frame images including a subject P at sleep chronologically; a difference information calculation unit that calculates difference information which is information indicating a difference between two frame images at different times; a subject attribute information acquisition unit that acquires subject attribute information which is information indicating an attribute of the subject; and a sleep state-related information calculation unit that, in accordance with the subject attribute information, and by using the difference information or secondary information obtained therefrom as an explanatory variable, calculates sleep state-related information which is information related to the sleep state of the subject.

Description

[Technical Field]

[0001] The present invention relates to a sleep state measurement system that measures the sleep state of a subject in a non-contact manner. [Background technology]

[0002] Sleep is closely related to the maintenance of brain and physical functions as well as growth and development, and in recent years there has been an increasing need to understand sleep status on a daily basis. However, sleep disorders such as sleep deprivation and insomnia are becoming increasingly problematic. Childhood sleep is particularly important as it is said to affect not only the growth of the subject but also their lifestyle habits in adulthood. Common screening / diagnosis of sleep disorders involves measurements using wearable devices, which are a burden for patients, and assessment of sleep status through interviews at sleep clinics.

[0003] Measurement using a wearable device places a heavy burden on patients and there is a risk that the same sleep data as normal will not be obtained. In addition, patients cannot communicate their own sleep state during interviews, so information must be obtained from caregivers or spouses, which makes it difficult to make an accurate judgment.

[0004] Therefore, recently, there has been a demand for a non-contact sleep state measurement system that can be easily installed and used for measurement at home, etc. In Patent Document 1, parameters related to body movement (amount of body movement, duration of body movement, duration of stillness, etc.) are calculated from the difference between image frames of a video taken by a camera of a sleeping subject, and the depth of sleep is estimated by machine learning based on these parameters. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-164615 Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional sleep depth estimation models do not take into account factors such as the subject's age, and there is room for improvement in accuracy.

[0007] The present invention was first made with this in mind, and aims to provide a non-contact sleep state measurement system that can accurately measure the sleep state of the person being measured, regardless of their age, etc. [Means for solving the problem]

[0008] This sleep state measurement system is characterized by comprising a frame image acquisition unit that acquires frame images including the subject while sleeping in chronological order; a difference information calculation unit that calculates difference information, which is information indicating the difference between two frame images taken at different times; a subject attribute information acquisition unit that acquires subject attribute information, which is information indicating the attributes of the subject; and a sleep state related information calculation unit that calculates sleep state related information, which is information regarding the sleep state of the subject, using the difference information and subject attribute information as explanatory variables. [Effects of the Invention]

[0009] With the above configuration, it is possible to measure the subject in a natural sleeping state without placing a burden on the subject. In addition, the sleeping state is measured using the subject's attribute information, such as age, as an explanatory variable, thereby improving measurement accuracy compared to conventional methods. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an overall schematic diagram of a sleep depth measurement system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a functional block diagram of the information processing apparatus according to the embodiment. [Figure 3] 10A and 10B are explanatory diagrams for explaining a process of correcting difference information according to the embodiment; [Figure 4] 10 is a flowchart illustrating the operation of the sleep depth measurement system according to the embodiment. [Figure 5]FIG. 10 is a diagram illustrating the operation of a sleep depth measurement system according to another embodiment of the present invention. [Figure 6] 3A to 3C are diagrams illustrating the operation of the sleep depth measurement system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] 1st Overview As shown in FIG. 1, the sleep depth measurement system 100 according to this embodiment includes a camera 1 installed in a room such as a bedroom, and an information processing device 2 that receives image data captured by the camera 1, analyzes the image data, and calculates the sleep depth of the subject P.

[0013] Part 2 Structure of each part 1. Camera The camera 1 is equipped with a lens and a two-dimensional sensor such as a CCD, and is configured to capture an image of the sleeping subject P and output the image data (video data). The camera 1 may be a fixed camera, or may be one whose imaging angle and zoom can be adjusted remotely, or may be a depth camera capable of acquiring three-dimensional information.

[0014] 2. Information processing equipment The information processing device 2 is equipped with a CPU, GPU, memory (including volatile memory such as DRAM and non-volatile memory such as HDD), input / output interface, communication interface, display, and input means (touch panel, mouse, keyboard, etc.), and in this embodiment, as shown in the same figure, it is composed of a main unit 21 connected to the camera 1 so as to be able to communicate with it via wired or wireless communication, and a (mobile) terminal 22 connected to this main unit 21 so as to be able to communicate with it via the Internet, etc.

[0015] The information processing device 2 may be a physically integrated device, such as a personal computer, or may further include a server system called a cloud or the like interposed between the main device 21 and the mobile terminal 22. The information processing device 2 may also be integrated with the camera 1. In short, the camera 1 and the information processing device 2 are not limited to their physical forms as long as they perform equivalent functions.

[0016] 3. Program Structure The information processing device 2 functions as a frame image acquisition unit, a target region identification unit, a size-related information acquisition unit, a subject attribute information acquisition unit, a difference information calculation unit, a sleep state-related information calculation unit, etc., as shown in Fig. 2, by the CPU and its peripheral devices cooperating in accordance with the program stored in the memory. Each of these units will be described below.

[0017] (1) Frame image acquisition section This frame image acquisition unit receives, for example, video data from the camera 1, extracts or generates a series of still images (frame images) arranged in time series at regular intervals (here, for example, every 0.5 seconds) from this video data, and stores the data in a specified area of ​​memory. Note that it is also possible to generate frame images on the camera 1 side, and have the frame image acquisition unit receive and acquire these.

[0018] (2) Target area identification part This target region specifying unit specifies a target region (ROI), which is a fixed image region that includes the entirety or a specific part of the subject P, in the frame images acquired by the frame image acquiring unit. The position and range of this target region are common to each frame image.

[0019] As shown in Figure 3, the target area identification unit here identifies as the target area a rectangular area that the operator has set and input so as to include the entire subject P in the captured image (initial frame image) displayed on the display.

[0020] Instead of relying on setting input by an operator, the area in which the subject P is captured may be identified by image recognition using machine learning such as semantic segmentation, and an appropriate fixed-shape area such as a rectangle containing this captured area may be automatically set as the target area. The size and position of the target area relative to the entire frame image may vary for each frame image depending on the subject's movement, etc. However, it is preferable that the size of the target area be set so that the ratio to the size of the subject P captured in the frame is equal. Furthermore, this target area is not limited to a rectangle, but may be a shape measured in pixels.

[0021] (3) Size-related information acquisition unit This size-related information acquisition section acquires size-related information, which is information relating to the relative size of the entire or specific part of the subject P in the frame image.

[0022] Here, the size-related information is simply the number of pixels in the area of ​​the target area or the number of pixels in the length of the target area along a specified direction, but as mentioned above, it is also possible to determine the image range of the subject P using a machine learning image recognition method such as semantic segmentation, and then calculate more accurate size-related information from there, such as by calculating the number of pixels corresponding to the height of the subject P.

[0023] (4) Measured person attribute information acquisition unit This subject attribute information acquisition unit acquires subject attribute information, which is information input by, for example, an operator and indicates the attributes of subject P. The subject attribute information here includes age, sex, height, weight, presence or absence of illness, physical condition, etc.

[0024] It is also possible to obtain this subject attribute information, for example, by linking with an existing PHR (Personal Health Record), without relying on operator input. As long as it is handled together with the measurement data, it can be obtained either at the time of measurement or analysis. Furthermore, image recognition technology may be used to extract physical features of the subject P, and from these, attribute information of the subject, such as age, may be estimated.

[0025] For example, image recognition can be used to extract the eyeball position of subject P and calculate the interpupillary distance. It has been found that interpupillary distance correlates with a child's height, build, age, and level of development, making it possible to estimate the subject's attribute information.

[0026] (5) Difference information acquisition section The difference information acquisition unit calculates difference information for each frame image, which is information indicating the difference between two frame images taken at different times, more specifically, the difference between the first frame image of the two frame images (hereinafter also referred to as the reference frame image) and the frame image taken a certain time interval later (hereinafter also referred to as the comparison frame image).

[0027] The difference information here refers not only to the difference information between the entire reference frame image and the entire comparison frame image, but also to the difference information between a portion of the reference frame image that is to be compared. In this embodiment, to reduce the computational load and noise, the difference information is calculated for only the target region.

[0028] In this embodiment, the difference information refers to the number of pixels whose pixel values ​​differ by more than a predetermined threshold when comparing corresponding pixels in a target area of ​​a reference frame image and a target area of ​​a comparison frame image. If the frame image is grayscale, the pixel value is represented by a one-dimensional value of brightness only, and if the frame image is a color image, it is represented by a three-dimensional value such as RGB or CYK. Note that the difference information may not only be the number of pixels, but may also be the ratio of the number of different pixels to the total number of strokes in the frame image. Furthermore, the amount of movement of a feature point (for example, various body parts or the pupil as described above) may be used as the basis instead of a change in pixel value.

[0029] In addition, in this embodiment, multiple (two) time intervals are set as the time intervals between frame images for obtaining difference information. Here, one of the time intervals, the first time interval, is set to, for example, 0.5 seconds, and the other, the second time interval, is set to 3 seconds. The difference information between frame images at the first time interval (hereinafter also referred to as first difference information) and the difference information between frame images at the second time interval (hereinafter also referred to as second difference information) are sequentially calculated by this difference information calculation unit. Note that, hereinafter, when there is no need to distinguish between the first difference information and the second difference information, they may be simply referred to as difference information.

[0030] (6) Sleep state related information calculation unit The sleep state related information calculation unit uses the difference information and subject attribute information as explanatory variables to calculate sleep state related information, which is information related to the sleep state of the subject P. Note that the sleep state related information referred to here may include body movement, posture, pulse rate, breathing, facial expression, sleep depth, sleep depth occupancy rate (the proportion of time periods occupied by each sleep depth in one sleep session), sleep cycle, etc.

[0031] Third action Next, the operation of the sleep depth measurement system 100 will be described in detail with reference to the flowchart of FIG. 4, which also serves as a more detailed description of each of the above-mentioned components.

[0032] 1. Initial Setup First, when the operator starts the sleep depth measurement system 100 , the image from the camera 1 is displayed on the display of the terminal 22 . The operator adjusts the field of view, angle, and position of the camera 1 while checking the image on the terminal 22 so that the entire subject P is captured. This adjustment can be made by directly operating the camera 1, or by using the PTZ camera 1 to make the adjustment remotely from the terminal 22.

[0033] 2. Acquisition of target area and subject attribute information Next, the operator specifies and inputs a rectangular target area on the screen of terminal 22 so that it includes subject P. This specifies target area information indicating the position and range of this target area in the image, which is acquired by the target area information acquisition unit and stored in a predetermined area of ​​memory (step S1).

[0034] The operator also inputs the subject attribute information at the terminal 22. The input subject attribute information is acquired by the subject attribute information acquisition unit and stored in a predetermined area of ​​the memory (step S1). 3. Acquisition of size-related information

[0035] Next, the size-related information acquisition unit acquires size-related information about the area of ​​the subject P in the frame image (step S2). As described above, the size-related information here is the number of pixels in the area of ​​the target region or the number of pixels in the length direction of the target region.

[0036] When measurement is started by the operator, the frame image acquisition unit first sequentially extracts or generates still images, i.e., frame image data, arranged in chronological order at regular intervals (for example, every 0.5 seconds) from the video data transmitted from camera 1, and stores these frame image data in a specified area of ​​memory (step S3).

[0037] 4. Calculation of differential information Next, the difference information calculation unit calculates difference information between each frame image and a frame image after a certain time interval (step S5). As described above, two time intervals (first time interval and second time interval) are set, and two types of difference information (first difference information and second difference information) are calculated for each target region of the series of frame images.

[0038] At this time, the difference information calculation unit removes obvious noise and errors. For example, if an impossible difference in pixel count (movement) is detected within the first time interval (0.5 seconds), the unit will not count the frame image as an error.

[0039] On the other hand, the sleep state related information calculation unit normalizes the difference information using the magnitude related information. In this embodiment, the normalization of the difference information is performed prior to the calculation of the difference information (step S4).

[0040] Here, the number of pixels in the area and the number of pixels in length and width of the target area are used as size-related information, and the size ratio of the target area to the size of the subject P on the image is made as equal as possible, so this target area is set to the size of the image frame. Then, the difference information calculation unit calculates difference information for this target area.

[0041] Here, we will briefly explain the necessity of normalizing the difference information. When comparing a subject P who appears small in an image with a subject P who appears large, even if the same body movement occurs, the value of the difference information, expressed as the number of pixels, will be smaller for the subject P who appears small. As a result, even if 10 pixels of difference information under one condition corresponds to the movement of one arm, under another condition it may correspond to trunk movement (turning over in sleep) or may be considered noise rather than body movement. If such difference information is used as is to calculate sleep state-related information, it will lead to a decrease in measurement accuracy. Therefore, such problems are solved by correcting the difference information through the normalization described above.

[0042] The normalization method is not limited to the above, and any equivalent calculation process may be performed. For example, the normalization may be performed after calculating the difference information. As an example, a reference value for the magnitude-related information is set in advance, and the difference information is corrected by multiplying the value of the difference information by the reciprocal of the ratio of the acquired value of the magnitude-related information to the reference value.

[0043] Furthermore, after calculating the difference information to be analyzed (time series transition of body movement amount), the entire series of difference information can be normalized to have a mean of 0 and a variance of 1, and used as an input parameter (explanatory variable) for the learning device. Normalizing for each data to be analyzed can suppress variation in the value of the difference information for each data.

[0044] 5. Calculation of sleep state information Next, the sleep state related information calculation unit calculates intermediate parameters (an example of information obtained by processing the difference information referred to in the claims) based on the two types of corrected (normalized) difference information (step S6). Specifically, the first difference information (difference pixel count) is considered to indicate fast body movement, and the second difference information is considered to indicate slow body movement. Then, the unit calculates the average value, variance, and duration for which the difference pixel count remains below a predetermined threshold of the values ​​of the first difference information and the second difference information over a certain period (a period corresponding to a number of epochs, for example, 30 seconds) as intermediate parameters. Note that these intermediate parameters are also included in the sleep state related information. On the other hand, the sleep state related information calculation unit selects a trained model based on the subject attribute information (step S7).

[0045] The trained model is an algorithm generated in advance by machine learning that outputs a sleep depth when the difference information and / or intermediate parameters are input. Here, the training data (the difference information and / or intermediate parameters) are divided into multiple groups according to the values ​​of the subject attribute information, and machine learning is performed on each group to generate a trained model for each value of the subject attribute information.

[0046] More specifically, in this embodiment, age is used as subject attribute information, and a trained model is generated in advance for each age group, such as 0-2 years old, 2-8 years old, etc. Next, the sleep state related information calculation unit calculates the depth of sleep by inputting the intermediate parameters into the trained model (step S8).

[0047] Fourth Alternative Embodiment The present invention is not limited to the above-described embodiment. For example, body movement may be calculated using fluctuations in the coordinates of key points of the body estimated by skeletal structure estimation. The sleep state related information may be measured not only from the entire body of the subject P but also from specific parts such as the face, hands, and feet. Although the body movement information is calculated from difference information between camera video frames, it may also be calculated from fluctuations in output values ​​from a contact sensor or the like at predetermined intervals.

[0048] In the above embodiment, the size-related information is set by the number of pixels in the area reflected in the image of the subject P, but this may also be corrected using the distance to the subject P as a parameter. For example, when capturing an image from an oblique angle, the distance from camera 1 is shorter at the front of the image, and the further back the subject P is, the longer the distance from camera 1 becomes. Therefore, even with the same subject P, if he or she is in the front of the image, he or she will appear larger, and if he or she is in the back, he or she will appear smaller. Therefore, the value of the size-related information can be corrected depending on the position of the subject P in the image. Specifically, if the subject P is in the front, the number of reflected pixels is corrected to be smaller, and the opposite is done if the subject P is in the back.

[0049] In this case, it is more preferable to calculate the size-related information for each image frame. For example, if the subject P is a child, the range of movement due to sleeping position is large, and changes in the relative position (especially distance) with the camera 1 that occur while sleeping cannot be ignored. Furthermore, a body part specifying unit, which is a program for specifying a body part of the person being measured P, may be provided. For example, when the operator designates a face area by tapping or the like, the body part identification unit can detect a face on the premise that the face is located within the designated area.

[0050] Attribute information may also be acquired by estimating it from size-related information. For example, attribute information such as height and age can be estimated from the interpupillary distance (or interocular distance) and head size of the subject P. Through this step, a trained model may be selected based on size-related information.

[0051] One of the trained models may be selected depending on age, etc., or multiple models may be selected and ensemble learning may be performed. For example, if subject P is growing faster than average, it may be possible to use both a trained model based on age and a trained model based on body type (height, weight).

[0052] The user interface may include input items such as "is the subject P's body shape larger / smaller than average?" in addition to input items for age and body shape. The subject P's body shape may be determined automatically by linking it to data on a physical growth curve.

[0053] Regarding "selecting multiple trained models," it is possible to prepare learning devices (models) using the same machine learning method but with different training data, or to use learning devices (models) created using different machine learning methods.

[0054] Depending on attribute information such as age, different machine learning methods may provide higher accuracy in estimating sleep state-related information. For example, from infancy onwards, when a child has acquired a sleep rhythm, characteristics of body movement manifestations according to sleep stages become clear. Because each sleep stage has clear characteristics, it is well suited to determining sleep depth using ExtraTree (majority voting). On the other hand, when estimating sleep depth in infants who have not yet acquired a sleep rhythm, estimating sleep depth using a learning model using a neural network has a higher accuracy rate.

[0055] Furthermore, a trained model that has been trained in advance may be updated with information about the subject P. For example, it is conceivable that the parent of the subject P may annotate information about the depth of sleep. All training data may be prepared for the subject. In this case, it is preferable to increase the training data by data augmentation.

[0056] In the above embodiment, after a series of frame images are acquired, difference information is calculated and the depth of sleep is measured, but it is also possible to calculate the depth of sleep in parallel while capturing the frame images, as in streaming. In the above configuration, the sleep state related information is calculated based on image data of the subject, but it is not limited to image data.

[0057] For example, biometric information may be extracted from the subject, time-varying information of the biometric information may be calculated, and the sleep state-related information may be calculated according to the subject's attribute information using this time-varying information or information obtained by processing the time-varying information as an explanatory variable.

[0058] The biological information referred to here is, for example, one or more of body movement, respiratory rate (or its variance), amplitude of respiratory waveform, heart rate, pulse rate, pulse interval, and time variation of pulse interval. Examples of biological information sensors for measuring this biological information include electromagnetic wave sensors that detect heart rate and pulse waves from the amount of transmitted and reflected electromagnetic waves such as infrared rays, and pressure sensors that detect load changes and load shifts. The biological information sensors may be in the form of a sheet type, a wearable type, a microwave radar type, or the like.

[0059] The biometric information may be extracted by detecting landmarks in the frame images, determining specific regions based on the landmarks, and extracting the specific regions. If the biometric information is, for example, respiratory information (respiratory rate or amplitude of respiratory waveform), the landmarks can be the face or neck. The landmark positions can be based on operator input, or can be determined by machine learning using a trained model that inputs frame images and outputs landmark positions.

[0060] A specific example is shown in Figure 5(a) to (d). Here, a frame image (shown in Figure 5(a)) is rotated at a certain angle within the plane (shown in Figure 5(b)), and landmarks are estimated at multiple angles using machine learning (shown in Figure 5(c)). The landmark detection result at the angle with the highest confidence is adopted (shown in Figure 5(d)).

[0061] If the landmark is a face and the biometric information is respiratory information, the thoracic and abdominal regions are determined from the position of the face region. Respiratory information may be extracted as biometric information from the movement of the estimated thoracic and abdominal regions. However, if the face is tilted relative to the body axis, the actual position of the thoracic and abdominal regions may deviate from the determined position. Therefore, the area around the determined position of the thoracic and abdominal regions may be determined as a target region, and respiratory information may be extracted as biometric information from this target region.

[0062] By extracting biological information from the thoracic and abdominal regions and the target region, noise movements from other regions can be eliminated. The shape of the target region may be set as a doughnut shape within a certain angular range with the determined landmark position as the center of rotation. A specific example will be described with reference to FIG. 6. The ROI (region of interest) of the face detection result in the angle image with high confidence is calculated (FIG. 6(a)), and its center point is calculated (FIG. 6(b)). Then, double concentric circles are drawn from the center point, and a fan shape with a certain angle is set from the doughnut region surrounded by the double concentric circles (FIG. 6(c)). This shape is applied to the angle image with high confidence, and the range of the fan shape is set to the range where the chest (abdomen) may be present (FIG. 6(d)). Limiting the target region in this way has the advantage of reducing the amount of calculation and eliminating noise caused by movements of other parts.

[0063] When the camera described above is considered together with an information processing device that processes the image data, it can also be said to function as a biometric information sensor that detects biometric information.

[0064] Furthermore, since the time variation information or the information obtained by processing the time variation information refers to the time variation of the biometric information, it can be said that the aforementioned difference information or the information obtained by processing this difference information is also included in this time variation information, etc.

[0065] Furthermore, the present invention is not limited to the above-described embodiments and illustrated examples, and can be modified within the scope of the spirit thereof, for example, by combining parts of the embodiments and modified examples.

[0066] 5. Summary The features of the above-described embodiment can be summarized as follows:

[0067] [1] A sleep state measurement system characterized by comprising a frame image acquisition unit that acquires frame images including a subject while sleeping in a chronological order; a difference information calculation unit that calculates difference information, which is information indicating the difference between two frame images taken at different times; a subject attribute information acquisition unit that acquires subject attribute information, which is information indicating the attributes of the subject; and a sleep state related information calculation unit that calculates sleep state related information, which is information regarding the sleep state of the subject, using the difference information and subject attribute information as explanatory variables.

[0068] Such a device not only places no burden on the person being measured and allows measurements to be taken in a natural sleeping state, but also measures the sleep state using the person's attribute information, such as age, as explanatory variables, thereby improving measurement accuracy compared to conventional methods.

[0069] [2] The sleep state measurement system described in [1] is characterized in that the sleep state related information calculation unit selects one or more trained models from among multiple trained models according to the content of the subject attribute information, and provides the difference information to the trained model to calculate the sleep state related information.

[0070] In this case, the sleep state measurement system can be realized by generating multiple types of trained models through machine learning, making it highly feasible and ensuring measurement accuracy. In particular, increasing the amount of training data through data augmentation can promote improvement in measurement accuracy.

[0071] [3] The sleep state measuring system according to [1] or [2], characterized in that the subject's age is used as the subject's attribute information. In such a case, the change patterns of sleep depth (sleep cycle, occupancy rate, etc.) can differ between children, infants, adults, and the elderly. Therefore, by using age (or age group) as attribute information of the person being measured and changing the algorithm for calculating sleep state, such as generating a trained model based on that value, a significant improvement in measurement accuracy can be expected.

[0072] [4] The sleep state measurement system described in any one of [1] to [3] is characterized in that it further comprises a size-related information acquisition unit that acquires size-related information, which is information regarding the relative size of the entire subject or a specific part of the subject in the frame image, and the sleep state-related information calculation unit calculates the sleep state-related information using the size-related information as an explanatory variable.

[0073] [5] The sleep state measurement system described in [4] is characterized in that the sleep state related information calculation unit normalizes the difference information using the magnitude related information, and calculates the sleep state related information based on this normalized difference information and the subject attribute information.

[0074] With this, the difference information and the body movement amount standard calculated based on it can be stabilized regardless of the size of the person being measured who appears in the image, thereby improving the measurement accuracy of sleep state-related information such as sleep depth.

[0075] [6] The sleep state measurement system described in [4] or [5] is characterized in that it further includes a target area identification unit that identifies a target area of ​​a predetermined shape in the frame images that includes the entire subject or a specific part, and the difference information calculation unit is configured to calculate difference information between the target areas in two frame images, and the size-related information acquisition unit acquires the number of area pixels of the target area or the number of length pixels along the number of length pixels in a predetermined direction of the target area as the size-related information. With this, the configuration of [4] or [5] above can be easily realized.

[0076] [7] The sleep state related information includes sleep depth, sleep cycle, and occupancy rate. In this case, the effect of this configuration becomes more pronounced.

[0077] [8] an extraction unit that extracts biological information from the subject while sleeping; a calculation unit that calculates time variation information of the biological information; a subject attribute information acquisition unit that acquires subject attribute information that is information indicating the attributes of the subject; The sleep state measurement system described in [1] is characterized in that it includes a sleep state related information calculation unit that calculates sleep state related information, which is information about the sleep state of the subject, using the time-varying information or information obtained by processing the time-varying information as explanatory variables in accordance with the subject attribute information. With this, it is possible to calculate sleep state related information from biological information.

[0078] [9] The sleep state measurement system described in [8], wherein the biological information includes one or more of body movement, respiratory rate, respiratory rate variance, respiratory waveform amplitude, heart rate, pulse rate, pulse interval, and time variation of pulse interval. By using such biological information, it is possible to improve the accuracy of measuring the sleep state.

[0079]

[10] The sleep state measuring system according to any one of [1] to [9], wherein the extraction unit detects landmarks in the frame images, determines a specific part based on the landmarks, and extracts the biological information from the specific part. By limiting the target region in this way, the amount of calculation can be reduced and noise movements of other parts can be eliminated.

[0080]

[11] The sleep state measuring system according to any one of [1] to [9], wherein the extraction unit detects landmarks in the frame image, determines a specific part based on the landmark, determines the area around the specific part as a target area, and extracts the biological information in the target area. In this way, the target region can be limited, and as described above, the amount of calculation can be reduced and noise movements of other parts can be eliminated.

[0081]

[12] the landmark is the face of the subject, the specific site is the thoracic and abdominal regions, The sleep state measuring system according to

[11] , wherein the biological information is respiratory information. This will improve the accuracy of measuring the sleep state.

[0082]

[13] an extraction unit that extracts biological information from the subject while sleeping; a calculation unit that calculates time variation information of the biological information; a subject attribute information acquisition unit that acquires subject attribute information that is information indicating the attributes of the subject; A sleep state measurement system characterized by comprising a sleep state related information calculation unit that calculates sleep state related information, which is information about the sleep state of the subject, using the time-varying information or information obtained by processing the time-varying information as explanatory variables in accordance with the subject attribute information. This can provide the same effects as the configuration [1] described above. [Industrial Applicability]

[0083] In addition to the fact that it places no burden on the person being measured and allows measurements to be taken in a natural sleeping state, the sleep state is measured using the person's attribute information, such as age, as explanatory variables, which makes it possible to improve measurement accuracy compared to conventional methods. [Explanation of symbols]

[0084] 100 Sleep state measurement system 1. Camera 2. Information processing device P...Person to be measured

Claims

1. a frame image acquisition unit that acquires frame images including the subject during sleep in a time series; a difference information calculation unit that calculates difference information that indicates the difference between two frame images taken at different times; a subject attribute information acquisition unit that acquires subject attribute information that is information indicating the attributes of the subject; a sleep state related information calculation unit that calculates sleep state related information, which is information about the sleep state of the subject, using the difference information or information obtained by processing the difference information as an explanatory variable in accordance with the subject attribute information, a size-related information acquiring unit that acquires size-related information that is information regarding the relative size of the entire subject or a specific part of the subject in the frame image; The sleep state measuring system is characterized in that the sleep state related information calculation unit calculates the sleep state related information using the magnitude related information as an explanatory variable.

2. The sleep state measurement system of claim 1, wherein the sleep state related information calculation unit selects one or more trained models from among a plurality of trained models that are the same or different from each other, according to the content of the subject attribute information, and provides the difference information to the trained model to calculate the sleep state related information.

3. 3. The sleep condition measuring system according to claim 2, wherein the subject's age is used as the subject's attribute information.

4. 2. The sleep state measuring system according to claim 1, wherein the sleep state related information calculation unit normalizes the difference information using the magnitude related information, and calculates the sleep state related information based on this normalized difference information and the subject attribute information.

5. a target area specifying unit that specifies a target area of ​​a predetermined shape that includes the entire subject or a specific part in the frame image, the difference information calculation unit is configured to calculate difference information between the target regions in two frame images, The sleep state measurement system according to claim 1 , wherein the size-related information acquisition unit acquires, as the size-related information, the number of pixels in the area of ​​the target region or the number of pixels in the length of the target region along a predetermined direction.

6. The sleep state measuring system according to claim 1 , wherein the sleep state related information includes a sleep depth.

7. an extraction unit that extracts biological information from the subject while sleeping; a calculation unit that calculates time variation information of the biological information, The sleep state measurement system of claim 1, wherein the sleep state related information calculation unit calculates sleep state related information, which is information about the sleep state of the subject, using the difference information or information obtained by processing the difference information, and the time-varying information or information obtained by processing the time-varying information as explanatory variables, depending on the subject attribute information.

8. The sleep state measuring system according to claim 7 , wherein the biological information includes one or more of body movement, respiratory rate, variance of respiratory rate, amplitude of respiratory waveform, heart rate, pulse rate, pulse interval, and time variation of pulse interval.

9. The sleep state measuring system according to claim 7 , wherein the extracting unit detects landmarks in the frame images, determines a specific region based on the landmarks, and extracts the biological information from the specific region.

10. The sleep state measurement system of claim 7, wherein the extraction unit detects landmarks in the frame images, determines a specific part based on the landmarks, determines the area around the specific part as a target area, and extracts the biological information from the target area.

11. the landmark is the face of the subject, the specific site is the thoracic and abdominal regions, The sleep condition measuring system according to claim 10, wherein the biological information is respiratory information.

12. Frame images including the subject sleeping are acquired in chronological order. Calculating difference information that indicates the difference between the frame images at different times based on the comparison results between the frame images; Acquire subject attribute information which is information indicating the attributes of the subject, Calculating sleep state-related information, which is information about the sleep state of the subject, using the difference information or information obtained by processing the difference information as an explanatory variable according to the subject attribute information; acquiring size-related information that is information regarding the relative size of the entire or specific part of the subject in the frame image; A sleep state measuring method, characterized in that, in calculating the sleep state related information, the magnitude related information is also used as an explanatory variable to calculate the sleep state related information.

13. a frame image acquisition unit that acquires frame images including the subject during sleep in a time series; a difference information calculation unit that calculates difference information indicating the difference between frame images taken at different times based on the comparison results between the frame images; a subject attribute information acquisition unit that acquires subject attribute information that is information indicating the attributes of the subject; a sleep state related information calculation unit that calculates sleep state related information, which is information about the sleep state of the subject, using the difference information or information obtained by processing the difference information as an explanatory variable in accordance with the subject attribute information; a size-related information acquisition unit that acquires size-related information, which is information about the relative size of the entire subject or a specific part of the subject in the frame image; The sleep state measuring program, wherein the sleep state related information calculation unit calculates the sleep state related information using the magnitude related information as an explanatory variable.

Images