Sleep state and sleep disease detection system based on human body pressure distribution image

Abstract

This invention belongs to the field of sleep state recognition technology, specifically a sleep state and sleep disorder detection system based on human body pressure distribution images. This invention uses a flexible pressure-sensitive mattress to collect sleep pressure distribution images of the subject; uses an infrared camera to record sleep posture; uses a PSG (Pressure Sensory Organisms) to record sleep stages during the sleep process; uses the PSG to obtain the subject's AHI (Awake Hypoesthesia Index) during sleep and classifies its severity; uses the camera-recorded sleep posture, PSG-obtained sleep stages, and AHI index as the gold standard, and constructs a dataset using pressure distribution images; employs a multi-task learning framework with shared hard parameters to build multiple deep learning models for automatic feature extraction and classification. The models are deployed in an STM32 microcontroller, directly monitoring sleep posture, sleep stages, and OSA (Obstructive Sleep Awareness) within the STM32. This invention enables faster, lower-cost, and more accurate sleep posture recognition, sleep staging, and OSA discrimination.

Description

Technical Field

[0001] This invention belongs to the field of sleep state recognition technology, specifically relating to a sleep state and sleep disorder detection system. Background Technology

[0002] During sleep, various changes occur on the electroencephalogram (EEG). Based on different EEG characteristics, sleep can be divided into several stages: wakefulness (W), non-rapid eye movement (NREM) sleep, and rapid eye movement (REM) sleep. The NREM stage can be further divided into N1 (sleep onset), N2 (light sleep), and N3 (deep sleep). Sleep staging is a crucial step in electrophysiological signal processing used in routine clinical testing and sleep research. Sleep posture, or simply sleeping position, is an important parameter in sleep monitoring and is closely related to human health. Incorrect sleeping posture can lead to diseases of the cervical spine, lumbar spine, and back. Prolonged improper sleeping posture can cause pressure sores due to prolonged pressure on certain parts of the body. Pressure sores, also known as pressure ulcers, refer to tissue necrosis caused by prolonged pressure, resulting in persistent ischemia, hypoxia, and malnutrition. Patients who are bedridden for extended periods often maintain the same posture, making them a high-risk group for this disease; approximately 60,000 people die from pressure sore complications each year. Furthermore, incorrect sleeping postures can obstruct breathing, compress muscles and blood vessels, and affect blood circulation, leading to symptoms such as poor blood flow and blood clots. It can also trigger obstructive sleep apnea (OSA), one of the most common sleep disorders characterized by repeated collapse of the upper airway during sleep, resulting in repeated pauses in breathing and insufficient ventilation, accompanied by snoring and frequent awakenings. The intermittent hypoxia and poor sleep quality caused by OSA increase the risk of malignant tumors in this population. The apnea caused by OSA prevents oxygen from reaching the brain and bloodstream, increasing blood viscosity and potentially inducing arrhythmias, hypertension, and coronary heart disease. In severe cases, it can even lead to sudden death during the night. Statistics show that more than one million people die from coronary heart disease in my country each year, with 30% of these sudden deaths occurring between midnight and 6 a.m. Therefore, long-term monitoring of sleeping posture and prevention of incorrect postures is crucial for reducing the incidence of diseases such as cervical spondylosis, pressure sores, and obstructive sleep apnea syndrome.

[0003] Currently, sleep staging is usually accomplished through visual analysis of polysomnography (PSG). However, due to the high cost of PSG equipment and the need for operation and evaluation by professionals in specialized settings, sleep staging may be subject to subjective errors. Furthermore, the use of PSG requires subjects to wear numerous sensors, which can disrupt their normal sleep.

[0004] In sleep posture monitoring (PSG), wearable body position sensors are used. This method also suffers from the aforementioned drawbacks, such as the sensor's foreign body sensation, reducing sleep comfort and affecting the wearer's normal sleep. Furthermore, accelerometers are significantly affected by motion artifacts, making their accuracy unreliable; PSG devices are expensive and require professional operation and analysis. Another type of sleep posture monitoring method is based on camera recording. While this method does not directly affect the subject's comfort, it still has some drawbacks, such as being greatly affected by changes in ambient light, privacy concerns regarding recording, and potentially increasing the subject's psychological burden.

[0005] The gold standard for diagnosing OSA is also PSG (Physiostomia-Revised Sleep Gynecology), which diagnoses OSA by recording the frequency of apnea and hypopnea events during sleep. An apnea-hypopnea index (AHI) of ≥5 times / hour is considered OSA. Based on the AHI level, OSA can be further classified as mild, moderate, or severe. However, PSG also has its drawbacks: it requires professional operation and analysis, necessitates connecting numerous sensors, and is expensive and time-consuming. Currently, initial screening for OSA in clinical practice uses questionnaires, including the ESS questionnaire, the Berlin questionnaire, and the STOP-Bang questionnaire, to inquire about the subject's medical history, physical parameters, and whether they snore, thereby conducting an assessment. While questionnaires are highly efficient, they clearly suffer from excessive subjectivity.

[0006] Sleep pressure distribution mapping records the relative pressure levels in various parts of the body during sleep. Spatially, the image shows the body's outline and the pressure distribution in different areas, thus enabling sleep posture identification. Temporally, as the chest and abdomen expand and contract with inhalation and exhalation, pressure changes occur, which are reflected in the temporal domain. When sleep apnea or hypoventilation occurs, the pressure changes differ from normal conditions, allowing for the detection of obstructive sleep apnea (OSA). The frequency of these abnormalities can further determine the severity of OSA. Body movement during sleep causes abrupt changes in the pressure distribution image, closely related to the sleep state, which can be used to predict sleep stages. Furthermore, there are correlations between sleep posture, sleep stage, and OSA. For example, regarding OSA and sleep posture: the difference between postural and non-postural OSA. Due to gravity and the attachment point of the tongue and soft palate, patients with postural OSA experience greater severity in the supine position than in the lateral position, while the opposite is true for non-postural OSA. Between OSA and Sleep Stages: Because OSA causes repeated apnea or hypopnea, leading to poor sleep quality, the sleep cycle diagrams of OSA patients differ from those of normal individuals.

[0007] To address some shortcomings of the aforementioned methods, this invention develops a smart mattress, consisting of an array of pressure sensors paired with peripheral control circuitry and corresponding software algorithms. This allows for the acquisition, processing, and analysis of images of human body pressure distribution during sleep. The smart mattress is a relatively stable device, minimally affected by environmental changes. Placed under the bed sheet, it avoids direct contact with the body, ensuring comfort and eliminating privacy concerns. We hope this novel method can quickly, economically, and contactlessly achieve sleep posture recognition, sleep staging, and OSA (Excessive Sleep Acuity) analysis. Summary of the Invention

[0008] In view of the harm to human health caused by improper sleeping posture and the obstructive sleep apnea it may cause, as well as the instability, high cost, and low efficiency of conventional sleep posture monitoring methods, sleep staging and the gold standard PSG for OSA diagnosis, this invention aims to provide a sleep state and sleep disorder detection system based on human pressure distribution images, so as to improve detection quality and efficiency and reduce detection costs.

[0009] This invention provides a sleep state and sleep disorder detection system based on human pressure distribution images. According to the correlation between sleeping posture, sleep stage, and obstructive sleep apnea (OSA), it uses a multi-task learning method based on hard parameter sharing and a deep learning model to classify and discriminate sleeping posture, sleep stage, and OSA, providing faster, more efficient, and lower-cost non-contact discrimination and diagnosis. This invention addresses the shortcomings of current sleeping posture recognition schemes, such as instability, discomfort, and privacy concerns, as well as the high cost and low efficiency of the gold standard for sleep staging and OSA—PSG.

[0010] The sleep state and sleep disorder detection system based on human body pressure distribution images provided by this invention, see [link to relevant documentation]. Figure 2 As shown, it includes: a data acquisition module, a preprocessing module, an automatic feature extraction and shared parameter network module, a sleep posture classification module, a sleep staging module, and an OSA classification module; among which:

[0011] The data acquisition module is used to acquire images of the human body pressure distribution of the subject;

[0012] The preprocessing module is used to preprocess the human body pressure distribution data, filter out noise and interference, enhance useful information, and divide the dataset, using the short-term experimental dataset as the training set and the all-night experimental dataset as the test set.

[0013] The automatic feature extraction and parameter sharing network module is used to extract features from the input image and share these features in subsequent sleep posture classification, sleep staging, and OSA classification tasks.

[0014] The sleeping posture classification module is used to classify sleeping postures, including four types: supine, prone, left lateral, or right lateral.

[0015] The sleep staging module is used to classify sleep stages, including three types: assessment of wakefulness, deep sleep, or light sleep.

[0016] The OSA classification module classifies OSA into four categories: no OSA, mild OSA, moderate OSA, or severe OSA.

[0017] Furthermore:

[0018] The data acquisition module specifically employs an array-type flexible pressure-sensitive mattress. This mattress comprises two flexible printed electrode layers with 32 channels horizontally and 32 channels vertically, and a sensing layer sandwiched between them, made of a polyolefin film. The polyolefin film exhibits pressure-sensitive characteristics, meaning its resistance decreases as the applied pressure increases. The channels in the horizontal and vertical directions intersect and overlap, forming a circuit loop with the central sensing layer, creating 32×32=1024 sensing points. The on / off state of each channel is controlled by an external circuit. For example, when a channel in the horizontal electrode layer is activated, the 32 channels in the vertical electrode layer are sequentially activated. After all channels are activated, the next channel in the horizontal electrode layer is activated, and so on, until the voltage at each sensing point is collected cyclically, yielding the relative pressure at each point. These 1024 values ​​are then mapped to a range of 0-255, reconstructing a 32×32 resolution image. Placing the intelligent pressure-sensitive mattress under the human torso allows for the acquisition of an image of the human body's pressure distribution.

[0019] The preprocessing module includes threshold filtering, peak filtering, Gaussian filtering, and / or opening operation. Wherein:

[0020] The threshold filtering process is as follows: For each pressure distribution image, calculate the average pixel value of all 1024 pixels, set this as the threshold, and identify pixels with pixel values ​​below this threshold as background, setting their pixel values ​​to 0 (pure black); while pixels with pixel values ​​above this threshold remain unchanged. The purpose of threshold filtering is to eliminate the influence of pressure on non-human body parts.

[0021] The peak filtering process is as follows: First, find the peak value of each row and column of pixels in the pressure distribution image, that is, the pixel value that is greater than the adjacent pixel value. After obtaining a series of peak values, set a minimum peak value of 0.75 times the threshold, clear pixel values ​​below this threshold, and leave pixel values ​​above this threshold unchanged. The purpose of peak filtering is to reduce the impact of pressure at a certain location on the surrounding area.

[0022] The process of Gaussian filtering is as follows: a sliding window uses convolution to perform a weighted average on the image, which smooths the image.

[0023] The opening operation process is as follows: first, the image is eroded, and then the image is dilated to remove isolated points, burrs, etc.

[0024] The automatic feature extraction and shared parameter network module designs a lightweight convolutional neural network model based on the ConcatBlock module for the three tasks of sleep posture classification, sleep staging, and OSA classification, and incorporates multi-layer feature fusion. This model is named MSTNet. MSTNet stacks four ConcatBlock layers, one global average pooling layer, and one flattening layer. Each ConcatBlock module contains one Inception layer, one convolutional layer, one BN layer, and one ReLU activation function, which extracts image features. The output feature maps of each ConcatBlock module are subjected to global average pooling and concatenated along the channel direction to achieve feature fusion. The flattening layer flattens the output feature map of the aforementioned automatic feature extraction and shared parameter network module into a one-dimensional feature vector for subsequent classification. The function of this module is to extract features from the input stress distribution image for use in the three tasks of sleep posture classification, sleep staging, and OSA classification.

[0025] The sleep posture classification module specifically includes one Dropout layer and one fully connected layer. The Dropout layer is used to randomly deactivate neural network nodes to reduce model overfitting. The fully connected layer reduces the dimension of the feature vector to the number of sleep posture categories, i.e., a feature vector of length 4. The label corresponding to the largest element value of the feature vector is taken as the predicted sleep posture type, thus completing the sleep posture classification.

[0026] The sleep staging module specifically includes one concatenation layer, three convolutional layers, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The concatenation layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layers further extract sleep-stage-related features from the feature matrix. The BN layer normalizes the features to avoid gradient vanishing and mitigate overfitting. After passing through the non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 3, corresponding to the three sleep stages, thus achieving sleep staging.

[0027] The OSA classification module specifically includes one concatenation layer, one convolutional layer, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The concatenation layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layer further extracts OSA-related features from the feature matrix. The BN layer normalizes the features to avoid gradient vanishing and mitigate overfitting. After passing through a non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 4, corresponding to the four OSA types, thus achieving OSA classification.

[0028] This invention also provides a sleep state and sleep disorder detection system, the specific operation of which is as follows:

[0029] (1) Use an array-type flexible pressure-sensitive mattress to collect images of human body pressure distribution during the subject's sleep. The sleep distribution data format is an N×N two-dimensional array, or can be understood as an image with a resolution of N×N. Each element value represents the relative pressure magnitude at the corresponding position of the array-type mattress.

[0030] (2) Use an infrared camera to record the sleeping posture of the subject during sleep as the gold standard for sleeping posture; use a PSG to record the sleep stages of the subject during sleep as the gold standard for sleep stages;

[0031] (3) Use PSG to obtain the AHI index during the sleep process of the subjects, and classify the severity of OSA according to the AHI index as the gold standard for OSA severity.

[0032] (4) Preprocess the pressure distribution data;

[0033] (5) A dataset was constructed using the stress distribution image dataset, the gold standard for sleeping position, the gold standard for sleep stage, and the gold standard for OSA severity;

[0034] (6) Build several deep learning models, construct a multi-task learning network with shared hard parameters, perform automatic feature extraction and classification, evaluate the models, and select several models with high accuracy and relatively lightweight.

[0035] (7) Model quantization compression technology is adopted to achieve a significant reduction in model size and a significant increase in inference speed at the cost of slightly reducing model accuracy;

[0036] (8) The model is deployed in the STM32 microcontroller, and the sleeping posture, sleep stage and OSA can be monitored directly in the STM32.

[0037] In this invention, in step (2), the sleep state of the subject is classified according to the following criteria:

[0038] Sleeping position: supine, left lateral, right lateral, or prone;

[0039] Sleep stages: awake, deep sleep, or light sleep;

[0040] In this invention, in step (3), the severity of OSA in the subjects is graded according to the following criteria:

[0041] OSA classification: no OSA, mild OSA, moderate OSA, or severe OSA;

[0042] No OSA, AHI < 5 times / hour;

[0043] Mild OSA, 5 times / hour ≤ AHI < 15 times / hour;

[0044] Moderate OSA, 15 times / hour ≤ AHI < 30 times / hour;

[0045] Severe OSA, AHI ≥ 30 times / hour.

[0046] In this invention, the preprocessing methods in step (4) include threshold filtering, peak filtering, Gaussian filtering, opening operation, etc.

[0047] In this invention, in step (6), the deep learning models under the multi-task learning framework include ResNet18, MobileNetV2, MobileNeXt, ShuffleNet and MSTNet.

[0048] In this invention, in step (7), the model quantization compression technology uses the TensorFlow Lite API to perform post-training quantization (PTQ) on the model. Model evaluation metrics include accuracy, precision, and recall.

[0049] In this invention, in step (8), the model deployment technology uses the AI ​​library of STM32CubeMX software to deploy the compressed model to STM32.

[0050] This invention uses an array-type flexible pressure-sensitive mattress to collect sleep pressure distribution images of subjects; an infrared camera to record the subjects' sleeping postures during sleep; a PSG (Physical Sleep Gauge) to record the subjects' sleep stages during sleep; and the PSG to obtain the AHI (Awake Hypoesthesia Index) during sleep, classifying its severity. The pressure distribution images are preprocessed. The sleeping postures recorded by the camera, the sleep stages obtained by the PSG, and the AHI are used as the gold standard to construct a dataset using the pressure distribution images. A multi-task learning framework with shared hard parameters is employed to construct multiple deep learning models for automatic feature extraction and classification. The models are evaluated, and several models with high accuracy and lightweight design are selected. Model quantization compression technology is used to significantly improve inference speed. The models are deployed on an STM32 microcontroller, directly monitoring sleeping postures, sleep stages, and OSA (Obstructive Sleep Awareness) within the STM32. The model of this invention can achieve faster, lower-cost, and more accurate sleep posture recognition, sleep stage classification, and OSA discrimination.

[0051] Compared to existing sleep status and sleep disorder detection technologies, the advantages of this invention are:

[0052] 1. Using human pressure distribution images collected from a flexible pressure-sensitive smart mattress, a multi-task learning mode is adopted to simultaneously process three tasks: sleep posture classification, sleep stage and OSA discrimination. By sharing some parameters of the deep learning network, the overall model size is reduced, the model inference speed is accelerated, and the generalization ability of the model is improved.

[0053] 2. This invention uses human body pressure distribution data for a deep learning model, which can automatically extract features and classify and distinguish sleeping posture, sleep stage and OSA. This can help ordinary people understand their sleep status and assist doctors in the initial screening of OSA. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the sleep state and sleep disease detection process based on human body pressure distribution images, according to the present invention.

[0055] Figure 2 This is a block diagram of a sleep state and sleep disorder detection system based on human body pressure distribution images.

[0056] Figure 3 This is a flowchart of the training process for the sleeping posture recognition, sleep staging, and OSA detection and severity discrimination model based on human pressure distribution images in this embodiment. Implementation

[0057] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0058] The sleep state and sleep disorder detection system based on human body pressure distribution images provided by this invention, see [link to relevant documentation]. Figure 2 As shown, it includes: a data acquisition module, a preprocessing module, an automatic feature extraction and shared parameter network module, a sleep posture classification module, a sleep staging module, and an OSA classification module; among which:

[0059] The data acquisition module is used to acquire images of human body pressure distribution.

[0060] The preprocessing module is used to preprocess human body pressure distribution data, filter out noise and interference, and enhance useful information;

[0061] Automatic Feature Extraction and Parameter Sharing Network Module: Used to extract features from the input image and share these features in subsequent sleep posture classification, sleep staging, and OSA classification tasks;

[0062] Sleep position classification module: used to classify sleeping positions into four categories, namely, supine, prone, left lateral, or right lateral;

[0063] Sleep staging module: used to classify sleep stages into three categories, namely, waking, deep sleep, or light sleep;

[0064] OSA classification module: classifies OSA into four categories: no OSA, mild OSA, moderate OSA, or severe OSA.

[0065] This invention provides a sleep state and sleep disorder detection system based on human body pressure distribution images, the workflow of which is as follows (see...). Figure 3 )for:

[0066] (1) Collect images of human body pressure distribution of the subjects;

[0067] Subjects lay on a flexible pressure-sensitive mattress in a natural sleeping position. The smart pressure mattress collected images of human body pressure distribution at a sampling frequency of 1Hz and transmitted the data to a PC.

[0068] (2) The sleeping posture of the subjects throughout the night was recorded using an infrared camera, and the sleep stages of the subjects during the sleep process were recorded using a PSG. The sleep states of the subjects were classified according to the following criteria:

[0069] Sleeping positions: supine, prone, left lateral, right lateral;

[0070] Sleep stages: waking, deep sleep, light sleep;

[0071] (3) Obtain the AHI index of the subject after overnight PSG, and classify the severity of the subject according to the following criteria:

[0072] OSA classification: no OSA, mild OSA, moderate OSA, or severe OSA;

[0073] No OSA, AHI < 5 times / hour;

[0074] Mild OSA, 5 times / hour ≤ AHI < 15 times / hour;

[0075] Moderate OSA, 15 times / hour ≤ AHI < 30 times / hour;

[0076] Severe OSA, AHI ≥ 30 times / hour;

[0077] (4) Preprocessing the human body pressure distribution image, including threshold filtering, peak filtering, Gaussian filtering, and opening operation on the two-dimensional pressure distribution image. The threshold for threshold filtering is set to the average value of all pixels in the entire image; the threshold for peak filtering is set to 0.75 times the minimum peak value; the kernel size for Gaussian filtering is set to 3×3, and the values ​​are set to [[0.05, 0.1, 0.05], [0.1, 0.4, 0.1], [0.05, 0.1, 0.05]]; the kernel size for opening operation is set to 3×3.

[0078] (5) In this embodiment, a dataset is constructed using pressure distribution images, the gold standard for sleeping position, the gold standard for sleep stages, and the gold standard for OSA severity;

[0079] (6) In this embodiment, the dataset constructed in the previous step is input into a deep learning model under a multi-task learning framework for training. The model uses our designed MSTNet. During the training phase, the Adam optimizer is used, the learning rate is set to 0.001, the loss function is the cross-entropy loss function, the batch size is set to 60, and the number of iterations is set to 50. Since there are three tasks: sleep posture classification, sleep stage classification, and OSA severity determination, and there is a large correlation between sleep posture, sleep stage, and OSA, the hard parameter sharing method in multi-task learning is adopted.

[0080] Multi-task learning (MTL) refers to learning multiple related tasks simultaneously. Treating multiple related tasks as independent single tasks ignores the rich interrelationships between them. In MTL, however, multiple tasks share factors and can share learned information during the learning process. Because tasks complement each other's domain-specific information and mutually promote learning, MTL typically has stronger generalization capabilities. Furthermore, MTL offers advantages such as merging models, sharing partial structures to reduce overall model size and memory usage, and reducing redundant computations during inference, thus accelerating inference speed.

[0081] Hard parameter sharing is a widely used mechanism in multi-task learning. It refers to a model sharing parameters across its main body when handling multiple related tasks, while specific parts of the model operate independently for each task, employing different output structures. This structure reduces the risk of overfitting on a single task by learning shared features across different tasks.

[0082] A deep learning network model is constructed as a multi-task learning network structure with shared hard parameters. For the three tasks of sleep posture classification, sleep staging, and OSA classification, a general feature extraction and parameter sharing network module is first built. This module includes a lightweight convolutional neural network based on the ConcatBlock module, with multi-layer feature fusion added. This model is named MSTNet. MSTNet stacks four ConcatBlock layers, one global average pooling layer, and one flattening layer. Each ConcatBlock module contains one Inception layer, one convolutional layer, one BN layer, and one ReLU activation function, which extracts image features. The output feature maps of each ConcatBlock module are subjected to global average pooling and concatenated along the channel direction to achieve feature fusion. The flattening layer flattens the output feature map of the aforementioned automatic feature extraction and parameter sharing network module into a one-dimensional feature vector for subsequent classification. The function of this module is to extract features from the input stress distribution image for use in the three tasks of sleep posture classification, sleep staging, and OSA classification.

[0083] Secondly, corresponding classification modules were constructed for the three different tasks.

[0084] The sleeping posture classification module includes one Dropout layer and one fully connected layer. The Dropout layer is used to randomly deactivate neural network nodes to reduce model overfitting. The fully connected layer reduces the dimension of the feature vector to the number of sleeping posture categories, i.e., a feature vector of length 4. The label corresponding to the largest element value of the feature vector is taken as the predicted sleeping posture type, thus completing the sleeping posture classification.

[0085] The sleep staging module comprises one concatenation layer, three convolutional layers, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The concatenation layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layers further extract sleep-stage-related features from the feature matrix. The BN layer normalizes the features to avoid gradient vanishing and mitigate overfitting. After passing through a non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 3, corresponding to the three sleep stages, thus achieving sleep staging.

[0086] The OSA classification module consists of one concatenation layer, one convolutional layer, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The concatenation layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layer further extracts OSA-related features from the feature matrix. The BN layer normalizes the features to avoid gradient vanishing and mitigate overfitting. After passing through a non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 4, corresponding to the four OSA types, thus achieving OSA classification.

[0087] Model quantization and compression: The TensorFlow Lite API is used to quantize the model after training, significantly reducing the model size and number of parameters. Short-term experimental data is used as the training set, and all-night experimental data is used as the test set. Model evaluation metrics include accuracy, precision, and recall to verify model performance.

[0088] Model deployment: The model is deployed on the STM32 using the AI ​​library of the STM32CubeMX software.

[0089] This invention differs from separate identification methods for sleep posture, sleep stage, and OSA. It is the first to propose a multi-task learning approach based on hard parameter sharing, integrating sleep posture, sleep stage, and OSA. It employs an advanced lightweight deep learning neural network for sleep posture identification, sleep stage classification, and OSA detection and severity assessment, reducing model size and accelerating inference speed. This method can serve as a rapid preliminary identification method for sleep posture, sleep stage, and OSA, helping patients understand their sleep patterns and assisting doctors in diagnosing sleep disorders.

Claims

1. A sleep state and sleep disease detection method based on a human body pressure distribution image, characterized by, The detection system, based on the correlation between sleep posture, sleep stage, and obstructive sleep apnea (OSA), employs a multi-task learning method based on hard parameter sharing. Using a deep learning model, it classifies and distinguishes sleep posture, sleep stage, and OSA to provide non-contact diagnosis. Specifically, it includes: a data acquisition module, a preprocessing module, an automatic feature extraction and shared parameter network module, a sleep posture classification module, a sleep staging module, and an OSA classification module. The data acquisition module is used to acquire images of the human body pressure distribution of the subject; The preprocessing module is used to preprocess the human body pressure distribution data, filter out noise and interference, enhance useful information, and divide the dataset, using the short-term experimental dataset as the training set and the all-night experimental dataset as the test set. The automatic feature extraction and parameter sharing network module is used to extract features from the input image and share these features in subsequent sleep posture classification, sleep staging, and OSA classification tasks. The sleeping posture classification module is used to classify sleeping postures, including four types: supine, prone, left lateral, or right lateral. The sleep staging module is used to classify sleep stages, including three types: assessment of wakefulness, deep sleep, or light sleep. The OSA classification module classifies OSA into four categories: no OSA, mild OSA, moderate OSA, or severe OSA. The specific steps are as follows: (1) Use an array-type flexible pressure-sensitive mattress to collect images of human body pressure distribution during the subject's sleep. The sleep distribution data format is an N×N two-dimensional array, or can be understood as an image with a resolution of N×N. Each element value represents the relative pressure magnitude at the corresponding position of the array-type mattress. Collect human body pressure distribution images at a sampling frequency of 1Hz and transmit the data to a PC. (2) The sleeping posture of the subjects during sleep was recorded using an infrared camera as the gold standard for sleeping posture; the sleep stages of the subjects during sleep were recorded using a PSG as the gold standard for sleep stages; and the sleep states of the subjects were classified according to the following criteria: Sleeping positions: supine, prone, left lateral, right lateral; Sleep stages: waking, deep sleep, light sleep; (3) Use PSG to obtain the AHI index during the sleep process of the subjects, and classify the severity of OSA according to the AHI index as the gold standard for OSA severity. OSA classification: no OSA, mild OSA, moderate OSA, or severe OSA; No OSA, AHI < 5 times / hour; Mild OSA, 5 times / hour ≤ AHI < 15 times / hour; Moderate OSA, 15 times / hour ≤ AHI < 30 times / hour; Severe OSA, AHI ≥ 30 times / hour; (4) Preprocessing the pressure distribution data, including threshold filtering, peak filtering, Gaussian filtering and opening operation on the two-dimensional pressure distribution image; wherein the threshold for threshold filtering is set to the mean value of all pixel values ​​in the entire image; the threshold for peak filtering is set to 0.75 times the minimum peak value; the kernel size for Gaussian filtering is set to 3×3, and the values ​​are set to [[0.05,0.1, 0.05], [0.1, 0.4, 0.1], [0.05, 0.1, 0.05]]; the kernel size for opening operation is set to 3×3; (5) A dataset was constructed using the stress distribution image dataset, the gold standard for sleeping position, the gold standard for sleep stage, and the gold standard for OSA severity; (6) Build several deep learning models, construct a multi-task learning network with shared hard parameters, perform automatic feature extraction and classification, evaluate the models, and select several models with high accuracy and relatively lightweight features; that is: The dataset constructed in the previous step is input into a deep learning model under a multi-task learning framework for training. The model uses MSTNet. During the training phase, the Adam optimizer is used, the learning rate is set to 0.001, the loss function is the cross-entropy loss function, the batch size is set to 60, and the number of iterations is set to 50. Since there are three tasks: sleep position classification, sleep stage classification, and OSA severity determination, and there is a correlation between sleep position, sleep stage, and OSA, the method of sharing hard parameters in multi-task learning is adopted. A deep learning network model is constructed as a multi-task learning network structure with shared hard parameters. For the three tasks of sleep posture classification, sleep staging, and OSA classification, a general feature extraction and parameter sharing network module is first built. This module includes a lightweight convolutional neural network based on the ConcatBlock module, with multi-layer feature fusion added. This model is named MSTNet. MSTNet stacks four ConcatBlock layers, one global average pooling layer, and one flattening layer. Each ConcatBlock module contains one Inception layer, one convolutional layer, one BN layer, and one ReLU activation function, which extracts image features. The output feature maps of each ConcatBlock module are subjected to global average pooling and concatenated along the channel direction to achieve feature fusion. The flattening layer flattens the output feature maps of the aforementioned automatic feature extraction and parameter sharing network module into a one-dimensional feature vector for subsequent classification. The function of this module is to extract features from the input stress distribution image for use in the three tasks of sleep posture classification, sleep staging, and OSA classification. Secondly, corresponding classification modules were constructed for the three different tasks; The sleeping posture classification module includes one Dropout layer and one fully connected layer. The Dropout layer is used to randomly deactivate neural network nodes to reduce model overfitting. The fully connected layer reduces the dimension of the feature vector to the number of sleeping posture categories, that is, a feature vector of length 4. The label corresponding to the largest element value of the feature vector is taken as the predicted sleeping posture type, thus completing the sleeping posture classification. The sleep staging module consists of one stitching layer, three convolutional layers, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The stitching layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layers further extract sleep-stage-related features from the feature matrix, and the BN layer normalizes the features. After passing through a non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 3, corresponding to the three sleep stages, thus achieving sleep staging. The OSA classification module consists of one concatenation layer, one convolutional layer, one batch normalization (BN) layer, one ReLU activation function layer, one flattening layer, one dropout layer, and one fully connected layer. The concatenation layer concatenates the 30 one-dimensional feature vectors output from 30 consecutive stress distribution images processed by the aforementioned automatic feature extraction and shared parameter network module, forming a 30-channel feature matrix. The convolutional layer further extracts OSA-related features from the feature matrix, and the BN layer normalizes the features. After passing through a non-linear activation function, the output feature matrix is ​​flattened, and the subsequent fully connected layer reduces the feature vector dimension to 4, corresponding to the four OSA types, thus achieving OSA classification. (7) Model quantization compression technology is adopted. The model quantization compression technology uses TensorFlow Lite API to quantize PTQ after training the model; the model evaluation metrics include accuracy, precision, and recall. (8) The model is deployed in the STM32 microcontroller, and the sleeping posture, sleep stage and OSA can be monitored directly in the STM32.

Images