Methods, devices, equipment and media for joint monitoring and analysis of sleep posture and respiratory status

Abstract

This invention provides a method, apparatus, device, and medium for joint monitoring and analysis of sleep posture and respiratory status, relating to the field of artificial intelligence technology. The method includes: acquiring multi-channel signals; the multi-channel signals are mixed vibration signals of the chest and abdomen during sleep; inputting the multi-channel signals into a pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the model; wherein, the joint monitoring and analysis results are the correlation analysis results between sleep posture and OSA (Exhaustive Sleep Acuity). Through the above method, synchronous joint monitoring of sleep posture and OSA is achieved, effectively analyzing the correlation between the two.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for joint monitoring and analysis of sleeping posture and breathing status. Background Technology

[0002] Sleep quality is one of the important indicators of human health. Good sleep can not only improve work efficiency, but also play a positive role in the immune system, heart health, and mental state. In recent years, obstructive sleep apnea syndrome (OSA) has gradually become a major problem affecting people's sleep health.

[0003] The main characteristic of OSA is that during sleep, breathing is interrupted due to upper airway collapse or obstruction, and patients often experience multiple brief awakenings at night. This condition can cause sleep structure disruption, leading to daytime sleepiness, memory loss, poor concentration, and even cardiovascular disease and other health problems.

[0004] Meanwhile, sleeping posture also plays a crucial role in sleep quality and health. Studies have shown that different sleeping postures directly affect the openness of the airway, thus influencing the likelihood of OSA (Obstructive Sleep Apnea): sleeping on one's back can easily lead to upper airway obstruction, worsening OSA symptoms, while sleeping on one's side is considered to help maintain airway patency and alleviate the negative effects of OSA. Therefore, adjusting sleeping posture can help relieve OSA symptoms.

[0005] Although practice suggests that there may be an interaction between sleeping position and OSA, there is currently a lack of quantitative analysis and in-depth research on the relationship between the two. This is because most existing monitoring systems focus only on the independent monitoring of OSA or sleeping position, making it difficult to capture the dynamic relationship between the two simultaneously, thus limiting the breadth and depth of research.

[0006] Currently, there is a lack of a method that can simultaneously monitor and analyze sleeping posture and OSA, making it difficult to analyze the correlation between the two. Summary of the Invention

[0007] This invention provides a method, device, equipment, and medium for joint monitoring and analysis of sleep posture and breathing status, in order to solve the deficiency in the prior art of lacking a method that can simultaneously monitor and analyze sleep posture and OSA, and the difficulty in analyzing the correlation between the two.

[0008] This invention provides a method for joint monitoring and analysis of sleep posture and breathing status, applied to a monitoring system, comprising: acquiring multi-channel signals; the multi-channel signals being mixed vibration signals of the chest and abdomen of a person in a sleep state; inputting the multi-channel signals into a pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the joint monitoring and analysis model; wherein, the joint monitoring and analysis results are the correlation analysis results between sleep posture and OSA.

[0009] According to the present invention, a method for joint monitoring and analysis of sleep posture and breathing status is provided. The joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleep posture subtask network and an OSA subtask network. The feature sharing layer is used to extract features from multi-channel signals to obtain a fused feature matrix. The sleep posture subtask network is used to identify sleep posture based on the fused feature matrix to obtain sleep posture identification features and sleep posture identification results. The OSA subtask network is used to perform OSA detection based on the fused feature matrix to obtain OSA detection features and OSA detection results. The joint attention mechanism layer is used to generate joint monitoring and analysis results based on the sleep posture identification features and OSA detection features.

[0010] According to the present invention, a method for joint monitoring and analysis of sleep posture and breathing status is provided. The feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer. The graph convolutional network is used to extract features from multi-channel signals to obtain differential entropy features, signal similarity features, and channel position features of the multi-channel signals. An adjacency matrix is ​​generated based on the differential entropy features, signal similarity features, and channel position features. Based on the adjacency matrix, a hybrid feature with sensor position information is generated. The temporal coding network is used to extract features from the hybrid feature to obtain temporal features. The spatial coding network is used to extract features from the hybrid feature to obtain spatial features. The spatiotemporal attention mechanism layer is used to fuse the temporal and spatial features to obtain a fused feature matrix.

[0011] According to the present invention, a method for joint monitoring and analysis of sleeping posture and breathing status is provided. The sleeping posture subtask network includes a first convolutional network, a first fully connected layer, a global pooling layer, and a first activation layer. The first convolutional network is used to extract features based on the fused feature matrix to obtain posture features. The first fully connected layer, the global pooling layer, and the first activation layer are used to perform sleeping posture recognition based on the posture features to obtain sleeping posture recognition features and sleeping posture recognition results.

[0012] According to the present invention, a method for joint monitoring and analysis of sleep posture and breathing status is provided. The OSA subtask network includes a second convolutional network, a third convolutional network, a second fully connected layer, and a second activation layer. The second convolutional network is used to extract features based on the fused feature matrix to obtain local features. The third convolutional network is used to extract features based on local features to obtain global features. The second fully connected layer and the second activation layer are used to perform OSA detection based on global features to obtain OSA detection features and OSA detection results.

[0013] According to the present invention, a method for joint monitoring and analysis of sleep posture and respiratory status is provided, which generates joint monitoring and analysis results based on sleep posture recognition features and OSA detection features. The method includes: mapping sleep posture recognition features to a query matrix, and mapping OSA detection features to a key matrix and a value matrix, respectively; calculating similarity based on the query matrix and the key matrix to obtain an attention weight matrix; using the attention weight matrix to characterize the correlation between sleep posture and OSA; generating joint features based on the value matrix and the attention weight matrix; and fusing and updating the sleep posture recognition features and OSA detection features based on the joint features to generate joint monitoring and analysis results.

[0014] The present invention also provides a device for joint monitoring and analysis of sleep posture and breathing status, comprising: an acquisition module for acquiring multi-channel signals; the multi-channel signals being mixed vibration signals of the chest and abdomen of a human body in a sleep state; and a joint monitoring and analysis module for inputting the multi-channel signals into a pre-trained joint monitoring and analysis model to obtain joint monitoring and analysis results output by the joint monitoring and analysis model; wherein the joint monitoring and analysis results are correlation analysis results between sleep posture and OSA.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for joint monitoring and analysis of sleeping posture and breathing status.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for joint monitoring and analysis of sleeping posture and breathing status.

[0017] The present invention provides a method, device, equipment, and medium for joint monitoring and analysis of sleep posture and respiratory status. Since the mixed vibration signal of the chest and abdomen during sleep reflects both the overall cardiopulmonary activity intensity distribution and the dynamic changes in local heart rate and respiratory activity, and the cardiopulmonary activity intensity distribution reflects changes in sleep posture, while the dynamic changes in local heart rate and respiratory activity are key indicators for OSA feature detection, a pre-trained joint monitoring and analysis model is used to analyze the multi-channel mixed vibration signal of the chest and abdomen during sleep. This allows for the simultaneous extraction of sleep posture features and OSA features from the multi-channel signal, leading to a correlation analysis result between sleep posture and OSA. This achieves synchronous joint monitoring of sleep posture and OSA, effectively analyzing the correlation between the two. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the method for joint monitoring and analysis of sleeping posture and breathing status provided by the present invention.

[0020] Figure 2 This is the second flowchart of the method for joint monitoring and analysis of sleeping posture and breathing status provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the generation process of the dynamic graph structure provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the feature sharing layer provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the independent subtask network layer and the joint attention mechanism layer provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the structure of the sleep posture and breathing state joint monitoring and analysis device provided by the present invention.

[0025] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0028] Please see Figure 1 , Figure 1 This is one of the flowcharts illustrating the method for joint monitoring and analysis of sleep posture and breathing status provided by the present invention. In this embodiment, the method for joint monitoring and analysis of sleep posture and breathing status is applied to a monitoring system. The method includes steps S110 to S120, each step of which is detailed below: S110: Acquire multi-channel signals.

[0029] The multi-channel signal is a mixed vibration signal of the chest and abdomen when the human body is in a sleep state.

[0030] Please see Figure 2 , Figure 2 This is the second flowchart of the method for joint monitoring and analysis of sleeping posture and breathing status provided by the present invention.

[0031] like Figure 2 As shown, a multi-channel (e.g., 16-channel, 32-channel, etc.) pressure acquisition system can be formed using a piezoelectric ceramic sensor array and an MSP430 chip to monitor multi-channel mixed vibration signals of the chest and abdomen in real time when the human body is asleep.

[0032] Precisely capturing mixed vibration signals from the human chest and abdomen provides a high-quality data foundation for both sleep posture recognition and OSA detection. The high sensitivity of the sensor array ensures the simultaneous acquisition of multimodal physiological signals such as heart rate, respiration, and changes in body surface pressure, which are crucial for monitoring sleep posture and OSA. Through the system's mixed acquisition and processing, these signals reflect the overall cardiopulmonary activity intensity distribution over the sensor area for sleep posture recognition, while also recording dynamic changes in local heart rate and respiratory activity, focusing on OSA feature detection. By combining these dynamic and static distribution features, subsequent models can simultaneously extract and separate key sleep posture and OSA information from the mixed signals, achieving efficient and accurate multi-task monitoring and providing reliable basic data support for subsequent correlation analysis.

[0033] like Figure 2 As shown, the monitoring system in this embodiment includes a signal processing module for signal preprocessing and a joint monitoring and analysis model. The joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network.

[0034] Specifically, after the piezoelectric ceramic sensor array of the pressure acquisition system acquires multi-channel mixed vibration signals of the chest and abdomen of the human body in a sleep state, the signal processing module can obtain multi-channel signals from the piezoelectric ceramic sensor array and perform preprocessing and signal separation on the multi-channel signals to extract key physiological signals such as cardiopulmonary activity and respiratory rhythm.

[0035] It's important to note that piezoelectric ceramic sensors exhibit high-frequency response characteristics, producing linear output signals for periodic high-frequency vibrations such as breathing, heartbeat, and pulse. However, they produce nonlinear distortion for low-frequency, non-periodic activities like turning over. These non-linearly distorted signals can overwhelm valid physiological information, affecting subsequent calculations. Furthermore, these turning over movements are crucial for changes in sleep posture. Therefore, to locate turning over events while eliminating interference from non-periodic movements, the signal processing stage distinguishes between interfering signals and normal physiological signals by calculating features such as energy entropy, approximate entropy (ApEn), kurtosis, and skewness. For interfering signals, the system acquires accurate body movement and turning over information; for undisturbed signals, the system inputs these normal samples into a feature-sharing layer, ensuring that these high-quality signals can be used for subsequent model training or applications, thereby improving the accuracy and reliability of OSA detection and sleep posture monitoring.

[0036] S120: Input the multi-channel signals into the pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the joint monitoring and analysis model.

[0037] Among them, the joint monitoring and analysis results are the correlation analysis results between sleeping posture and OSA.

[0038] Specifically, to achieve multi-target recognition of OSA and sleeping posture within the same network model and quantitatively analyze the correlation between the two, a feature-sharing layer based on Graph Convolutional Network (GCN), spatial self-attention, and temporal self-attention can be constructed to capture the temporal, spatial, and channel position features of the signal. The GCN is responsible for capturing the positional dependencies between multiple channels of the sensor, generating a dynamic adjacency matrix through differential entropy, and optimizing the graph network structure by combining sensor position information to obtain mixed features of the signal with sensor position information. The spatiotemporal self-attention mechanism is used to further enhance the output features, dynamically evaluating the weight distribution in the temporal and spatial dimensions to provide more modal information for subsequent multi-target detection. This feature optimization mechanism significantly improves the model's accuracy and robustness in multi-task monitoring. Based on the feature-sharing layer, independent sub-task networks for sleeping posture and OSA can be trained separately. Each sub-network focuses on feature extraction and analysis for its respective task. The sleeping posture sub-task network primarily handles sensor channel position features, while the OSA sub-task network focuses on processing temporal and spatial features related to sleep apnea. To quantitatively analyze the correlation between the two tasks, a joint attention mechanism layer is introduced to handle feature fusion between the sleeping posture and OSA sub-task networks. By dynamically adjusting the weights of features in each task, the contribution of each feature to different tasks is analyzed, ultimately achieving joint detection and correlation analysis of sleeping posture and OSA. Specifically, the model can calculate an attention weight matrix based on sleeping posture and OSA features, thereby identifying the correlation between the two, and achieving joint analysis through feature fusion. Through this mechanism, the model can analyze the interaction between sleeping posture and OSA and quantify their correlation.

[0039] Optionally, the sleep posture detection subnetwork can further optimize sleep posture detection by utilizing the features of body turning over obtained during the signal processing stage. A turning over event is a necessary and sufficient condition for a change in sleep posture; by utilizing the features of the current turning over event, the continuity and accuracy of sleep posture recognition can be improved.

[0040] The joint monitoring and analysis method provided in this embodiment can reflect both the overall cardiopulmonary activity intensity distribution and the dynamic changes in local heart rate and respiratory activity in the mixed chest and abdomen vibration signals of a person in a sleep state. The cardiopulmonary activity intensity distribution can reflect changes in sleeping posture, and the dynamic changes in local heart rate and respiratory activity are key indicators for OSA feature detection. Therefore, by using a pre-trained joint monitoring and analysis model to analyze the multi-channel mixed chest and abdomen vibration signals of a person in a sleep state, the sleeping posture features and OSA features of the person can be extracted simultaneously from the multi-channel signals. This allows for the analysis of the correlation between sleeping posture and OSA, achieving synchronous joint monitoring of sleeping posture and OSA, and effectively analyzing the correlation between the two.

[0041] In some embodiments, the joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network. The feature sharing layer is used to extract features from multi-channel signals to obtain a fused feature matrix. The sleeping posture subtask network is used to perform sleeping posture recognition based on the fused feature matrix to obtain sleeping posture recognition features and sleeping posture recognition results. The OSA subtask network is used to perform OSA detection based on the fused feature matrix to obtain OSA detection features and OSA detection results. The joint attention mechanism layer is used to generate joint monitoring and analysis results based on the sleeping posture recognition features and OSA detection features.

[0042] Please continue reading. Figure 2 ,like Figure 2 As shown, the joint monitoring and analysis model in this embodiment includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network.

[0043] The feature sharing layer includes graph convolutional networks, temporal coding networks, spatial coding networks, and spatiotemporal attention mechanism layers.

[0044] The feature sharing layer is used to extract features from multi-channel signals and obtain a fused feature matrix.

[0045] The sleeping posture subtask network is used to perform sleeping posture recognition based on the fused feature matrix, and obtain sleeping posture recognition features and sleeping posture recognition results; wherein, the sleeping posture recognition results are determined based on the sleeping posture recognition features.

[0046] The OSA subtask network is used to perform OSA detection based on the fused feature matrix, and obtain OSA detection features and OSA detection results; wherein, the OSA detection results are determined based on the OSA detection features.

[0047] The joint attention mechanism layer is used to generate joint monitoring and analysis results based on sleeping posture recognition features and OSA detection features.

[0048] In some embodiments, the feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer. The graph convolutional network is used to extract features from multi-channel signals to obtain differential entropy features, signal similarity features, and channel position features of the multi-channel signals. Based on the differential entropy features, signal similarity features, and channel position features, an adjacency matrix is ​​generated, and based on the adjacency matrix, a hybrid feature with sensor position information is generated. The temporal coding network is used to extract features from the hybrid feature to obtain temporal features. The spatial coding network is used to extract features from the hybrid feature to obtain spatial features. The spatiotemporal attention mechanism layer is used to fuse the temporal and spatial features to obtain a fused feature matrix.

[0049] like Figure 2 As shown, the feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer. Please see Figure 3 , Figure 3 This is a schematic diagram of the generation process of the dynamic graph structure provided by the present invention.

[0050] like Figure 3 As shown, the convolutional network is used to extract features from multi-channel signals, obtaining differential entropy features, signal similarity features, and channel position features of the multi-channel signals. Based on the differential entropy features, signal similarity features, and channel position features, an adjacency matrix is ​​generated, and based on the adjacency matrix, a hybrid feature with sensor position information is generated.

[0051] Specifically, in this embodiment, each sensor can provide accurate spatial location information on human posture and cardiopulmonary activity intensity by monitoring mixed vibration signals at different locations in the chest and abdomen. Graph Convolutional Networks (GCNs) effectively capture the complex interaction features between human posture and signals by modeling the spatial dependencies between multi-channel signals, ensuring accurate correlation between sensor distribution and posture information. In GCNs, the network structure and adjacency matrix play a crucial role; the adjacency matrix defines the connection strength between sensor nodes and directly affects network performance. However, because human sleeping posture and force distribution change over time, static graph structures cannot adequately adapt to these dynamic characteristics. Therefore, differential entropy can be introduced during the feature extraction process of GCNs. Differential entropy can assess the uncertainty and complexity of the signals received by each sensor, helping to generate a dynamic adjacency matrix, enabling the graph structure of the GCN to adjust in real time, thereby improving the accuracy and tracking effect of sleeping posture recognition.

[0052] Differential entropy quantifies the complexity of each channel's signal, reflecting its level of uncertainty, thus helping to identify salient signals. In multi-sensor systems, the signals received by sensors are not only affected by spatial location but also closely related to the dynamic environment around the sensor and changes in human posture. By calculating the differential entropy of each sensor channel's signal, it is possible to determine which sensors receive signals with higher complexity and uncertainty, thereby reflecting the dynamic changes in the system's current state. Due to the limited contact area of ​​the body and the fact that the position changes with posture, differential entropy can identify sensor channels that receive more signal variations. These channels typically indicate changes in the contact area or pressure. Higher differential entropy indicates that the channel detects stronger signal activity, reflecting that the sensor is located at a key position in body posture or physiological activity, while lower differential entropy indicates that the channel signal is relatively stable or contains less information, and therefore does not need to be given too much weight in the dynamic graph structure.

[0053] In this embodiment, differential entropy is used to dynamically generate edge weights in the adjacency matrix, thereby optimizing the graph structure of the Graph Convolutional Network (GCN). Differential entropy measures the uncertainty of sensor signals and helps identify which sensor signals are active. The formula for calculating differential entropy is: ; in, It is the probability density function of the signal. Based on the information of differential entropy, the edge weight between each pair of sensor nodes can be determined by the differential entropy between the nodes. The formula for calculating the edge weight between sensor node m and sensor node n is as follows: ; in, Represents the differential entropy of sensor node m; Represents the differential entropy of sensor node n; This represents a measure of the difference between sensor node m and sensor node n. As an adjustment factor, it can control the influence of signal differences or distances on edge weight calculation. By leveraging the effect of differential entropy in weight calculation, an adjacency matrix can be dynamically generated, allowing the graph structure to be adjusted in real time.

[0054] The spatial location information of the sensors and the signal similarity between channels also affect the graph structure relationships. The location information of different channels has a direct impact on signal propagation characteristics; therefore, location information can be used as an additional feature (i.e., channel location features) to help correct the edge weights between sensors. The Euclidean distance formula between the channel positions of sensor node m and sensor node n is: ; in, Represents the coordinate position of sensor node m; This represents the coordinate position of sensor node n.

[0055] Due to changes in human sleeping posture and force application, the signal strength and patterns of different channels will change over time. Signal similarity measures the correlation between these signals. Optionally, the cosine similarity between any two sensor nodes m and n can be used as a signal similarity feature. Cosine similarity can be used to measure the similarity between two signals, and the formula is: ; in, It is the signal vector of sensor node m; It is the signal vector of sensor node n.

[0056] Finally, during the feature extraction process of multi-channel signals, the graph convolutional network obtains the differential entropy features, signal similarity features, and channel position features of the multi-channel signals, and generates an adjacency matrix based on these features. The formula for calculating the adjacency matrix is ​​as follows: ; The above formula combines differential entropy features, channel location features, and signal similarity features, which measure signal complexity, to dynamically generate an adjacency matrix, thereby optimizing the graph structure of the graph convolutional network (GCN) and improving the ability to perceive sleeping posture and signal changes.

[0057] Based on dynamically generated adjacency matrices, Graph Convolutional Networks (GCNs) can capture the spatial characteristics between sensors, integrating channel position features and signal features between adjacent sensors through multiple convolutional operations: ; in, Indicates the first The feature matrix of layer 0 These are the initial input features. It is an adjacency matrix that includes self-connections, where A is a dynamically generated adjacency matrix and I is the identity matrix. It is a node degree matrix. It is a normalized adjacency matrix used to ensure balanced propagation of features. It is the first The weight matrix of the layer, It is an activation function.

[0058] By stacking multiple layers of graph convolutional networks (GCNs), the features of nodes are gradually propagated and fused in space, capturing the dynamic characteristics of pose changes and physiological signal intensity at different locations. In the final output layer, the generated hybrid features with sensor location information are passed to a classifier to achieve accurate identification and classification of physiological signals such as sleeping posture and breathing.

[0059] Please see Figure 4 , Figure 4 This is a schematic diagram of the feature sharing layer provided by the present invention.

[0060] like Figure 4 As shown, in the feature sharing layer of this embodiment, the Graph Convolutional Network (GCN) has fully extracted the mixed features with location information from each sensor through differential entropy and spatial distance optimization. At this point, spatial attention and temporal attention mechanisms can be further utilized to optimize its embedding process. The goal of the shared feature layer is to further fuse temporal and spatial information to ensure that the model can accurately capture the dynamic relationships and temporal changes between signals from different sensors.

[0061] Specifically, a temporal coding network is set up, which uses a temporal attention mechanism to extract features from the mixed features with sensor location information to obtain temporal features; a spatial coding network is set up, which uses a spatial attention mechanism to extract features from the mixed features with sensor location information to obtain spatial features.

[0062] By using a temporal coding network (i.e., a temporal embedding module), the model can capture the changes in signals over time, which is beneficial for OSA detection. The temporal coding network combines Conv1D (one-dimensional convolutional network) and LSTM (Long Short-Term Memory). The one-dimensional convolutional network extracts local temporal features, while the LSTM captures long-term dependencies in the time series. The feature extraction process of the temporal coding network can be represented as follows: ; in, Represents the timing signal of sensor node m; LSTM is used to extract local features, while LSTM is used to capture global temporal changes.

[0063] Furthermore, a spatiotemporal attention mechanism layer is set up to perform feature fusion of temporal and spatial features to obtain a fused feature matrix.

[0064] The spatiotemporal self-attention mechanism can capture the global interactions between sensors, fuse and calculate the spatial attention weights of features. The formula for the spatial attention weights is: ; in, K is the query vector, K is the key vector, and V is the value vector. For the dimension of attention head.

[0065] To embed these features into a unified representation space, a feedforward network (FFN) can be used in the spatiotemporal attention mechanism layer to perform nonlinear transformations and feature fusion on the input features, generating a fused feature matrix. The calculation formula for the feedforward network (FFN) is as follows: ; in, and For learnable weights, and For bias terms, is the activation function. Through the synergistic effect of these modules, the feature sharing layer can effectively integrate the spatial and temporal features of the sensor, providing the model with richer input features, thereby improving the accuracy of sleep posture recognition and OSA detection.

[0066] In some embodiments, the sleeping posture subtask network includes a first convolutional network, a first fully connected layer, a global pooling layer, and a first activation layer; the first convolutional network is used to extract features based on the fused feature matrix to obtain posture features; the first fully connected layer, the global pooling layer, and the first activation layer are used to perform sleeping posture recognition based on the posture features to obtain sleeping posture recognition features and sleeping posture recognition results.

[0067] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of the independent subtask network layer and the joint attention mechanism layer provided by the present invention.

[0068] like Figure 5 As shown, the main objective of the sleeping posture sub-task network is to further extract information related to human sleeping posture from the fused features extracted by the feature sharing layer, thereby achieving efficient posture classification. The feature sharing layer has already extracted and encoded the spatial dependencies and channel characteristics between sensors; therefore, in the sleeping posture sub-task network, a two-dimensional convolutional network can be used to conduct in-depth analysis of these spatial features, and combined with a fully connected layer to complete posture classification.

[0069] In this embodiment, the sleeping posture subtask network includes a first convolutional network, a first fully connected layer, a global pooling layer, and a first activation layer.

[0070] The first convolutional network is a two-dimensional convolutional network used to extract features based on the fused feature matrix and obtain pose features.

[0071] The fused feature matrix output by the feature sharing layer C represents the number of sensor channels, and T represents the time step, indicating the pose characteristics of each sensor channel over time. Two-dimensional convolutional networks can capture key pose features from these fused features. The formula is as follows: ; in, This refers to the kernel size of a two-dimensional convolutional network. The convolution stride of a 2D convolutional network. This is a two-dimensional convolutional network. The purpose of convolution is to find local correlations between data from different sensors, such as the response patterns of different sensors under certain poses.

[0072] Furthermore, by utilizing the first fully connected layer, the global pooling layer, and the first activation layer, sleeping posture recognition is performed based on pose features to obtain sleeping posture recognition features and sleeping posture recognition results.

[0073] Specifically, a classic fully connected layer, a global pooling layer, and a softmax layer (i.e., the first activation layer, using softmax as the activation function) are used to classify pose feature information, obtaining sleeping posture recognition features and sleeping posture recognition results, as shown in the following formula: ; in, It is the mapping function of the fully connected layer. It is the mapping function of the softmax layer. These are the parameters of the fully connected layer. These are the parameters of the softmax layer. This is the result of sleeping posture recognition.

[0074] The sleeping posture subtask network can effectively combine local convolutional feature extraction and global pooling operations, enabling the model to accurately extract and classify posture information in the sleeping posture recognition task.

[0075] Among them, the sleeping posture recognition result is determined based on the sleeping posture recognition features.

[0076] In some embodiments, the OSA subtask network includes a second convolutional network, a third convolutional network, a second fully connected layer, and a second activation layer; the second convolutional network is used to extract features based on the fused feature matrix to obtain local features; the third convolutional network is used to extract features based on the local features to obtain global features; the second fully connected layer and the second activation layer are used to perform OSA detection based on the global features to obtain OSA detection features and OSA detection results.

[0077] The goal of the OSA subtask network is to combine temporal and spatial features. Unlike sleep posture recognition, OSA detection primarily relies on identifying anomalous events (such as sleep apnea) in the temporal dimension and their relationship to body posture. Therefore, the OSA subtask network focuses on extracting key respiratory rhythm changes from temporal signals and combining them with spatial features to detect OSA events.

[0078] Specifically, the OSA subtask network includes a second convolutional network, a third convolutional network, a second fully connected layer, and a second activation layer.

[0079] The second convolutional network is a one-dimensional convolutional network (Conv1D), used for feature extraction based on the fused feature matrix to obtain local features. The one-dimensional convolutional network (Conv1D) is used to extract local features in the time dimension. It can capture short-term changes in breathing apnea over time, and its formula is as follows: ; in, This represents the kernel size of the second convolutional network. This represents the stride of the second convolutional network. The purpose of convolution is to preserve the temporal feature dimension and extract local features within the convolution window.

[0080] After capturing the temporal features, a third convolutional network is used to further extract features based on local features to obtain global features.

[0081] The third convolutional network is a two-dimensional convolutional network (Conv2D), which can combine the correlation between channels and extract global features through a global average pooling layer. : ; in, This is the kernel size of the third convolutional network. is the convolution stride of the third convolutional network.

[0082] The OSA subtask network is similar to the sleeping posture subtask network, using fully connected layers and softmax layers (i.e., the second activation layer, using softmax as the activation function) for OSA detection to obtain OSA detection features and OSA detection results, as shown in the following formula: ; in, For OSA test results, It is the mapping function of the fully connected layer. It is the mapping function of the softmax layer. These are the parameters of the fully connected layer. These are the parameters of the softmax layer.

[0083] The OSA detection results are determined based on OSA detection features.

[0084] In some embodiments, a joint monitoring and analysis result is generated based on sleep posture recognition features and OSA detection features, including: mapping sleep posture recognition features to a query matrix, and mapping OSA detection features to a key matrix and a value matrix, respectively; performing similarity calculation based on the query matrix and the key matrix to obtain an attention weight matrix; the attention weight matrix is ​​used to characterize the correlation between sleep posture and OSA; generating joint features based on the value matrix and the attention weight matrix; and fusing and updating the sleep posture recognition features and OSA detection features based on the joint features to generate the joint monitoring and analysis result.

[0085] Please continue reading. Figure 5 ,like Figure 5 As shown, the joint attention mechanism layer can use the joint attention mechanism to generate joint monitoring and analysis results based on sleeping posture recognition features and OSA detection features.

[0086] In the joint attention mechanism layer, correlation analysis can be performed on the features of the sleeping posture recognition task and the OSA detection task. Sleeping posture recognition features are extracted from the sleeping posture sub-task network. for The feature matrix, representing the OSA detection features extracted from the OSA subtask network. for eigenmatrix , and , This refers to the feature dimension corresponding to the task. To calculate the correlation between the sleeping posture recognition features and the OSA detection features, these two features need to be mapped to the same dimension.

[0087] Specifically, through the linear transformation matrix The sleep posture recognition features are mapped to a query matrix suitable for attention calculation. ,Right now: ; Similarly, through linear transformation matrix and OSA detection features Mapped to key matrices respectively Sum matrix The mapping formula is: ; ; Through the linear transformation described above, the features of the two tasks are transformed into the same dimensional space, thus enabling the capture of their interactions through a joint attention mechanism. The purpose of this step is to provide a unified feature representation for subsequent joint attention computation.

[0088] Furthermore, the query matrix based on the sleeping posture task Key matrix of OSA tasks Similarity calculation is performed to obtain the attention weight matrix. This is used to measure the degree of correlation between two tasks in the feature space. ; in, It is a key matrix The dimension is used as a scaling factor to ensure that the calculated similarity is neither too large nor too small. It is the attention weight matrix, which represents the degree of similarity between two task features. The higher the similarity, the stronger the correlation between the features of the two tasks.

[0089] Furthermore, based on the value matrix Attention weight matrix Weighted summation is performed to generate joint features related to the sleeping posture task. The calculation formula is as follows: .

[0090] Furthermore, based on joint features The sleep posture recognition features and OSA detection features are fused and updated to generate joint monitoring and analysis results.

[0091] Specifically, the joint features They are respectively integrated into sleeping posture recognition features and OSA detection features In this process, updated sleeping posture recognition features can be obtained. and updated OSA detection features These two features serve as the results of the joint monitoring and analysis. They respectively contain joint information from the original task features and the counterpart task features. By analyzing these two updated features, the contribution of the joint features to each task can be obtained: ; ; in, It can reflect the contribution of joint features in the sleeping posture task and can be used to analyze the association between OSA and sleeping posture; It can reflect the contribution of joint features in OSA detection tasks and can be used to analyze the association between sleeping posture and OSA.

[0092] This invention also provides a device for joint monitoring and analysis of sleeping posture and respiratory status. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of the structure of the sleep posture and breathing state joint monitoring and analysis device provided by the present invention. In this embodiment, the sleep posture and breathing state joint monitoring and analysis device includes an acquisition module 610 and a joint monitoring and analysis module 620.

[0093] The acquisition module 610 is used to acquire multi-channel signals.

[0094] The multi-channel signal is a mixed vibration signal of the chest and abdomen when the human body is in a sleep state.

[0095] The joint monitoring and analysis module 620 is used to input multi-channel signals into a pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the joint monitoring and analysis model.

[0096] Among them, the joint monitoring and analysis results are the correlation analysis results between sleeping posture and OSA.

[0097] In some embodiments, the joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network. The feature sharing layer is used to extract features from multi-channel signals to obtain a fused feature matrix. The sleeping posture subtask network is used to perform sleeping posture recognition based on the fused feature matrix to obtain sleeping posture recognition features and sleeping posture recognition results. The OSA subtask network is used to perform OSA detection based on the fused feature matrix to obtain OSA detection features and OSA detection results. The joint attention mechanism layer is used to generate joint monitoring and analysis results based on the sleeping posture recognition features and OSA detection features.

[0098] In some embodiments, the feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer. The graph convolutional network is used to extract features from multi-channel signals to obtain differential entropy features, signal similarity features, and channel position features of the multi-channel signals. Based on the differential entropy features, signal similarity features, and channel position features, an adjacency matrix is ​​generated, and based on the adjacency matrix, a hybrid feature with sensor position information is generated. The temporal coding network is used to extract features from the hybrid feature to obtain temporal features. The spatial coding network is used to extract features from the hybrid feature to obtain spatial features. The spatiotemporal attention mechanism layer is used to fuse the temporal and spatial features to obtain a fused feature matrix.

[0099] In some embodiments, the sleeping posture subtask network includes a first convolutional network, a first fully connected layer, a global pooling layer, and a first activation layer; the first convolutional network is used to extract features based on the fused feature matrix to obtain posture features; the first fully connected layer, the global pooling layer, and the first activation layer are used to perform sleeping posture recognition based on the posture features to obtain sleeping posture recognition features and sleeping posture recognition results.

[0100] In some embodiments, the OSA subtask network includes a second convolutional network, a third convolutional network, a second fully connected layer, and a second activation layer; the second convolutional network is used to extract features based on the fused feature matrix to obtain local features; the third convolutional network is used to extract features based on the local features to obtain global features; the second fully connected layer and the second activation layer are used to perform OSA detection based on the global features to obtain OSA detection features and OSA detection results.

[0101] In some embodiments, the joint attention mechanism layer is specifically used to map sleep posture recognition features to a query matrix and OSA detection features to a key matrix and a value matrix, respectively; based on the query matrix and the key matrix, similarity calculation is performed to obtain an attention weight matrix; the attention weight matrix is ​​used to characterize the correlation between sleep posture and OSA; based on the value matrix and the attention weight matrix, joint features are generated; based on the joint features, the sleep posture recognition features and OSA detection features are fused and updated to generate joint monitoring and analysis results.

[0102] The present invention also provides an electronic device. Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions from the memory 730 to execute a method for jointly monitoring and analyzing sleep posture and breathing status.

[0103] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the sleep posture and breathing state joint monitoring and analysis method provided by the above methods.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for joint monitoring and analysis of sleep posture and respiratory status, characterized in that, Applied to monitoring systems, including: Acquire multi-channel signals; the multi-channel signals are mixed vibration signals of the chest and abdomen of a human body in a sleep state; The multi-channel signals are input into a pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the joint monitoring and analysis model. The joint monitoring and analysis results are the correlation analysis results between sleeping posture and OSA; The joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network. The feature sharing layer is used to extract features from the multi-channel signal to obtain a fused feature matrix; The sleeping posture subtask network is used to perform sleeping posture recognition based on the fused feature matrix, and obtain sleeping posture recognition features and sleeping posture recognition results; The OSA subtask network is used to perform OSA detection based on the fused feature matrix to obtain OSA detection features and OSA detection results. The joint attention mechanism layer is used to generate the joint monitoring and analysis results based on the sleeping posture recognition features and the OSA detection features; The feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer; The graph convolutional network is used to extract features from the multi-channel signal to obtain differential entropy features, signal similarity features, and channel position features of the multi-channel signal. Based on the differential entropy features, signal similarity features, and channel position features, an adjacency matrix is ​​generated, and based on the adjacency matrix, a hybrid feature with sensor position information is generated. The temporal coding network is used to extract features from the hybrid features to obtain temporal features; The spatial coding network is used to extract features from the hybrid features to obtain spatial features; The spatiotemporal attention mechanism layer is used to fuse the temporal features and the spatial features to obtain the fused feature matrix. The process of generating the joint monitoring and analysis results based on the sleeping posture recognition features and the OSA detection features includes: The sleeping posture recognition features are mapped to a query matrix, and the OSA detection features are mapped to a key matrix and a value matrix, respectively. Based on the query matrix and the key matrix, similarity calculation is performed to obtain an attention weight matrix; the attention weight matrix is ​​used to characterize the correlation between sleeping posture and OSA. Based on the value matrix and the attention weight matrix, joint features are generated; Based on the joint features, the sleeping posture recognition features and the OSA detection features are fused and updated to generate the joint monitoring and analysis results; The joint monitoring and analysis results include updated sleeping posture recognition features and updated OSA detection features. The updated sleeping posture recognition features can reflect the contribution of the joint features in the sleeping posture task and can be used to analyze the correlation between OSA and sleeping posture. The updated OSA detection features can reflect the contribution of the joint features in the OSA detection task and can be used to analyze the correlation between sleeping posture and OSA.

2. The method for joint monitoring and analysis of sleep posture and respiratory status according to claim 1, characterized in that, The sleeping posture subtask network includes a first convolutional network, a first fully connected layer, a global pooling layer, and a first activation layer; The first convolutional network is used to extract features based on the fused feature matrix to obtain pose features; The first fully connected layer, the global pooling layer, and the first activation layer are used to perform sleeping posture recognition based on the posture features, and obtain sleeping posture recognition features and sleeping posture recognition results.

3. The method for joint monitoring and analysis of sleep posture and respiratory status according to claim 1, characterized in that, The OSA subtask network includes a second convolutional network, a third convolutional network, a second fully connected layer, and a second activation layer. The second convolutional network is used to extract features based on the fused feature matrix to obtain local features; The third convolutional network is used to extract features based on the local features to obtain global features; The second fully connected layer and the second activation layer are used to perform OSA detection based on the global features to obtain OSA detection features and OSA detection results.

4. A device for joint monitoring and analysis of sleeping posture and respiratory status, characterized in that, include: The acquisition module is used to acquire multi-channel signals; the multi-channel signals are mixed vibration signals of the chest and abdomen of a human body in a sleep state. The joint monitoring and analysis module is used to input the multi-channel signals into a pre-trained joint monitoring and analysis model to obtain the joint monitoring and analysis results output by the joint monitoring and analysis model. The joint monitoring and analysis results are the correlation analysis results between sleeping posture and OSA; The joint monitoring and analysis model includes a feature sharing layer, an independent subtask network layer, and a joint attention mechanism layer connected in sequence. The independent subtask network layer includes a sleeping posture subtask network and an OSA subtask network. The feature sharing layer is used to extract features from the multi-channel signal to obtain a fused feature matrix; The sleeping posture subtask network is used to perform sleeping posture recognition based on the fused feature matrix, and obtain sleeping posture recognition features and sleeping posture recognition results; The OSA subtask network is used to perform OSA detection based on the fused feature matrix to obtain OSA detection features and OSA detection results. The joint attention mechanism layer is used to generate the joint monitoring and analysis results based on the sleeping posture recognition features and the OSA detection features; The feature sharing layer includes a graph convolutional network, a temporal coding network, a spatial coding network, and a spatiotemporal attention mechanism layer; The graph convolutional network is used to extract features from the multi-channel signal to obtain differential entropy features, signal similarity features, and channel position features of the multi-channel signal. Based on the differential entropy features, signal similarity features, and channel position features, an adjacency matrix is ​​generated, and based on the adjacency matrix, a hybrid feature with sensor position information is generated. The temporal coding network is used to extract features from the hybrid features to obtain temporal features; The spatial coding network is used to extract features from the hybrid features to obtain spatial features; The spatiotemporal attention mechanism layer is used to fuse the temporal features and the spatial features to obtain the fused feature matrix. The process of generating the joint monitoring and analysis results based on the sleeping posture recognition features and the OSA detection features includes: The sleeping posture recognition features are mapped to a query matrix, and the OSA detection features are mapped to a key matrix and a value matrix, respectively. Based on the query matrix and the key matrix, similarity calculation is performed to obtain an attention weight matrix; the attention weight matrix is ​​used to characterize the correlation between sleeping posture and OSA. Based on the value matrix and the attention weight matrix, joint features are generated; Based on the joint features, the sleeping posture recognition features and the OSA detection features are fused and updated to generate the joint monitoring and analysis results; The joint monitoring and analysis results include updated sleeping posture recognition features and updated OSA detection features. The updated sleeping posture recognition features can reflect the contribution of the joint features in the sleeping posture task and can be used to analyze the correlation between OSA and sleeping posture. The updated OSA detection features can reflect the contribution of the joint features in the OSA detection task and can be used to analyze the correlation between sleeping posture and OSA.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for joint monitoring and analysis of sleeping posture and breathing status as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for joint monitoring and analysis of sleeping posture and breathing status as described in any one of claims 1 to 3.

Images